diff --git a/README.rst b/README.rst index bc43ac7..4a9e3b0 100644 --- a/README.rst +++ b/README.rst @@ -113,9 +113,9 @@ because it is the main way to communicate with other parts of the bot. A `earwigbot.config.BotConfig`_ stores configuration information for the bot. Its docstring explains what each attribute is used for, but essentially each "node" -(one of ``config.components``, ``wiki``, ``tasks``, ``irc``, and ``metadata``) -maps to a section of the bot's ``config.yml`` file. For example, if -``config.yml`` includes something like:: +(one of ``config.components``, ``wiki``, ``irc``, ``commands``, ``tasks``, and +``metadata``) maps to a section of the bot's ``config.yml`` file. For example, +if ``config.yml`` includes something like:: irc: frontend: @@ -133,7 +133,8 @@ Custom IRC commands ~~~~~~~~~~~~~~~~~~~ Custom commands are subclasses of `earwigbot.commands.Command`_ that override -``Command``'s ``process()`` (and optionally ``check()``) methods. +``Command``'s ``process()`` (and optionally ``check()`` or ``setup()``) +methods. The bot has a wide selection of built-in commands and plugins to act as sample code and/or to give ideas. Start with test_, and then check out chanops_ and diff --git a/docs/customizing.rst b/docs/customizing.rst index 3c8f2bc..3336353 100644 --- a/docs/customizing.rst +++ b/docs/customizing.rst @@ -58,8 +58,13 @@ The most useful attributes are: :py:class:`earwigbot.config.BotConfig` stores configuration information for the bot. Its docstrings explains what each attribute is used for, but essentially -each "node" (one of :py:attr:`config.components`, :py:attr:`wiki`, -:py:attr:`tasks`, :py:attr:`tasks`, or :py:attr:`metadata`) maps to a section +each "node" (one of :py:attr:`config.components +`, +:py:attr:`~earwigbot.config.BotConfig.wiki`, +:py:attr:`~earwigbot.config.BotConfig.irc`, +:py:attr:`~earwigbot.config.BotConfig.commands`, +:py:attr:`~earwigbot.config.BotConfig.tasks`, or +:py:attr:`~earwigbot.config.BotConfig.metadata`) maps to a section of the bot's :file:`config.yml` file. For example, if :file:`config.yml` includes something like:: @@ -81,7 +86,8 @@ Custom IRC commands Custom commands are subclasses of :py:class:`earwigbot.commands.Command` that override :py:class:`~earwigbot.commands.Command`'s :py:meth:`~earwigbot.commands.Command.process` (and optionally -:py:meth:`~earwigbot.commands.Command.check`) methods. +:py:meth:`~earwigbot.commands.Command.check` or +:py:meth:`~earwigbot.commands.Command.setup`) methods. :py:class:`~earwigbot.commands.Command`'s docstrings should explain what each attribute and method is for and what they should be overridden with, but these @@ -90,6 +96,15 @@ are the basics: - Class attribute :py:attr:`~earwigbot.commands.Command.name` is the name of the command. This must be specified. +- Class attribute :py:attr:`~earwigbot.commands.Command.commands` is a list of + names that will trigger this command. It defaults to the command's + :py:attr:`~earwigbot.commands.Command.name`, but you can override it with + multiple names to serve as aliases. This is handled by the default + :py:meth:`~earwigbot.commands.Command.check` implementation (see below), so + if :py:meth:`~earwigbot.commands.Command.check` is overridden, this is + ignored by everything except the help_ command (so ``!help alias`` will + trigger help for the actual command). + - Class attribute :py:attr:`~earwigbot.commands.Command.hooks` is a list of the "IRC events" that this command might respond to. It defaults to ``["msg"]``, but options include ``"msg_private"`` (for private messages only), @@ -97,15 +112,25 @@ are the basics: joins a channel). See the afc_status_ plugin for a command that responds to other hook types. +- Method :py:meth:`~earwigbot.commands.Command.setup` is called *once* with no + arguments immediately after the command is first loaded. Does nothing by + default; treat it like an :py:meth:`__init__` if you want + (:py:meth:`~earwigbot.tasks.Command.__init__` does things by default and a + dedicated setup method is often easier than overriding + :py:meth:`~earwigbot.tasks.Command.__init__` and using :py:obj:`super`). + - Method :py:meth:`~earwigbot.commands.Command.check` is passed a :py:class:`~earwigbot.irc.data.Data` object, and should return ``True`` if you want to respond to this message, or ``False`` otherwise. The default behavior is to return ``True`` only if :py:attr:`data.is_command` is ``True`` - and :py:attr:`data.command` == :py:attr:`~earwigbot.commands.Command.name`, - which is suitable for most cases. A common, straightforward reason for - overriding is if a command has aliases (see chanops_ for an example). Note - that by returning ``True``, you prevent any other commands from responding to - this message. + and :py:attr:`data.command` ``==`` + :py:attr:`~earwigbot.commands.Command.name` (or :py:attr:`data.command + ` is in + :py:attr:`~earwigbot.commands.Command.commands` if that list is overriden; + see above), which is suitable for most cases. A possible reason for + overriding is if you want to do something in response to events from a + specific channel only. Note that by returning ``True``, you prevent any other + commands from responding to this message. - Method :py:meth:`~earwigbot.commands.Command.process` is passed the same :py:class:`~earwigbot.irc.data.Data` object as @@ -128,6 +153,12 @@ are the basics: `, and :py:meth:`part(chan) `. +Commands have access to :py:attr:`config.commands[command_name]` for config +information, which is a node in :file:`config.yml` like every other attribute +of :py:attr:`bot.config`. This can be used to store, for example, API keys or +SQL connection info, so that these can be easily changed without modifying the +command itself. + The command *class* doesn't need a specific name, but it should logically follow the command's name. The filename doesn't matter, but it is recommended to match the command name for readability. Multiple command classes are allowed @@ -190,7 +221,7 @@ are the basics: Tasks have access to :py:attr:`config.tasks[task_name]` for config information, which is a node in :file:`config.yml` like every other attribute of -:py:attr:`bot.config`. This can be used to store, for example, edit summaries, +:py:attr:`bot.config`. This can be used to store, for example, edit summaries or templates to append to user talk pages, so that these can be easily changed without modifying the task itself. @@ -201,8 +232,9 @@ the task name for readability. Multiple tasks classes are allowed in one file. See the built-in wikiproject_tagger_ task for a relatively straightforward task, or the afc_statistics_ plugin for a more complicated one. +.. _help: https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/help.py .. _afc_status: https://github.com/earwig/earwigbot-plugins/blob/develop/commands/afc_status.py -.. _chanops: https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/chanops.py .. _test: https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/test.py +.. _chanops: https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/chanops.py .. _wikiproject_tagger: https://github.com/earwig/earwigbot/blob/develop/earwigbot/tasks/wikiproject_tagger.py .. _afc_statistics: https://github.com/earwig/earwigbot-plugins/blob/develop/tasks/afc_statistics.py diff --git a/docs/toolset.rst b/docs/toolset.rst index a1db42b..d22b4af 100644 --- a/docs/toolset.rst +++ b/docs/toolset.rst @@ -80,6 +80,8 @@ following attributes: ``"en"`` - :py:attr:`~earwigbot.wiki.site.Site.domain`: the site's web domain, like ``"en.wikipedia.org"`` +- :py:attr:`~earwigbot.wiki.site.Site.url`: the site's full base URL, like + ``"https://en.wikipedia.org"`` and the following methods: @@ -97,11 +99,11 @@ and the following methods: - :py:meth:`namespace_name_to_id(name) `: given a namespace name, returns the associated namespace ID -- :py:meth:`get_page(title, follow_redirects=False) +- :py:meth:`get_page(title, follow_redirects=False, ...) `: returns a ``Page`` object for the given title (or a :py:class:`~earwigbot.wiki.category.Category` object if the page's namespace is "``Category:``") -- :py:meth:`get_category(catname, follow_redirects=False) +- :py:meth:`get_category(catname, follow_redirects=False, ...) `: returns a ``Category`` object for the given title (sans namespace) - :py:meth:`get_user(username) `: returns a @@ -120,7 +122,7 @@ provide the following attributes: - :py:attr:`~earwigbot.wiki.page.Page.site`: the page's corresponding :py:class:`~earwigbot.wiki.site.Site` object - :py:attr:`~earwigbot.wiki.page.Page.title`: the page's title, or pagename -- :py:attr:`~earwigbot.wiki.page.Page.exists`: whether the page exists +- :py:attr:`~earwigbot.wiki.page.Page.exists`: whether or not the page exists - :py:attr:`~earwigbot.wiki.page.Page.pageid`: an integer ID representing the page - :py:attr:`~earwigbot.wiki.page.Page.url`: the page's URL @@ -166,9 +168,10 @@ or :py:meth:`site.get_page(title) ` where ``title`` is in the ``Category:`` namespace) provide the following additional method: -- :py:meth:`get_members(use_sql=False, limit=None) - `: returns a list of page - titles in the category (limit is ``50`` by default if using the API) +- :py:meth:`get_members(use_sql=False, limit=None, ...) + `: iterates over + :py:class:`~earwigbot.wiki.page.Page`\ s in the category, until either the + category is exhausted or (if given) ``limit`` is reached Users ~~~~~ @@ -178,6 +181,8 @@ Create :py:class:`earwigbot.wiki.User ` objects with :py:meth:`page.get_creator() `. They provide the following attributes: +- :py:attr:`~earwigbot.wiki.user.User.site`: the user's corresponding + :py:class:`~earwigbot.wiki.site.Site` object - :py:attr:`~earwigbot.wiki.user.User.name`: the user's username - :py:attr:`~earwigbot.wiki.user.User.exists`: ``True`` if the user exists, or ``False`` if they do not diff --git a/earwigbot/commands/__init__.py b/earwigbot/commands/__init__.py index 7a89fb0..ee58e95 100644 --- a/earwigbot/commands/__init__.py +++ b/earwigbot/commands/__init__.py @@ -36,9 +36,13 @@ class Command(object): This docstring is reported to the user when they type ``"!help "``. """ - # This is the command's name, as reported to the user when they use !help: + # The command's name, as reported to the user when they use !help: name = None + # A list of names that will trigger this command. If left empty, it will + # be triggered by the command's name and its name only: + commands = [] + # Hooks are "msg", "msg_private", "msg_public", and "join". "msg" is the # default behavior; if you wish to override that, change the value in your # command subclass: @@ -49,9 +53,10 @@ class Command(object): This is called once when the command is loaded (from :py:meth:`commands.load() `). - *bot* is out base :py:class:`~earwigbot.bot.Bot` object. Generally you - shouldn't need to override this; if you do, call - ``super(Command, self).__init__()`` first. + *bot* is out base :py:class:`~earwigbot.bot.Bot` object. Don't override + this directly; if you do, remember to place + ``super(Command, self).__init()`` first. Use :py:meth:`setup` for + typical command-init/setup needs. """ self.bot = bot self.config = bot.config @@ -67,6 +72,15 @@ class Command(object): self.mode = lambda t, level, msg: self.bot.frontend.mode(t, level, msg) self.pong = lambda target: self.bot.frontend.pong(target) + self.setup() + + def setup(self): + """Hook called immediately after the command is loaded. + + Does nothing by default; feel free to override. + """ + pass + def check(self, data): """Return whether this command should be called in response to *data*. @@ -76,11 +90,15 @@ class Command(object): sent on IRC, it should be cheap to execute and unlikely to throw exceptions. - Most commands return ``True`` if :py:attr:`data.command + Most commands return ``True`` only if :py:attr:`data.command ` ``==`` :py:attr:`self.name `, - otherwise they return ``False``. This is the default behavior of - :py:meth:`check`; you need only override it if you wish to change that. + or :py:attr:`data.command ` is in + :py:attr:`self.commands ` if that list is overriden. This is + the default behavior; you should only override it if you wish to change + that. """ + if self.commands: + return data.is_command and data.command in self.commands return data.is_command and data.command == self.name def process(self, data): diff --git a/earwigbot/commands/_old.py b/earwigbot/commands/_old.py deleted file mode 100644 index 6cbb7c5..0000000 --- a/earwigbot/commands/_old.py +++ /dev/null @@ -1,401 +0,0 @@ -# -*- coding: utf-8 -*- -###### -###### NOTE: -###### This is an old commands file from the previous version of EarwigBot. -###### It is not used by the new EarwigBot and is simply here for reference -###### when developing new commands. -###### -### EarwigBot - -def parse(command, line, line2, nick, chan, host, auth, notice, say, reply, s): - authy = auth(host) - if command == "access": - a = 'The bot\'s owner is "%s".' % OWNER - b = 'The bot\'s admins are "%s".' % ', '.join(ADMINS_R) - reply(a, chan, nick) - reply(b, chan, nick) - return - if command == "tock": - u = urllib.urlopen('http://tycho.usno.navy.mil/cgi-bin/timer.pl') - info = u.info() - u.close() - say('"' + info['Date'] + '" - tycho.usno.navy.mil', chan) - return - if command == "beats": - beats = ((time.time() + 3600) % 86400) / 86.4 - beats = int(math.floor(beats)) - say('@%03i' % beats, chan) - return - if command == "dict" or command == "dictionary": - def trim(thing): - if thing.endswith(' '): - thing = thing[:-6] - return thing.strip(' :.') - r_li = re.compile(r'(?ims)
  • .*?
    • ') - r_tag = re.compile(r'<[^>]+>') - r_parens = re.compile(r'(?<=\()(?:[^()]+|\([^)]+\))*(?=\))') - r_word = re.compile(r'^[A-Za-z0-9\' -]+$') - uri = 'http://encarta.msn.com/dictionary_/%s.html' - r_info = re.compile(r'(?:ResultBody">

    (.*?) )|(?:(.*?))') - try: - word = line2[4] - except Exception: - reply("Please enter a word.", chan, nick) - return - word = urllib.quote(word.encode('utf-8')) - bytes = web.get(uri % word) - results = {} - wordkind = None - for kind, sense in r_info.findall(bytes): - kind, sense = trim(kind), trim(sense) - if kind: wordkind = kind - elif sense: - results.setdefault(wordkind, []).append(sense) - result = word.encode('utf-8') + ' - ' - for key in sorted(results.keys()): - if results[key]: - result += (key or '') + ' 1. ' + results[key][0] - if len(results[key]) > 1: - result += ', 2. ' + results[key][1] - result += '; ' - result = result.rstrip('; ') - if result.endswith('-') and (len(result) < 30): - reply('Sorry, no definition found.', chan, nick) - else: say(result, chan) - return - if command == "ety" or command == "etymology": - etyuri = 'http://etymonline.com/?term=%s' - etysearch = 'http://etymonline.com/?search=%s' - r_definition = re.compile(r'(?ims)]*>.*?') - r_tag = re.compile(r'<(?!!)[^>]+>') - r_whitespace = re.compile(r'[\t\r\n ]+') - abbrs = [ - 'cf', 'lit', 'etc', 'Ger', 'Du', 'Skt', 'Rus', 'Eng', 'Amer.Eng', 'Sp', - 'Fr', 'N', 'E', 'S', 'W', 'L', 'Gen', 'J.C', 'dial', 'Gk', - '19c', '18c', '17c', '16c', 'St', 'Capt', 'obs', 'Jan', 'Feb', 'Mar', - 'Apr', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec', 'c', 'tr', 'e', 'g' - ] - t_sentence = r'^.*?(?') - s = s.replace('<', '<') - s = s.replace('&', '&') - return s - def text(html): - html = r_tag.sub('', html) - html = r_whitespace.sub(' ', html) - return unescape(html).strip() - try: - word = line2[4] - except Exception: - reply("Please enter a word.", chan, nick) - return - def ety(word): - if len(word) > 25: - raise ValueError("Word too long: %s[...]" % word[:10]) - word = {'axe': 'ax/axe'}.get(word, word) - bytes = web.get(etyuri % word) - definitions = r_definition.findall(bytes) - if not definitions: - return None - defn = text(definitions[0]) - m = r_sentence.match(defn) - if not m: - return None - sentence = m.group(0) - try: - sentence = unicode(sentence, 'iso-8859-1') - sentence = sentence.encode('utf-8') - except: pass - maxlength = 275 - if len(sentence) > maxlength: - sentence = sentence[:maxlength] - words = sentence[:-5].split(' ') - words.pop() - sentence = ' '.join(words) + ' [...]' - sentence = '"' + sentence.replace('"', "'") + '"' - return sentence + ' - ' + (etyuri % word) - try: - result = ety(word.encode('utf-8')) - except IOError: - msg = "Can't connect to etymonline.com (%s)" % (etyuri % word) - reply(msg, chan, nick) - return - except AttributeError: - result = None - if result is not None: - reply(result, chan, nick) - else: - uri = etysearch % word - msg = 'Can\'t find the etymology for "%s". Try %s' % (word, uri) - reply(msg, chan, nick) - return - if command == "pend" or command == "pending": - say("Pending submissions status page: .", chan) - say("Pending submissions category: .", chan) - return - if command == "sub" or command == "submissions": - try: - number = int(line2[4]) - except Exception: - reply("Please enter a number.", chan, nick) - return - do_url = False - try: - if "url" in line2[5:]: do_url = True - except Exception: - pass - url = "http://en.wikipedia.org/w/api.php?action=query&list=categorymembers&cmtitle=Category:Pending_AfC_submissions&cmlimit=500&cmsort=timestamp" - query = urllib.urlopen(url) - data = query.read() - pages = re.findall("title="(.*?)"", data) - try: - pages.remove("Wikipedia:Articles for creation/Redirects") - except Exception: - pass - try: - pages.remove("Wikipedia:Files for upload") - except Exception: - pass - pages.reverse() - pages = pages[:number] - if not do_url: - s = string.join(pages, "]], [[") - s = "[[%s]]" % s - else: - s = string.join(pages, ">, ,_<", ">, <", s) - report = "\x02First %s pending AfC submissions:\x0F %s" % (number, s) - say(report, chan) - return - if command == "trout": - try: - user = line2[4] - user = ' '.join(line2[4:]) - except Exception: - reply("Hahahahahahahaha...", chan, nick) - return - normal = unicodedata.normalize('NFKD', unicode(string.lower(user))) - if "itself" in normal: - reply("I'm not that stupid ;)", chan, nick) - return - elif "earwigbot" in normal: - reply("I'm not that stupid ;)", chan, nick) - elif "earwig" not in normal and "ear wig" not in normal: - text = 'slaps %s around a bit with a large trout.' % user - msg = '\x01ACTION %s\x01' % text - say(msg, chan) - else: - reply("I refuse to hurt anything with \"Earwig\" in its name :P", chan, nick) - return - if command == "mysql": - if authy != "owner": - reply("You aren't authorized to use this command.", chan, nick) - return - import MySQLdb - try: - strings = line2[4] - strings = ' '.join(line2[4:]) - if "db:" in strings: - database = re.findall("db\:(.*?)\s", strings)[0] - else: - database = "enwiki_p" - if "time:" in strings: - times = int(re.findall("time\:(.*?)\s", strings)[0]) - else: - times = 60 - file = re.findall("file\:(.*?)\s", strings)[0] - sqlquery = re.findall("query\:(.*?)\Z", strings)[0] - except Exception: - reply("You did not specify enough data for the bot to continue.", chan, nick) - return - database2 = database[:-2] + "-p" - db = MySQLdb.connect(db=database, host="%s.rrdb.toolserver.org" % database2, read_default_file="/home/earwig/.my.cnf") - db.query(sqlquery) - r = db.use_result() - data = r.fetch_row(0) - try: - f = codecs.open("/home/earwig/public_html/reports/%s/%s" % (database[:-2], file), 'r') - reply("A file already exists with that name.", chan, nick) - return - except Exception: - pass - f = codecs.open("/home/earwig/public_html/reports/%s/%s" % (database[:-2], file), 'a', 'utf-8') - for line in data: - new_line = [] - for l in line: - new_line.append(str(l)) - f.write(' '.join(new_line) + "\n") - f.close() - reply("Query completed successfully. See http://toolserver.org/~earwig/reports/%s/%s. I will delete the report in %s seconds." % (database[:-2], file, times), chan, nick) - time.sleep(times) - os.remove("/home/earwig/public_html/reports/%s/%s" % (database[:-2], file)) - return - if command == "notes" or command == "note" or command == "about" or command == "data" or command == "database": - try: - action = line2[4] - except BaseException: - reply("What do you want me to do? Type \"!notes help\" for more information.", chan, nick) - return - import MySQLdb - db = MySQLdb.connect(db="u_earwig_ircbot", host="sql", read_default_file="/home/earwig/.my.cnf") - specify = ' '.join(line2[5:]) - if action == "help" or action == "manual": - shortCommandList = "read, write, change, undo, delete, move, author, category, list, report, developer" - if specify == "read": - say("To read an entry, type \"!notes read \".", chan) - elif specify == "write": - say("To write a new entry, type \"!notes write \". This will create a new entry only if one does not exist, see the below command...", chan) - elif specify == "change": - say("To change an entry, type \"!notes change \". The old entry will be stored in the database, so it can be undone later.", chan) - elif specify == "undo": - say("To undo a change, type \"!notes undo \".", chan) - elif specify == "delete": - say("To delete an entry, type \"!notes delete \". For security reasons, only bot admins can do this.", chan) - elif specify == "move": - say("To move an entry, type \"!notes move \".", chan) - elif specify == "author": - say("To return the author of an entry, type \"!notes author \".", chan) - elif specify == "category" or specify == "cat": - say("To change an entry's category, type \"!notes category \".", chan) - elif specify == "list": - say("To list all categories in the database, type \"!notes list\". Type \"!notes list \" to get all entries in a certain category.", chan) - elif specify == "report": - say("To give some statistics about the mini-wiki, including some debugging information, type \"!notes report\" in a PM.", chan) - elif specify == "developer": - say("To do developer work, such as writing to the database directly, type \"!notes developer \". This can only be done by the bot owner.", chan) - else: - db.query("SELECT * FROM version;") - r = db.use_result() - data = r.fetch_row(0) - version = data[0] - reply("The Earwig Mini-Wiki: running v%s." % version, chan, nick) - reply("The full list of commands, for reference, are: %s." % shortCommandList, chan, nick) - reply("For an explaination of a certain command, type \"!notes help \".", chan, nick) - reply("You can also access the database from the Toolserver: http://toolserver.org/~earwig/cgi-bin/irc_database.py", chan, nick) - time.sleep(0.4) - return - elif action == "read": - specify = string.lower(specify) - if " " in specify: specify = string.split(specify, " ")[0] - if not specify or "\"" in specify: - reply("Please include the name of the entry you would like to read after the command, e.g. !notes read earwig", chan, nick) - return - try: - db.query("SELECT entry_content FROM entries WHERE entry_title = \"%s\";" % specify) - r = db.use_result() - data = r.fetch_row(0) - entry = data[0][0] - say("Entry \"\x02%s\x0F\": %s" % (specify, entry), chan) - except Exception: - reply("There is no entry titled \"\x02%s\x0F\"." % specify, chan, nick) - return - elif action == "delete" or action == "remove": - specify = string.lower(specify) - if " " in specify: specify = string.split(specify, " ")[0] - if not specify or "\"" in specify: - reply("Please include the name of the entry you would like to delete after the command, e.g. !notes delete earwig", chan, nick) - return - if authy == "owner" or authy == "admin": - try: - db.query("DELETE from entries where entry_title = \"%s\";" % specify) - r = db.use_result() - db.commit() - reply("The entry on \"\x02%s\x0F\" has been removed." % specify, chan, nick) - except Exception: - phenny.reply("Unable to remove the entry on \"\x02%s\x0F\", because it doesn't exist." % specify, chan, nick) - else: - reply("Only bot admins can remove entries.", chan, nick) - return - elif action == "developer": - if authy == "owner": - db.query(specify) - r = db.use_result() - try: - print r.fetch_row(0) - except Exception: - pass - db.commit() - reply("Done.", chan, nick) - else: - reply("Only the bot owner can modify the raw database.", chan, nick) - return - elif action == "write": - try: - write = line2[5] - content = ' '.join(line2[6:]) - except Exception: - reply("Please include some content in your entry.", chan, nick) - return - db.query("SELECT * from entries WHERE entry_title = \"%s\";" % write) - r = db.use_result() - data = r.fetch_row(0) - if data: - reply("An entry on %s already exists; please use \"!notes change %s %s\"." % (write, write, content), chan, nick) - return - content2 = content.replace('"', '\\' + '"') - db.query("INSERT INTO entries (entry_title, entry_author, entry_category, entry_content, entry_content_old) VALUES (\"%s\", \"%s\", \"uncategorized\", \"%s\", NULL);" % (write, nick, content2)) - db.commit() - reply("You have written an entry titled \"\x02%s\x0F\", with the following content: \"%s\"" % (write, content), chan, nick) - return - elif action == "change": - reply("NotImplementedError", chan, nick) - elif action == "undo": - reply("NotImplementedError", chan, nick) - elif action == "move": - reply("NotImplementedError", chan, nick) - elif action == "author": - try: - entry = line2[5] - except Exception: - reply("Please include the name of the entry you would like to get information for after the command, e.g. !notes author earwig", chan, nick) - return - db.query("SELECT entry_author from entries WHERE entry_title = \"%s\";" % entry) - r = db.use_result() - data = r.fetch_row(0) - if data: - say("The author of \"\x02%s\x0F\" is \x02%s\x0F." % (entry, data[0][0]), chan) - return - reply("There is no entry titled \"\x02%s\x0F\"." % entry, chan, nick) - return - elif action == "cat" or action == "category": - reply("NotImplementedError", chan, nick) - elif action == "list": - reply("NotImplementedError", chan, nick) - elif action == "report": - reply("NotImplementedError", chan, nick) - if command == "langcode" or command == "lang" or command == "language": - try: - lang = line2[4] - except Exception: - reply("Please specify an ISO code.", chan, nick) - return - data = urllib.urlopen("http://toolserver.org/~earwig/cgi-bin/swmt.py?action=iso").read() - data = string.split(data, "\n") - result = False - for datum in data: - if datum.startswith(lang): - result = re.findall(".*? (.*)", datum)[0] - break - if result: - reply(result, chan, nick) - return - reply("Not found.", chan, nick) - return - if command == "lookup" or command == "ip": - try: - hexIP = line2[4] - except Exception: - reply("Please specify a hex IP address.", chan, nick) - return - hexes = [hexIP[:2], hexIP[2:4], hexIP[4:6], hexIP[6:8]] - hashes = [] - for hexHash in hexes: - newHex = int(hexHash, 16) - hashes.append(newHex) - normalizedIP = "%s.%s.%s.%s" % (hashes[0], hashes[1], hashes[2], hashes[3]) - reply(normalizedIP, chan, nick) - return diff --git a/earwigbot/commands/afc_pending.py b/earwigbot/commands/afc_pending.py new file mode 100644 index 0000000..6a2fec0 --- /dev/null +++ b/earwigbot/commands/afc_pending.py @@ -0,0 +1,34 @@ +# -*- coding: utf-8 -*- +# +# Copyright (C) 2009-2012 by Ben Kurtovic +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +from earwigbot.commands import BaseCommand + +class Command(BaseCommand): + """Link the user to the pending AFC submissions page and category.""" + name = "pending" + commands = ["pending", "pend"] + + def process(self, data): + msg1 = "pending submissions status page: http://enwp.org/WP:AFC/ST" + msg2 = "pending submissions category: http://enwp.org/CAT:PEND" + self.reply(data, msg1) + self.reply(data, msg2) diff --git a/earwigbot/commands/afc_report.py b/earwigbot/commands/afc_report.py index a6f201e..c4ecf79 100644 --- a/earwigbot/commands/afc_report.py +++ b/earwigbot/commands/afc_report.py @@ -72,7 +72,7 @@ class AFCReport(Command): def get_page(self, title): page = self.site.get_page(title, follow_redirects=False) - if page.exists[0]: + if page.exists == page.PAGE_EXISTS: return page def report(self, page): diff --git a/earwigbot/commands/afc_status.py b/earwigbot/commands/afc_status.py index c1b77f8..4f517d5 100644 --- a/earwigbot/commands/afc_status.py +++ b/earwigbot/commands/afc_status.py @@ -30,13 +30,12 @@ class AFCStatus(Command): """Get the number of pending AfC submissions, open redirect requests, and open file upload requests.""" name = "status" + commands = ["status", "count", "num", "number"] hooks = ["join", "msg"] def check(self, data): - commands = ["status", "count", "num", "number"] - if data.is_command and data.command in commands: + if data.is_command and data.command in self.commands: return True - try: if data.line[1] == "JOIN" and data.chan == "#wikipedia-en-afc": if data.nick != self.config.irc["frontend"]["nick"]: diff --git a/earwigbot/commands/afc_submissions.py b/earwigbot/commands/afc_submissions.py new file mode 100644 index 0000000..2c8ce9f --- /dev/null +++ b/earwigbot/commands/afc_submissions.py @@ -0,0 +1,60 @@ +# -*- coding: utf-8 -*- +# +# Copyright (C) 2009-2012 by Ben Kurtovic +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +from earwigbot.commands import Command + +__all__ = ["AFCSubmissions"] + +class AFCSubmissions(Command): + """Link the user directly to some pending AFC submissions.""" + name = "submissions" + commands = ["submissions", "subs"] + + def setup(self): + try: + self.ignore_list = self.config.commands[self.name]["ignoreList"] + except KeyError: + try: + ignores = self.config.tasks["afc_statistics"]["ignoreList"] + self.ignore_list = ignores + except KeyError: + self.ignore_list = [] + + def process(self, data): + if data.args: + try: + number = int(data.args[0]) + except ValueError: + self.reply(data, "argument must be a number.") + return + if number > 5: + msg = "cannot get more than five submissions at a time." + self.reply(data, msg) + return + else: + number = 3 + + site = self.bot.wiki.get_site() + category = site.get_category("Pending AfC submissions") + members = category.get_members(use_sql=True, limit=number) + pages = ", ".join([member.url for member in members]) + self.reply(data, "{0} pending AfC subs: {1}".format(number, pages)) diff --git a/earwigbot/commands/chanops.py b/earwigbot/commands/chanops.py index fa20b2e..7d48670 100644 --- a/earwigbot/commands/chanops.py +++ b/earwigbot/commands/chanops.py @@ -28,10 +28,7 @@ class ChanOps(Command): """Voice, devoice, op, or deop users in the channel, or join or part from other channels.""" name = "chanops" - - def check(self, data): - cmnds = ["chanops", "voice", "devoice", "op", "deop", "join", "part"] - return data.is_command and data.command in cmnds + commands = ["chanops", "voice", "devoice", "op", "deop", "join", "part"] def process(self, data): if data.command == "chanops": diff --git a/earwigbot/commands/crypt.py b/earwigbot/commands/crypt.py index 0d37eb0..cd64787 100644 --- a/earwigbot/commands/crypt.py +++ b/earwigbot/commands/crypt.py @@ -32,12 +32,7 @@ class Crypt(Command): """Provides hash functions with !hash (!hash list for supported algorithms) and blowfish encryption with !encrypt and !decrypt.""" name = "crypt" - - def check(self, data): - commands = ["crypt", "hash", "encrypt", "decrypt"] - if data.is_command and data.command in commands: - return True - return False + commands = ["crypt", "hash", "encrypt", "decrypt"] def process(self, data): if data.command == "crypt": diff --git a/earwigbot/commands/editcount.py b/earwigbot/commands/editcount.py index 8223004..288b656 100644 --- a/earwigbot/commands/editcount.py +++ b/earwigbot/commands/editcount.py @@ -30,12 +30,7 @@ __all__ = ["Editcount"] class Editcount(Command): """Return a user's edit count.""" name = "editcount" - - def check(self, data): - commands = ["ec", "editcount"] - if data.is_command and data.command in commands: - return True - return False + commands = ["ec", "editcount"] def process(self, data): if not data.args: @@ -54,6 +49,7 @@ class Editcount(Command): return safe = quote_plus(user.name) - url = "http://toolserver.org/~tparis/pcount/index.php?name={0}&lang=en&wiki=wikipedia" + url = "http://toolserver.org/~tparis/pcount/index.php?name={0}&lang={1}&wiki={2}" + fullurl = url.format(safe, site.lang, site.project) msg = "\x0302{0}\x0301 has {1} edits ({2})." - self.reply(data, msg.format(name, count, url.format(safe))) + self.reply(data, msg.format(name, count, fullurl)) diff --git a/earwigbot/commands/geolocate.py b/earwigbot/commands/geolocate.py new file mode 100644 index 0000000..371e73c --- /dev/null +++ b/earwigbot/commands/geolocate.py @@ -0,0 +1,72 @@ +# -*- coding: utf-8 -*- +# +# Copyright (C) 2009-2012 by Ben Kurtovic +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +import json +import urllib2 + +from earwigbot.commands import BaseCommand + +class Command(BaseCommand): + """Geolocate an IP address (via http://ipinfodb.com/).""" + name = "geolocate" + commands = ["geolocate", "locate", "geo", "ip"] + + def setup(self): + self.config.decrypt(self.config.commands, (self.name, "apiKey")) + try: + self.key = self.config.commands[self.name]["apiKey"] + except KeyError: + self.key = None + log = 'Cannot use without an API key for http://ipinfodb.com/ stored as config.commands["{0}"]["apiKey"]' + self.logger.warn(log.format(self.name)) + + def process(self, data): + if not data.args: + self.reply(data, "please specify an IP to lookup.") + return + + if not self.key: + msg = 'I need an API key for http://ipinfodb.com/ stored as \x0303config.commands["{0}"]["apiKey"]\x0301.' + log = 'Need an API key for http://ipinfodb.com/ stored as config.commands["{0}"]["apiKey"]' + self.reply(data, msg.format(self.name) + ".") + self.logger.error(log.format(self.name)) + return + + address = data.args[0] + url = "http://api.ipinfodb.com/v3/ip-city/?key={0}&ip={1}&format=json" + query = urllib2.urlopen(url.format(self.key, address)).read() + res = json.loads(query) + + try: + country = res["countryName"] + region = res["regionName"] + city = res["cityName"] + latitude = res["latitude"] + longitude = res["longitude"] + utcoffset = res["timeZone"] + except KeyError: + self.reply(data, "IP \x0302{0}\x0301 not found.".format(address)) + return + + msg = "{0}, {1}, {2} ({3}, {4}), UTC {5}" + geo = msg.format(country, region, city, latitude, longitude, utcoffset) + self.reply(data, geo) diff --git a/earwigbot/commands/git.py b/earwigbot/commands/git.py index 23a5daf..4c2a29c 100644 --- a/earwigbot/commands/git.py +++ b/earwigbot/commands/git.py @@ -32,10 +32,12 @@ class Git(Command): """Commands to interface with the bot's git repository; use '!git' for a sub-command list.""" name = "git" - repos = { - "core": "/home/earwig/git/earwigbot", - "plugins": "/home/earwig/git/earwigbot-plugins", - } + + def setup(self): + try: + self.repos = self.config.commands[self.name]["repos"] + except KeyError: + self.repos = None def process(self, data): self.data = data @@ -46,6 +48,9 @@ class Git(Command): if not data.args or data.args[0] == "help": self.do_help() return + if not self.repos: + self.reply(data, "no repos are specified in the config file.") + return command = data.args[0] try: @@ -57,7 +62,7 @@ class Git(Command): return if repo_name not in self.repos: repos = self.get_repos() - msg = "repository must be one of the following: {0}" + msg = "repository must be one of the following: {0}." self.reply(data, msg.format(repos)) return self.repo = git.Repo(self.repos[repo_name]) @@ -91,7 +96,7 @@ class Git(Command): try: return getattr(self.repo.remotes, remote_name) except AttributeError: - msg = "unknown remote: \x0302{0}\x0301".format(remote_name) + msg = "unknown remote: \x0302{0}\x0301.".format(remote_name) self.reply(self.data, msg) def get_time_since(self, date): diff --git a/earwigbot/commands/help.py b/earwigbot/commands/help.py index 5fd2d59..463b910 100644 --- a/earwigbot/commands/help.py +++ b/earwigbot/commands/help.py @@ -23,7 +23,6 @@ import re from earwigbot.commands import Command -from earwigbot.irc import Data __all__ = ["Help"] @@ -50,34 +49,24 @@ class Help(Command): def do_main_help(self, data): """Give the user a general help message with a list of all commands.""" msg = "Hi, I'm a bot! I have {0} commands loaded: {1}. You can get help for any command with '!help '." - cmnds = sorted(self.bot.commands) + cmnds = sorted([cmnd.name for cmnd in self.bot.commands]) msg = msg.format(len(cmnds), ', '.join(cmnds)) self.reply(data, msg) def do_command_help(self, data): """Give the user help for a specific command.""" - command = data.args[0] + target = data.args[0] - # Create a dummy message to test which commands pick up the user's - # input: - msg = ":foo!bar@example.com PRIVMSG #channel :msg".split() - dummy = Data(self.bot, msg) - dummy.command = command.lower() - dummy.is_command = True + for command in self.bot.commands: + if command.name == target or target in command.commands: + if command.__doc__: + doc = command.__doc__.replace("\n", "") + doc = re.sub("\s\s+", " ", doc) + msg = "help for command \x0303{0}\x0301: \"{1}\"" + self.reply(data, msg.format(target, doc)) + return - for cmnd_name in self.bot.commands: - cmnd = self.bot.commands.get(cmnd_name) - if not cmnd.check(dummy): - continue - if cmnd.__doc__: - doc = cmnd.__doc__.replace("\n", "") - doc = re.sub("\s\s+", " ", doc) - msg = "help for command \x0303{0}\x0301: \"{1}\"" - self.reply(data, msg.format(command, doc)) - return - break - - msg = "sorry, no help for \x0303{0}\x0301.".format(command) + msg = "sorry, no help for \x0303{0}\x0301.".format(target) self.reply(data, msg) def do_hello(self, data): diff --git a/earwigbot/commands/langcode.py b/earwigbot/commands/langcode.py new file mode 100644 index 0000000..434e72b --- /dev/null +++ b/earwigbot/commands/langcode.py @@ -0,0 +1,49 @@ +# -*- coding: utf-8 -*- +# +# Copyright (C) 2009-2012 by Ben Kurtovic +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +from earwigbot.commands import BaseCommand + +class Command(BaseCommand): + """Convert a language code into its name and a list of WMF sites in that + language.""" + name = "langcode" + commands = ["langcode", "lang", "language"] + + def process(self, data): + if not data.args: + self.reply(data, "please specify a language code.") + return + + code = data.args[0] + site = self.bot.wiki.get_site() + matrix = site.api_query(action="sitematrix")["sitematrix"] + del matrix["specials"] + + for site in matrix.itervalues(): + if site["code"] == code: + name = site["name"] + sites = ", ".join([s["url"] for s in site["site"]]) + msg = "\x0302{0}\x0301 is {1} ({2})".format(code, name, sites) + self.reply(data, msg) + return + + self.reply(data, "site \x0302{0}\x0301 not found.".format(code)) diff --git a/earwigbot/commands/link.py b/earwigbot/commands/link.py index 08734d9..7799b0f 100644 --- a/earwigbot/commands/link.py +++ b/earwigbot/commands/link.py @@ -31,14 +31,6 @@ class Link(Command): """Convert a Wikipedia page name into a URL.""" name = "link" - def check(self, data): - # if ((data.is_command and data.command == "link") or - # (("[[" in data.msg and "]]" in data.msg) or - # ("{{" in data.msg and "}}" in data.msg))): - if data.is_command and data.command == "link": - return True - return False - def process(self, data): msg = data.msg diff --git a/earwigbot/commands/notes.py b/earwigbot/commands/notes.py new file mode 100644 index 0000000..29d387f --- /dev/null +++ b/earwigbot/commands/notes.py @@ -0,0 +1,169 @@ +# -*- coding: utf-8 -*- +# +# Copyright (C) 2009-2012 by Ben Kurtovic +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +from earwigbot.commands import Command + +__all__ = ["Notes"] + +class Notes(Command): + """A mini IRC-based wiki for storing notes, tips, and reminders.""" + name = "notes" + + def process(self, data): + pass + + +class OldCommand(object): + def parse(self): + if command == "notes" or command == "note" or command == "about" or command == "data" or command == "database": + try: + action = line2[4] + except BaseException: + reply("What do you want me to do? Type \"!notes help\" for more information.", chan, nick) + return + import MySQLdb + db = MySQLdb.connect(db="u_earwig_ircbot", host="sql", read_default_file="/home/earwig/.my.cnf") + specify = ' '.join(line2[5:]) + if action == "help" or action == "manual": + shortCommandList = "read, write, change, undo, delete, move, author, category, list, report, developer" + if specify == "read": + say("To read an entry, type \"!notes read \".", chan) + elif specify == "write": + say("To write a new entry, type \"!notes write \". This will create a new entry only if one does not exist, see the below command...", chan) + elif specify == "change": + say("To change an entry, type \"!notes change \". The old entry will be stored in the database, so it can be undone later.", chan) + elif specify == "undo": + say("To undo a change, type \"!notes undo \".", chan) + elif specify == "delete": + say("To delete an entry, type \"!notes delete \". For security reasons, only bot admins can do this.", chan) + elif specify == "move": + say("To move an entry, type \"!notes move \".", chan) + elif specify == "author": + say("To return the author of an entry, type \"!notes author \".", chan) + elif specify == "category" or specify == "cat": + say("To change an entry's category, type \"!notes category \".", chan) + elif specify == "list": + say("To list all categories in the database, type \"!notes list\". Type \"!notes list \" to get all entries in a certain category.", chan) + elif specify == "report": + say("To give some statistics about the mini-wiki, including some debugging information, type \"!notes report\" in a PM.", chan) + elif specify == "developer": + say("To do developer work, such as writing to the database directly, type \"!notes developer \". This can only be done by the bot owner.", chan) + else: + db.query("SELECT * FROM version;") + r = db.use_result() + data = r.fetch_row(0) + version = data[0] + reply("The Earwig Mini-Wiki: running v%s." % version, chan, nick) + reply("The full list of commands, for reference, are: %s." % shortCommandList, chan, nick) + reply("For an explaination of a certain command, type \"!notes help \".", chan, nick) + reply("You can also access the database from the Toolserver: http://toolserver.org/~earwig/cgi-bin/irc_database.py", chan, nick) + time.sleep(0.4) + return + elif action == "read": + specify = string.lower(specify) + if " " in specify: specify = string.split(specify, " ")[0] + if not specify or "\"" in specify: + reply("Please include the name of the entry you would like to read after the command, e.g. !notes read earwig", chan, nick) + return + try: + db.query("SELECT entry_content FROM entries WHERE entry_title = \"%s\";" % specify) + r = db.use_result() + data = r.fetch_row(0) + entry = data[0][0] + say("Entry \"\x02%s\x0F\": %s" % (specify, entry), chan) + except Exception: + reply("There is no entry titled \"\x02%s\x0F\"." % specify, chan, nick) + return + elif action == "delete" or action == "remove": + specify = string.lower(specify) + if " " in specify: specify = string.split(specify, " ")[0] + if not specify or "\"" in specify: + reply("Please include the name of the entry you would like to delete after the command, e.g. !notes delete earwig", chan, nick) + return + if authy == "owner" or authy == "admin": + try: + db.query("DELETE from entries where entry_title = \"%s\";" % specify) + r = db.use_result() + db.commit() + reply("The entry on \"\x02%s\x0F\" has been removed." % specify, chan, nick) + except Exception: + phenny.reply("Unable to remove the entry on \"\x02%s\x0F\", because it doesn't exist." % specify, chan, nick) + else: + reply("Only bot admins can remove entries.", chan, nick) + return + elif action == "developer": + if authy == "owner": + db.query(specify) + r = db.use_result() + try: + print r.fetch_row(0) + except Exception: + pass + db.commit() + reply("Done.", chan, nick) + else: + reply("Only the bot owner can modify the raw database.", chan, nick) + return + elif action == "write": + try: + write = line2[5] + content = ' '.join(line2[6:]) + except Exception: + reply("Please include some content in your entry.", chan, nick) + return + db.query("SELECT * from entries WHERE entry_title = \"%s\";" % write) + r = db.use_result() + data = r.fetch_row(0) + if data: + reply("An entry on %s already exists; please use \"!notes change %s %s\"." % (write, write, content), chan, nick) + return + content2 = content.replace('"', '\\' + '"') + db.query("INSERT INTO entries (entry_title, entry_author, entry_category, entry_content, entry_content_old) VALUES (\"%s\", \"%s\", \"uncategorized\", \"%s\", NULL);" % (write, nick, content2)) + db.commit() + reply("You have written an entry titled \"\x02%s\x0F\", with the following content: \"%s\"" % (write, content), chan, nick) + return + elif action == "change": + reply("NotImplementedError", chan, nick) + elif action == "undo": + reply("NotImplementedError", chan, nick) + elif action == "move": + reply("NotImplementedError", chan, nick) + elif action == "author": + try: + entry = line2[5] + except Exception: + reply("Please include the name of the entry you would like to get information for after the command, e.g. !notes author earwig", chan, nick) + return + db.query("SELECT entry_author from entries WHERE entry_title = \"%s\";" % entry) + r = db.use_result() + data = r.fetch_row(0) + if data: + say("The author of \"\x02%s\x0F\" is \x02%s\x0F." % (entry, data[0][0]), chan) + return + reply("There is no entry titled \"\x02%s\x0F\"." % entry, chan, nick) + return + elif action == "cat" or action == "category": + reply("NotImplementedError", chan, nick) + elif action == "list": + reply("NotImplementedError", chan, nick) + elif action == "report": + reply("NotImplementedError", chan, nick) diff --git a/earwigbot/commands/praise.py b/earwigbot/commands/praise.py index 693b58d..13ce35e 100644 --- a/earwigbot/commands/praise.py +++ b/earwigbot/commands/praise.py @@ -28,24 +28,23 @@ class Praise(Command): """Praise people!""" name = "praise" + def setup(self): + try: + self.praises = self.config.commands[self.name]["praises"] + except KeyError: + self.praises = [] + def check(self, data): - commands = ["praise", "earwig", "leonard", "leonard^bloom", "groove", - "groovedog"] - return data.is_command and data.command in commands + check = data.command == "praise" or data.command in self.praises + return data.is_command and check def process(self, data): - if data.command == "earwig": - msg = "\x02Earwig\x0F is the bestest Python programmer ever!" - elif data.command in ["leonard", "leonard^bloom"]: - msg = "\x02Leonard^Bloom\x0F is the biggest slacker ever!" - elif data.command in ["groove", "groovedog"]: - msg = "\x02GrooveDog\x0F is the bestest heh evar!" - else: - if not data.args: - msg = "You use this command to praise certain people. Who they are is a secret." - else: - msg = "You're doing it wrong." - self.reply(data, msg) + if data.command in self.praises: + msg = self.praises[data.command] + self.say(data.chan, msg) return - - self.say(data.chan, msg) + if not data.args: + msg = "You use this command to praise certain people. Who they are is a secret." + else: + msg = "you're doing it wrong." + self.reply(data, msg) diff --git a/earwigbot/commands/quit.py b/earwigbot/commands/quit.py index 9ccede0..6b165b2 100644 --- a/earwigbot/commands/quit.py +++ b/earwigbot/commands/quit.py @@ -28,10 +28,7 @@ class Quit(Command): """Quit, restart, or reload components from the bot. Only the owners can run this command.""" name = "quit" - - def check(self, data): - commands = ["quit", "restart", "reload"] - return data.is_command and data.command in commands + commands = ["quit", "restart", "reload"] def process(self, data): if data.host not in self.config.irc["permissions"]["owners"]: diff --git a/earwigbot/commands/registration.py b/earwigbot/commands/registration.py index f6fce3c..cd498c1 100644 --- a/earwigbot/commands/registration.py +++ b/earwigbot/commands/registration.py @@ -30,12 +30,7 @@ __all__ = ["Registration"] class Registration(Command): """Return when a user registered.""" name = "registration" - - def check(self, data): - commands = ["registration", "reg", "age"] - if data.is_command and data.command in commands: - return True - return False + commands = ["registration", "reg", "age"] def process(self, data): if not data.args: @@ -61,7 +56,7 @@ class Registration(Command): elif user.gender == "female": gender = "She's" else: - gender = "They're" + gender = "They're" # Singluar they? msg = "\x0302{0}\x0301 registered on {1}. {2} {3} old." self.reply(data, msg.format(name, date, gender, age)) diff --git a/earwigbot/commands/remind.py b/earwigbot/commands/remind.py index 19c5b32..3a44e3c 100644 --- a/earwigbot/commands/remind.py +++ b/earwigbot/commands/remind.py @@ -20,7 +20,7 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. -import threading +from threading import Timer import time from earwigbot.commands import Command @@ -30,11 +30,7 @@ __all__ = ["Remind"] class Remind(Command): """Set a message to be repeated to you in a certain amount of time.""" name = "remind" - - def check(self, data): - if data.is_command and data.command in ["remind", "reminder"]: - return True - return False + commands = ["remind", "reminder"] def process(self, data): if not data.args: @@ -62,12 +58,7 @@ class Remind(Command): msg = msg.format(message, wait, end_time_with_timezone) self.reply(data, msg) - t_reminder = threading.Thread(target=self.reminder, - args=(data, message, wait)) + t_reminder = Timer(wait, self.reply, args=(data, message)) t_reminder.name = "reminder " + end_time t_reminder.daemon = True t_reminder.start() - - def reminder(self, data, message, wait): - time.sleep(wait) - self.reply(data, message) diff --git a/earwigbot/commands/replag.py b/earwigbot/commands/replag.py index 8a12dde..bb970aa 100644 --- a/earwigbot/commands/replag.py +++ b/earwigbot/commands/replag.py @@ -32,10 +32,16 @@ class Replag(Command): """Return the replag for a specific database on the Toolserver.""" name = "replag" + def setup(self): + try: + self.default = self.config.commands[self.name]["default"] + except KeyError: + self.default = None + def process(self, data): args = {} if not data.args: - args["db"] = "enwiki_p" + args["db"] = self.default or self.bot.wiki.get_site().name + "_p" else: args["db"] = data.args[0] args["host"] = args["db"].replace("_", "-") + ".rrdb.toolserver.org" @@ -43,10 +49,11 @@ class Replag(Command): conn = oursql.connect(**args) with conn.cursor() as cursor: - query = "SELECT UNIX_TIMESTAMP() - UNIX_TIMESTAMP(rc_timestamp) FROM recentchanges ORDER BY rc_timestamp DESC LIMIT 1" + query = """SELECT UNIX_TIMESTAMP() - UNIX_TIMESTAMP(rc_timestamp) + FROM recentchanges ORDER BY rc_timestamp DESC LIMIT 1""" cursor.execute(query) replag = int(cursor.fetchall()[0][0]) conn.close() - msg = "Replag on \x0302{0}\x0301 is \x02{1}\x0F seconds." + msg = "replag on \x0302{0}\x0301 is \x02{1}\x0F seconds." self.reply(data, msg.format(args["db"], replag)) diff --git a/earwigbot/commands/rights.py b/earwigbot/commands/rights.py index cbdb765..f768ff7 100644 --- a/earwigbot/commands/rights.py +++ b/earwigbot/commands/rights.py @@ -28,12 +28,7 @@ __all__ = ["Rights"] class Rights(Command): """Retrieve a list of rights for a given username.""" name = "rights" - - def check(self, data): - commands = ["rights", "groups", "permissions", "privileges"] - if data.is_command and data.command in commands: - return True - return False + commands = ["rights", "groups", "permissions", "privileges"] def process(self, data): if not data.args: diff --git a/earwigbot/commands/threads.py b/earwigbot/commands/threads.py index bd2fed9..97a9738 100644 --- a/earwigbot/commands/threads.py +++ b/earwigbot/commands/threads.py @@ -30,12 +30,7 @@ __all__ = ["Threads"] class Threads(Command): """Manage wiki tasks from IRC, and check on thread status.""" name = "threads" - - def check(self, data): - commands = ["tasks", "task", "threads", "tasklist"] - if data.is_command and data.command in commands: - return True - return False + commands = ["tasks", "task", "threads", "tasklist"] def process(self, data): self.data = data @@ -106,7 +101,7 @@ class Threads(Command): whether they are currently running or idle.""" threads = threading.enumerate() tasklist = [] - for task in sorted(self.bot.tasks): + for task in sorted([task.name for task in self.bot.tasks]): threadlist = [t for t in threads if t.name.startswith(task)] ids = [str(t.ident) for t in threadlist] if not ids: @@ -134,7 +129,7 @@ class Threads(Command): self.reply(data, "what task do you want me to start?") return - if task_name not in self.bot.tasks: + if task_name not in [task.name for task in self.bot.tasks]: # This task does not exist or hasn't been loaded: msg = "task could not be found; either it doesn't exist, or it wasn't loaded correctly." self.reply(data, msg.format(task_name)) diff --git a/earwigbot/commands/time.py b/earwigbot/commands/time.py new file mode 100644 index 0000000..53dafe8 --- /dev/null +++ b/earwigbot/commands/time.py @@ -0,0 +1,68 @@ +# -*- coding: utf-8 -*- +# +# Copyright (C) 2009-2012 by Ben Kurtovic +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +from datetime import datetime +from math import floor +from time import time + +try: + import pytz +except ImportError: + pytz = None + +from earwigbot.commands import BaseCommand + +class Command(BaseCommand): + """Report the current time in any timezone (UTC default), or in beats.""" + name = "time" + commands = ["time", "beats", "swatch"] + + def process(self, data): + if data.command in ["beats", "swatch"]: + self.do_beats(data) + return + if data.args: + timezone = data.args[0] + else: + timezone = "UTC" + if timezone in ["beats", "swatch"]: + self.do_beats(data) + else: + self.do_time(data, timezone) + + def do_beats(self, data): + beats = ((time() + 3600) % 86400) / 86.4 + beats = int(floor(beats)) + self.reply(data, "@{0:0>3}".format(beats)) + + def do_time(self, data, timezone): + if not pytz: + msg = "this command requires the 'pytz' module: http://pytz.sourceforge.net/" + self.reply(data, msg) + return + try: + tzinfo = pytz.timezone(timezone) + except pytz.exceptions.UnknownTimeZoneError: + self.reply(data, "unknown timezone: {0}.".format(timezone)) + return + now = pytz.utc.localize(datetime.utcnow()).astimezone(tzinfo) + self.reply(data, now.strftime("%Y-%m-%d %H:%M:%S %Z")) diff --git a/earwigbot/commands/trout.py b/earwigbot/commands/trout.py new file mode 100644 index 0000000..6449d62 --- /dev/null +++ b/earwigbot/commands/trout.py @@ -0,0 +1,48 @@ +# -*- coding: utf-8 -*- +# +# Copyright (C) 2009-2012 by Ben Kurtovic +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +from unicodedata import normalize + +from earwigbot.commands import Command + +__all__ = ["Trout"] + +class Trout(Command): + """Slap someone with a trout, or related fish.""" + name = "trout" + commands = ["trout", "whale"] + + def setup(self): + try: + self.exceptions = self.config.commands[self.name]["exceptions"] + except KeyError: + self.exceptions = {} + + def process(self, data): + animal = data.command + target = " ".join(data.args) or data.nick + normal = normalize("NFKD", target.decode("utf8")).lower() + if normal in self.exceptions: + self.reply(data, self.exceptions["normal"]) + else: + msg = "slaps {0} around a bit with a large {1}." + self.action(data.chan, msg.format(target, animal)) diff --git a/earwigbot/config.py b/earwigbot/config.py index ab9eecd..6e7387f 100644 --- a/earwigbot/config.py +++ b/earwigbot/config.py @@ -48,8 +48,9 @@ class BotConfig(object): - :py:attr:`path`: path to the bot's config file - :py:attr:`components`: enabled components - :py:attr:`wiki`: information about wiki-editing - - :py:attr:`tasks`: information for bot tasks - :py:attr:`irc`: information about IRC + - :py:attr:`commands`: information about IRC commands + - :py:attr:`tasks`: information for bot tasks - :py:attr:`metadata`: miscellaneous information - :py:meth:`schedule`: tasks scheduled to run at a given time @@ -69,12 +70,13 @@ class BotConfig(object): self._components = _ConfigNode() self._wiki = _ConfigNode() - self._tasks = _ConfigNode() self._irc = _ConfigNode() + self._commands = _ConfigNode() + self._tasks = _ConfigNode() self._metadata = _ConfigNode() - self._nodes = [self._components, self._wiki, self._tasks, self._irc, - self._metadata] + self._nodes = [self._components, self._wiki, self._irc, self._commands, + self._tasks, self._metadata] self._decryptable_nodes = [ # Default nodes to decrypt (self._wiki, ("password",)), @@ -196,16 +198,21 @@ class BotConfig(object): return self._wiki @property - def tasks(self): - """A dict of information for bot tasks.""" - return self._tasks - - @property def irc(self): """A dict of information about IRC.""" return self._irc @property + def commands(self): + """A dict of information for IRC commands.""" + return self._commands + + @property + def tasks(self): + """A dict of information for bot tasks.""" + return self._tasks + + @property def metadata(self): """A dict of miscellaneous information.""" return self._metadata @@ -225,14 +232,14 @@ class BotConfig(object): user. If there is no config file at all, offer to make one, otherwise exit. - Data from the config file is stored in five + Data from the config file is stored in six :py:class:`~earwigbot.config._ConfigNode`\ s (:py:attr:`components`, - :py:attr:`wiki`, :py:attr:`tasks`, :py:attr:`irc`, :py:attr:`metadata`) - for easy access (as well as the lower-level :py:attr:`data` attribute). - If passwords are encrypted, we'll use :py:func:`~getpass.getpass` for - the key and then decrypt them. If the config is being reloaded, - encrypted items will be automatically decrypted if they were decrypted - earlier. + :py:attr:`wiki`, :py:attr:`irc`, :py:attr:`commands`, :py:attr:`tasks`, + :py:attr:`metadata`) for easy access (as well as the lower-level + :py:attr:`data` attribute). If passwords are encrypted, we'll use + :py:func:`~getpass.getpass` for the key and then decrypt them. If the + config is being reloaded, encrypted items will be automatically + decrypted if they were decrypted earlier. """ if not path.exists(self._config_path): print "Config file not found:", self._config_path @@ -246,8 +253,9 @@ class BotConfig(object): data = self._data self.components._load(data.get("components", {})) self.wiki._load(data.get("wiki", {})) - self.tasks._load(data.get("tasks", {})) self.irc._load(data.get("irc", {})) + self.commands._load(data.get("commands", {})) + self.tasks._load(data.get("tasks", {})) self.metadata._load(data.get("metadata", {})) self._setup_logging() @@ -273,7 +281,10 @@ class BotConfig(object): >>> config.decrypt(config.irc, "frontend", "nickservPassword") # decrypts config.irc["frontend"]["nickservPassword"] """ - self._decryptable_nodes.append((node, nodes)) + signature = (node, nodes) + if signature in self._decryptable_nodes: + return # Already decrypted + self._decryptable_nodes.append(signature) if self.is_encrypted(): self._decrypt(node, nodes) diff --git a/earwigbot/managers.py b/earwigbot/managers.py index 0cb34c2..f71fee1 100644 --- a/earwigbot/managers.py +++ b/earwigbot/managers.py @@ -24,7 +24,7 @@ import imp from os import listdir, path from re import sub -from threading import Lock, Thread +from threading import RLock, Thread from time import gmtime, strftime from earwigbot.commands import Command @@ -46,11 +46,7 @@ class _ResourceManager(object): This class handles the low-level tasks of (re)loading resources via :py:meth:`load`, retrieving specific resources via :py:meth:`get`, and - iterating over all resources via :py:meth:`__iter__`. If iterating over - resources, it is recommended to acquire :py:attr:`self.lock ` - beforehand and release it afterwards (alternatively, wrap your code in a - ``with`` statement) so an attempt at reloading resources in another thread - won't disrupt your iteration. + iterating over all resources via :py:meth:`__iter__`. """ def __init__(self, bot, name, base): self.bot = bot @@ -62,8 +58,9 @@ class _ResourceManager(object): self._resource_access_lock = Lock() def __iter__(self): - for name in self._resources: - yield name + with self.lock: + for resource in self._resources.itervalues(): + yield resource def _load_resource(self, name, path, klass): """Instantiate a resource class and add it to the dictionary.""" @@ -140,7 +137,8 @@ class _ResourceManager(object): Will raise :py:exc:`KeyError` if the resource (a command or task) is not found. """ - return self._resources[key] + with self.lock: + return self._resources[key] class CommandManager(_ResourceManager): @@ -168,13 +166,10 @@ class CommandManager(_ResourceManager): def call(self, hook, data): """Respond to a hook type and a :py:class:`Data` object.""" - self.lock.acquire() - for command in self._resources.itervalues(): + for command in self: if hook in command.hooks and self._wrap_check(command, data): - self.lock.release() self._wrap_process(command, data) return - self.lock.release() class TaskManager(_ResourceManager): diff --git a/earwigbot/tasks/blptag.py b/earwigbot/tasks/blp_tag.py similarity index 98% rename from earwigbot/tasks/blptag.py rename to earwigbot/tasks/blp_tag.py index 80f8bab..57ce94e 100644 --- a/earwigbot/tasks/blptag.py +++ b/earwigbot/tasks/blp_tag.py @@ -27,7 +27,7 @@ __all__ = ["BLPTag"] class BLPTag(Task): """A task to add |blp=yes to ``{{WPB}}`` or ``{{WPBS}}`` when it is used along with ``{{WP Biography}}``.""" - name = "blptag" + name = "blp_tag" def setup(self): pass diff --git a/earwigbot/tasks/feed_dailycats.py b/earwigbot/tasks/image_display_resize.py similarity index 88% rename from earwigbot/tasks/feed_dailycats.py rename to earwigbot/tasks/image_display_resize.py index 83563dc..ebf2bdf 100644 --- a/earwigbot/tasks/feed_dailycats.py +++ b/earwigbot/tasks/image_display_resize.py @@ -22,11 +22,11 @@ from earwigbot.tasks import Task -__all__ = ["FeedDailyCats"] +__all__ = ["ImageDisplayResize"] -class FeedDailyCats(Task): - """A task to create daily categories for [[WP:FEED]].""" - name = "feed_dailycats" +class ImageDisplayResize(Task): + """A task to resize upscaled portraits in infoboxes.""" + name = "image_display_resize" def setup(self): pass diff --git a/earwigbot/tasks/wrongmime.py b/earwigbot/tasks/wrong_mime.py similarity index 98% rename from earwigbot/tasks/wrongmime.py rename to earwigbot/tasks/wrong_mime.py index 353dad2..c3c9aba 100644 --- a/earwigbot/tasks/wrongmime.py +++ b/earwigbot/tasks/wrong_mime.py @@ -27,7 +27,7 @@ __all__ = ["WrongMIME"] class WrongMIME(Task): """A task to tag files whose extensions do not agree with their MIME type.""" - name = "wrongmime" + name = "wrong_mime" def setup(self): pass diff --git a/earwigbot/wiki/category.py b/earwigbot/wiki/category.py index e953e70..2df9a0e 100644 --- a/earwigbot/wiki/category.py +++ b/earwigbot/wiki/category.py @@ -39,7 +39,7 @@ class Category(Page): *Public methods:* - - :py:meth:`get_members`: returns a list of page titles in the category + - :py:meth:`get_members`: iterates over Pages in the category """ def __repr__(self): @@ -51,8 +51,8 @@ class Category(Page): """Return a nice string representation of the Category.""" return ''.format(self.title, str(self._site)) - def _get_members_via_sql(self, limit): - """Return a list of tuples of (title, pageid) in the category.""" + def _get_members_via_sql(self, limit, follow): + """Iterate over Pages in the category using SQL.""" query = """SELECT page_title, page_namespace, page_id FROM page JOIN categorylinks ON page_id = cl_from WHERE cl_to = ?""" @@ -64,42 +64,66 @@ class Category(Page): else: result = self._site.sql_query(query, (title,)) - members = [] - for row in result: + members = list(result) + for row in members: base = row[0].replace("_", " ").decode("utf8") namespace = self._site.namespace_id_to_name(row[1]) if namespace: title = u":".join((namespace, base)) else: # Avoid doing a silly (albeit valid) ":Pagename" thing title = base - members.append((title, row[2])) - return members + yield self._site.get_page(title, follow_redirects=follow, + pageid=row[2]) - def _get_members_via_api(self, limit): - """Return a list of page titles in the category using the API.""" + def _get_members_via_api(self, limit, follow): + """Iterate over Pages in the category using the API.""" params = {"action": "query", "list": "categorymembers", - "cmlimit": limit, "cmtitle": self._title} - if not limit: - params["cmlimit"] = 50 # Default value - - result = self._site.api_query(**params) - members = result['query']['categorymembers'] - return [member["title"] for member in members] - - def get_members(self, use_sql=False, limit=None): - """Return a list of page titles in the category. + "cmtitle": self._title} + + while 1: + params["cmlimit"] = limit if limit else "max" + result = self._site.api_query(**params) + for member in result["query"]["categorymembers"]: + title = member["title"] + yield self._site.get_page(title, follow_redirects=follow) + + if "query-continue" in result: + qcontinue = result["query-continue"]["categorymembers"] + params["cmcontinue"] = qcontinue["cmcontinue"] + if limit: + limit -= len(result["query"]["categorymembers"]) + else: + break + + def get_members(self, use_sql=False, limit=None, follow_redirects=None): + """Iterate over Pages in the category. If *use_sql* is ``True``, we will use a SQL query instead of the API. - Pages will be returned as tuples of ``(title, pageid)`` instead of just - titles. - - If *limit* is provided, we will provide this many titles, or less if - the category is smaller. It defaults to 50 for API queries; normal - users can go up to 500, and bots can go up to 5,000 on a single API - query. If we're using SQL, the limit is ``None`` by default (returning - all pages in the category), but an arbitrary limit can still be chosen. + Note that pages are retrieved from the API in chunks (by default, in + 500-page chunks for normal users and 5000-page chunks for bots and + admins), so queries may be made as we go along. If *limit* is given, we + will provide this many pages, or less if the category is smaller. By + default, *limit* is ``None``, meaning we will keep iterating over + members until the category is exhausted. *follow_redirects* is passed + directly to :py:meth:`site.get_page() + `; it defaults to ``None``, which + will use the value passed to our :py:meth:`__init__`. + + .. note:: + Be careful when iterating over very large categories with no limit. + If using the API, at best, you will make one query per 5000 pages, + which can add up significantly for categories with hundreds of + thousands of members. As for SQL, note that *all page titles are + stored internally* as soon as the query is made, so the site-wide + SQL lock can be freed and unrelated queries can be made without + requiring a separate connection to be opened. This is generally not + an issue unless your category's size approaches several hundred + thousand, in which case the sheer number of titles in memory becomes + problematic. """ + if follow_redirects is None: + follow_redirects = self._follow_redirects if use_sql: - return self._get_members_via_sql(limit) + return self._get_members_via_sql(limit, follow_redirects) else: - return self._get_members_via_api(limit) + return self._get_members_via_api(limit, follow_redirects) diff --git a/earwigbot/wiki/page.py b/earwigbot/wiki/page.py index ab18952..3ada7ec 100644 --- a/earwigbot/wiki/page.py +++ b/earwigbot/wiki/page.py @@ -48,7 +48,7 @@ class Page(CopyrightMixIn): - :py:attr:`site`: the page's corresponding Site object - :py:attr:`title`: the page's title, or pagename - - :py:attr:`exists`: whether the page exists + - :py:attr:`exists`: whether or not the page exists - :py:attr:`pageid`: an integer ID representing the page - :py:attr:`url`: the page's URL - :py:attr:`namespace`: the page's namespace as an integer @@ -75,17 +75,20 @@ class Page(CopyrightMixIn): checks the page like :py:meth:`copyvio_check`, but against a specific URL """ - re_redirect = "^\s*\#\s*redirect\s*\[\[(.*?)\]\]" + PAGE_UNKNOWN = 0 + PAGE_INVALID = 1 + PAGE_MISSING = 2 + PAGE_EXISTS = 3 - def __init__(self, site, title, follow_redirects=False): + def __init__(self, site, title, follow_redirects=False, pageid=None): """Constructor for new Page instances. - Takes three arguments: a Site object, the Page's title (or pagename), - and whether or not to follow redirects (optional, defaults to False). + Takes four arguments: a Site object, the Page's title (or pagename), + whether or not to follow redirects (optional, defaults to False), and + a page ID to supplement the title (optional, defaults to None - i.e., + we will have to query the API to get it). - As with User, site.get_page() is preferred. Site's method has support - for a default *follow_redirects* value in our config, while __init__() - always defaults to False. + As with User, site.get_page() is preferred. __init__() will not do any API queries, but it will use basic namespace logic to determine our namespace ID and if we are a talkpage. @@ -94,9 +97,9 @@ class Page(CopyrightMixIn): self._site = site self._title = title.strip() self._follow_redirects = self._keep_following = follow_redirects + self._pageid = pageid - self._exists = 0 - self._pageid = None + self._exists = self.PAGE_UNKNOWN self._is_redirect = None self._lastrevid = None self._protection = None @@ -145,7 +148,7 @@ class Page(CopyrightMixIn): Note that validity != existence. If a page's title is invalid (e.g, it contains "[") it will always be invalid, and cannot be edited. """ - if self._exists == 1: + if self._exists == self.PAGE_INVALID: e = "Page '{0}' is invalid.".format(self._title) raise exceptions.InvalidPageError(e) @@ -157,7 +160,7 @@ class Page(CopyrightMixIn): It will also call _assert_validity() beforehand. """ self._assert_validity() - if self._exists == 2: + if self._exists == self.PAGE_MISSING: e = "Page '{0}' does not exist.".format(self._title) raise exceptions.PageNotFoundError(e) @@ -218,14 +221,14 @@ class Page(CopyrightMixIn): if "missing" in res: # If it has a negative ID and it's missing; we can still get # data like the namespace, protection, and URL: - self._exists = 2 + self._exists = self.PAGE_MISSING else: # If it has a negative ID and it's invalid, then break here, # because there's no other data for us to get: - self._exists = 1 + self._exists = self.PAGE_INVALID return else: - self._exists = 3 + self._exists = self.PAGE_EXISTS self._fullurl = res["fullurl"] self._protection = res["protection"] @@ -317,7 +320,7 @@ class Page(CopyrightMixIn): if result["edit"]["result"] == "Success": self._content = None self._basetimestamp = None - self._exists = 0 + self._exists = self.PAGE_UNKNOWN return # If we're here, then the edit failed. If it's because of AssertEdit, @@ -351,7 +354,7 @@ class Page(CopyrightMixIn): params["starttimestamp"] = self._starttimestamp if self._basetimestamp: params["basetimestamp"] = self._basetimestamp - if self._exists == 2: + if self._exists == self.PAGE_MISSING: # Page does not exist; don't edit if it already exists: params["createonly"] = "true" else: @@ -389,7 +392,7 @@ class Page(CopyrightMixIn): # These attributes are now invalidated: self._content = None self._basetimestamp = None - self._exists = 0 + self._exists = self.PAGE_UNKNOWN raise exceptions.EditConflictError(error.info) elif error.code in ["emptypage", "emptynewsection"]: @@ -437,12 +440,12 @@ class Page(CopyrightMixIn): @property def site(self): - """The Page's corresponding Site object.""" + """The page's corresponding Site object.""" return self._site @property def title(self): - """The Page's title, or "pagename". + """The page's title, or "pagename". This won't do any API queries on its own. Any other attributes or methods that do API queries will reload the title, however, like @@ -453,37 +456,36 @@ class Page(CopyrightMixIn): @property def exists(self): - """Information about whether the Page exists or not. + """Whether or not the page exists. - The "information" is a tuple with two items. The first is a bool, - either ``True`` if the page exists or ``False`` if it does not. The - second is a string giving more information, either ``"invalid"``, - (title is invalid, e.g. it contains ``"["``), ``"missing"``, or - ``"exists"``. + This will be a number; its value does not matter, but it will equal + one of :py:attr:`self.PAGE_INVALID `, + :py:attr:`self.PAGE_MISSING `, or + :py:attr:`self.PAGE_EXISTS `. Makes an API query only if we haven't already made one. """ - cases = { - 0: (None, "unknown"), - 1: (False, "invalid"), - 2: (False, "missing"), - 3: (True, "exists"), - } - if self._exists == 0: + if self._exists == self.PAGE_UNKNOWN: self._load() - return cases[self._exists] + return self._exists @property def pageid(self): - """An integer ID representing the Page. + """An integer ID representing the page. - Makes an API query only if we haven't already made one. + Makes an API query only if we haven't already made one and the *pageid* + parameter to :py:meth:`__init__` was left as ``None``, which should be + true for all cases except when pages are returned by an SQL generator + (like :py:meth:`category.get_members(use_sql=True) + `). Raises :py:exc:`~earwigbot.exceptions.InvalidPageError` or :py:exc:`~earwigbot.exceptions.PageNotFoundError` if the page name is invalid or the page does not exist, respectively. """ - if self._exists == 0: + if self._pageid: + return self._pageid + if self._exists == self.PAGE_UNKNOWN: self._load() self._assert_existence() # Missing pages do not have IDs return self._pageid @@ -501,7 +503,7 @@ class Page(CopyrightMixIn): else: slug = quote(self._title.replace(" ", "_"), safe="/:") path = self._site._article_path.replace("$1", slug) - return ''.join((self._site._base_url, path)) + return ''.join((self._site.url, path)) @property def namespace(self): @@ -523,7 +525,7 @@ class Page(CopyrightMixIn): name is invalid. Won't raise an error if the page is missing because those can still be create-protected. """ - if self._exists == 0: + if self._exists == self.PAGE_UNKNOWN: self._load() self._assert_validity() # Invalid pages cannot be protected return self._protection @@ -546,7 +548,7 @@ class Page(CopyrightMixIn): We will return ``False`` even if the page does not exist or is invalid. """ - if self._exists == 0: + if self._exists == self.PAGE_UNKNOWN: self._load() return self._is_redirect @@ -611,7 +613,7 @@ class Page(CopyrightMixIn): Raises InvalidPageError or PageNotFoundError if the page name is invalid or the page does not exist, respectively. """ - if self._exists == 0: + if self._exists == self.PAGE_UNKNOWN: # Kill two birds with one stone by doing an API query for both our # attributes and our page content: query = self._site.api_query @@ -626,7 +628,7 @@ class Page(CopyrightMixIn): if self._keep_following and self._is_redirect: self._title = self.get_redirect_target() self._keep_following = False # Don't follow double redirects - self._exists = 0 # Force another API query + self._exists = self.PAGE_UNKNOWN # Force another API query self.get() return self._content @@ -650,9 +652,10 @@ class Page(CopyrightMixIn): :py:exc:`~earwigbot.exceptions.RedirectError` if the page is not a redirect. """ + re_redirect = "^\s*\#\s*redirect\s*\[\[(.*?)\]\]" content = self.get() try: - return re.findall(self.re_redirect, content, flags=re.I)[0] + return re.findall(re_redirect, content, flags=re.I)[0] except IndexError: e = "The page does not appear to have a redirect target." raise exceptions.RedirectError(e) @@ -671,7 +674,7 @@ class Page(CopyrightMixIn): :py:exc:`~earwigbot.exceptions.PageNotFoundError` if the page name is invalid or the page does not exist, respectively. """ - if self._exists == 0: + if self._exists == self.PAGE_UNKNOWN: self._load() self._assert_existence() if not self._creator: diff --git a/earwigbot/wiki/site.py b/earwigbot/wiki/site.py index 4ec5beb..005c0d5 100644 --- a/earwigbot/wiki/site.py +++ b/earwigbot/wiki/site.py @@ -69,6 +69,7 @@ class Site(object): - :py:attr:`project`: the site's project name, like ``"wikipedia"`` - :py:attr:`lang`: the site's language code, like ``"en"`` - :py:attr:`domain`: the site's web domain, like ``"en.wikipedia.org"`` + - :py:attr:`url`: the site's URL, like ``"https://en.wikipedia.org"`` *Public methods:* @@ -184,6 +185,12 @@ class Site(object): res = "" return res.format(self.name, self.project, self.lang, self.domain) + def _unicodeify(self, value, encoding="utf8"): + """Return input as unicode if it's not unicode to begin with.""" + if isinstance(value, unicode): + return value + return unicode(value, encoding) + def _urlencode_utf8(self, params): """Implement urllib.urlencode() with support for unicode input.""" enc = lambda s: s.encode("utf8") if isinstance(s, unicode) else str(s) @@ -237,14 +244,7 @@ class Site(object): e = "Tried to do an API query, but no API URL is known." raise exceptions.SiteAPIError(e) - base_url = self._base_url - if base_url.startswith("//"): # Protocol-relative URLs from 1.18 - if self._use_https: - base_url = "https:" + base_url - else: - base_url = "http:" + base_url - url = ''.join((base_url, self._script_path, "/api.php")) - + url = ''.join((self.url, self._script_path, "/api.php")) params["format"] = "json" # This is the only format we understand if self._assert_edit: # If requested, ensure that we're logged in params["assert"] = self._assert_edit @@ -542,6 +542,17 @@ class Site(object): """The Site's web domain, like ``"en.wikipedia.org"``.""" return urlparse(self._base_url).netloc + @property + def url(self): + """The Site's full base URL, like ``"https://en.wikipedia.org"``.""" + url = self._base_url + if url.startswith("//"): # Protocol-relative URLs from 1.18 + if self._use_https: + url = "https:" + url + else: + url = "http:" + url + return url + def api_query(self, **kwargs): """Do an API query with `kwargs` as the parameters. @@ -682,7 +693,7 @@ class Site(object): e = "There is no namespace with name '{0}'.".format(name) raise exceptions.NamespaceNotFoundError(e) - def get_page(self, title, follow_redirects=False): + def get_page(self, title, follow_redirects=False, pageid=None): """Return a :py:class:`Page` object for the given title. *follow_redirects* is passed directly to @@ -696,23 +707,26 @@ class Site(object): redirect-following: :py:class:`~earwigbot.wiki.page.Page`'s methods provide that. """ + title = self._unicodeify(title) prefixes = self.namespace_id_to_name(constants.NS_CATEGORY, all=True) prefix = title.split(":", 1)[0] if prefix != title: # Avoid a page that is simply "Category" if prefix in prefixes: - return Category(self, title, follow_redirects) - return Page(self, title, follow_redirects) + return Category(self, title, follow_redirects, pageid) + return Page(self, title, follow_redirects, pageid) - def get_category(self, catname, follow_redirects=False): + def get_category(self, catname, follow_redirects=False, pageid=None): """Return a :py:class:`Category` object for the given category name. *catname* should be given *without* a namespace prefix. This method is really just shorthand for :py:meth:`get_page("Category:" + catname) `. """ + catname = self._unicodeify(catname) + name = name if isinstance(name, unicode) else name.decode("utf8") prefix = self.namespace_id_to_name(constants.NS_CATEGORY) - pagename = ':'.join((prefix, catname)) - return Category(self, pagename, follow_redirects) + pagename = u':'.join((prefix, catname)) + return Category(self, pagename, follow_redirects, pageid) def get_user(self, username=None): """Return a :py:class:`User` object for the given username. @@ -721,6 +735,7 @@ class Site(object): :py:class:`~earwigbot.wiki.user.User` object representing the currently logged-in (or anonymous!) user is returned. """ + username = self._unicodeify(username) if not username: username = self._get_username() return User(self, username) diff --git a/earwigbot/wiki/user.py b/earwigbot/wiki/user.py index 619e6ad..9256a52 100644 --- a/earwigbot/wiki/user.py +++ b/earwigbot/wiki/user.py @@ -39,6 +39,7 @@ class User(object): *Attributes:* + - :py:attr:`site`: the user's corresponding Site object - :py:attr:`name`: the user's username - :py:attr:`exists`: ``True`` if the user exists, else ``False`` - :py:attr:`userid`: an integer ID representing the user @@ -155,6 +156,11 @@ class User(object): self._gender = res["gender"] @property + def site(self): + """The user's corresponding Site object.""" + return self._site + + @property def name(self): """The user's username. diff --git a/setup.py b/setup.py index dcbc0a7..d687e96 100644 --- a/setup.py +++ b/setup.py @@ -38,6 +38,7 @@ setup( "oursql >= 0.9.3", # Talking with MediaWiki databases "oauth2 >= 1.5.211", # Talking with Yahoo BOSS Search "pycrypto >= 2.5", # Storing bot passwords and keys + "pytz >= 2012c", # Timezone handling ], test_suite = "tests", version = __version__,