Merge branch 'feature/commands' into develop

Conflicts:
	README.rst
	docs/customizing.rst
	earwigbot/commands/help.py
	earwigbot/commands/threads.py
	earwigbot/managers.py
	earwigbot/tasks/image_display_resize.py
	setup.py

README.rst

@@ -113,9 +113,9 @@ because it is the main way to communicate with other parts of the bot. A
 
 `earwigbot.config.BotConfig`_ stores configuration information for the bot. Its
 docstring explains what each attribute is used for, but essentially each "node"
-(one of ``config.components``, ``wiki``, ``tasks``, ``irc``, and ``metadata``)
-maps to a section of the bot's ``config.yml`` file. For example, if
-``config.yml`` includes something like::
+(one of ``config.components``, ``wiki``, ``irc``, ``commands``, ``tasks``, and
+``metadata``) maps to a section of the bot's ``config.yml`` file. For example,
+if ``config.yml`` includes something like::
 
     irc:
         frontend:
@@ -133,7 +133,8 @@ Custom IRC commands
 ~~~~~~~~~~~~~~~~~~~
 
 Custom commands are subclasses of `earwigbot.commands.Command`_ that override
-``Command``'s ``process()`` (and optionally ``check()``) methods.
+``Command``'s ``process()`` (and optionally ``check()`` or ``setup()``)
+methods.
 
 The bot has a wide selection of built-in commands and plugins to act as sample
 code and/or to give ideas. Start with test_, and then check out chanops_ and

docs/customizing.rst

@@ -58,8 +58,13 @@ The most useful attributes are:
 
 :py:class:`earwigbot.config.BotConfig` stores configuration information for the
 bot. Its docstrings explains what each attribute is used for, but essentially
-each "node" (one of :py:attr:`config.components`, :py:attr:`wiki`,
-:py:attr:`tasks`, :py:attr:`tasks`, or :py:attr:`metadata`) maps to a section
+each "node" (one of :py:attr:`config.components
+<earwigbot.config.BotConfig.components>`,
+:py:attr:`~earwigbot.config.BotConfig.wiki`,
+:py:attr:`~earwigbot.config.BotConfig.irc`,
+:py:attr:`~earwigbot.config.BotConfig.commands`,
+:py:attr:`~earwigbot.config.BotConfig.tasks`, or
+:py:attr:`~earwigbot.config.BotConfig.metadata`) maps to a section
 of the bot's :file:`config.yml` file. For example, if :file:`config.yml`
 includes something like::
 
@@ -81,7 +86,8 @@ Custom IRC commands
 Custom commands are subclasses of :py:class:`earwigbot.commands.Command` that
 override :py:class:`~earwigbot.commands.Command`'s
 :py:meth:`~earwigbot.commands.Command.process` (and optionally
-:py:meth:`~earwigbot.commands.Command.check`) methods.
+:py:meth:`~earwigbot.commands.Command.check` or
+:py:meth:`~earwigbot.commands.Command.setup`) methods.
 
 :py:class:`~earwigbot.commands.Command`'s docstrings should explain what each
 attribute and method is for and what they should be overridden with, but these
@@ -90,6 +96,15 @@ are the basics:
 - Class attribute :py:attr:`~earwigbot.commands.Command.name` is the name of
   the command. This must be specified.
 
+- Class attribute :py:attr:`~earwigbot.commands.Command.commands` is a list of
+  names that will trigger this command. It defaults to the command's
+  :py:attr:`~earwigbot.commands.Command.name`, but you can override it with
+  multiple names to serve as aliases. This is handled by the default
+  :py:meth:`~earwigbot.commands.Command.check` implementation (see below), so
+  if :py:meth:`~earwigbot.commands.Command.check` is overridden, this is
+  ignored by everything except the help_ command (so ``!help alias`` will
+  trigger help for the actual command).
+
 - Class attribute :py:attr:`~earwigbot.commands.Command.hooks` is a list of the
   "IRC events" that this command might respond to. It defaults to ``["msg"]``,
   but options include ``"msg_private"`` (for private messages only),
@@ -97,15 +112,25 @@ are the basics:
   joins a channel). See the afc_status_ plugin for a command that responds to
   other hook types.
 
+- Method :py:meth:`~earwigbot.commands.Command.setup` is called *once* with no
+  arguments immediately after the command is first loaded. Does nothing by
+  default; treat it like an :py:meth:`__init__` if you want
+  (:py:meth:`~earwigbot.tasks.Command.__init__` does things by default and a
+  dedicated setup method is often easier than overriding
+  :py:meth:`~earwigbot.tasks.Command.__init__` and using :py:obj:`super`).
+
 - Method :py:meth:`~earwigbot.commands.Command.check` is passed a
   :py:class:`~earwigbot.irc.data.Data` object, and should return ``True`` if
   you want to respond to this message, or ``False`` otherwise. The default
   behavior is to return ``True`` only if :py:attr:`data.is_command` is ``True``
-  and :py:attr:`data.command` == :py:attr:`~earwigbot.commands.Command.name`,
-  which is suitable for most cases. A common, straightforward reason for
-  overriding is if a command has aliases (see chanops_ for an example). Note
-  that by returning ``True``, you prevent any other commands from responding to
-  this message.
+  and :py:attr:`data.command` ``==``
+  :py:attr:`~earwigbot.commands.Command.name` (or :py:attr:`data.command
+  <earwigbot.irc.data.Data.command>` is in
+  :py:attr:`~earwigbot.commands.Command.commands` if that list is overriden;
+  see above), which is suitable for most cases. A possible reason for
+  overriding is if you want to do something in response to events from a
+  specific channel only. Note that by returning ``True``, you prevent any other
+  commands from responding to this message.
 
 - Method :py:meth:`~earwigbot.commands.Command.process` is passed the same
   :py:class:`~earwigbot.irc.data.Data` object as
@@ -128,6 +153,12 @@ are the basics:
   <earwigbot.irc.connection.IRCConnection.join>`, and
   :py:meth:`part(chan) <earwigbot.irc.connection.IRCConnection.part>`.
 
+Commands have access to :py:attr:`config.commands[command_name]` for config
+information, which is a node in :file:`config.yml` like every other attribute
+of :py:attr:`bot.config`. This can be used to store, for example, API keys or
+SQL connection info, so that these can be easily changed without modifying the
+command itself.
+
 The command *class* doesn't need a specific name, but it should logically
 follow the command's name. The filename doesn't matter, but it is recommended
 to match the command name for readability. Multiple command classes are allowed
@@ -190,7 +221,7 @@ are the basics:
 
 Tasks have access to :py:attr:`config.tasks[task_name]` for config information,
 which is a node in :file:`config.yml` like every other attribute of
-:py:attr:`bot.config`. This can be used to store, for example, edit summaries,
+:py:attr:`bot.config`. This can be used to store, for example, edit summaries
 or templates to append to user talk pages, so that these can be easily changed
 without modifying the task itself.
 
@@ -201,8 +232,9 @@ the task name for readability. Multiple tasks classes are allowed in one file.
 See the built-in wikiproject_tagger_ task for a relatively straightforward
 task, or the afc_statistics_ plugin for a more complicated one.
 
+.. _help:               https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/help.py
 .. _afc_status:         https://github.com/earwig/earwigbot-plugins/blob/develop/commands/afc_status.py
-.. _chanops:            https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/chanops.py
 .. _test:               https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/test.py
+.. _chanops:            https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/chanops.py
 .. _wikiproject_tagger: https://github.com/earwig/earwigbot/blob/develop/earwigbot/tasks/wikiproject_tagger.py
 .. _afc_statistics:     https://github.com/earwig/earwigbot-plugins/blob/develop/tasks/afc_statistics.py

docs/toolset.rst

@@ -80,6 +80,8 @@ following attributes:
   ``"en"``
 - :py:attr:`~earwigbot.wiki.site.Site.domain`: the site's web domain, like
   ``"en.wikipedia.org"``
+- :py:attr:`~earwigbot.wiki.site.Site.url`: the site's full base URL, like
+  ``"https://en.wikipedia.org"``
 
 and the following methods:
 
@@ -97,11 +99,11 @@ and the following methods:
 - :py:meth:`namespace_name_to_id(name)
   <earwigbot.wiki.site.Site.namespace_name_to_id>`: given a namespace name,
   returns the associated namespace ID
-- :py:meth:`get_page(title, follow_redirects=False)
+- :py:meth:`get_page(title, follow_redirects=False, ...)
   <earwigbot.wiki.site.Site.get_page>`: returns a ``Page`` object for the given
   title (or a :py:class:`~earwigbot.wiki.category.Category` object if the
   page's namespace is "``Category:``")
-- :py:meth:`get_category(catname, follow_redirects=False)
+- :py:meth:`get_category(catname, follow_redirects=False, ...)
   <earwigbot.wiki.site.Site.get_category>`: returns a ``Category`` object for
   the given title (sans namespace)
 - :py:meth:`get_user(username) <earwigbot.wiki.site.Site.get_user>`: returns a
@@ -120,7 +122,7 @@ provide the following attributes:
 - :py:attr:`~earwigbot.wiki.page.Page.site`: the page's corresponding
   :py:class:`~earwigbot.wiki.site.Site` object
 - :py:attr:`~earwigbot.wiki.page.Page.title`: the page's title, or pagename
-- :py:attr:`~earwigbot.wiki.page.Page.exists`: whether the page exists
+- :py:attr:`~earwigbot.wiki.page.Page.exists`: whether or not the page exists
 - :py:attr:`~earwigbot.wiki.page.Page.pageid`: an integer ID representing the
   page
 - :py:attr:`~earwigbot.wiki.page.Page.url`: the page's URL
@@ -166,9 +168,10 @@ or :py:meth:`site.get_page(title) <earwigbot.wiki.site.Site.get_page>` where
 ``title`` is in the ``Category:`` namespace) provide the following additional
 method:
 
-- :py:meth:`get_members(use_sql=False, limit=None)
-  <earwigbot.wiki.category.Category.get_members>`: returns a list of page
-  titles in the category (limit is ``50`` by default if using the API)
+- :py:meth:`get_members(use_sql=False, limit=None, ...)
+  <earwigbot.wiki.category.Category.get_members>`: iterates over
+  :py:class:`~earwigbot.wiki.page.Page`\ s in the category, until either the
+  category is exhausted or (if given) ``limit`` is reached
 
 Users
 ~~~~~
@@ -178,6 +181,8 @@ Create :py:class:`earwigbot.wiki.User <earwigbot.wiki.user.User>` objects with
 :py:meth:`page.get_creator() <earwigbot.wiki.page.Page.get_creator>`. They
 provide the following attributes:
 
+- :py:attr:`~earwigbot.wiki.user.User.site`: the user's corresponding
+  :py:class:`~earwigbot.wiki.site.Site` object
 - :py:attr:`~earwigbot.wiki.user.User.name`: the user's username
 - :py:attr:`~earwigbot.wiki.user.User.exists`: ``True`` if the user exists, or
   ``False`` if they do not

earwigbot/commands/__init__.py

@@ -36,9 +36,13 @@ class Command(object):
     This docstring is reported to the user when they type ``"!help
     <command>"``.
     """
-    # This is the command's name, as reported to the user when they use !help:
+    # The command's name, as reported to the user when they use !help:
     name = None
 
+    # A list of names that will trigger this command. If left empty, it will
+    # be triggered by the command's name and its name only:
+    commands = []
+
     # Hooks are "msg", "msg_private", "msg_public", and "join". "msg" is the
     # default behavior; if you wish to override that, change the value in your
     # command subclass:
@@ -49,9 +53,10 @@ class Command(object):
 
         This is called once when the command is loaded (from
         :py:meth:`commands.load() <earwigbot.managers._ResourceManager.load>`).
-        *bot* is out base :py:class:`~earwigbot.bot.Bot` object. Generally you
-        shouldn't need to override this; if you do, call
-        ``super(Command, self).__init__()`` first.
+        *bot* is out base :py:class:`~earwigbot.bot.Bot` object. Don't override
+        this directly; if you do, remember to place
+        ``super(Command, self).__init()`` first. Use :py:meth:`setup` for
+        typical command-init/setup needs.
         """
         self.bot = bot
         self.config = bot.config
@@ -67,6 +72,15 @@ class Command(object):
         self.mode = lambda t, level, msg: self.bot.frontend.mode(t, level, msg)
         self.pong = lambda target: self.bot.frontend.pong(target)
 
+        self.setup()
+
+    def setup(self):
+        """Hook called immediately after the command is loaded.
+
+        Does nothing by default; feel free to override.
+        """
+        pass
+
     def check(self, data):
         """Return whether this command should be called in response to *data*.
 
@@ -76,11 +90,15 @@ class Command(object):
         sent on IRC, it should be cheap to execute and unlikely to throw
         exceptions.
 
-        Most commands return ``True`` if :py:attr:`data.command
+        Most commands return ``True`` only if :py:attr:`data.command
         <earwigbot.irc.data.Data.command>` ``==`` :py:attr:`self.name <name>`,
-        otherwise they return ``False``. This is the default behavior of
-        :py:meth:`check`; you need only override it if you wish to change that.
+        or :py:attr:`data.command <earwigbot.irc.data.Data.command>` is in
+        :py:attr:`self.commands <commands>` if that list is overriden. This is
+        the default behavior; you should only override it if you wish to change
+        that.
         """
+        if self.commands:
+            return data.is_command and data.command in self.commands
         return data.is_command and data.command == self.name
 
     def process(self, data):

earwigbot/commands/_old.py

@@ -1,401 +0,0 @@
-# -*- coding: utf-8  -*-
-######
-###### NOTE:
-###### This is an old commands file from the previous version of EarwigBot.
-###### It is not used by the new EarwigBot and is simply here for reference
-###### when developing new commands.
-######
-### EarwigBot
-
-def parse(command, line, line2, nick, chan, host, auth, notice, say, reply, s):
-	authy = auth(host)
-	if command == "access":
-		a = 'The bot\'s owner is "%s".' % OWNER
-		b = 'The bot\'s admins are "%s".' % ', '.join(ADMINS_R)
-		reply(a, chan, nick)
-		reply(b, chan, nick)
-		return
-	if command == "tock":
-		u = urllib.urlopen('http://tycho.usno.navy.mil/cgi-bin/timer.pl')
-		info = u.info()
-		u.close()
-		say('"' + info['Date'] + '" - tycho.usno.navy.mil', chan)
-		return
-	if command == "beats":
-		beats = ((time.time() + 3600) % 86400) / 86.4
-		beats = int(math.floor(beats))
-		say('@%03i' % beats, chan)
-		return
-	if command == "dict" or command == "dictionary":
-		def trim(thing):
-			if thing.endswith(' '):
-				thing = thing[:-6]
-			return thing.strip(' :.')
-		r_li = re.compile(r'(?ims)<li>.*?</li>')
-		r_tag = re.compile(r'<[^>]+>')
-		r_parens = re.compile(r'(?<=\()(?:[^()]+|\([^)]+\))*(?=\))')
-		r_word = re.compile(r'^[A-Za-z0-9\' -]+$')
-		uri = 'http://encarta.msn.com/dictionary_/%s.html'
-		r_info = re.compile(r'(?:ResultBody"><br /><br />(.*?)&nbsp;)|(?:<b>(.*?)</b>)')
-		try:
-			word = line2[4]
-		except Exception:
-			reply("Please enter a word.", chan, nick)
-			return
-		word = urllib.quote(word.encode('utf-8'))
-		bytes = web.get(uri % word)
-		results = {}
-		wordkind = None
-		for kind, sense in r_info.findall(bytes):
-			kind, sense = trim(kind), trim(sense)
-			if kind: wordkind = kind
-			elif sense:
-				results.setdefault(wordkind, []).append(sense)
-		result = word.encode('utf-8') + ' - '
-		for key in sorted(results.keys()):
-			if results[key]:
-				result += (key or '') + ' 1. ' + results[key][0]
-				if len(results[key]) > 1:
-					result += ', 2. ' + results[key][1]
-				result += '; '
-		result = result.rstrip('; ')
-		if result.endswith('-') and (len(result) < 30):
-			reply('Sorry, no definition found.', chan, nick)
-		else: say(result, chan)
-		return
-	if command == "ety" or command == "etymology":
-		etyuri = 'http://etymonline.com/?term=%s'
-		etysearch = 'http://etymonline.com/?search=%s'
-		r_definition = re.compile(r'(?ims)<dd[^>]*>.*?</dd>')
-		r_tag = re.compile(r'<(?!!)[^>]+>')
-		r_whitespace = re.compile(r'[\t\r\n ]+')
-		abbrs = [
-			'cf', 'lit', 'etc', 'Ger', 'Du', 'Skt', 'Rus', 'Eng', 'Amer.Eng', 'Sp',
-			'Fr', 'N', 'E', 'S', 'W', 'L', 'Gen', 'J.C', 'dial', 'Gk',
-			'19c', '18c', '17c', '16c', 'St', 'Capt', 'obs', 'Jan', 'Feb', 'Mar',
-			'Apr', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec', 'c', 'tr', 'e', 'g'
-		]
-		t_sentence = r'^.*?(?<!%s)(?:\.(?= [A-Z0-9]|\Z)|\Z)'
-		r_sentence = re.compile(t_sentence % ')(?<!'.join(abbrs))
-		def unescape(s):
-			s = s.replace('&gt;', '>')
-			s = s.replace('&lt;', '<')
-			s = s.replace('&amp;', '&')
-			return s
-		def text(html):
-			html = r_tag.sub('', html)
-			html = r_whitespace.sub(' ', html)
-			return unescape(html).strip()
-		try:
-			word = line2[4]
-		except Exception:
-			reply("Please enter a word.", chan, nick)
-			return
-		def ety(word):
-			if len(word) > 25:
-				raise ValueError("Word too long: %s[...]" % word[:10])
-			word = {'axe': 'ax/axe'}.get(word, word)
-			bytes = web.get(etyuri % word)
-			definitions = r_definition.findall(bytes)
-			if not definitions:
-				return None
-			defn = text(definitions[0])
-			m = r_sentence.match(defn)
-			if not m:
-				return None
-			sentence = m.group(0)
-			try:
-				sentence = unicode(sentence, 'iso-8859-1')
-				sentence = sentence.encode('utf-8')
-			except: pass
-			maxlength = 275
-			if len(sentence) > maxlength:
-				sentence = sentence[:maxlength]
-				words = sentence[:-5].split(' ')
-				words.pop()
-				sentence = ' '.join(words) + ' [...]'
-			sentence = '"' + sentence.replace('"', "'") + '"'
-			return sentence + ' - ' + (etyuri % word)
-		try:
-			result = ety(word.encode('utf-8'))
-		except IOError:
-			msg = "Can't connect to etymonline.com (%s)" % (etyuri % word)
-			reply(msg, chan, nick)
-			return
-		except AttributeError:
-			result = None
-		if result is not None:
-			reply(result, chan, nick)
-		else:
-			uri = etysearch % word
-			msg = 'Can\'t find the etymology for "%s". Try %s' % (word, uri)
-			reply(msg, chan, nick)
-		return
-	if command == "pend" or command == "pending":
-		say("Pending submissions status page: <http://en.wikipedia.org/wiki/WP:AFC/S>.", chan)
-		say("Pending submissions category: <http://en.wikipedia.org/wiki/Category:Pending_AfC_submissions>.", chan)
-		return
-	if command == "sub" or command == "submissions":
-		try:
-			number = int(line2[4])
-		except Exception:
-			reply("Please enter a number.", chan, nick)
-			return
-		do_url = False
-		try:
-			if "url" in line2[5:]: do_url = True
-		except Exception:
-			pass
-		url = "http://en.wikipedia.org/w/api.php?action=query&list=categorymembers&cmtitle=Category:Pending_AfC_submissions&cmlimit=500&cmsort=timestamp"
-		query = urllib.urlopen(url)
-		data = query.read()
-		pages = re.findall("title=&quot;(.*?)&quot;", data)
-		try:
-			pages.remove("Wikipedia:Articles for creation/Redirects")
-		except Exception:
-			pass
-		try:
-			pages.remove("Wikipedia:Files for upload")
-		except Exception:
-			pass
-		pages.reverse()
-		pages = pages[:number]
-		if not do_url:
-			s = string.join(pages, "]], [[")
-			s = "[[%s]]" % s
-		else:
-			s = string.join(pages, ">, <http://en.wikipedia.org/wiki/")
-			s = "<http://en.wikipedia.org/wiki/%s>" % s
-			s = re.sub(" ", "_", s)
-			s = re.sub(">,_<", ">, <", s)
-		report = "\x02First %s pending AfC submissions:\x0F %s" % (number, s)
-		say(report, chan)
-		return
-	if command == "trout":
-		try:
-			user = line2[4]
-			user = ' '.join(line2[4:])
-		except Exception:
-			reply("Hahahahahahahaha...", chan, nick)
-			return
-		normal = unicodedata.normalize('NFKD', unicode(string.lower(user)))
-		if "itself" in normal:
-			reply("I'm not that stupid ;)", chan, nick)
-			return
-		elif "earwigbot" in normal:
-			reply("I'm not that stupid ;)", chan, nick)
-		elif "earwig" not in normal and "ear wig" not in normal:
-			text = 'slaps %s around a bit with a large trout.' % user
-			msg = '\x01ACTION %s\x01' % text
-			say(msg, chan)
-		else:
-			reply("I refuse to hurt anything with \"Earwig\" in its name :P", chan, nick)
-		return
-	if command == "mysql":
-		if authy != "owner":
-			reply("You aren't authorized to use this command.", chan, nick)
-			return
-		import MySQLdb
-		try:
-			strings = line2[4]
-			strings = ' '.join(line2[4:])
-			if "db:" in strings:
-				database = re.findall("db\:(.*?)\s", strings)[0]
-			else:
-				database = "enwiki_p"
-			if "time:" in strings:
-				times = int(re.findall("time\:(.*?)\s", strings)[0])
-			else:
-				times = 60
-			file = re.findall("file\:(.*?)\s", strings)[0]
-			sqlquery = re.findall("query\:(.*?)\Z", strings)[0]
-		except Exception:
-			reply("You did not specify enough data for the bot to continue.", chan, nick)
-			return
-		database2 = database[:-2] + "-p"
-		db = MySQLdb.connect(db=database, host="%s.rrdb.toolserver.org" % database2, read_default_file="/home/earwig/.my.cnf")
-		db.query(sqlquery)
-		r = db.use_result()
-		data = r.fetch_row(0)
-		try:
-			f = codecs.open("/home/earwig/public_html/reports/%s/%s" % (database[:-2], file), 'r')
-			reply("A file already exists with that name.", chan, nick)
-			return
-		except Exception:
-			pass
-		f = codecs.open("/home/earwig/public_html/reports/%s/%s" % (database[:-2], file), 'a', 'utf-8')
-		for line in data:
-			new_line = []
-			for l in line:
-				new_line.append(str(l))
-			f.write('	'.join(new_line) + "\n")
-		f.close()
-		reply("Query completed successfully. See http://toolserver.org/~earwig/reports/%s/%s. I will delete the report in %s seconds." % (database[:-2], file, times), chan, nick)
-		time.sleep(times)
-		os.remove("/home/earwig/public_html/reports/%s/%s" % (database[:-2], file))
-		return
-	if command == "notes" or command == "note" or command == "about" or command == "data" or command == "database":
-		try:
-			action = line2[4]
-		except BaseException:
-			reply("What do you want me to do? Type \"!notes help\" for more information.", chan, nick)
-			return
-		import MySQLdb
-		db = MySQLdb.connect(db="u_earwig_ircbot", host="sql", read_default_file="/home/earwig/.my.cnf")
-		specify = ' '.join(line2[5:])
-		if action == "help" or action == "manual":
-			shortCommandList = "read, write, change, undo, delete, move, author, category, list, report, developer"
-			if specify == "read":
-				say("To read an entry, type \"!notes read <entry>\".", chan)
-			elif specify == "write":
-				say("To write a new entry, type \"!notes write <entry> <content>\". This will create a new entry only if one does not exist, see the below command...", chan)
-			elif specify == "change":
-				say("To change an entry, type \"!notes change <entry> <new content>\". The old entry will be stored in the database, so it can be undone later.", chan)
-			elif specify == "undo":
-				say("To undo a change, type \"!notes undo <entry>\".", chan)
-			elif specify == "delete":
-				say("To delete an entry, type \"!notes delete <entry>\". For security reasons, only bot admins can do this.", chan)
-			elif specify == "move":
-				say("To move an entry, type \"!notes move <old_title> <new_title>\".", chan)
-			elif specify == "author":
-				say("To return the author of an entry, type \"!notes author <entry>\".", chan)
-			elif specify == "category" or specify == "cat":
-				say("To change an entry's category, type \"!notes category <entry> <category>\".", chan)
-			elif specify == "list":
-				say("To list all categories in the database, type \"!notes list\". Type \"!notes list <category>\" to get all entries in a certain category.", chan)
-			elif specify == "report":
-				say("To give some statistics about the mini-wiki, including some debugging information, type \"!notes report\" in a PM.", chan)
-			elif specify == "developer":
-				say("To do developer work, such as writing to the database directly, type \"!notes developer <command>\". This can only be done by the bot owner.", chan)
-			else:
-				db.query("SELECT * FROM version;")
-				r = db.use_result()
-				data = r.fetch_row(0)
-				version = data[0]
-				reply("The Earwig Mini-Wiki: running v%s." % version, chan, nick)
-				reply("The full list of commands, for reference, are: %s." % shortCommandList, chan, nick)
-				reply("For an explaination of a certain command, type \"!notes help <command>\".", chan, nick)
-				reply("You can also access the database from the Toolserver: http://toolserver.org/~earwig/cgi-bin/irc_database.py", chan, nick)
-				time.sleep(0.4)
-			return
-		elif action == "read":
-			specify = string.lower(specify)
-			if " " in specify: specify = string.split(specify, " ")[0]
-			if not specify or "\"" in specify:
-				reply("Please include the name of the entry you would like to read after the command, e.g. !notes read earwig", chan, nick)
-				return
-			try:
-				db.query("SELECT entry_content FROM entries WHERE entry_title = \"%s\";" % specify)
-				r = db.use_result()
-				data = r.fetch_row(0)
-				entry = data[0][0]
-				say("Entry \"\x02%s\x0F\": %s" % (specify, entry), chan)
-			except Exception:
-				reply("There is no entry titled \"\x02%s\x0F\"." % specify, chan, nick)
-			return
-		elif action == "delete" or action == "remove":
-			specify = string.lower(specify)
-			if " " in specify: specify = string.split(specify, " ")[0]
-			if not specify or "\"" in specify:
-				reply("Please include the name of the entry you would like to delete after the command, e.g. !notes delete earwig", chan, nick)
-				return
-			if authy == "owner" or authy == "admin":
-				try:
-					db.query("DELETE from entries where entry_title = \"%s\";" % specify)
-					r = db.use_result()
-					db.commit()
-					reply("The entry on \"\x02%s\x0F\" has been removed." % specify, chan, nick)
-				except Exception:
-					phenny.reply("Unable to remove the entry on \"\x02%s\x0F\", because it doesn't exist." % specify, chan, nick)
-			else:
-				reply("Only bot admins can remove entries.", chan, nick)
-			return
-		elif action == "developer":
-			if authy == "owner":
-				db.query(specify)
-				r = db.use_result()
-				try:
-					print r.fetch_row(0)
-				except Exception:
-					pass
-				db.commit()
-				reply("Done.", chan, nick)
-			else:
-				reply("Only the bot owner can modify the raw database.", chan, nick)
-			return
-		elif action == "write":
-			try:
-				write = line2[5]
-				content = ' '.join(line2[6:])
-			except Exception:
-				reply("Please include some content in your entry.", chan, nick)
-				return
-			db.query("SELECT * from entries WHERE entry_title = \"%s\";" % write)
-			r = db.use_result()
-			data = r.fetch_row(0)
-			if data:
-				reply("An entry on %s already exists; please use \"!notes change %s %s\"." % (write, write, content), chan, nick)
-				return
-			content2 = content.replace('"', '\\' + '"')
-			db.query("INSERT INTO entries (entry_title, entry_author, entry_category, entry_content, entry_content_old) VALUES (\"%s\", \"%s\", \"uncategorized\", \"%s\", NULL);" % (write, nick, content2))
-			db.commit()
-			reply("You have written an entry titled \"\x02%s\x0F\", with the following content: \"%s\"" % (write, content), chan, nick)
-			return
-		elif action == "change":
-			reply("NotImplementedError", chan, nick)
-		elif action == "undo":
-			reply("NotImplementedError", chan, nick)
-		elif action == "move":
-			reply("NotImplementedError", chan, nick)
-		elif action == "author":
-			try:
-				entry = line2[5]
-			except Exception:
-				reply("Please include the name of the entry you would like to get information for after the command, e.g. !notes author earwig", chan, nick)
-				return
-			db.query("SELECT entry_author from entries WHERE entry_title = \"%s\";" % entry)
-			r = db.use_result()
-			data = r.fetch_row(0)
-			if data:
-				say("The author of \"\x02%s\x0F\" is \x02%s\x0F." % (entry, data[0][0]), chan)
-				return
-			reply("There is no entry titled \"\x02%s\x0F\"." % entry, chan, nick)
-			return
-		elif action == "cat" or action == "category":
-			reply("NotImplementedError", chan, nick)
-		elif action == "list":
-			reply("NotImplementedError", chan, nick)
-		elif action == "report":
-			reply("NotImplementedError", chan, nick)
-	if command == "langcode" or command == "lang" or command == "language":
-		try:
-			lang = line2[4]
-		except Exception:
-			reply("Please specify an ISO code.", chan, nick)
-			return
-		data = urllib.urlopen("http://toolserver.org/~earwig/cgi-bin/swmt.py?action=iso").read()
-		data = string.split(data, "\n")
-		result = False
-		for datum in data:
-			if datum.startswith(lang):
-				result = re.findall(".*? (.*)", datum)[0]
-				break
-		if result:
-			reply(result, chan, nick)
-			return
-		reply("Not found.", chan, nick)
-		return
-	if command == "lookup" or command == "ip":
-		try:
-			hexIP = line2[4]
-		except Exception:
-			reply("Please specify a hex IP address.", chan, nick)
-			return
-		hexes = [hexIP[:2], hexIP[2:4], hexIP[4:6], hexIP[6:8]]
-		hashes = []
-		for hexHash in hexes:
-			newHex = int(hexHash, 16)
-			hashes.append(newHex)
-		normalizedIP = "%s.%s.%s.%s" % (hashes[0], hashes[1], hashes[2], hashes[3])
-		reply(normalizedIP, chan, nick)
-		return

earwigbot/commands/afc_pending.py

@@ -0,0 +1,34 @@
+# -*- coding: utf-8  -*-
+#
+# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net>
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+from earwigbot.commands import BaseCommand
+
+class Command(BaseCommand):
+    """Link the user to the pending AFC submissions page and category."""
+    name = "pending"
+    commands = ["pending", "pend"]
+
+    def process(self, data):
+        msg1 = "pending submissions status page: http://enwp.org/WP:AFC/ST"
+        msg2 = "pending submissions category: http://enwp.org/CAT:PEND"
+        self.reply(data, msg1)
+        self.reply(data, msg2)

earwigbot/commands/afc_report.py

@@ -72,7 +72,7 @@ class AFCReport(Command):
 
     def get_page(self, title):
         page = self.site.get_page(title, follow_redirects=False)
-        if page.exists[0]:
+        if page.exists == page.PAGE_EXISTS:
             return page
 
     def report(self, page):

earwigbot/commands/afc_status.py

@@ -30,13 +30,12 @@ class AFCStatus(Command):
     """Get the number of pending AfC submissions, open redirect requests, and
     open file upload requests."""
     name = "status"
+    commands = ["status", "count", "num", "number"]
     hooks = ["join", "msg"]
 
     def check(self, data):
-        commands = ["status", "count", "num", "number"]
-        if data.is_command and data.command in commands:
+        if data.is_command and data.command in self.commands:
             return True
-
         try:
             if data.line[1] == "JOIN" and data.chan == "#wikipedia-en-afc":
                 if data.nick != self.config.irc["frontend"]["nick"]:

earwigbot/commands/afc_submissions.py

@@ -0,0 +1,60 @@
+# -*- coding: utf-8  -*-
+#
+# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net>
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+from earwigbot.commands import Command
+
+__all__ = ["AFCSubmissions"]
+
+class AFCSubmissions(Command):
+    """Link the user directly to some pending AFC submissions."""
+    name = "submissions"
+    commands = ["submissions", "subs"]
+
+    def setup(self):
+        try:
+            self.ignore_list = self.config.commands[self.name]["ignoreList"]
+        except KeyError:
+            try:
+                ignores = self.config.tasks["afc_statistics"]["ignoreList"]
+                self.ignore_list = ignores
+            except KeyError:
+                self.ignore_list = []
+
+    def process(self, data):
+        if data.args:
+            try:
+                number = int(data.args[0])
+            except ValueError:
+                self.reply(data, "argument must be a number.")
+                return
+            if number > 5:
+                msg = "cannot get more than five submissions at a time."
+                self.reply(data, msg)
+                return
+        else:
+            number = 3
+
+        site = self.bot.wiki.get_site()
+        category = site.get_category("Pending AfC submissions")
+        members = category.get_members(use_sql=True, limit=number)
+        pages = ", ".join([member.url for member in members])
+        self.reply(data, "{0} pending AfC subs: {1}".format(number, pages))

earwigbot/commands/chanops.py

@@ -28,10 +28,7 @@ class ChanOps(Command):
     """Voice, devoice, op, or deop users in the channel, or join or part from
     other channels."""
     name = "chanops"
-
-    def check(self, data):
-        cmnds = ["chanops", "voice", "devoice", "op", "deop", "join", "part"]
-        return data.is_command and data.command in cmnds
+    commands = ["chanops", "voice", "devoice", "op", "deop", "join", "part"]
 
     def process(self, data):
         if data.command == "chanops":

earwigbot/commands/crypt.py

@@ -32,12 +32,7 @@ class Crypt(Command):
     """Provides hash functions with !hash (!hash list for supported algorithms)
     and blowfish encryption with !encrypt and !decrypt."""
     name = "crypt"
-
-    def check(self, data):
-        commands = ["crypt", "hash", "encrypt", "decrypt"]
-        if data.is_command and data.command in commands:
-            return True
-        return False
+    commands = ["crypt", "hash", "encrypt", "decrypt"]
 
     def process(self, data):
         if data.command == "crypt":

earwigbot/commands/editcount.py

@@ -30,12 +30,7 @@ __all__ = ["Editcount"]
 class Editcount(Command):
     """Return a user's edit count."""
     name = "editcount"
-
-    def check(self, data):
-        commands = ["ec", "editcount"]
-        if data.is_command and data.command in commands:
-            return True
-        return False
+    commands = ["ec", "editcount"]
 
     def process(self, data):
         if not data.args:
@@ -54,6 +49,7 @@ class Editcount(Command):
             return
 
         safe = quote_plus(user.name)
-        url = "http://toolserver.org/~tparis/pcount/index.php?name={0}&lang=en&wiki=wikipedia"
+        url = "http://toolserver.org/~tparis/pcount/index.php?name={0}&lang={1}&wiki={2}"
+        fullurl = url.format(safe, site.lang, site.project)
         msg = "\x0302{0}\x0301 has {1} edits ({2})."
-        self.reply(data, msg.format(name, count, url.format(safe)))
+        self.reply(data, msg.format(name, count, fullurl))

earwigbot/commands/geolocate.py

@@ -0,0 +1,72 @@
+# -*- coding: utf-8  -*-
+#
+# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net>
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+import json
+import urllib2
+
+from earwigbot.commands import BaseCommand
+
+class Command(BaseCommand):
+    """Geolocate an IP address (via http://ipinfodb.com/)."""
+    name = "geolocate"
+    commands = ["geolocate", "locate", "geo", "ip"]
+
+    def setup(self):
+        self.config.decrypt(self.config.commands, (self.name, "apiKey"))
+        try:
+            self.key = self.config.commands[self.name]["apiKey"]
+        except KeyError:
+            self.key = None
+            log = 'Cannot use without an API key for http://ipinfodb.com/ stored as config.commands["{0}"]["apiKey"]'
+            self.logger.warn(log.format(self.name))
+
+    def process(self, data):
+        if not data.args:
+            self.reply(data, "please specify an IP to lookup.")
+            return
+
+        if not self.key:
+            msg = 'I need an API key for http://ipinfodb.com/ stored as \x0303config.commands["{0}"]["apiKey"]\x0301.'
+            log = 'Need an API key for http://ipinfodb.com/ stored as config.commands["{0}"]["apiKey"]'
+            self.reply(data, msg.format(self.name) + ".")
+            self.logger.error(log.format(self.name))
+            return
+
+        address = data.args[0]
+        url = "http://api.ipinfodb.com/v3/ip-city/?key={0}&ip={1}&format=json"
+        query = urllib2.urlopen(url.format(self.key, address)).read()
+        res = json.loads(query)
+
+        try:
+            country = res["countryName"]
+            region = res["regionName"]
+            city = res["cityName"]
+            latitude = res["latitude"]
+            longitude = res["longitude"]
+            utcoffset = res["timeZone"]
+        except KeyError:
+            self.reply(data, "IP \x0302{0}\x0301 not found.".format(address))
+            return
+
+        msg = "{0}, {1}, {2} ({3}, {4}), UTC {5}"
+        geo = msg.format(country, region, city, latitude, longitude, utcoffset)
+        self.reply(data, geo)

earwigbot/commands/git.py

@@ -32,10 +32,12 @@ class Git(Command):
     """Commands to interface with the bot's git repository; use '!git' for a
     sub-command list."""
     name = "git"
-    repos = {
-        "core": "/home/earwig/git/earwigbot",
-        "plugins": "/home/earwig/git/earwigbot-plugins",
-    }
+
+    def setup(self):
+        try:
+            self.repos = self.config.commands[self.name]["repos"]
+        except KeyError:
+            self.repos = None
 
     def process(self, data):
         self.data = data
@@ -46,6 +48,9 @@ class Git(Command):
         if not data.args or data.args[0] == "help":
             self.do_help()
             return
+        if not self.repos:
+            self.reply(data, "no repos are specified in the config file.")
+            return
 
         command = data.args[0]
         try:
@@ -57,7 +62,7 @@ class Git(Command):
             return
         if repo_name not in self.repos:
             repos = self.get_repos()
-            msg = "repository must be one of the following: {0}"
+            msg = "repository must be one of the following: {0}."
             self.reply(data, msg.format(repos))
             return
         self.repo = git.Repo(self.repos[repo_name])
@@ -91,7 +96,7 @@ class Git(Command):
         try:
             return getattr(self.repo.remotes, remote_name)
         except AttributeError:
-            msg = "unknown remote: \x0302{0}\x0301".format(remote_name)
+            msg = "unknown remote: \x0302{0}\x0301.".format(remote_name)
             self.reply(self.data, msg)
 
     def get_time_since(self, date):

earwigbot/commands/help.py

@@ -23,7 +23,6 @@
 import re
 
 from earwigbot.commands import Command
-from earwigbot.irc import Data
 
 __all__ = ["Help"]
 
@@ -50,34 +49,24 @@ class Help(Command):
     def do_main_help(self, data):
         """Give the user a general help message with a list of all commands."""
         msg = "Hi, I'm a bot! I have {0} commands loaded: {1}. You can get help for any command with '!help <command>'."
-        cmnds = sorted(self.bot.commands)
+        cmnds = sorted([cmnd.name for cmnd in self.bot.commands])
         msg = msg.format(len(cmnds), ', '.join(cmnds))
         self.reply(data, msg)
 
     def do_command_help(self, data):
         """Give the user help for a specific command."""
-        command = data.args[0]
+        target = data.args[0]
 
-        # Create a dummy message to test which commands pick up the user's
-        # input:
-        msg = ":foo!bar@example.com PRIVMSG #channel :msg".split()
-        dummy = Data(self.bot, msg)
-        dummy.command = command.lower()
-        dummy.is_command = True
+        for command in self.bot.commands:
+            if command.name == target or target in command.commands:
+                if command.__doc__:
+                    doc = command.__doc__.replace("\n", "")
+                    doc = re.sub("\s\s+", " ", doc)
+                    msg = "help for command \x0303{0}\x0301: \"{1}\""
+                    self.reply(data, msg.format(target, doc))
+                    return
 
-        for cmnd_name in self.bot.commands:
-            cmnd = self.bot.commands.get(cmnd_name)
-            if not cmnd.check(dummy):
-                continue
-            if cmnd.__doc__:
-                doc = cmnd.__doc__.replace("\n", "")
-                doc = re.sub("\s\s+", " ", doc)
-                msg = "help for command \x0303{0}\x0301: \"{1}\""
-                self.reply(data, msg.format(command, doc))
-                return
-            break
-
-        msg = "sorry, no help for \x0303{0}\x0301.".format(command)
+        msg = "sorry, no help for \x0303{0}\x0301.".format(target)
         self.reply(data, msg)
 
     def do_hello(self, data):

earwigbot/commands/langcode.py

@@ -0,0 +1,49 @@
+# -*- coding: utf-8  -*-
+#
+# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net>
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+from earwigbot.commands import BaseCommand
+
+class Command(BaseCommand):
+    """Convert a language code into its name and a list of WMF sites in that
+    language."""
+    name = "langcode"
+    commands = ["langcode", "lang", "language"]
+
+    def process(self, data):
+        if not data.args:
+            self.reply(data, "please specify a language code.")
+            return
+
+        code = data.args[0]
+        site = self.bot.wiki.get_site()
+        matrix = site.api_query(action="sitematrix")["sitematrix"]
+        del matrix["specials"]
+
+        for site in matrix.itervalues():
+            if site["code"] == code:
+                name = site["name"]
+                sites = ", ".join([s["url"] for s in site["site"]])
+                msg = "\x0302{0}\x0301 is {1} ({2})".format(code, name, sites)
+                self.reply(data, msg)
+                return
+
+        self.reply(data, "site \x0302{0}\x0301 not found.".format(code))

earwigbot/commands/link.py

@@ -31,14 +31,6 @@ class Link(Command):
     """Convert a Wikipedia page name into a URL."""
     name = "link"
 
-    def check(self, data):
-        # if ((data.is_command and data.command == "link") or
-        # (("[[" in data.msg and "]]" in data.msg) or
-        # ("{{" in data.msg and "}}" in data.msg))):
-        if data.is_command and data.command == "link":
-            return True
-        return False
-
     def process(self, data):
         msg = data.msg
 

earwigbot/commands/notes.py

@@ -0,0 +1,169 @@
+# -*- coding: utf-8  -*-
+#
+# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net>
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+from earwigbot.commands import Command
+
+__all__ = ["Notes"]
+
+class Notes(Command):
+    """A mini IRC-based wiki for storing notes, tips, and reminders."""
+    name = "notes"
+
+    def process(self, data):
+        pass
+
+
+class OldCommand(object):
+    def parse(self):
+        if command == "notes" or command == "note" or command == "about" or command == "data" or command == "database":
+            try:
+                action = line2[4]
+            except BaseException:
+                reply("What do you want me to do? Type \"!notes help\" for more information.", chan, nick)
+                return
+            import MySQLdb
+            db = MySQLdb.connect(db="u_earwig_ircbot", host="sql", read_default_file="/home/earwig/.my.cnf")
+            specify = ' '.join(line2[5:])
+            if action == "help" or action == "manual":
+                shortCommandList = "read, write, change, undo, delete, move, author, category, list, report, developer"
+                if specify == "read":
+                    say("To read an entry, type \"!notes read <entry>\".", chan)
+                elif specify == "write":
+                    say("To write a new entry, type \"!notes write <entry> <content>\". This will create a new entry only if one does not exist, see the below command...", chan)
+                elif specify == "change":
+                    say("To change an entry, type \"!notes change <entry> <new content>\". The old entry will be stored in the database, so it can be undone later.", chan)
+                elif specify == "undo":
+                    say("To undo a change, type \"!notes undo <entry>\".", chan)
+                elif specify == "delete":
+                    say("To delete an entry, type \"!notes delete <entry>\". For security reasons, only bot admins can do this.", chan)
+                elif specify == "move":
+                    say("To move an entry, type \"!notes move <old_title> <new_title>\".", chan)
+                elif specify == "author":
+                    say("To return the author of an entry, type \"!notes author <entry>\".", chan)
+                elif specify == "category" or specify == "cat":
+                    say("To change an entry's category, type \"!notes category <entry> <category>\".", chan)
+                elif specify == "list":
+                    say("To list all categories in the database, type \"!notes list\". Type \"!notes list <category>\" to get all entries in a certain category.", chan)
+                elif specify == "report":
+                    say("To give some statistics about the mini-wiki, including some debugging information, type \"!notes report\" in a PM.", chan)
+                elif specify == "developer":
+                    say("To do developer work, such as writing to the database directly, type \"!notes developer <command>\". This can only be done by the bot owner.", chan)
+                else:
+                    db.query("SELECT * FROM version;")
+                    r = db.use_result()
+                    data = r.fetch_row(0)
+                    version = data[0]
+                    reply("The Earwig Mini-Wiki: running v%s." % version, chan, nick)
+                    reply("The full list of commands, for reference, are: %s." % shortCommandList, chan, nick)
+                    reply("For an explaination of a certain command, type \"!notes help <command>\".", chan, nick)
+                    reply("You can also access the database from the Toolserver: http://toolserver.org/~earwig/cgi-bin/irc_database.py", chan, nick)
+                    time.sleep(0.4)
+                return
+            elif action == "read":
+                specify = string.lower(specify)
+                if " " in specify: specify = string.split(specify, " ")[0]
+                if not specify or "\"" in specify:
+                    reply("Please include the name of the entry you would like to read after the command, e.g. !notes read earwig", chan, nick)
+                    return
+                try:
+                    db.query("SELECT entry_content FROM entries WHERE entry_title = \"%s\";" % specify)
+                    r = db.use_result()
+                    data = r.fetch_row(0)
+                    entry = data[0][0]
+                    say("Entry \"\x02%s\x0F\": %s" % (specify, entry), chan)
+                except Exception:
+                    reply("There is no entry titled \"\x02%s\x0F\"." % specify, chan, nick)
+                return
+            elif action == "delete" or action == "remove":
+                specify = string.lower(specify)
+                if " " in specify: specify = string.split(specify, " ")[0]
+                if not specify or "\"" in specify:
+                    reply("Please include the name of the entry you would like to delete after the command, e.g. !notes delete earwig", chan, nick)
+                    return
+                if authy == "owner" or authy == "admin":
+                    try:
+                        db.query("DELETE from entries where entry_title = \"%s\";" % specify)
+                        r = db.use_result()
+                        db.commit()
+                        reply("The entry on \"\x02%s\x0F\" has been removed." % specify, chan, nick)
+                    except Exception:
+                        phenny.reply("Unable to remove the entry on \"\x02%s\x0F\", because it doesn't exist." % specify, chan, nick)
+                else:
+                    reply("Only bot admins can remove entries.", chan, nick)
+                return
+            elif action == "developer":
+                if authy == "owner":
+                    db.query(specify)
+                    r = db.use_result()
+                    try:
+                        print r.fetch_row(0)
+                    except Exception:
+                        pass
+                    db.commit()
+                    reply("Done.", chan, nick)
+                else:
+                    reply("Only the bot owner can modify the raw database.", chan, nick)
+                return
+            elif action == "write":
+                try:
+                    write = line2[5]
+                    content = ' '.join(line2[6:])
+                except Exception:
+                    reply("Please include some content in your entry.", chan, nick)
+                    return
+                db.query("SELECT * from entries WHERE entry_title = \"%s\";" % write)
+                r = db.use_result()
+                data = r.fetch_row(0)
+                if data:
+                    reply("An entry on %s already exists; please use \"!notes change %s %s\"." % (write, write, content), chan, nick)
+                    return
+                content2 = content.replace('"', '\\' + '"')
+                db.query("INSERT INTO entries (entry_title, entry_author, entry_category, entry_content, entry_content_old) VALUES (\"%s\", \"%s\", \"uncategorized\", \"%s\", NULL);" % (write, nick, content2))
+                db.commit()
+                reply("You have written an entry titled \"\x02%s\x0F\", with the following content: \"%s\"" % (write, content), chan, nick)
+                return
+            elif action == "change":
+                reply("NotImplementedError", chan, nick)
+            elif action == "undo":
+                reply("NotImplementedError", chan, nick)
+            elif action == "move":
+                reply("NotImplementedError", chan, nick)
+            elif action == "author":
+                try:
+                    entry = line2[5]
+                except Exception:
+                    reply("Please include the name of the entry you would like to get information for after the command, e.g. !notes author earwig", chan, nick)
+                    return
+                db.query("SELECT entry_author from entries WHERE entry_title = \"%s\";" % entry)
+                r = db.use_result()
+                data = r.fetch_row(0)
+                if data:
+                    say("The author of \"\x02%s\x0F\" is \x02%s\x0F." % (entry, data[0][0]), chan)
+                    return
+                reply("There is no entry titled \"\x02%s\x0F\"." % entry, chan, nick)
+                return
+            elif action == "cat" or action == "category":
+                reply("NotImplementedError", chan, nick)
+            elif action == "list":
+                reply("NotImplementedError", chan, nick)
+            elif action == "report":
+                reply("NotImplementedError", chan, nick)

earwigbot/commands/praise.py

@@ -28,24 +28,23 @@ class Praise(Command):
     """Praise people!"""
     name = "praise"
 
+    def setup(self):
+        try:
+            self.praises = self.config.commands[self.name]["praises"]
+        except KeyError:
+            self.praises = []
+
     def check(self, data):
-        commands = ["praise", "earwig", "leonard", "leonard^bloom", "groove",
-                  "groovedog"]
-        return data.is_command and data.command in commands
+        check = data.command == "praise" or data.command in self.praises
+        return data.is_command and check
 
     def process(self, data):
-        if data.command == "earwig":
-            msg = "\x02Earwig\x0F is the bestest Python programmer ever!"
-        elif data.command in ["leonard", "leonard^bloom"]:
-            msg = "\x02Leonard^Bloom\x0F is the biggest slacker ever!"
-        elif data.command in ["groove", "groovedog"]:
-            msg = "\x02GrooveDog\x0F is the bestest heh evar!"
-        else:
-            if not data.args:
-                msg = "You use this command to praise certain people. Who they are is a secret."
-            else:
-                msg = "You're doing it wrong."
-            self.reply(data, msg)
+        if data.command in self.praises:
+            msg = self.praises[data.command]
+            self.say(data.chan, msg)
             return
-
-        self.say(data.chan, msg)
+        if not data.args:
+            msg = "You use this command to praise certain people. Who they are is a secret."
+        else:
+            msg = "you're doing it wrong."
+        self.reply(data, msg)

earwigbot/commands/quit.py

@@ -28,10 +28,7 @@ class Quit(Command):
     """Quit, restart, or reload components from the bot. Only the owners can
     run this command."""
     name = "quit"
-
-    def check(self, data):
-        commands = ["quit", "restart", "reload"]
-        return data.is_command and data.command in commands
+    commands = ["quit", "restart", "reload"]
 
     def process(self, data):
         if data.host not in self.config.irc["permissions"]["owners"]:

earwigbot/commands/registration.py

@@ -30,12 +30,7 @@ __all__ = ["Registration"]
 class Registration(Command):
     """Return when a user registered."""
     name = "registration"
-
-    def check(self, data):
-        commands = ["registration", "reg", "age"]
-        if data.is_command and data.command in commands:
-            return True
-        return False
+    commands = ["registration", "reg", "age"]
 
     def process(self, data):
         if not data.args:
@@ -61,7 +56,7 @@ class Registration(Command):
         elif user.gender == "female":
             gender = "She's"
         else:
-            gender = "They're"
+            gender = "They're"  # Singluar they?
 
         msg = "\x0302{0}\x0301 registered on {1}. {2} {3} old."
         self.reply(data, msg.format(name, date, gender, age))

earwigbot/commands/remind.py

@@ -20,7 +20,7 @@
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 # SOFTWARE.
 
-import threading
+from threading import Timer
 import time
 
 from earwigbot.commands import Command
@@ -30,11 +30,7 @@ __all__ = ["Remind"]
 class Remind(Command):
     """Set a message to be repeated to you in a certain amount of time."""
     name = "remind"
-
-    def check(self, data):
-        if data.is_command and data.command in ["remind", "reminder"]:
-            return True
-        return False
+    commands = ["remind", "reminder"]
 
     def process(self, data):
         if not data.args:
@@ -62,12 +58,7 @@ class Remind(Command):
         msg = msg.format(message, wait, end_time_with_timezone)
         self.reply(data, msg)
 
-        t_reminder = threading.Thread(target=self.reminder,
-                                      args=(data, message, wait))
+        t_reminder = Timer(wait, self.reply, args=(data, message))
         t_reminder.name = "reminder " + end_time
         t_reminder.daemon = True
         t_reminder.start()
-
-    def reminder(self, data, message, wait):
-        time.sleep(wait)
-        self.reply(data, message)

earwigbot/commands/replag.py

@@ -32,10 +32,16 @@ class Replag(Command):
     """Return the replag for a specific database on the Toolserver."""
     name = "replag"
 
+    def setup(self):
+        try:
+            self.default = self.config.commands[self.name]["default"]
+        except KeyError:
+            self.default = None
+
     def process(self, data):
         args = {}
         if not data.args:
-            args["db"] = "enwiki_p"
+            args["db"] = self.default or self.bot.wiki.get_site().name + "_p"
         else:
             args["db"] = data.args[0]
         args["host"] = args["db"].replace("_", "-") + ".rrdb.toolserver.org"
@@ -43,10 +49,11 @@ class Replag(Command):
 
         conn = oursql.connect(**args)
         with conn.cursor() as cursor:
-            query = "SELECT UNIX_TIMESTAMP() - UNIX_TIMESTAMP(rc_timestamp) FROM recentchanges ORDER BY rc_timestamp DESC LIMIT 1"
+            query = """SELECT UNIX_TIMESTAMP() - UNIX_TIMESTAMP(rc_timestamp)
+                       FROM recentchanges ORDER BY rc_timestamp DESC LIMIT 1"""
             cursor.execute(query)
             replag = int(cursor.fetchall()[0][0])
         conn.close()
 
-        msg = "Replag on \x0302{0}\x0301 is \x02{1}\x0F seconds."
+        msg = "replag on \x0302{0}\x0301 is \x02{1}\x0F seconds."
         self.reply(data, msg.format(args["db"], replag))

earwigbot/commands/rights.py

@@ -28,12 +28,7 @@ __all__ = ["Rights"]
 class Rights(Command):
     """Retrieve a list of rights for a given username."""
     name = "rights"
-
-    def check(self, data):
-        commands = ["rights", "groups", "permissions", "privileges"]
-        if data.is_command and data.command in commands:
-            return True
-        return False
+    commands = ["rights", "groups", "permissions", "privileges"]
 
     def process(self, data):
         if not data.args:

earwigbot/commands/threads.py

@@ -30,12 +30,7 @@ __all__ = ["Threads"]
 class Threads(Command):
     """Manage wiki tasks from IRC, and check on thread status."""
     name = "threads"
-
-    def check(self, data):
-        commands = ["tasks", "task", "threads", "tasklist"]
-        if data.is_command and data.command in commands:
-            return True
-        return False
+    commands = ["tasks", "task", "threads", "tasklist"]
 
     def process(self, data):
         self.data = data
@@ -106,7 +101,7 @@ class Threads(Command):
         whether they are currently running or idle."""
         threads = threading.enumerate()
         tasklist = []
-        for task in sorted(self.bot.tasks):
+        for task in sorted([task.name for task in self.bot.tasks]):
             threadlist = [t for t in threads if t.name.startswith(task)]
             ids = [str(t.ident) for t in threadlist]
             if not ids:
@@ -134,7 +129,7 @@ class Threads(Command):
             self.reply(data, "what task do you want me to start?")
             return
 
-        if task_name not in self.bot.tasks:
+        if task_name not in [task.name for task in self.bot.tasks]:
             # This task does not exist or hasn't been loaded:
             msg = "task could not be found; either it doesn't exist, or it wasn't loaded correctly."
             self.reply(data, msg.format(task_name))

earwigbot/commands/time.py

@@ -0,0 +1,68 @@
+# -*- coding: utf-8  -*-
+#
+# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net>
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+from datetime import datetime
+from math import floor
+from time import time
+
+try:
+    import pytz
+except ImportError:
+    pytz = None
+
+from earwigbot.commands import BaseCommand
+
+class Command(BaseCommand):
+    """Report the current time in any timezone (UTC default), or in beats."""
+    name = "time"
+    commands = ["time", "beats", "swatch"]
+
+    def process(self, data):
+        if data.command in ["beats", "swatch"]:
+            self.do_beats(data)
+            return
+        if data.args:
+            timezone = data.args[0]
+        else:
+            timezone = "UTC"
+        if timezone in ["beats", "swatch"]:
+            self.do_beats(data)
+        else:
+            self.do_time(data, timezone)
+
+    def do_beats(self, data):
+        beats = ((time() + 3600) % 86400) / 86.4
+        beats = int(floor(beats))
+        self.reply(data, "@{0:0>3}".format(beats))
+
+    def do_time(self, data, timezone):
+        if not pytz:
+            msg = "this command requires the 'pytz' module: http://pytz.sourceforge.net/"
+            self.reply(data, msg)
+            return
+        try:
+            tzinfo = pytz.timezone(timezone)
+        except pytz.exceptions.UnknownTimeZoneError:
+            self.reply(data, "unknown timezone: {0}.".format(timezone))
+            return
+        now = pytz.utc.localize(datetime.utcnow()).astimezone(tzinfo)
+        self.reply(data, now.strftime("%Y-%m-%d %H:%M:%S %Z"))

earwigbot/commands/trout.py

@@ -0,0 +1,48 @@
+# -*- coding: utf-8  -*-
+#
+# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net>
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+from unicodedata import normalize
+
+from earwigbot.commands import Command
+
+__all__ = ["Trout"]
+
+class Trout(Command):
+    """Slap someone with a trout, or related fish."""
+    name = "trout"
+    commands = ["trout", "whale"]
+
+    def setup(self):
+        try:
+            self.exceptions = self.config.commands[self.name]["exceptions"]
+        except KeyError:
+            self.exceptions = {}
+
+    def process(self, data):
+        animal = data.command
+        target = " ".join(data.args) or data.nick
+        normal = normalize("NFKD", target.decode("utf8")).lower()
+        if normal in self.exceptions:
+            self.reply(data, self.exceptions["normal"])
+        else:
+            msg = "slaps {0} around a bit with a large {1}."
+            self.action(data.chan, msg.format(target, animal))

earwigbot/config.py

@@ -48,8 +48,9 @@ class BotConfig(object):
     - :py:attr:`path`:       path to the bot's config file
     - :py:attr:`components`: enabled components
     - :py:attr:`wiki`:       information about wiki-editing
-    - :py:attr:`tasks`:      information for bot tasks
     - :py:attr:`irc`:        information about IRC
+    - :py:attr:`commands`:   information about IRC commands
+    - :py:attr:`tasks`:      information for bot tasks
     - :py:attr:`metadata`:   miscellaneous information
     - :py:meth:`schedule`:   tasks scheduled to run at a given time
 
@@ -69,12 +70,13 @@ class BotConfig(object):
 
         self._components = _ConfigNode()
         self._wiki = _ConfigNode()
-        self._tasks = _ConfigNode()
         self._irc = _ConfigNode()
+        self._commands = _ConfigNode()
+        self._tasks = _ConfigNode()
         self._metadata = _ConfigNode()
 
-        self._nodes = [self._components, self._wiki, self._tasks, self._irc,
-                       self._metadata]
+        self._nodes = [self._components, self._wiki, self._irc, self._commands,
+                       self._tasks, self._metadata]
 
         self._decryptable_nodes = [  # Default nodes to decrypt
             (self._wiki, ("password",)),
@@ -196,16 +198,21 @@ class BotConfig(object):
         return self._wiki
 
     @property
-    def tasks(self):
-        """A dict of information for bot tasks."""
-        return self._tasks
-
-    @property
     def irc(self):
         """A dict of information about IRC."""
         return self._irc
 
     @property
+    def commands(self):
+        """A dict of information for IRC commands."""
+        return self._commands
+
+    @property
+    def tasks(self):
+        """A dict of information for bot tasks."""
+        return self._tasks
+
+    @property
     def metadata(self):
         """A dict of miscellaneous information."""
         return self._metadata
@@ -225,14 +232,14 @@ class BotConfig(object):
         user. If there is no config file at all, offer to make one, otherwise
         exit.
 
-        Data from the config file is stored in five
+        Data from the config file is stored in six
         :py:class:`~earwigbot.config._ConfigNode`\ s (:py:attr:`components`,
-        :py:attr:`wiki`, :py:attr:`tasks`, :py:attr:`irc`, :py:attr:`metadata`)
-        for easy access (as well as the lower-level :py:attr:`data` attribute).
-        If passwords are encrypted, we'll use :py:func:`~getpass.getpass` for
-        the key and then decrypt them. If the config is being reloaded,
-        encrypted items will be automatically decrypted if they were decrypted
-        earlier.
+        :py:attr:`wiki`, :py:attr:`irc`, :py:attr:`commands`, :py:attr:`tasks`,
+        :py:attr:`metadata`) for easy access (as well as the lower-level
+        :py:attr:`data` attribute). If passwords are encrypted, we'll use
+        :py:func:`~getpass.getpass` for the key and then decrypt them. If the
+        config is being reloaded, encrypted items will be automatically
+        decrypted if they were decrypted earlier.
         """
         if not path.exists(self._config_path):
             print "Config file not found:", self._config_path
@@ -246,8 +253,9 @@ class BotConfig(object):
         data = self._data
         self.components._load(data.get("components", {}))
         self.wiki._load(data.get("wiki", {}))
-        self.tasks._load(data.get("tasks", {}))
         self.irc._load(data.get("irc", {}))
+        self.commands._load(data.get("commands", {}))
+        self.tasks._load(data.get("tasks", {}))
         self.metadata._load(data.get("metadata", {}))
 
         self._setup_logging()
@@ -273,7 +281,10 @@ class BotConfig(object):
             >>> config.decrypt(config.irc, "frontend", "nickservPassword")
             # decrypts config.irc["frontend"]["nickservPassword"]
         """
-        self._decryptable_nodes.append((node, nodes))
+        signature = (node, nodes)
+        if signature in self._decryptable_nodes:
+            return  # Already decrypted
+        self._decryptable_nodes.append(signature)
         if self.is_encrypted():
             self._decrypt(node, nodes)
 

earwigbot/managers.py

@@ -24,7 +24,7 @@
 import imp
 from os import listdir, path
 from re import sub
-from threading import Lock, Thread
+from threading import RLock, Thread
 from time import gmtime, strftime
 
 from earwigbot.commands import Command
@@ -46,11 +46,7 @@ class _ResourceManager(object):
 
     This class handles the low-level tasks of (re)loading resources via
     :py:meth:`load`, retrieving specific resources via :py:meth:`get`, and
-    iterating over all resources via :py:meth:`__iter__`. If iterating over
-    resources, it is recommended to acquire :py:attr:`self.lock <lock>`
-    beforehand and release it afterwards (alternatively, wrap your code in a
-    ``with`` statement) so an attempt at reloading resources in another thread
-    won't disrupt your iteration.
+    iterating over all resources via :py:meth:`__iter__`.
     """
     def __init__(self, bot, name, base):
         self.bot = bot
@@ -62,8 +58,9 @@ class _ResourceManager(object):
         self._resource_access_lock = Lock()
 
     def __iter__(self):
-        for name in self._resources:
-            yield name
+        with self.lock:
+            for resource in self._resources.itervalues():
+                yield resource
 
     def _load_resource(self, name, path, klass):
         """Instantiate a resource class and add it to the dictionary."""
@@ -140,7 +137,8 @@ class _ResourceManager(object):
         Will raise :py:exc:`KeyError` if the resource (a command or task) is
         not found.
         """
-        return self._resources[key]
+        with self.lock:
+            return self._resources[key]
 
 
 class CommandManager(_ResourceManager):
@@ -168,13 +166,10 @@ class CommandManager(_ResourceManager):
 
     def call(self, hook, data):
         """Respond to a hook type and a :py:class:`Data` object."""
-        self.lock.acquire()
-        for command in self._resources.itervalues():
+        for command in self:
             if hook in command.hooks and self._wrap_check(command, data):
-                self.lock.release()
                 self._wrap_process(command, data)
                 return
-        self.lock.release()
 
 
 class TaskManager(_ResourceManager):

earwigbot/tasks/blptag.py → earwigbot/tasks/blp_tag.py

@@ -27,7 +27,7 @@ __all__ = ["BLPTag"]
 class BLPTag(Task):
     """A task to add |blp=yes to ``{{WPB}}`` or ``{{WPBS}}`` when it is used
     along with ``{{WP Biography}}``."""
-    name = "blptag"
+    name = "blp_tag"
 
     def setup(self):
         pass

earwigbot/tasks/feed_dailycats.py → earwigbot/tasks/image_display_resize.py

@@ -22,11 +22,11 @@
 
 from earwigbot.tasks import Task
 
-__all__ = ["FeedDailyCats"]
+__all__ = ["ImageDisplayResize"]
 
-class FeedDailyCats(Task):
-    """A task to create daily categories for [[WP:FEED]]."""
-    name = "feed_dailycats"
+class ImageDisplayResize(Task):
+    """A task to resize upscaled portraits in infoboxes."""
+    name = "image_display_resize"
 
     def setup(self):
         pass

earwigbot/tasks/wrongmime.py → earwigbot/tasks/wrong_mime.py

@@ -27,7 +27,7 @@ __all__ = ["WrongMIME"]
 class WrongMIME(Task):
     """A task to tag files whose extensions do not agree with their MIME
     type."""
-    name = "wrongmime"
+    name = "wrong_mime"
 
     def setup(self):
         pass

earwigbot/wiki/category.py

@@ -39,7 +39,7 @@ class Category(Page):
 
     *Public methods:*
 
-    - :py:meth:`get_members`: returns a list of page titles in the category
+    - :py:meth:`get_members`: iterates over Pages in the category
     """
 
     def __repr__(self):
@@ -51,8 +51,8 @@ class Category(Page):
         """Return a nice string representation of the Category."""
         return '<Category "{0}" of {1}>'.format(self.title, str(self._site))
 
-    def _get_members_via_sql(self, limit):
-        """Return a list of tuples of (title, pageid) in the category."""
+    def _get_members_via_sql(self, limit, follow):
+        """Iterate over Pages in the category using SQL."""
         query = """SELECT page_title, page_namespace, page_id FROM page
                    JOIN categorylinks ON page_id = cl_from
                    WHERE cl_to = ?"""
@@ -64,42 +64,66 @@ class Category(Page):
         else:
             result = self._site.sql_query(query, (title,))
 
-        members = []
-        for row in result:
+        members = list(result)
+        for row in members:
             base = row[0].replace("_", " ").decode("utf8")
             namespace = self._site.namespace_id_to_name(row[1])
             if namespace:
                 title = u":".join((namespace, base))
             else:  # Avoid doing a silly (albeit valid) ":Pagename" thing
                 title = base
-            members.append((title, row[2]))
-        return members
+            yield self._site.get_page(title, follow_redirects=follow,
+                                      pageid=row[2])
 
-    def _get_members_via_api(self, limit):
-        """Return a list of page titles in the category using the API."""
+    def _get_members_via_api(self, limit, follow):
+        """Iterate over Pages in the category using the API."""
         params = {"action": "query", "list": "categorymembers",
-                  "cmlimit": limit, "cmtitle": self._title}
-        if not limit:
-            params["cmlimit"] = 50  # Default value
-
-        result = self._site.api_query(**params)
-        members = result['query']['categorymembers']
-        return [member["title"] for member in members]
-
-    def get_members(self, use_sql=False, limit=None):
-        """Return a list of page titles in the category.
+                  "cmtitle": self._title}
+
+        while 1:
+            params["cmlimit"] = limit if limit else "max"
+            result = self._site.api_query(**params)
+            for member in result["query"]["categorymembers"]:
+                title = member["title"]
+                yield self._site.get_page(title, follow_redirects=follow)
+
+            if "query-continue" in result:
+                qcontinue = result["query-continue"]["categorymembers"]
+                params["cmcontinue"] = qcontinue["cmcontinue"]
+                if limit:
+                    limit -= len(result["query"]["categorymembers"])
+            else:
+                break
+
+    def get_members(self, use_sql=False, limit=None, follow_redirects=None):
+        """Iterate over Pages in the category.
 
         If *use_sql* is ``True``, we will use a SQL query instead of the API.
-        Pages will be returned as tuples of ``(title, pageid)`` instead of just
-        titles.
-
-        If *limit* is provided, we will provide this many titles, or less if
-        the category is smaller. It defaults to 50 for API queries; normal
-        users can go up to 500, and bots can go up to 5,000 on a single API
-        query. If we're using SQL, the limit is ``None`` by default (returning
-        all pages in the category), but an arbitrary limit can still be chosen.
+        Note that pages are retrieved from the API in chunks (by default, in
+        500-page chunks for normal users and 5000-page chunks for bots and
+        admins), so queries may be made as we go along. If *limit* is given, we
+        will provide this many pages, or less if the category is smaller. By
+        default, *limit* is ``None``, meaning we will keep iterating over
+        members until the category is exhausted. *follow_redirects* is passed
+        directly to :py:meth:`site.get_page()
+        <earwigbot.wiki.site.Site.get_page>`; it defaults to ``None``, which
+        will use the value passed to our :py:meth:`__init__`.
+
+        .. note::
+           Be careful when iterating over very large categories with no limit.
+           If using the API, at best, you will make one query per 5000 pages,
+           which can add up significantly for categories with hundreds of
+           thousands of members. As for SQL, note that *all page titles are
+           stored internally* as soon as the query is made, so the site-wide
+           SQL lock can be freed and unrelated queries can be made without
+           requiring a separate connection to be opened. This is generally not
+           an issue unless your category's size approaches several hundred
+           thousand, in which case the sheer number of titles in memory becomes
+           problematic.
         """
+        if follow_redirects is None:
+            follow_redirects = self._follow_redirects
         if use_sql:
-            return self._get_members_via_sql(limit)
+            return self._get_members_via_sql(limit, follow_redirects)
         else:
-            return self._get_members_via_api(limit)
+            return self._get_members_via_api(limit, follow_redirects)

earwigbot/wiki/page.py

@@ -48,7 +48,7 @@ class Page(CopyrightMixIn):
 
     - :py:attr:`site`:        the page's corresponding Site object
     - :py:attr:`title`:       the page's title, or pagename
-    - :py:attr:`exists`:      whether the page exists
+    - :py:attr:`exists`:      whether or not the page exists
     - :py:attr:`pageid`:      an integer ID representing the page
     - :py:attr:`url`:         the page's URL
     - :py:attr:`namespace`:   the page's namespace as an integer
@@ -75,17 +75,20 @@ class Page(CopyrightMixIn):
       checks the page like :py:meth:`copyvio_check`, but against a specific URL
     """
 
-    re_redirect = "^\s*\#\s*redirect\s*\[\[(.*?)\]\]"
+    PAGE_UNKNOWN = 0
+    PAGE_INVALID = 1
+    PAGE_MISSING = 2
+    PAGE_EXISTS = 3
 
-    def __init__(self, site, title, follow_redirects=False):
+    def __init__(self, site, title, follow_redirects=False, pageid=None):
         """Constructor for new Page instances.
 
-        Takes three arguments: a Site object, the Page's title (or pagename),
-        and whether or not to follow redirects (optional, defaults to False).
+        Takes four arguments: a Site object, the Page's title (or pagename),
+        whether or not to follow redirects (optional, defaults to False), and
+        a page ID to supplement the title (optional, defaults to None - i.e.,
+        we will have to query the API to get it).
 
-        As with User, site.get_page() is preferred. Site's method has support
-        for a default *follow_redirects* value in our config, while __init__()
-        always defaults to False.
+        As with User, site.get_page() is preferred.
 
         __init__() will not do any API queries, but it will use basic namespace
         logic to determine our namespace ID and if we are a talkpage.
@@ -94,9 +97,9 @@ class Page(CopyrightMixIn):
         self._site = site
         self._title = title.strip()
         self._follow_redirects = self._keep_following = follow_redirects
+        self._pageid = pageid
 
-        self._exists = 0
-        self._pageid = None
+        self._exists = self.PAGE_UNKNOWN
         self._is_redirect = None
         self._lastrevid = None
         self._protection = None
@@ -145,7 +148,7 @@ class Page(CopyrightMixIn):
         Note that validity != existence. If a page's title is invalid (e.g, it
         contains "[") it will always be invalid, and cannot be edited.
         """
-        if self._exists == 1:
+        if self._exists == self.PAGE_INVALID:
             e = "Page '{0}' is invalid.".format(self._title)
             raise exceptions.InvalidPageError(e)
 
@@ -157,7 +160,7 @@ class Page(CopyrightMixIn):
         It will also call _assert_validity() beforehand.
         """
         self._assert_validity()
-        if self._exists == 2:
+        if self._exists == self.PAGE_MISSING:
             e = "Page '{0}' does not exist.".format(self._title)
             raise exceptions.PageNotFoundError(e)
 
@@ -218,14 +221,14 @@ class Page(CopyrightMixIn):
             if "missing" in res:
                 # If it has a negative ID and it's missing; we can still get
                 # data like the namespace, protection, and URL:
-                self._exists = 2
+                self._exists = self.PAGE_MISSING
             else:
                 # If it has a negative ID and it's invalid, then break here,
                 # because there's no other data for us to get:
-                self._exists = 1
+                self._exists = self.PAGE_INVALID
                 return
         else:
-            self._exists = 3
+            self._exists = self.PAGE_EXISTS
 
         self._fullurl = res["fullurl"]
         self._protection = res["protection"]
@@ -317,7 +320,7 @@ class Page(CopyrightMixIn):
         if result["edit"]["result"] == "Success":
             self._content = None
             self._basetimestamp = None
-            self._exists = 0
+            self._exists = self.PAGE_UNKNOWN
             return
 
         # If we're here, then the edit failed. If it's because of AssertEdit,
@@ -351,7 +354,7 @@ class Page(CopyrightMixIn):
             params["starttimestamp"] = self._starttimestamp
             if self._basetimestamp:
                 params["basetimestamp"] = self._basetimestamp
-            if self._exists == 2:
+            if self._exists == self.PAGE_MISSING:
                 # Page does not exist; don't edit if it already exists:
                 params["createonly"] = "true"
         else:
@@ -389,7 +392,7 @@ class Page(CopyrightMixIn):
             # These attributes are now invalidated:
             self._content = None
             self._basetimestamp = None
-            self._exists = 0
+            self._exists = self.PAGE_UNKNOWN
             raise exceptions.EditConflictError(error.info)
 
         elif error.code in ["emptypage", "emptynewsection"]:
@@ -437,12 +440,12 @@ class Page(CopyrightMixIn):
 
     @property
     def site(self):
-        """The Page's corresponding Site object."""
+        """The page's corresponding Site object."""
         return self._site
 
     @property
     def title(self):
-        """The Page's title, or "pagename".
+        """The page's title, or "pagename".
 
         This won't do any API queries on its own. Any other attributes or
         methods that do API queries will reload the title, however, like
@@ -453,37 +456,36 @@ class Page(CopyrightMixIn):
 
     @property
     def exists(self):
-        """Information about whether the Page exists or not.
+        """Whether or not the page exists.
 
-        The "information" is a tuple with two items. The first is a bool,
-        either ``True`` if the page exists or ``False`` if it does not. The
-        second is a string giving more information, either ``"invalid"``,
-        (title is invalid, e.g. it contains ``"["``), ``"missing"``, or
-        ``"exists"``.
+        This will be a number; its value does not matter, but it will equal
+        one of :py:attr:`self.PAGE_INVALID <PAGE_INVALID>`,
+        :py:attr:`self.PAGE_MISSING <PAGE_MISSING>`, or
+        :py:attr:`self.PAGE_EXISTS <PAGE_EXISTS>`.
 
         Makes an API query only if we haven't already made one.
         """
-        cases = {
-            0: (None, "unknown"),
-            1: (False, "invalid"),
-            2: (False, "missing"),
-            3: (True, "exists"),
-        }
-        if self._exists == 0:
+        if self._exists == self.PAGE_UNKNOWN:
             self._load()
-        return cases[self._exists]
+        return self._exists
 
     @property
     def pageid(self):
-        """An integer ID representing the Page.
+        """An integer ID representing the page.
 
-        Makes an API query only if we haven't already made one.
+        Makes an API query only if we haven't already made one and the *pageid*
+        parameter to :py:meth:`__init__` was left as ``None``, which should be
+        true for all cases except when pages are returned by an SQL generator
+        (like :py:meth:`category.get_members(use_sql=True)
+        <earwigbot.wiki.category.Category.get_members>`).
 
         Raises :py:exc:`~earwigbot.exceptions.InvalidPageError` or
         :py:exc:`~earwigbot.exceptions.PageNotFoundError` if the page name is
         invalid or the page does not exist, respectively.
         """
-        if self._exists == 0:
+        if self._pageid:
+            return self._pageid
+        if self._exists == self.PAGE_UNKNOWN:
             self._load()
         self._assert_existence()  # Missing pages do not have IDs
         return self._pageid
@@ -501,7 +503,7 @@ class Page(CopyrightMixIn):
         else:
             slug = quote(self._title.replace(" ", "_"), safe="/:")
             path = self._site._article_path.replace("$1", slug)
-            return ''.join((self._site._base_url, path))
+            return ''.join((self._site.url, path))
 
     @property
     def namespace(self):
@@ -523,7 +525,7 @@ class Page(CopyrightMixIn):
         name is invalid. Won't raise an error if the page is missing because
         those can still be create-protected.
         """
-        if self._exists == 0:
+        if self._exists == self.PAGE_UNKNOWN:
             self._load()
         self._assert_validity()  # Invalid pages cannot be protected
         return self._protection
@@ -546,7 +548,7 @@ class Page(CopyrightMixIn):
 
         We will return ``False`` even if the page does not exist or is invalid.
         """
-        if self._exists == 0:
+        if self._exists == self.PAGE_UNKNOWN:
             self._load()
         return self._is_redirect
 
@@ -611,7 +613,7 @@ class Page(CopyrightMixIn):
         Raises InvalidPageError or PageNotFoundError if the page name is
         invalid or the page does not exist, respectively.
         """
-        if self._exists == 0:
+        if self._exists == self.PAGE_UNKNOWN:
             # Kill two birds with one stone by doing an API query for both our
             # attributes and our page content:
             query = self._site.api_query
@@ -626,7 +628,7 @@ class Page(CopyrightMixIn):
             if self._keep_following and self._is_redirect:
                 self._title = self.get_redirect_target()
                 self._keep_following = False  # Don't follow double redirects
-                self._exists = 0  # Force another API query
+                self._exists = self.PAGE_UNKNOWN  # Force another API query
                 self.get()
 
             return self._content
@@ -650,9 +652,10 @@ class Page(CopyrightMixIn):
         :py:exc:`~earwigbot.exceptions.RedirectError` if the page is not a
         redirect.
         """
+        re_redirect = "^\s*\#\s*redirect\s*\[\[(.*?)\]\]"
         content = self.get()
         try:
-            return re.findall(self.re_redirect, content, flags=re.I)[0]
+            return re.findall(re_redirect, content, flags=re.I)[0]
         except IndexError:
             e = "The page does not appear to have a redirect target."
             raise exceptions.RedirectError(e)
@@ -671,7 +674,7 @@ class Page(CopyrightMixIn):
         :py:exc:`~earwigbot.exceptions.PageNotFoundError` if the page name is
         invalid or the page does not exist, respectively.
         """
-        if self._exists == 0:
+        if self._exists == self.PAGE_UNKNOWN:
             self._load()
         self._assert_existence()
         if not self._creator:

earwigbot/wiki/site.py

@@ -69,6 +69,7 @@ class Site(object):
     - :py:attr:`project`: the site's project name, like ``"wikipedia"``
     - :py:attr:`lang`:    the site's language code, like ``"en"``
     - :py:attr:`domain`:  the site's web domain, like ``"en.wikipedia.org"``
+    - :py:attr:`url`:     the site's URL, like ``"https://en.wikipedia.org"``
 
     *Public methods:*
 
@@ -184,6 +185,12 @@ class Site(object):
         res = "<Site {0} ({1}:{2}) at {3}>"
         return res.format(self.name, self.project, self.lang, self.domain)
 
+    def _unicodeify(self, value, encoding="utf8"):
+        """Return input as unicode if it's not unicode to begin with."""
+        if isinstance(value, unicode):
+            return value
+        return unicode(value, encoding)
+
     def _urlencode_utf8(self, params):
         """Implement urllib.urlencode() with support for unicode input."""
         enc = lambda s: s.encode("utf8") if isinstance(s, unicode) else str(s)
@@ -237,14 +244,7 @@ class Site(object):
             e = "Tried to do an API query, but no API URL is known."
             raise exceptions.SiteAPIError(e)
 
-        base_url = self._base_url
-        if base_url.startswith("//"): # Protocol-relative URLs from 1.18
-            if self._use_https:
-                base_url = "https:" + base_url
-            else:
-                base_url = "http:" + base_url
-        url = ''.join((base_url, self._script_path, "/api.php"))
-
+        url = ''.join((self.url, self._script_path, "/api.php"))
         params["format"] = "json"  # This is the only format we understand
         if self._assert_edit:  # If requested, ensure that we're logged in
             params["assert"] = self._assert_edit
@@ -542,6 +542,17 @@ class Site(object):
         """The Site's web domain, like ``"en.wikipedia.org"``."""
         return urlparse(self._base_url).netloc
 
+    @property
+    def url(self):
+        """The Site's full base URL, like ``"https://en.wikipedia.org"``."""
+        url = self._base_url
+        if url.startswith("//"):  # Protocol-relative URLs from 1.18
+            if self._use_https:
+                url = "https:" + url
+            else:
+                url = "http:" + url
+        return url
+
     def api_query(self, **kwargs):
         """Do an API query with `kwargs` as the parameters.
 
@@ -682,7 +693,7 @@ class Site(object):
         e = "There is no namespace with name '{0}'.".format(name)
         raise exceptions.NamespaceNotFoundError(e)
 
-    def get_page(self, title, follow_redirects=False):
+    def get_page(self, title, follow_redirects=False, pageid=None):
         """Return a :py:class:`Page` object for the given title.
 
         *follow_redirects* is passed directly to
@@ -696,23 +707,26 @@ class Site(object):
         redirect-following: :py:class:`~earwigbot.wiki.page.Page`'s methods
         provide that.
         """
+        title = self._unicodeify(title)
         prefixes = self.namespace_id_to_name(constants.NS_CATEGORY, all=True)
         prefix = title.split(":", 1)[0]
         if prefix != title:  # Avoid a page that is simply "Category"
             if prefix in prefixes:
-                return Category(self, title, follow_redirects)
-        return Page(self, title, follow_redirects)
+                return Category(self, title, follow_redirects, pageid)
+        return Page(self, title, follow_redirects, pageid)
 
-    def get_category(self, catname, follow_redirects=False):
+    def get_category(self, catname, follow_redirects=False, pageid=None):
         """Return a :py:class:`Category` object for the given category name.
 
         *catname* should be given *without* a namespace prefix. This method is
         really just shorthand for :py:meth:`get_page("Category:" + catname)
         <get_page>`.
         """
+        catname = self._unicodeify(catname)
+        name = name if isinstance(name, unicode) else name.decode("utf8")
         prefix = self.namespace_id_to_name(constants.NS_CATEGORY)
-        pagename = ':'.join((prefix, catname))
-        return Category(self, pagename, follow_redirects)
+        pagename = u':'.join((prefix, catname))
+        return Category(self, pagename, follow_redirects, pageid)
 
     def get_user(self, username=None):
         """Return a :py:class:`User` object for the given username.
@@ -721,6 +735,7 @@ class Site(object):
         :py:class:`~earwigbot.wiki.user.User` object representing the currently
         logged-in (or anonymous!) user is returned.
         """
+        username = self._unicodeify(username)
         if not username:
             username = self._get_username()
         return User(self, username)

earwigbot/wiki/user.py

@@ -39,6 +39,7 @@ class User(object):
 
     *Attributes:*
 
+    - :py:attr:`site`:         the user's corresponding Site object
     - :py:attr:`name`:         the user's username
     - :py:attr:`exists`:       ``True`` if the user exists, else ``False``
     - :py:attr:`userid`:       an integer ID representing the user
@@ -155,6 +156,11 @@ class User(object):
         self._gender = res["gender"]
 
     @property
+    def site(self):
+        """The user's corresponding Site object."""
+        return self._site
+
+    @property
     def name(self):
         """The user's username.
 

setup.py

@@ -38,6 +38,7 @@ setup(
                         "oursql >= 0.9.3",  # Talking with MediaWiki databases
                         "oauth2 >= 1.5.211",  # Talking with Yahoo BOSS Search
                         "pycrypto >= 2.5",  # Storing bot passwords and keys
+                        "pytz >= 2012c",  # Timezone handling
                         ],
     test_suite = "tests",
     version = __version__,