Browse Source

Merge branch 'feature/commands' into develop

Conflicts:
	README.rst
	docs/customizing.rst
	earwigbot/commands/help.py
	earwigbot/commands/threads.py
	earwigbot/managers.py
	earwigbot/tasks/image_display_resize.py
	setup.py
tags/v0.1^2
Ben Kurtovic 12 years ago
parent
commit
160890b464
37 changed files with 829 additions and 667 deletions
  1. +5
    -4
      README.rst
  2. +42
    -10
      docs/customizing.rst
  3. +11
    -6
      docs/toolset.rst
  4. +25
    -7
      earwigbot/commands/__init__.py
  5. +0
    -401
      earwigbot/commands/_old.py
  6. +34
    -0
      earwigbot/commands/afc_pending.py
  7. +1
    -1
      earwigbot/commands/afc_report.py
  8. +2
    -3
      earwigbot/commands/afc_status.py
  9. +60
    -0
      earwigbot/commands/afc_submissions.py
  10. +1
    -4
      earwigbot/commands/chanops.py
  11. +1
    -6
      earwigbot/commands/crypt.py
  12. +4
    -8
      earwigbot/commands/editcount.py
  13. +72
    -0
      earwigbot/commands/geolocate.py
  14. +11
    -6
      earwigbot/commands/git.py
  15. +11
    -22
      earwigbot/commands/help.py
  16. +49
    -0
      earwigbot/commands/langcode.py
  17. +0
    -8
      earwigbot/commands/link.py
  18. +169
    -0
      earwigbot/commands/notes.py
  19. +16
    -17
      earwigbot/commands/praise.py
  20. +1
    -4
      earwigbot/commands/quit.py
  21. +2
    -7
      earwigbot/commands/registration.py
  22. +3
    -12
      earwigbot/commands/remind.py
  23. +10
    -3
      earwigbot/commands/replag.py
  24. +1
    -6
      earwigbot/commands/rights.py
  25. +3
    -8
      earwigbot/commands/threads.py
  26. +68
    -0
      earwigbot/commands/time.py
  27. +48
    -0
      earwigbot/commands/trout.py
  28. +29
    -18
      earwigbot/config.py
  29. +8
    -13
      earwigbot/managers.py
  30. +1
    -1
      earwigbot/tasks/blp_tag.py
  31. +4
    -4
      earwigbot/tasks/image_display_resize.py
  32. +1
    -1
      earwigbot/tasks/wrong_mime.py
  33. +53
    -29
      earwigbot/wiki/category.py
  34. +47
    -44
      earwigbot/wiki/page.py
  35. +29
    -14
      earwigbot/wiki/site.py
  36. +6
    -0
      earwigbot/wiki/user.py
  37. +1
    -0
      setup.py

+ 5
- 4
README.rst View File

@@ -113,9 +113,9 @@ because it is the main way to communicate with other parts of the bot. A

`earwigbot.config.BotConfig`_ stores configuration information for the bot. Its
docstring explains what each attribute is used for, but essentially each "node"
(one of ``config.components``, ``wiki``, ``tasks``, ``irc``, and ``metadata``)
maps to a section of the bot's ``config.yml`` file. For example, if
``config.yml`` includes something like::
(one of ``config.components``, ``wiki``, ``irc``, ``commands``, ``tasks``, and
``metadata``) maps to a section of the bot's ``config.yml`` file. For example,
if ``config.yml`` includes something like::

irc:
frontend:
@@ -133,7 +133,8 @@ Custom IRC commands
~~~~~~~~~~~~~~~~~~~

Custom commands are subclasses of `earwigbot.commands.Command`_ that override
``Command``'s ``process()`` (and optionally ``check()``) methods.
``Command``'s ``process()`` (and optionally ``check()`` or ``setup()``)
methods.

The bot has a wide selection of built-in commands and plugins to act as sample
code and/or to give ideas. Start with test_, and then check out chanops_ and


+ 42
- 10
docs/customizing.rst View File

@@ -58,8 +58,13 @@ The most useful attributes are:

:py:class:`earwigbot.config.BotConfig` stores configuration information for the
bot. Its docstrings explains what each attribute is used for, but essentially
each "node" (one of :py:attr:`config.components`, :py:attr:`wiki`,
:py:attr:`tasks`, :py:attr:`tasks`, or :py:attr:`metadata`) maps to a section
each "node" (one of :py:attr:`config.components
<earwigbot.config.BotConfig.components>`,
:py:attr:`~earwigbot.config.BotConfig.wiki`,
:py:attr:`~earwigbot.config.BotConfig.irc`,
:py:attr:`~earwigbot.config.BotConfig.commands`,
:py:attr:`~earwigbot.config.BotConfig.tasks`, or
:py:attr:`~earwigbot.config.BotConfig.metadata`) maps to a section
of the bot's :file:`config.yml` file. For example, if :file:`config.yml`
includes something like::

@@ -81,7 +86,8 @@ Custom IRC commands
Custom commands are subclasses of :py:class:`earwigbot.commands.Command` that
override :py:class:`~earwigbot.commands.Command`'s
:py:meth:`~earwigbot.commands.Command.process` (and optionally
:py:meth:`~earwigbot.commands.Command.check`) methods.
:py:meth:`~earwigbot.commands.Command.check` or
:py:meth:`~earwigbot.commands.Command.setup`) methods.

:py:class:`~earwigbot.commands.Command`'s docstrings should explain what each
attribute and method is for and what they should be overridden with, but these
@@ -90,6 +96,15 @@ are the basics:
- Class attribute :py:attr:`~earwigbot.commands.Command.name` is the name of
the command. This must be specified.

- Class attribute :py:attr:`~earwigbot.commands.Command.commands` is a list of
names that will trigger this command. It defaults to the command's
:py:attr:`~earwigbot.commands.Command.name`, but you can override it with
multiple names to serve as aliases. This is handled by the default
:py:meth:`~earwigbot.commands.Command.check` implementation (see below), so
if :py:meth:`~earwigbot.commands.Command.check` is overridden, this is
ignored by everything except the help_ command (so ``!help alias`` will
trigger help for the actual command).

- Class attribute :py:attr:`~earwigbot.commands.Command.hooks` is a list of the
"IRC events" that this command might respond to. It defaults to ``["msg"]``,
but options include ``"msg_private"`` (for private messages only),
@@ -97,15 +112,25 @@ are the basics:
joins a channel). See the afc_status_ plugin for a command that responds to
other hook types.

- Method :py:meth:`~earwigbot.commands.Command.setup` is called *once* with no
arguments immediately after the command is first loaded. Does nothing by
default; treat it like an :py:meth:`__init__` if you want
(:py:meth:`~earwigbot.tasks.Command.__init__` does things by default and a
dedicated setup method is often easier than overriding
:py:meth:`~earwigbot.tasks.Command.__init__` and using :py:obj:`super`).

- Method :py:meth:`~earwigbot.commands.Command.check` is passed a
:py:class:`~earwigbot.irc.data.Data` object, and should return ``True`` if
you want to respond to this message, or ``False`` otherwise. The default
behavior is to return ``True`` only if :py:attr:`data.is_command` is ``True``
and :py:attr:`data.command` == :py:attr:`~earwigbot.commands.Command.name`,
which is suitable for most cases. A common, straightforward reason for
overriding is if a command has aliases (see chanops_ for an example). Note
that by returning ``True``, you prevent any other commands from responding to
this message.
and :py:attr:`data.command` ``==``
:py:attr:`~earwigbot.commands.Command.name` (or :py:attr:`data.command
<earwigbot.irc.data.Data.command>` is in
:py:attr:`~earwigbot.commands.Command.commands` if that list is overriden;
see above), which is suitable for most cases. A possible reason for
overriding is if you want to do something in response to events from a
specific channel only. Note that by returning ``True``, you prevent any other
commands from responding to this message.

- Method :py:meth:`~earwigbot.commands.Command.process` is passed the same
:py:class:`~earwigbot.irc.data.Data` object as
@@ -128,6 +153,12 @@ are the basics:
<earwigbot.irc.connection.IRCConnection.join>`, and
:py:meth:`part(chan) <earwigbot.irc.connection.IRCConnection.part>`.

Commands have access to :py:attr:`config.commands[command_name]` for config
information, which is a node in :file:`config.yml` like every other attribute
of :py:attr:`bot.config`. This can be used to store, for example, API keys or
SQL connection info, so that these can be easily changed without modifying the
command itself.

The command *class* doesn't need a specific name, but it should logically
follow the command's name. The filename doesn't matter, but it is recommended
to match the command name for readability. Multiple command classes are allowed
@@ -190,7 +221,7 @@ are the basics:

Tasks have access to :py:attr:`config.tasks[task_name]` for config information,
which is a node in :file:`config.yml` like every other attribute of
:py:attr:`bot.config`. This can be used to store, for example, edit summaries,
:py:attr:`bot.config`. This can be used to store, for example, edit summaries
or templates to append to user talk pages, so that these can be easily changed
without modifying the task itself.

@@ -201,8 +232,9 @@ the task name for readability. Multiple tasks classes are allowed in one file.
See the built-in wikiproject_tagger_ task for a relatively straightforward
task, or the afc_statistics_ plugin for a more complicated one.

.. _help: https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/help.py
.. _afc_status: https://github.com/earwig/earwigbot-plugins/blob/develop/commands/afc_status.py
.. _chanops: https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/chanops.py
.. _test: https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/test.py
.. _chanops: https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/chanops.py
.. _wikiproject_tagger: https://github.com/earwig/earwigbot/blob/develop/earwigbot/tasks/wikiproject_tagger.py
.. _afc_statistics: https://github.com/earwig/earwigbot-plugins/blob/develop/tasks/afc_statistics.py

+ 11
- 6
docs/toolset.rst View File

@@ -80,6 +80,8 @@ following attributes:
``"en"``
- :py:attr:`~earwigbot.wiki.site.Site.domain`: the site's web domain, like
``"en.wikipedia.org"``
- :py:attr:`~earwigbot.wiki.site.Site.url`: the site's full base URL, like
``"https://en.wikipedia.org"``

and the following methods:

@@ -97,11 +99,11 @@ and the following methods:
- :py:meth:`namespace_name_to_id(name)
<earwigbot.wiki.site.Site.namespace_name_to_id>`: given a namespace name,
returns the associated namespace ID
- :py:meth:`get_page(title, follow_redirects=False)
- :py:meth:`get_page(title, follow_redirects=False, ...)
<earwigbot.wiki.site.Site.get_page>`: returns a ``Page`` object for the given
title (or a :py:class:`~earwigbot.wiki.category.Category` object if the
page's namespace is "``Category:``")
- :py:meth:`get_category(catname, follow_redirects=False)
- :py:meth:`get_category(catname, follow_redirects=False, ...)
<earwigbot.wiki.site.Site.get_category>`: returns a ``Category`` object for
the given title (sans namespace)
- :py:meth:`get_user(username) <earwigbot.wiki.site.Site.get_user>`: returns a
@@ -120,7 +122,7 @@ provide the following attributes:
- :py:attr:`~earwigbot.wiki.page.Page.site`: the page's corresponding
:py:class:`~earwigbot.wiki.site.Site` object
- :py:attr:`~earwigbot.wiki.page.Page.title`: the page's title, or pagename
- :py:attr:`~earwigbot.wiki.page.Page.exists`: whether the page exists
- :py:attr:`~earwigbot.wiki.page.Page.exists`: whether or not the page exists
- :py:attr:`~earwigbot.wiki.page.Page.pageid`: an integer ID representing the
page
- :py:attr:`~earwigbot.wiki.page.Page.url`: the page's URL
@@ -166,9 +168,10 @@ or :py:meth:`site.get_page(title) <earwigbot.wiki.site.Site.get_page>` where
``title`` is in the ``Category:`` namespace) provide the following additional
method:

- :py:meth:`get_members(use_sql=False, limit=None)
<earwigbot.wiki.category.Category.get_members>`: returns a list of page
titles in the category (limit is ``50`` by default if using the API)
- :py:meth:`get_members(use_sql=False, limit=None, ...)
<earwigbot.wiki.category.Category.get_members>`: iterates over
:py:class:`~earwigbot.wiki.page.Page`\ s in the category, until either the
category is exhausted or (if given) ``limit`` is reached

Users
~~~~~
@@ -178,6 +181,8 @@ Create :py:class:`earwigbot.wiki.User <earwigbot.wiki.user.User>` objects with
:py:meth:`page.get_creator() <earwigbot.wiki.page.Page.get_creator>`. They
provide the following attributes:

- :py:attr:`~earwigbot.wiki.user.User.site`: the user's corresponding
:py:class:`~earwigbot.wiki.site.Site` object
- :py:attr:`~earwigbot.wiki.user.User.name`: the user's username
- :py:attr:`~earwigbot.wiki.user.User.exists`: ``True`` if the user exists, or
``False`` if they do not


+ 25
- 7
earwigbot/commands/__init__.py View File

@@ -36,9 +36,13 @@ class Command(object):
This docstring is reported to the user when they type ``"!help
<command>"``.
"""
# This is the command's name, as reported to the user when they use !help:
# The command's name, as reported to the user when they use !help:
name = None

# A list of names that will trigger this command. If left empty, it will
# be triggered by the command's name and its name only:
commands = []

# Hooks are "msg", "msg_private", "msg_public", and "join". "msg" is the
# default behavior; if you wish to override that, change the value in your
# command subclass:
@@ -49,9 +53,10 @@ class Command(object):

This is called once when the command is loaded (from
:py:meth:`commands.load() <earwigbot.managers._ResourceManager.load>`).
*bot* is out base :py:class:`~earwigbot.bot.Bot` object. Generally you
shouldn't need to override this; if you do, call
``super(Command, self).__init__()`` first.
*bot* is out base :py:class:`~earwigbot.bot.Bot` object. Don't override
this directly; if you do, remember to place
``super(Command, self).__init()`` first. Use :py:meth:`setup` for
typical command-init/setup needs.
"""
self.bot = bot
self.config = bot.config
@@ -67,6 +72,15 @@ class Command(object):
self.mode = lambda t, level, msg: self.bot.frontend.mode(t, level, msg)
self.pong = lambda target: self.bot.frontend.pong(target)

self.setup()

def setup(self):
"""Hook called immediately after the command is loaded.

Does nothing by default; feel free to override.
"""
pass

def check(self, data):
"""Return whether this command should be called in response to *data*.

@@ -76,11 +90,15 @@ class Command(object):
sent on IRC, it should be cheap to execute and unlikely to throw
exceptions.

Most commands return ``True`` if :py:attr:`data.command
Most commands return ``True`` only if :py:attr:`data.command
<earwigbot.irc.data.Data.command>` ``==`` :py:attr:`self.name <name>`,
otherwise they return ``False``. This is the default behavior of
:py:meth:`check`; you need only override it if you wish to change that.
or :py:attr:`data.command <earwigbot.irc.data.Data.command>` is in
:py:attr:`self.commands <commands>` if that list is overriden. This is
the default behavior; you should only override it if you wish to change
that.
"""
if self.commands:
return data.is_command and data.command in self.commands
return data.is_command and data.command == self.name

def process(self, data):


+ 0
- 401
earwigbot/commands/_old.py View File

@@ -1,401 +0,0 @@
# -*- coding: utf-8 -*-
######
###### NOTE:
###### This is an old commands file from the previous version of EarwigBot.
###### It is not used by the new EarwigBot and is simply here for reference
###### when developing new commands.
######
### EarwigBot

def parse(command, line, line2, nick, chan, host, auth, notice, say, reply, s):
authy = auth(host)
if command == "access":
a = 'The bot\'s owner is "%s".' % OWNER
b = 'The bot\'s admins are "%s".' % ', '.join(ADMINS_R)
reply(a, chan, nick)
reply(b, chan, nick)
return
if command == "tock":
u = urllib.urlopen('http://tycho.usno.navy.mil/cgi-bin/timer.pl')
info = u.info()
u.close()
say('"' + info['Date'] + '" - tycho.usno.navy.mil', chan)
return
if command == "beats":
beats = ((time.time() + 3600) % 86400) / 86.4
beats = int(math.floor(beats))
say('@%03i' % beats, chan)
return
if command == "dict" or command == "dictionary":
def trim(thing):
if thing.endswith('&nbsp;'):
thing = thing[:-6]
return thing.strip(' :.')
r_li = re.compile(r'(?ims)<li>.*?</li>')
r_tag = re.compile(r'<[^>]+>')
r_parens = re.compile(r'(?<=\()(?:[^()]+|\([^)]+\))*(?=\))')
r_word = re.compile(r'^[A-Za-z0-9\' -]+$')
uri = 'http://encarta.msn.com/dictionary_/%s.html'
r_info = re.compile(r'(?:ResultBody"><br /><br />(.*?)&nbsp;)|(?:<b>(.*?)</b>)')
try:
word = line2[4]
except Exception:
reply("Please enter a word.", chan, nick)
return
word = urllib.quote(word.encode('utf-8'))
bytes = web.get(uri % word)
results = {}
wordkind = None
for kind, sense in r_info.findall(bytes):
kind, sense = trim(kind), trim(sense)
if kind: wordkind = kind
elif sense:
results.setdefault(wordkind, []).append(sense)
result = word.encode('utf-8') + ' - '
for key in sorted(results.keys()):
if results[key]:
result += (key or '') + ' 1. ' + results[key][0]
if len(results[key]) > 1:
result += ', 2. ' + results[key][1]
result += '; '
result = result.rstrip('; ')
if result.endswith('-') and (len(result) < 30):
reply('Sorry, no definition found.', chan, nick)
else: say(result, chan)
return
if command == "ety" or command == "etymology":
etyuri = 'http://etymonline.com/?term=%s'
etysearch = 'http://etymonline.com/?search=%s'
r_definition = re.compile(r'(?ims)<dd[^>]*>.*?</dd>')
r_tag = re.compile(r'<(?!!)[^>]+>')
r_whitespace = re.compile(r'[\t\r\n ]+')
abbrs = [
'cf', 'lit', 'etc', 'Ger', 'Du', 'Skt', 'Rus', 'Eng', 'Amer.Eng', 'Sp',
'Fr', 'N', 'E', 'S', 'W', 'L', 'Gen', 'J.C', 'dial', 'Gk',
'19c', '18c', '17c', '16c', 'St', 'Capt', 'obs', 'Jan', 'Feb', 'Mar',
'Apr', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec', 'c', 'tr', 'e', 'g'
]
t_sentence = r'^.*?(?<!%s)(?:\.(?= [A-Z0-9]|\Z)|\Z)'
r_sentence = re.compile(t_sentence % ')(?<!'.join(abbrs))
def unescape(s):
s = s.replace('&gt;', '>')
s = s.replace('&lt;', '<')
s = s.replace('&amp;', '&')
return s
def text(html):
html = r_tag.sub('', html)
html = r_whitespace.sub(' ', html)
return unescape(html).strip()
try:
word = line2[4]
except Exception:
reply("Please enter a word.", chan, nick)
return
def ety(word):
if len(word) > 25:
raise ValueError("Word too long: %s[...]" % word[:10])
word = {'axe': 'ax/axe'}.get(word, word)
bytes = web.get(etyuri % word)
definitions = r_definition.findall(bytes)
if not definitions:
return None
defn = text(definitions[0])
m = r_sentence.match(defn)
if not m:
return None
sentence = m.group(0)
try:
sentence = unicode(sentence, 'iso-8859-1')
sentence = sentence.encode('utf-8')
except: pass
maxlength = 275
if len(sentence) > maxlength:
sentence = sentence[:maxlength]
words = sentence[:-5].split(' ')
words.pop()
sentence = ' '.join(words) + ' [...]'
sentence = '"' + sentence.replace('"', "'") + '"'
return sentence + ' - ' + (etyuri % word)
try:
result = ety(word.encode('utf-8'))
except IOError:
msg = "Can't connect to etymonline.com (%s)" % (etyuri % word)
reply(msg, chan, nick)
return
except AttributeError:
result = None
if result is not None:
reply(result, chan, nick)
else:
uri = etysearch % word
msg = 'Can\'t find the etymology for "%s". Try %s' % (word, uri)
reply(msg, chan, nick)
return
if command == "pend" or command == "pending":
say("Pending submissions status page: <http://en.wikipedia.org/wiki/WP:AFC/S>.", chan)
say("Pending submissions category: <http://en.wikipedia.org/wiki/Category:Pending_AfC_submissions>.", chan)
return
if command == "sub" or command == "submissions":
try:
number = int(line2[4])
except Exception:
reply("Please enter a number.", chan, nick)
return
do_url = False
try:
if "url" in line2[5:]: do_url = True
except Exception:
pass
url = "http://en.wikipedia.org/w/api.php?action=query&list=categorymembers&cmtitle=Category:Pending_AfC_submissions&cmlimit=500&cmsort=timestamp"
query = urllib.urlopen(url)
data = query.read()
pages = re.findall("title=&quot;(.*?)&quot;", data)
try:
pages.remove("Wikipedia:Articles for creation/Redirects")
except Exception:
pass
try:
pages.remove("Wikipedia:Files for upload")
except Exception:
pass
pages.reverse()
pages = pages[:number]
if not do_url:
s = string.join(pages, "]], [[")
s = "[[%s]]" % s
else:
s = string.join(pages, ">, <http://en.wikipedia.org/wiki/")
s = "<http://en.wikipedia.org/wiki/%s>" % s
s = re.sub(" ", "_", s)
s = re.sub(">,_<", ">, <", s)
report = "\x02First %s pending AfC submissions:\x0F %s" % (number, s)
say(report, chan)
return
if command == "trout":
try:
user = line2[4]
user = ' '.join(line2[4:])
except Exception:
reply("Hahahahahahahaha...", chan, nick)
return
normal = unicodedata.normalize('NFKD', unicode(string.lower(user)))
if "itself" in normal:
reply("I'm not that stupid ;)", chan, nick)
return
elif "earwigbot" in normal:
reply("I'm not that stupid ;)", chan, nick)
elif "earwig" not in normal and "ear wig" not in normal:
text = 'slaps %s around a bit with a large trout.' % user
msg = '\x01ACTION %s\x01' % text
say(msg, chan)
else:
reply("I refuse to hurt anything with \"Earwig\" in its name :P", chan, nick)
return
if command == "mysql":
if authy != "owner":
reply("You aren't authorized to use this command.", chan, nick)
return
import MySQLdb
try:
strings = line2[4]
strings = ' '.join(line2[4:])
if "db:" in strings:
database = re.findall("db\:(.*?)\s", strings)[0]
else:
database = "enwiki_p"
if "time:" in strings:
times = int(re.findall("time\:(.*?)\s", strings)[0])
else:
times = 60
file = re.findall("file\:(.*?)\s", strings)[0]
sqlquery = re.findall("query\:(.*?)\Z", strings)[0]
except Exception:
reply("You did not specify enough data for the bot to continue.", chan, nick)
return
database2 = database[:-2] + "-p"
db = MySQLdb.connect(db=database, host="%s.rrdb.toolserver.org" % database2, read_default_file="/home/earwig/.my.cnf")
db.query(sqlquery)
r = db.use_result()
data = r.fetch_row(0)
try:
f = codecs.open("/home/earwig/public_html/reports/%s/%s" % (database[:-2], file), 'r')
reply("A file already exists with that name.", chan, nick)
return
except Exception:
pass
f = codecs.open("/home/earwig/public_html/reports/%s/%s" % (database[:-2], file), 'a', 'utf-8')
for line in data:
new_line = []
for l in line:
new_line.append(str(l))
f.write(' '.join(new_line) + "\n")
f.close()
reply("Query completed successfully. See http://toolserver.org/~earwig/reports/%s/%s. I will delete the report in %s seconds." % (database[:-2], file, times), chan, nick)
time.sleep(times)
os.remove("/home/earwig/public_html/reports/%s/%s" % (database[:-2], file))
return
if command == "notes" or command == "note" or command == "about" or command == "data" or command == "database":
try:
action = line2[4]
except BaseException:
reply("What do you want me to do? Type \"!notes help\" for more information.", chan, nick)
return
import MySQLdb
db = MySQLdb.connect(db="u_earwig_ircbot", host="sql", read_default_file="/home/earwig/.my.cnf")
specify = ' '.join(line2[5:])
if action == "help" or action == "manual":
shortCommandList = "read, write, change, undo, delete, move, author, category, list, report, developer"
if specify == "read":
say("To read an entry, type \"!notes read <entry>\".", chan)
elif specify == "write":
say("To write a new entry, type \"!notes write <entry> <content>\". This will create a new entry only if one does not exist, see the below command...", chan)
elif specify == "change":
say("To change an entry, type \"!notes change <entry> <new content>\". The old entry will be stored in the database, so it can be undone later.", chan)
elif specify == "undo":
say("To undo a change, type \"!notes undo <entry>\".", chan)
elif specify == "delete":
say("To delete an entry, type \"!notes delete <entry>\". For security reasons, only bot admins can do this.", chan)
elif specify == "move":
say("To move an entry, type \"!notes move <old_title> <new_title>\".", chan)
elif specify == "author":
say("To return the author of an entry, type \"!notes author <entry>\".", chan)
elif specify == "category" or specify == "cat":
say("To change an entry's category, type \"!notes category <entry> <category>\".", chan)
elif specify == "list":
say("To list all categories in the database, type \"!notes list\". Type \"!notes list <category>\" to get all entries in a certain category.", chan)
elif specify == "report":
say("To give some statistics about the mini-wiki, including some debugging information, type \"!notes report\" in a PM.", chan)
elif specify == "developer":
say("To do developer work, such as writing to the database directly, type \"!notes developer <command>\". This can only be done by the bot owner.", chan)
else:
db.query("SELECT * FROM version;")
r = db.use_result()
data = r.fetch_row(0)
version = data[0]
reply("The Earwig Mini-Wiki: running v%s." % version, chan, nick)
reply("The full list of commands, for reference, are: %s." % shortCommandList, chan, nick)
reply("For an explaination of a certain command, type \"!notes help <command>\".", chan, nick)
reply("You can also access the database from the Toolserver: http://toolserver.org/~earwig/cgi-bin/irc_database.py", chan, nick)
time.sleep(0.4)
return
elif action == "read":
specify = string.lower(specify)
if " " in specify: specify = string.split(specify, " ")[0]
if not specify or "\"" in specify:
reply("Please include the name of the entry you would like to read after the command, e.g. !notes read earwig", chan, nick)
return
try:
db.query("SELECT entry_content FROM entries WHERE entry_title = \"%s\";" % specify)
r = db.use_result()
data = r.fetch_row(0)
entry = data[0][0]
say("Entry \"\x02%s\x0F\": %s" % (specify, entry), chan)
except Exception:
reply("There is no entry titled \"\x02%s\x0F\"." % specify, chan, nick)
return
elif action == "delete" or action == "remove":
specify = string.lower(specify)
if " " in specify: specify = string.split(specify, " ")[0]
if not specify or "\"" in specify:
reply("Please include the name of the entry you would like to delete after the command, e.g. !notes delete earwig", chan, nick)
return
if authy == "owner" or authy == "admin":
try:
db.query("DELETE from entries where entry_title = \"%s\";" % specify)
r = db.use_result()
db.commit()
reply("The entry on \"\x02%s\x0F\" has been removed." % specify, chan, nick)
except Exception:
phenny.reply("Unable to remove the entry on \"\x02%s\x0F\", because it doesn't exist." % specify, chan, nick)
else:
reply("Only bot admins can remove entries.", chan, nick)
return
elif action == "developer":
if authy == "owner":
db.query(specify)
r = db.use_result()
try:
print r.fetch_row(0)
except Exception:
pass
db.commit()
reply("Done.", chan, nick)
else:
reply("Only the bot owner can modify the raw database.", chan, nick)
return
elif action == "write":
try:
write = line2[5]
content = ' '.join(line2[6:])
except Exception:
reply("Please include some content in your entry.", chan, nick)
return
db.query("SELECT * from entries WHERE entry_title = \"%s\";" % write)
r = db.use_result()
data = r.fetch_row(0)
if data:
reply("An entry on %s already exists; please use \"!notes change %s %s\"." % (write, write, content), chan, nick)
return
content2 = content.replace('"', '\\' + '"')
db.query("INSERT INTO entries (entry_title, entry_author, entry_category, entry_content, entry_content_old) VALUES (\"%s\", \"%s\", \"uncategorized\", \"%s\", NULL);" % (write, nick, content2))
db.commit()
reply("You have written an entry titled \"\x02%s\x0F\", with the following content: \"%s\"" % (write, content), chan, nick)
return
elif action == "change":
reply("NotImplementedError", chan, nick)
elif action == "undo":
reply("NotImplementedError", chan, nick)
elif action == "move":
reply("NotImplementedError", chan, nick)
elif action == "author":
try:
entry = line2[5]
except Exception:
reply("Please include the name of the entry you would like to get information for after the command, e.g. !notes author earwig", chan, nick)
return
db.query("SELECT entry_author from entries WHERE entry_title = \"%s\";" % entry)
r = db.use_result()
data = r.fetch_row(0)
if data:
say("The author of \"\x02%s\x0F\" is \x02%s\x0F." % (entry, data[0][0]), chan)
return
reply("There is no entry titled \"\x02%s\x0F\"." % entry, chan, nick)
return
elif action == "cat" or action == "category":
reply("NotImplementedError", chan, nick)
elif action == "list":
reply("NotImplementedError", chan, nick)
elif action == "report":
reply("NotImplementedError", chan, nick)
if command == "langcode" or command == "lang" or command == "language":
try:
lang = line2[4]
except Exception:
reply("Please specify an ISO code.", chan, nick)
return
data = urllib.urlopen("http://toolserver.org/~earwig/cgi-bin/swmt.py?action=iso").read()
data = string.split(data, "\n")
result = False
for datum in data:
if datum.startswith(lang):
result = re.findall(".*? (.*)", datum)[0]
break
if result:
reply(result, chan, nick)
return
reply("Not found.", chan, nick)
return
if command == "lookup" or command == "ip":
try:
hexIP = line2[4]
except Exception:
reply("Please specify a hex IP address.", chan, nick)
return
hexes = [hexIP[:2], hexIP[2:4], hexIP[4:6], hexIP[6:8]]
hashes = []
for hexHash in hexes:
newHex = int(hexHash, 16)
hashes.append(newHex)
normalizedIP = "%s.%s.%s.%s" % (hashes[0], hashes[1], hashes[2], hashes[3])
reply(normalizedIP, chan, nick)
return

+ 34
- 0
earwigbot/commands/afc_pending.py View File

@@ -0,0 +1,34 @@
# -*- coding: utf-8 -*-
#
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net>
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.

from earwigbot.commands import BaseCommand

class Command(BaseCommand):
"""Link the user to the pending AFC submissions page and category."""
name = "pending"
commands = ["pending", "pend"]

def process(self, data):
msg1 = "pending submissions status page: http://enwp.org/WP:AFC/ST"
msg2 = "pending submissions category: http://enwp.org/CAT:PEND"
self.reply(data, msg1)
self.reply(data, msg2)

+ 1
- 1
earwigbot/commands/afc_report.py View File

@@ -72,7 +72,7 @@ class AFCReport(Command):

def get_page(self, title):
page = self.site.get_page(title, follow_redirects=False)
if page.exists[0]:
if page.exists == page.PAGE_EXISTS:
return page

def report(self, page):


+ 2
- 3
earwigbot/commands/afc_status.py View File

@@ -30,13 +30,12 @@ class AFCStatus(Command):
"""Get the number of pending AfC submissions, open redirect requests, and
open file upload requests."""
name = "status"
commands = ["status", "count", "num", "number"]
hooks = ["join", "msg"]

def check(self, data):
commands = ["status", "count", "num", "number"]
if data.is_command and data.command in commands:
if data.is_command and data.command in self.commands:
return True

try:
if data.line[1] == "JOIN" and data.chan == "#wikipedia-en-afc":
if data.nick != self.config.irc["frontend"]["nick"]:


+ 60
- 0
earwigbot/commands/afc_submissions.py View File

@@ -0,0 +1,60 @@
# -*- coding: utf-8 -*-
#
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net>
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.

from earwigbot.commands import Command

__all__ = ["AFCSubmissions"]

class AFCSubmissions(Command):
"""Link the user directly to some pending AFC submissions."""
name = "submissions"
commands = ["submissions", "subs"]

def setup(self):
try:
self.ignore_list = self.config.commands[self.name]["ignoreList"]
except KeyError:
try:
ignores = self.config.tasks["afc_statistics"]["ignoreList"]
self.ignore_list = ignores
except KeyError:
self.ignore_list = []

def process(self, data):
if data.args:
try:
number = int(data.args[0])
except ValueError:
self.reply(data, "argument must be a number.")
return
if number > 5:
msg = "cannot get more than five submissions at a time."
self.reply(data, msg)
return
else:
number = 3

site = self.bot.wiki.get_site()
category = site.get_category("Pending AfC submissions")
members = category.get_members(use_sql=True, limit=number)
pages = ", ".join([member.url for member in members])
self.reply(data, "{0} pending AfC subs: {1}".format(number, pages))

+ 1
- 4
earwigbot/commands/chanops.py View File

@@ -28,10 +28,7 @@ class ChanOps(Command):
"""Voice, devoice, op, or deop users in the channel, or join or part from
other channels."""
name = "chanops"

def check(self, data):
cmnds = ["chanops", "voice", "devoice", "op", "deop", "join", "part"]
return data.is_command and data.command in cmnds
commands = ["chanops", "voice", "devoice", "op", "deop", "join", "part"]

def process(self, data):
if data.command == "chanops":


+ 1
- 6
earwigbot/commands/crypt.py View File

@@ -32,12 +32,7 @@ class Crypt(Command):
"""Provides hash functions with !hash (!hash list for supported algorithms)
and blowfish encryption with !encrypt and !decrypt."""
name = "crypt"

def check(self, data):
commands = ["crypt", "hash", "encrypt", "decrypt"]
if data.is_command and data.command in commands:
return True
return False
commands = ["crypt", "hash", "encrypt", "decrypt"]

def process(self, data):
if data.command == "crypt":


+ 4
- 8
earwigbot/commands/editcount.py View File

@@ -30,12 +30,7 @@ __all__ = ["Editcount"]
class Editcount(Command):
"""Return a user's edit count."""
name = "editcount"

def check(self, data):
commands = ["ec", "editcount"]
if data.is_command and data.command in commands:
return True
return False
commands = ["ec", "editcount"]

def process(self, data):
if not data.args:
@@ -54,6 +49,7 @@ class Editcount(Command):
return

safe = quote_plus(user.name)
url = "http://toolserver.org/~tparis/pcount/index.php?name={0}&lang=en&wiki=wikipedia"
url = "http://toolserver.org/~tparis/pcount/index.php?name={0}&lang={1}&wiki={2}"
fullurl = url.format(safe, site.lang, site.project)
msg = "\x0302{0}\x0301 has {1} edits ({2})."
self.reply(data, msg.format(name, count, url.format(safe)))
self.reply(data, msg.format(name, count, fullurl))

+ 72
- 0
earwigbot/commands/geolocate.py View File

@@ -0,0 +1,72 @@
# -*- coding: utf-8 -*-
#
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net>
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.

import json
import urllib2

from earwigbot.commands import BaseCommand

class Command(BaseCommand):
"""Geolocate an IP address (via http://ipinfodb.com/)."""
name = "geolocate"
commands = ["geolocate", "locate", "geo", "ip"]

def setup(self):
self.config.decrypt(self.config.commands, (self.name, "apiKey"))
try:
self.key = self.config.commands[self.name]["apiKey"]
except KeyError:
self.key = None
log = 'Cannot use without an API key for http://ipinfodb.com/ stored as config.commands["{0}"]["apiKey"]'
self.logger.warn(log.format(self.name))

def process(self, data):
if not data.args:
self.reply(data, "please specify an IP to lookup.")
return

if not self.key:
msg = 'I need an API key for http://ipinfodb.com/ stored as \x0303config.commands["{0}"]["apiKey"]\x0301.'
log = 'Need an API key for http://ipinfodb.com/ stored as config.commands["{0}"]["apiKey"]'
self.reply(data, msg.format(self.name) + ".")
self.logger.error(log.format(self.name))
return

address = data.args[0]
url = "http://api.ipinfodb.com/v3/ip-city/?key={0}&ip={1}&format=json"
query = urllib2.urlopen(url.format(self.key, address)).read()
res = json.loads(query)

try:
country = res["countryName"]
region = res["regionName"]
city = res["cityName"]
latitude = res["latitude"]
longitude = res["longitude"]
utcoffset = res["timeZone"]
except KeyError:
self.reply(data, "IP \x0302{0}\x0301 not found.".format(address))
return

msg = "{0}, {1}, {2} ({3}, {4}), UTC {5}"
geo = msg.format(country, region, city, latitude, longitude, utcoffset)
self.reply(data, geo)

+ 11
- 6
earwigbot/commands/git.py View File

@@ -32,10 +32,12 @@ class Git(Command):
"""Commands to interface with the bot's git repository; use '!git' for a
sub-command list."""
name = "git"
repos = {
"core": "/home/earwig/git/earwigbot",
"plugins": "/home/earwig/git/earwigbot-plugins",
}

def setup(self):
try:
self.repos = self.config.commands[self.name]["repos"]
except KeyError:
self.repos = None

def process(self, data):
self.data = data
@@ -46,6 +48,9 @@ class Git(Command):
if not data.args or data.args[0] == "help":
self.do_help()
return
if not self.repos:
self.reply(data, "no repos are specified in the config file.")
return

command = data.args[0]
try:
@@ -57,7 +62,7 @@ class Git(Command):
return
if repo_name not in self.repos:
repos = self.get_repos()
msg = "repository must be one of the following: {0}"
msg = "repository must be one of the following: {0}."
self.reply(data, msg.format(repos))
return
self.repo = git.Repo(self.repos[repo_name])
@@ -91,7 +96,7 @@ class Git(Command):
try:
return getattr(self.repo.remotes, remote_name)
except AttributeError:
msg = "unknown remote: \x0302{0}\x0301".format(remote_name)
msg = "unknown remote: \x0302{0}\x0301.".format(remote_name)
self.reply(self.data, msg)

def get_time_since(self, date):


+ 11
- 22
earwigbot/commands/help.py View File

@@ -23,7 +23,6 @@
import re

from earwigbot.commands import Command
from earwigbot.irc import Data

__all__ = ["Help"]

@@ -50,34 +49,24 @@ class Help(Command):
def do_main_help(self, data):
"""Give the user a general help message with a list of all commands."""
msg = "Hi, I'm a bot! I have {0} commands loaded: {1}. You can get help for any command with '!help <command>'."
cmnds = sorted(self.bot.commands)
cmnds = sorted([cmnd.name for cmnd in self.bot.commands])
msg = msg.format(len(cmnds), ', '.join(cmnds))
self.reply(data, msg)

def do_command_help(self, data):
"""Give the user help for a specific command."""
command = data.args[0]
target = data.args[0]

# Create a dummy message to test which commands pick up the user's
# input:
msg = ":foo!bar@example.com PRIVMSG #channel :msg".split()
dummy = Data(self.bot, msg)
dummy.command = command.lower()
dummy.is_command = True
for command in self.bot.commands:
if command.name == target or target in command.commands:
if command.__doc__:
doc = command.__doc__.replace("\n", "")
doc = re.sub("\s\s+", " ", doc)
msg = "help for command \x0303{0}\x0301: \"{1}\""
self.reply(data, msg.format(target, doc))
return

for cmnd_name in self.bot.commands:
cmnd = self.bot.commands.get(cmnd_name)
if not cmnd.check(dummy):
continue
if cmnd.__doc__:
doc = cmnd.__doc__.replace("\n", "")
doc = re.sub("\s\s+", " ", doc)
msg = "help for command \x0303{0}\x0301: \"{1}\""
self.reply(data, msg.format(command, doc))
return
break

msg = "sorry, no help for \x0303{0}\x0301.".format(command)
msg = "sorry, no help for \x0303{0}\x0301.".format(target)
self.reply(data, msg)

def do_hello(self, data):


+ 49
- 0
earwigbot/commands/langcode.py View File

@@ -0,0 +1,49 @@
# -*- coding: utf-8 -*-
#
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net>
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.

from earwigbot.commands import BaseCommand

class Command(BaseCommand):
"""Convert a language code into its name and a list of WMF sites in that
language."""
name = "langcode"
commands = ["langcode", "lang", "language"]

def process(self, data):
if not data.args:
self.reply(data, "please specify a language code.")
return

code = data.args[0]
site = self.bot.wiki.get_site()
matrix = site.api_query(action="sitematrix")["sitematrix"]
del matrix["specials"]

for site in matrix.itervalues():
if site["code"] == code:
name = site["name"]
sites = ", ".join([s["url"] for s in site["site"]])
msg = "\x0302{0}\x0301 is {1} ({2})".format(code, name, sites)
self.reply(data, msg)
return

self.reply(data, "site \x0302{0}\x0301 not found.".format(code))

+ 0
- 8
earwigbot/commands/link.py View File

@@ -31,14 +31,6 @@ class Link(Command):
"""Convert a Wikipedia page name into a URL."""
name = "link"

def check(self, data):
# if ((data.is_command and data.command == "link") or
# (("[[" in data.msg and "]]" in data.msg) or
# ("{{" in data.msg and "}}" in data.msg))):
if data.is_command and data.command == "link":
return True
return False

def process(self, data):
msg = data.msg



+ 169
- 0
earwigbot/commands/notes.py View File

@@ -0,0 +1,169 @@
# -*- coding: utf-8 -*-
#
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net>
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.

from earwigbot.commands import Command

__all__ = ["Notes"]

class Notes(Command):
"""A mini IRC-based wiki for storing notes, tips, and reminders."""
name = "notes"

def process(self, data):
pass


class OldCommand(object):
def parse(self):
if command == "notes" or command == "note" or command == "about" or command == "data" or command == "database":
try:
action = line2[4]
except BaseException:
reply("What do you want me to do? Type \"!notes help\" for more information.", chan, nick)
return
import MySQLdb
db = MySQLdb.connect(db="u_earwig_ircbot", host="sql", read_default_file="/home/earwig/.my.cnf")
specify = ' '.join(line2[5:])
if action == "help" or action == "manual":
shortCommandList = "read, write, change, undo, delete, move, author, category, list, report, developer"
if specify == "read":
say("To read an entry, type \"!notes read <entry>\".", chan)
elif specify == "write":
say("To write a new entry, type \"!notes write <entry> <content>\". This will create a new entry only if one does not exist, see the below command...", chan)
elif specify == "change":
say("To change an entry, type \"!notes change <entry> <new content>\". The old entry will be stored in the database, so it can be undone later.", chan)
elif specify == "undo":
say("To undo a change, type \"!notes undo <entry>\".", chan)
elif specify == "delete":
say("To delete an entry, type \"!notes delete <entry>\". For security reasons, only bot admins can do this.", chan)
elif specify == "move":
say("To move an entry, type \"!notes move <old_title> <new_title>\".", chan)
elif specify == "author":
say("To return the author of an entry, type \"!notes author <entry>\".", chan)
elif specify == "category" or specify == "cat":
say("To change an entry's category, type \"!notes category <entry> <category>\".", chan)
elif specify == "list":
say("To list all categories in the database, type \"!notes list\". Type \"!notes list <category>\" to get all entries in a certain category.", chan)
elif specify == "report":
say("To give some statistics about the mini-wiki, including some debugging information, type \"!notes report\" in a PM.", chan)
elif specify == "developer":
say("To do developer work, such as writing to the database directly, type \"!notes developer <command>\". This can only be done by the bot owner.", chan)
else:
db.query("SELECT * FROM version;")
r = db.use_result()
data = r.fetch_row(0)
version = data[0]
reply("The Earwig Mini-Wiki: running v%s." % version, chan, nick)
reply("The full list of commands, for reference, are: %s." % shortCommandList, chan, nick)
reply("For an explaination of a certain command, type \"!notes help <command>\".", chan, nick)
reply("You can also access the database from the Toolserver: http://toolserver.org/~earwig/cgi-bin/irc_database.py", chan, nick)
time.sleep(0.4)
return
elif action == "read":
specify = string.lower(specify)
if " " in specify: specify = string.split(specify, " ")[0]
if not specify or "\"" in specify:
reply("Please include the name of the entry you would like to read after the command, e.g. !notes read earwig", chan, nick)
return
try:
db.query("SELECT entry_content FROM entries WHERE entry_title = \"%s\";" % specify)
r = db.use_result()
data = r.fetch_row(0)
entry = data[0][0]
say("Entry \"\x02%s\x0F\": %s" % (specify, entry), chan)
except Exception:
reply("There is no entry titled \"\x02%s\x0F\"." % specify, chan, nick)
return
elif action == "delete" or action == "remove":
specify = string.lower(specify)
if " " in specify: specify = string.split(specify, " ")[0]
if not specify or "\"" in specify:
reply("Please include the name of the entry you would like to delete after the command, e.g. !notes delete earwig", chan, nick)
return
if authy == "owner" or authy == "admin":
try:
db.query("DELETE from entries where entry_title = \"%s\";" % specify)
r = db.use_result()
db.commit()
reply("The entry on \"\x02%s\x0F\" has been removed." % specify, chan, nick)
except Exception:
phenny.reply("Unable to remove the entry on \"\x02%s\x0F\", because it doesn't exist." % specify, chan, nick)
else:
reply("Only bot admins can remove entries.", chan, nick)
return
elif action == "developer":
if authy == "owner":
db.query(specify)
r = db.use_result()
try:
print r.fetch_row(0)
except Exception:
pass
db.commit()
reply("Done.", chan, nick)
else:
reply("Only the bot owner can modify the raw database.", chan, nick)
return
elif action == "write":
try:
write = line2[5]
content = ' '.join(line2[6:])
except Exception:
reply("Please include some content in your entry.", chan, nick)
return
db.query("SELECT * from entries WHERE entry_title = \"%s\";" % write)
r = db.use_result()
data = r.fetch_row(0)
if data:
reply("An entry on %s already exists; please use \"!notes change %s %s\"." % (write, write, content), chan, nick)
return
content2 = content.replace('"', '\\' + '"')
db.query("INSERT INTO entries (entry_title, entry_author, entry_category, entry_content, entry_content_old) VALUES (\"%s\", \"%s\", \"uncategorized\", \"%s\", NULL);" % (write, nick, content2))
db.commit()
reply("You have written an entry titled \"\x02%s\x0F\", with the following content: \"%s\"" % (write, content), chan, nick)
return
elif action == "change":
reply("NotImplementedError", chan, nick)
elif action == "undo":
reply("NotImplementedError", chan, nick)
elif action == "move":
reply("NotImplementedError", chan, nick)
elif action == "author":
try:
entry = line2[5]
except Exception:
reply("Please include the name of the entry you would like to get information for after the command, e.g. !notes author earwig", chan, nick)
return
db.query("SELECT entry_author from entries WHERE entry_title = \"%s\";" % entry)
r = db.use_result()
data = r.fetch_row(0)
if data:
say("The author of \"\x02%s\x0F\" is \x02%s\x0F." % (entry, data[0][0]), chan)
return
reply("There is no entry titled \"\x02%s\x0F\"." % entry, chan, nick)
return
elif action == "cat" or action == "category":
reply("NotImplementedError", chan, nick)
elif action == "list":
reply("NotImplementedError", chan, nick)
elif action == "report":
reply("NotImplementedError", chan, nick)

+ 16
- 17
earwigbot/commands/praise.py View File

@@ -28,24 +28,23 @@ class Praise(Command):
"""Praise people!"""
name = "praise"

def setup(self):
try:
self.praises = self.config.commands[self.name]["praises"]
except KeyError:
self.praises = []

def check(self, data):
commands = ["praise", "earwig", "leonard", "leonard^bloom", "groove",
"groovedog"]
return data.is_command and data.command in commands
check = data.command == "praise" or data.command in self.praises
return data.is_command and check

def process(self, data):
if data.command == "earwig":
msg = "\x02Earwig\x0F is the bestest Python programmer ever!"
elif data.command in ["leonard", "leonard^bloom"]:
msg = "\x02Leonard^Bloom\x0F is the biggest slacker ever!"
elif data.command in ["groove", "groovedog"]:
msg = "\x02GrooveDog\x0F is the bestest heh evar!"
else:
if not data.args:
msg = "You use this command to praise certain people. Who they are is a secret."
else:
msg = "You're doing it wrong."
self.reply(data, msg)
if data.command in self.praises:
msg = self.praises[data.command]
self.say(data.chan, msg)
return

self.say(data.chan, msg)
if not data.args:
msg = "You use this command to praise certain people. Who they are is a secret."
else:
msg = "you're doing it wrong."
self.reply(data, msg)

+ 1
- 4
earwigbot/commands/quit.py View File

@@ -28,10 +28,7 @@ class Quit(Command):
"""Quit, restart, or reload components from the bot. Only the owners can
run this command."""
name = "quit"

def check(self, data):
commands = ["quit", "restart", "reload"]
return data.is_command and data.command in commands
commands = ["quit", "restart", "reload"]

def process(self, data):
if data.host not in self.config.irc["permissions"]["owners"]:


+ 2
- 7
earwigbot/commands/registration.py View File

@@ -30,12 +30,7 @@ __all__ = ["Registration"]
class Registration(Command):
"""Return when a user registered."""
name = "registration"

def check(self, data):
commands = ["registration", "reg", "age"]
if data.is_command and data.command in commands:
return True
return False
commands = ["registration", "reg", "age"]

def process(self, data):
if not data.args:
@@ -61,7 +56,7 @@ class Registration(Command):
elif user.gender == "female":
gender = "She's"
else:
gender = "They're"
gender = "They're" # Singluar they?

msg = "\x0302{0}\x0301 registered on {1}. {2} {3} old."
self.reply(data, msg.format(name, date, gender, age))


+ 3
- 12
earwigbot/commands/remind.py View File

@@ -20,7 +20,7 @@
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.

import threading
from threading import Timer
import time

from earwigbot.commands import Command
@@ -30,11 +30,7 @@ __all__ = ["Remind"]
class Remind(Command):
"""Set a message to be repeated to you in a certain amount of time."""
name = "remind"

def check(self, data):
if data.is_command and data.command in ["remind", "reminder"]:
return True
return False
commands = ["remind", "reminder"]

def process(self, data):
if not data.args:
@@ -62,12 +58,7 @@ class Remind(Command):
msg = msg.format(message, wait, end_time_with_timezone)
self.reply(data, msg)

t_reminder = threading.Thread(target=self.reminder,
args=(data, message, wait))
t_reminder = Timer(wait, self.reply, args=(data, message))
t_reminder.name = "reminder " + end_time
t_reminder.daemon = True
t_reminder.start()

def reminder(self, data, message, wait):
time.sleep(wait)
self.reply(data, message)

+ 10
- 3
earwigbot/commands/replag.py View File

@@ -32,10 +32,16 @@ class Replag(Command):
"""Return the replag for a specific database on the Toolserver."""
name = "replag"

def setup(self):
try:
self.default = self.config.commands[self.name]["default"]
except KeyError:
self.default = None

def process(self, data):
args = {}
if not data.args:
args["db"] = "enwiki_p"
args["db"] = self.default or self.bot.wiki.get_site().name + "_p"
else:
args["db"] = data.args[0]
args["host"] = args["db"].replace("_", "-") + ".rrdb.toolserver.org"
@@ -43,10 +49,11 @@ class Replag(Command):

conn = oursql.connect(**args)
with conn.cursor() as cursor:
query = "SELECT UNIX_TIMESTAMP() - UNIX_TIMESTAMP(rc_timestamp) FROM recentchanges ORDER BY rc_timestamp DESC LIMIT 1"
query = """SELECT UNIX_TIMESTAMP() - UNIX_TIMESTAMP(rc_timestamp)
FROM recentchanges ORDER BY rc_timestamp DESC LIMIT 1"""
cursor.execute(query)
replag = int(cursor.fetchall()[0][0])
conn.close()

msg = "Replag on \x0302{0}\x0301 is \x02{1}\x0F seconds."
msg = "replag on \x0302{0}\x0301 is \x02{1}\x0F seconds."
self.reply(data, msg.format(args["db"], replag))

+ 1
- 6
earwigbot/commands/rights.py View File

@@ -28,12 +28,7 @@ __all__ = ["Rights"]
class Rights(Command):
"""Retrieve a list of rights for a given username."""
name = "rights"

def check(self, data):
commands = ["rights", "groups", "permissions", "privileges"]
if data.is_command and data.command in commands:
return True
return False
commands = ["rights", "groups", "permissions", "privileges"]

def process(self, data):
if not data.args:


+ 3
- 8
earwigbot/commands/threads.py View File

@@ -30,12 +30,7 @@ __all__ = ["Threads"]
class Threads(Command):
"""Manage wiki tasks from IRC, and check on thread status."""
name = "threads"

def check(self, data):
commands = ["tasks", "task", "threads", "tasklist"]
if data.is_command and data.command in commands:
return True
return False
commands = ["tasks", "task", "threads", "tasklist"]

def process(self, data):
self.data = data
@@ -106,7 +101,7 @@ class Threads(Command):
whether they are currently running or idle."""
threads = threading.enumerate()
tasklist = []
for task in sorted(self.bot.tasks):
for task in sorted([task.name for task in self.bot.tasks]):
threadlist = [t for t in threads if t.name.startswith(task)]
ids = [str(t.ident) for t in threadlist]
if not ids:
@@ -134,7 +129,7 @@ class Threads(Command):
self.reply(data, "what task do you want me to start?")
return

if task_name not in self.bot.tasks:
if task_name not in [task.name for task in self.bot.tasks]:
# This task does not exist or hasn't been loaded:
msg = "task could not be found; either it doesn't exist, or it wasn't loaded correctly."
self.reply(data, msg.format(task_name))


+ 68
- 0
earwigbot/commands/time.py View File

@@ -0,0 +1,68 @@
# -*- coding: utf-8 -*-
#
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net>
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.

from datetime import datetime
from math import floor
from time import time

try:
import pytz
except ImportError:
pytz = None

from earwigbot.commands import BaseCommand

class Command(BaseCommand):
"""Report the current time in any timezone (UTC default), or in beats."""
name = "time"
commands = ["time", "beats", "swatch"]

def process(self, data):
if data.command in ["beats", "swatch"]:
self.do_beats(data)
return
if data.args:
timezone = data.args[0]
else:
timezone = "UTC"
if timezone in ["beats", "swatch"]:
self.do_beats(data)
else:
self.do_time(data, timezone)

def do_beats(self, data):
beats = ((time() + 3600) % 86400) / 86.4
beats = int(floor(beats))
self.reply(data, "@{0:0>3}".format(beats))

def do_time(self, data, timezone):
if not pytz:
msg = "this command requires the 'pytz' module: http://pytz.sourceforge.net/"
self.reply(data, msg)
return
try:
tzinfo = pytz.timezone(timezone)
except pytz.exceptions.UnknownTimeZoneError:
self.reply(data, "unknown timezone: {0}.".format(timezone))
return
now = pytz.utc.localize(datetime.utcnow()).astimezone(tzinfo)
self.reply(data, now.strftime("%Y-%m-%d %H:%M:%S %Z"))

+ 48
- 0
earwigbot/commands/trout.py View File

@@ -0,0 +1,48 @@
# -*- coding: utf-8 -*-
#
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net>
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.

from unicodedata import normalize

from earwigbot.commands import Command

__all__ = ["Trout"]

class Trout(Command):
"""Slap someone with a trout, or related fish."""
name = "trout"
commands = ["trout", "whale"]

def setup(self):
try:
self.exceptions = self.config.commands[self.name]["exceptions"]
except KeyError:
self.exceptions = {}

def process(self, data):
animal = data.command
target = " ".join(data.args) or data.nick
normal = normalize("NFKD", target.decode("utf8")).lower()
if normal in self.exceptions:
self.reply(data, self.exceptions["normal"])
else:
msg = "slaps {0} around a bit with a large {1}."
self.action(data.chan, msg.format(target, animal))

+ 29
- 18
earwigbot/config.py View File

@@ -48,8 +48,9 @@ class BotConfig(object):
- :py:attr:`path`: path to the bot's config file
- :py:attr:`components`: enabled components
- :py:attr:`wiki`: information about wiki-editing
- :py:attr:`tasks`: information for bot tasks
- :py:attr:`irc`: information about IRC
- :py:attr:`commands`: information about IRC commands
- :py:attr:`tasks`: information for bot tasks
- :py:attr:`metadata`: miscellaneous information
- :py:meth:`schedule`: tasks scheduled to run at a given time

@@ -69,12 +70,13 @@ class BotConfig(object):

self._components = _ConfigNode()
self._wiki = _ConfigNode()
self._tasks = _ConfigNode()
self._irc = _ConfigNode()
self._commands = _ConfigNode()
self._tasks = _ConfigNode()
self._metadata = _ConfigNode()

self._nodes = [self._components, self._wiki, self._tasks, self._irc,
self._metadata]
self._nodes = [self._components, self._wiki, self._irc, self._commands,
self._tasks, self._metadata]

self._decryptable_nodes = [ # Default nodes to decrypt
(self._wiki, ("password",)),
@@ -196,16 +198,21 @@ class BotConfig(object):
return self._wiki

@property
def tasks(self):
"""A dict of information for bot tasks."""
return self._tasks

@property
def irc(self):
"""A dict of information about IRC."""
return self._irc

@property
def commands(self):
"""A dict of information for IRC commands."""
return self._commands

@property
def tasks(self):
"""A dict of information for bot tasks."""
return self._tasks

@property
def metadata(self):
"""A dict of miscellaneous information."""
return self._metadata
@@ -225,14 +232,14 @@ class BotConfig(object):
user. If there is no config file at all, offer to make one, otherwise
exit.

Data from the config file is stored in five
Data from the config file is stored in six
:py:class:`~earwigbot.config._ConfigNode`\ s (:py:attr:`components`,
:py:attr:`wiki`, :py:attr:`tasks`, :py:attr:`irc`, :py:attr:`metadata`)
for easy access (as well as the lower-level :py:attr:`data` attribute).
If passwords are encrypted, we'll use :py:func:`~getpass.getpass` for
the key and then decrypt them. If the config is being reloaded,
encrypted items will be automatically decrypted if they were decrypted
earlier.
:py:attr:`wiki`, :py:attr:`irc`, :py:attr:`commands`, :py:attr:`tasks`,
:py:attr:`metadata`) for easy access (as well as the lower-level
:py:attr:`data` attribute). If passwords are encrypted, we'll use
:py:func:`~getpass.getpass` for the key and then decrypt them. If the
config is being reloaded, encrypted items will be automatically
decrypted if they were decrypted earlier.
"""
if not path.exists(self._config_path):
print "Config file not found:", self._config_path
@@ -246,8 +253,9 @@ class BotConfig(object):
data = self._data
self.components._load(data.get("components", {}))
self.wiki._load(data.get("wiki", {}))
self.tasks._load(data.get("tasks", {}))
self.irc._load(data.get("irc", {}))
self.commands._load(data.get("commands", {}))
self.tasks._load(data.get("tasks", {}))
self.metadata._load(data.get("metadata", {}))

self._setup_logging()
@@ -273,7 +281,10 @@ class BotConfig(object):
>>> config.decrypt(config.irc, "frontend", "nickservPassword")
# decrypts config.irc["frontend"]["nickservPassword"]
"""
self._decryptable_nodes.append((node, nodes))
signature = (node, nodes)
if signature in self._decryptable_nodes:
return # Already decrypted
self._decryptable_nodes.append(signature)
if self.is_encrypted():
self._decrypt(node, nodes)



+ 8
- 13
earwigbot/managers.py View File

@@ -24,7 +24,7 @@
import imp
from os import listdir, path
from re import sub
from threading import Lock, Thread
from threading import RLock, Thread
from time import gmtime, strftime

from earwigbot.commands import Command
@@ -46,11 +46,7 @@ class _ResourceManager(object):

This class handles the low-level tasks of (re)loading resources via
:py:meth:`load`, retrieving specific resources via :py:meth:`get`, and
iterating over all resources via :py:meth:`__iter__`. If iterating over
resources, it is recommended to acquire :py:attr:`self.lock <lock>`
beforehand and release it afterwards (alternatively, wrap your code in a
``with`` statement) so an attempt at reloading resources in another thread
won't disrupt your iteration.
iterating over all resources via :py:meth:`__iter__`.
"""
def __init__(self, bot, name, base):
self.bot = bot
@@ -62,8 +58,9 @@ class _ResourceManager(object):
self._resource_access_lock = Lock()

def __iter__(self):
for name in self._resources:
yield name
with self.lock:
for resource in self._resources.itervalues():
yield resource

def _load_resource(self, name, path, klass):
"""Instantiate a resource class and add it to the dictionary."""
@@ -140,7 +137,8 @@ class _ResourceManager(object):
Will raise :py:exc:`KeyError` if the resource (a command or task) is
not found.
"""
return self._resources[key]
with self.lock:
return self._resources[key]


class CommandManager(_ResourceManager):
@@ -168,13 +166,10 @@ class CommandManager(_ResourceManager):

def call(self, hook, data):
"""Respond to a hook type and a :py:class:`Data` object."""
self.lock.acquire()
for command in self._resources.itervalues():
for command in self:
if hook in command.hooks and self._wrap_check(command, data):
self.lock.release()
self._wrap_process(command, data)
return
self.lock.release()


class TaskManager(_ResourceManager):


earwigbot/tasks/blptag.py → earwigbot/tasks/blp_tag.py View File

@@ -27,7 +27,7 @@ __all__ = ["BLPTag"]
class BLPTag(Task):
"""A task to add |blp=yes to ``{{WPB}}`` or ``{{WPBS}}`` when it is used
along with ``{{WP Biography}}``."""
name = "blptag"
name = "blp_tag"

def setup(self):
pass

earwigbot/tasks/feed_dailycats.py → earwigbot/tasks/image_display_resize.py View File

@@ -22,11 +22,11 @@

from earwigbot.tasks import Task

__all__ = ["FeedDailyCats"]
__all__ = ["ImageDisplayResize"]

class FeedDailyCats(Task):
"""A task to create daily categories for [[WP:FEED]]."""
name = "feed_dailycats"
class ImageDisplayResize(Task):
"""A task to resize upscaled portraits in infoboxes."""
name = "image_display_resize"

def setup(self):
pass

earwigbot/tasks/wrongmime.py → earwigbot/tasks/wrong_mime.py View File

@@ -27,7 +27,7 @@ __all__ = ["WrongMIME"]
class WrongMIME(Task):
"""A task to tag files whose extensions do not agree with their MIME
type."""
name = "wrongmime"
name = "wrong_mime"

def setup(self):
pass

+ 53
- 29
earwigbot/wiki/category.py View File

@@ -39,7 +39,7 @@ class Category(Page):

*Public methods:*

- :py:meth:`get_members`: returns a list of page titles in the category
- :py:meth:`get_members`: iterates over Pages in the category
"""

def __repr__(self):
@@ -51,8 +51,8 @@ class Category(Page):
"""Return a nice string representation of the Category."""
return '<Category "{0}" of {1}>'.format(self.title, str(self._site))

def _get_members_via_sql(self, limit):
"""Return a list of tuples of (title, pageid) in the category."""
def _get_members_via_sql(self, limit, follow):
"""Iterate over Pages in the category using SQL."""
query = """SELECT page_title, page_namespace, page_id FROM page
JOIN categorylinks ON page_id = cl_from
WHERE cl_to = ?"""
@@ -64,42 +64,66 @@ class Category(Page):
else:
result = self._site.sql_query(query, (title,))

members = []
for row in result:
members = list(result)
for row in members:
base = row[0].replace("_", " ").decode("utf8")
namespace = self._site.namespace_id_to_name(row[1])
if namespace:
title = u":".join((namespace, base))
else: # Avoid doing a silly (albeit valid) ":Pagename" thing
title = base
members.append((title, row[2]))
return members
yield self._site.get_page(title, follow_redirects=follow,
pageid=row[2])

def _get_members_via_api(self, limit):
"""Return a list of page titles in the category using the API."""
def _get_members_via_api(self, limit, follow):
"""Iterate over Pages in the category using the API."""
params = {"action": "query", "list": "categorymembers",
"cmlimit": limit, "cmtitle": self._title}
if not limit:
params["cmlimit"] = 50 # Default value

result = self._site.api_query(**params)
members = result['query']['categorymembers']
return [member["title"] for member in members]

def get_members(self, use_sql=False, limit=None):
"""Return a list of page titles in the category.
"cmtitle": self._title}

while 1:
params["cmlimit"] = limit if limit else "max"
result = self._site.api_query(**params)
for member in result["query"]["categorymembers"]:
title = member["title"]
yield self._site.get_page(title, follow_redirects=follow)

if "query-continue" in result:
qcontinue = result["query-continue"]["categorymembers"]
params["cmcontinue"] = qcontinue["cmcontinue"]
if limit:
limit -= len(result["query"]["categorymembers"])
else:
break

def get_members(self, use_sql=False, limit=None, follow_redirects=None):
"""Iterate over Pages in the category.

If *use_sql* is ``True``, we will use a SQL query instead of the API.
Pages will be returned as tuples of ``(title, pageid)`` instead of just
titles.

If *limit* is provided, we will provide this many titles, or less if
the category is smaller. It defaults to 50 for API queries; normal
users can go up to 500, and bots can go up to 5,000 on a single API
query. If we're using SQL, the limit is ``None`` by default (returning
all pages in the category), but an arbitrary limit can still be chosen.
Note that pages are retrieved from the API in chunks (by default, in
500-page chunks for normal users and 5000-page chunks for bots and
admins), so queries may be made as we go along. If *limit* is given, we
will provide this many pages, or less if the category is smaller. By
default, *limit* is ``None``, meaning we will keep iterating over
members until the category is exhausted. *follow_redirects* is passed
directly to :py:meth:`site.get_page()
<earwigbot.wiki.site.Site.get_page>`; it defaults to ``None``, which
will use the value passed to our :py:meth:`__init__`.

.. note::
Be careful when iterating over very large categories with no limit.
If using the API, at best, you will make one query per 5000 pages,
which can add up significantly for categories with hundreds of
thousands of members. As for SQL, note that *all page titles are
stored internally* as soon as the query is made, so the site-wide
SQL lock can be freed and unrelated queries can be made without
requiring a separate connection to be opened. This is generally not
an issue unless your category's size approaches several hundred
thousand, in which case the sheer number of titles in memory becomes
problematic.
"""
if follow_redirects is None:
follow_redirects = self._follow_redirects
if use_sql:
return self._get_members_via_sql(limit)
return self._get_members_via_sql(limit, follow_redirects)
else:
return self._get_members_via_api(limit)
return self._get_members_via_api(limit, follow_redirects)

+ 47
- 44
earwigbot/wiki/page.py View File

@@ -48,7 +48,7 @@ class Page(CopyrightMixIn):

- :py:attr:`site`: the page's corresponding Site object
- :py:attr:`title`: the page's title, or pagename
- :py:attr:`exists`: whether the page exists
- :py:attr:`exists`: whether or not the page exists
- :py:attr:`pageid`: an integer ID representing the page
- :py:attr:`url`: the page's URL
- :py:attr:`namespace`: the page's namespace as an integer
@@ -75,17 +75,20 @@ class Page(CopyrightMixIn):
checks the page like :py:meth:`copyvio_check`, but against a specific URL
"""

re_redirect = "^\s*\#\s*redirect\s*\[\[(.*?)\]\]"
PAGE_UNKNOWN = 0
PAGE_INVALID = 1
PAGE_MISSING = 2
PAGE_EXISTS = 3

def __init__(self, site, title, follow_redirects=False):
def __init__(self, site, title, follow_redirects=False, pageid=None):
"""Constructor for new Page instances.

Takes three arguments: a Site object, the Page's title (or pagename),
and whether or not to follow redirects (optional, defaults to False).
Takes four arguments: a Site object, the Page's title (or pagename),
whether or not to follow redirects (optional, defaults to False), and
a page ID to supplement the title (optional, defaults to None - i.e.,
we will have to query the API to get it).

As with User, site.get_page() is preferred. Site's method has support
for a default *follow_redirects* value in our config, while __init__()
always defaults to False.
As with User, site.get_page() is preferred.

__init__() will not do any API queries, but it will use basic namespace
logic to determine our namespace ID and if we are a talkpage.
@@ -94,9 +97,9 @@ class Page(CopyrightMixIn):
self._site = site
self._title = title.strip()
self._follow_redirects = self._keep_following = follow_redirects
self._pageid = pageid

self._exists = 0
self._pageid = None
self._exists = self.PAGE_UNKNOWN
self._is_redirect = None
self._lastrevid = None
self._protection = None
@@ -145,7 +148,7 @@ class Page(CopyrightMixIn):
Note that validity != existence. If a page's title is invalid (e.g, it
contains "[") it will always be invalid, and cannot be edited.
"""
if self._exists == 1:
if self._exists == self.PAGE_INVALID:
e = "Page '{0}' is invalid.".format(self._title)
raise exceptions.InvalidPageError(e)

@@ -157,7 +160,7 @@ class Page(CopyrightMixIn):
It will also call _assert_validity() beforehand.
"""
self._assert_validity()
if self._exists == 2:
if self._exists == self.PAGE_MISSING:
e = "Page '{0}' does not exist.".format(self._title)
raise exceptions.PageNotFoundError(e)

@@ -218,14 +221,14 @@ class Page(CopyrightMixIn):
if "missing" in res:
# If it has a negative ID and it's missing; we can still get
# data like the namespace, protection, and URL:
self._exists = 2
self._exists = self.PAGE_MISSING
else:
# If it has a negative ID and it's invalid, then break here,
# because there's no other data for us to get:
self._exists = 1
self._exists = self.PAGE_INVALID
return
else:
self._exists = 3
self._exists = self.PAGE_EXISTS

self._fullurl = res["fullurl"]
self._protection = res["protection"]
@@ -317,7 +320,7 @@ class Page(CopyrightMixIn):
if result["edit"]["result"] == "Success":
self._content = None
self._basetimestamp = None
self._exists = 0
self._exists = self.PAGE_UNKNOWN
return

# If we're here, then the edit failed. If it's because of AssertEdit,
@@ -351,7 +354,7 @@ class Page(CopyrightMixIn):
params["starttimestamp"] = self._starttimestamp
if self._basetimestamp:
params["basetimestamp"] = self._basetimestamp
if self._exists == 2:
if self._exists == self.PAGE_MISSING:
# Page does not exist; don't edit if it already exists:
params["createonly"] = "true"
else:
@@ -389,7 +392,7 @@ class Page(CopyrightMixIn):
# These attributes are now invalidated:
self._content = None
self._basetimestamp = None
self._exists = 0
self._exists = self.PAGE_UNKNOWN
raise exceptions.EditConflictError(error.info)

elif error.code in ["emptypage", "emptynewsection"]:
@@ -437,12 +440,12 @@ class Page(CopyrightMixIn):

@property
def site(self):
"""The Page's corresponding Site object."""
"""The page's corresponding Site object."""
return self._site

@property
def title(self):
"""The Page's title, or "pagename".
"""The page's title, or "pagename".

This won't do any API queries on its own. Any other attributes or
methods that do API queries will reload the title, however, like
@@ -453,37 +456,36 @@ class Page(CopyrightMixIn):

@property
def exists(self):
"""Information about whether the Page exists or not.
"""Whether or not the page exists.

The "information" is a tuple with two items. The first is a bool,
either ``True`` if the page exists or ``False`` if it does not. The
second is a string giving more information, either ``"invalid"``,
(title is invalid, e.g. it contains ``"["``), ``"missing"``, or
``"exists"``.
This will be a number; its value does not matter, but it will equal
one of :py:attr:`self.PAGE_INVALID <PAGE_INVALID>`,
:py:attr:`self.PAGE_MISSING <PAGE_MISSING>`, or
:py:attr:`self.PAGE_EXISTS <PAGE_EXISTS>`.

Makes an API query only if we haven't already made one.
"""
cases = {
0: (None, "unknown"),
1: (False, "invalid"),
2: (False, "missing"),
3: (True, "exists"),
}
if self._exists == 0:
if self._exists == self.PAGE_UNKNOWN:
self._load()
return cases[self._exists]
return self._exists

@property
def pageid(self):
"""An integer ID representing the Page.
"""An integer ID representing the page.

Makes an API query only if we haven't already made one.
Makes an API query only if we haven't already made one and the *pageid*
parameter to :py:meth:`__init__` was left as ``None``, which should be
true for all cases except when pages are returned by an SQL generator
(like :py:meth:`category.get_members(use_sql=True)
<earwigbot.wiki.category.Category.get_members>`).

Raises :py:exc:`~earwigbot.exceptions.InvalidPageError` or
:py:exc:`~earwigbot.exceptions.PageNotFoundError` if the page name is
invalid or the page does not exist, respectively.
"""
if self._exists == 0:
if self._pageid:
return self._pageid
if self._exists == self.PAGE_UNKNOWN:
self._load()
self._assert_existence() # Missing pages do not have IDs
return self._pageid
@@ -501,7 +503,7 @@ class Page(CopyrightMixIn):
else:
slug = quote(self._title.replace(" ", "_"), safe="/:")
path = self._site._article_path.replace("$1", slug)
return ''.join((self._site._base_url, path))
return ''.join((self._site.url, path))

@property
def namespace(self):
@@ -523,7 +525,7 @@ class Page(CopyrightMixIn):
name is invalid. Won't raise an error if the page is missing because
those can still be create-protected.
"""
if self._exists == 0:
if self._exists == self.PAGE_UNKNOWN:
self._load()
self._assert_validity() # Invalid pages cannot be protected
return self._protection
@@ -546,7 +548,7 @@ class Page(CopyrightMixIn):

We will return ``False`` even if the page does not exist or is invalid.
"""
if self._exists == 0:
if self._exists == self.PAGE_UNKNOWN:
self._load()
return self._is_redirect

@@ -611,7 +613,7 @@ class Page(CopyrightMixIn):
Raises InvalidPageError or PageNotFoundError if the page name is
invalid or the page does not exist, respectively.
"""
if self._exists == 0:
if self._exists == self.PAGE_UNKNOWN:
# Kill two birds with one stone by doing an API query for both our
# attributes and our page content:
query = self._site.api_query
@@ -626,7 +628,7 @@ class Page(CopyrightMixIn):
if self._keep_following and self._is_redirect:
self._title = self.get_redirect_target()
self._keep_following = False # Don't follow double redirects
self._exists = 0 # Force another API query
self._exists = self.PAGE_UNKNOWN # Force another API query
self.get()

return self._content
@@ -650,9 +652,10 @@ class Page(CopyrightMixIn):
:py:exc:`~earwigbot.exceptions.RedirectError` if the page is not a
redirect.
"""
re_redirect = "^\s*\#\s*redirect\s*\[\[(.*?)\]\]"
content = self.get()
try:
return re.findall(self.re_redirect, content, flags=re.I)[0]
return re.findall(re_redirect, content, flags=re.I)[0]
except IndexError:
e = "The page does not appear to have a redirect target."
raise exceptions.RedirectError(e)
@@ -671,7 +674,7 @@ class Page(CopyrightMixIn):
:py:exc:`~earwigbot.exceptions.PageNotFoundError` if the page name is
invalid or the page does not exist, respectively.
"""
if self._exists == 0:
if self._exists == self.PAGE_UNKNOWN:
self._load()
self._assert_existence()
if not self._creator:


+ 29
- 14
earwigbot/wiki/site.py View File

@@ -69,6 +69,7 @@ class Site(object):
- :py:attr:`project`: the site's project name, like ``"wikipedia"``
- :py:attr:`lang`: the site's language code, like ``"en"``
- :py:attr:`domain`: the site's web domain, like ``"en.wikipedia.org"``
- :py:attr:`url`: the site's URL, like ``"https://en.wikipedia.org"``

*Public methods:*

@@ -184,6 +185,12 @@ class Site(object):
res = "<Site {0} ({1}:{2}) at {3}>"
return res.format(self.name, self.project, self.lang, self.domain)

def _unicodeify(self, value, encoding="utf8"):
"""Return input as unicode if it's not unicode to begin with."""
if isinstance(value, unicode):
return value
return unicode(value, encoding)

def _urlencode_utf8(self, params):
"""Implement urllib.urlencode() with support for unicode input."""
enc = lambda s: s.encode("utf8") if isinstance(s, unicode) else str(s)
@@ -237,14 +244,7 @@ class Site(object):
e = "Tried to do an API query, but no API URL is known."
raise exceptions.SiteAPIError(e)

base_url = self._base_url
if base_url.startswith("//"): # Protocol-relative URLs from 1.18
if self._use_https:
base_url = "https:" + base_url
else:
base_url = "http:" + base_url
url = ''.join((base_url, self._script_path, "/api.php"))

url = ''.join((self.url, self._script_path, "/api.php"))
params["format"] = "json" # This is the only format we understand
if self._assert_edit: # If requested, ensure that we're logged in
params["assert"] = self._assert_edit
@@ -542,6 +542,17 @@ class Site(object):
"""The Site's web domain, like ``"en.wikipedia.org"``."""
return urlparse(self._base_url).netloc

@property
def url(self):
"""The Site's full base URL, like ``"https://en.wikipedia.org"``."""
url = self._base_url
if url.startswith("//"): # Protocol-relative URLs from 1.18
if self._use_https:
url = "https:" + url
else:
url = "http:" + url
return url

def api_query(self, **kwargs):
"""Do an API query with `kwargs` as the parameters.

@@ -682,7 +693,7 @@ class Site(object):
e = "There is no namespace with name '{0}'.".format(name)
raise exceptions.NamespaceNotFoundError(e)

def get_page(self, title, follow_redirects=False):
def get_page(self, title, follow_redirects=False, pageid=None):
"""Return a :py:class:`Page` object for the given title.

*follow_redirects* is passed directly to
@@ -696,23 +707,26 @@ class Site(object):
redirect-following: :py:class:`~earwigbot.wiki.page.Page`'s methods
provide that.
"""
title = self._unicodeify(title)
prefixes = self.namespace_id_to_name(constants.NS_CATEGORY, all=True)
prefix = title.split(":", 1)[0]
if prefix != title: # Avoid a page that is simply "Category"
if prefix in prefixes:
return Category(self, title, follow_redirects)
return Page(self, title, follow_redirects)
return Category(self, title, follow_redirects, pageid)
return Page(self, title, follow_redirects, pageid)

def get_category(self, catname, follow_redirects=False):
def get_category(self, catname, follow_redirects=False, pageid=None):
"""Return a :py:class:`Category` object for the given category name.

*catname* should be given *without* a namespace prefix. This method is
really just shorthand for :py:meth:`get_page("Category:" + catname)
<get_page>`.
"""
catname = self._unicodeify(catname)
name = name if isinstance(name, unicode) else name.decode("utf8")
prefix = self.namespace_id_to_name(constants.NS_CATEGORY)
pagename = ':'.join((prefix, catname))
return Category(self, pagename, follow_redirects)
pagename = u':'.join((prefix, catname))
return Category(self, pagename, follow_redirects, pageid)

def get_user(self, username=None):
"""Return a :py:class:`User` object for the given username.
@@ -721,6 +735,7 @@ class Site(object):
:py:class:`~earwigbot.wiki.user.User` object representing the currently
logged-in (or anonymous!) user is returned.
"""
username = self._unicodeify(username)
if not username:
username = self._get_username()
return User(self, username)

+ 6
- 0
earwigbot/wiki/user.py View File

@@ -39,6 +39,7 @@ class User(object):

*Attributes:*

- :py:attr:`site`: the user's corresponding Site object
- :py:attr:`name`: the user's username
- :py:attr:`exists`: ``True`` if the user exists, else ``False``
- :py:attr:`userid`: an integer ID representing the user
@@ -155,6 +156,11 @@ class User(object):
self._gender = res["gender"]

@property
def site(self):
"""The user's corresponding Site object."""
return self._site

@property
def name(self):
"""The user's username.



+ 1
- 0
setup.py View File

@@ -38,6 +38,7 @@ setup(
"oursql >= 0.9.3", # Talking with MediaWiki databases
"oauth2 >= 1.5.211", # Talking with Yahoo BOSS Search
"pycrypto >= 2.5", # Storing bot passwords and keys
"pytz >= 2012c", # Timezone handling
],
test_suite = "tests",
version = __version__,


Loading…
Cancel
Save