diff --git a/.gitignore b/.gitignore index edef0ae..4984243 100644 --- a/.gitignore +++ b/.gitignore @@ -1,19 +1,6 @@ -# Ignore python bytecode: *.pyc - -# Ignore bot-specific config file: -config.json - -# Ignore logs directory: -logs/ - -# Ignore cookies file: -.cookies - -# Ignore OS X's crud: +*.egg +*.egg-info .DS_Store - -# Ignore pydev's nonsense: -.project -.pydevproject -.settings/ +build +docs/_build diff --git a/README.md b/README.md deleted file mode 100644 index 3ecc02b..0000000 --- a/README.md +++ /dev/null @@ -1,39 +0,0 @@ -[EarwigBot](http://en.wikipedia.org/wiki/User:EarwigBot) is a -[Python](http://python.org/) robot that edits -[Wikipedia](http://en.wikipedia.org/) and interacts with people over -[IRC](http://en.wikipedia.org/wiki/Internet_Relay_Chat). - -# History - -Development began, based on the -[Pywikipedia framework](http://pywikipediabot.sourceforge.net/), in early 2009. -Approval for its fist task, a -[copyright violation detector](http://en.wikipedia.org/wiki/Wikipedia:Bots/Requests_for_approval/EarwigBot_1), -was carried out in May, and the bot has been running consistently ever since -(with the exception of Jan/Feb 2011). It currently handles -[several ongoing tasks](http://en.wikipedia.org/wiki/User:EarwigBot#Tasks), -ranging from statistics generation to category cleanup, and on-demand tasks -such as WikiProject template tagging. Since it started running, the bot has -made over 45,000 edits. - -A project to rewrite it from scratch began in early April 2011, thus moving -away from the Pywikipedia framework and allowing for less overall code, better -integration between bot parts, and easier maintenance. - -# Installation - -## Dependencies - -EarwigBot uses the MySQL library -[oursql](http://packages.python.org/oursql/) (>= 0.9.2) for communicating with -MediaWiki databases, and some tasks use their own tables for storage. -Additionally, the afc_history task uses -[matplotlib](http://matplotlib.sourceforge.net/) and -[numpy](http://numpy.scipy.org/) for graphing AfC statistics. Neither of these -modules are required for the main bot itself. - -`earwigbot.wiki.copyright` requires access to a search engine for detecting -copyright violations. Currently, -[Yahoo! BOSS](http://developer.yahoo.com/search/boss/) is the only engine -supported, and this requires -[oauth2](https://github.com/simplegeo/python-oauth2). diff --git a/README.rst b/README.rst new file mode 100644 index 0000000..4a9e3b0 --- /dev/null +++ b/README.rst @@ -0,0 +1,205 @@ +EarwigBot +========= + +EarwigBot_ is a Python_ robot that edits Wikipedia_ and interacts with people +over IRC_. This file provides a basic overview of how to install and setup the +bot; more detailed information is located in the ``docs/`` directory (available +online at PyPI_). + +History +------- + +Development began, based on the `Pywikipedia framework`_, in early 2009. +Approval for its fist task, a `copyright violation detector`_, was carried out +in May, and the bot has been running consistently ever since (with the +exception of Jan/Feb 2011). It currently handles `several ongoing tasks`_ +ranging from statistics generation to category cleanup, and on-demand tasks +such as WikiProject template tagging. Since it started running, the bot has +made over 50,000 edits. + +A project to rewrite it from scratch began in early April 2011, thus moving +away from the Pywikipedia framework and allowing for less overall code, better +integration between bot parts, and easier maintenance. + +Installation +------------ + +This package contains the core ``earwigbot``, abstracted enough that it should +be usable and customizable by anyone running a bot on a MediaWiki site. Since +it is component-based, the IRC components can be disabled if desired. IRC +commands and bot tasks specific to `my instance of EarwigBot`_ that I don't +feel the average user will need are available from the repository +`earwigbot-plugins`_. + +It's recommended to run the bot's unit tests before installing. Run ``python +setup.py test`` from the project's root directory. Note that some +tests require an internet connection, and others may take a while to run. +Coverage is currently rather incomplete. + +Latest release (v0.1) +~~~~~~~~~~~~~~~~~~~~~ + +EarwigBot is available from the `Python Package Index`_, so you can install the +latest release with ``pip install earwigbot`` (`get pip`_). + +You can also install it from source [1]_ directly:: + + curl -Lo earwigbot.tgz https://github.com/earwig/earwigbot/tarball/v0.1 + tar -xf earwigbot.tgz + cd earwig-earwigbot-* + python setup.py install + cd .. + rm -r earwigbot.tgz earwig-earwigbot-* + +Development version +~~~~~~~~~~~~~~~~~~~ + +You can install the development version of the bot from ``git`` by using +setuptools/distribute's ``develop`` command [1]_, probably on the ``develop`` +branch which contains (usually) working code. ``master`` contains the latest +release. EarwigBot uses `git flow`_, so you're free to +browse by tags or by new features (``feature/*`` branches):: + + git clone git://github.com/earwig/earwigbot.git earwigbot + cd earwigbot + python setup.py develop + +Setup +----- + +The bot stores its data in a "working directory", including its config file and +databases. This is also the location where you will place custom IRC commands +and bot tasks, which will be explained later. It doesn't matter where this +directory is, as long as the bot can write to it. + +Start the bot with ``earwigbot path/to/working/dir``, or just ``earwigbot`` if +the working directory is the current directory. It will notice that no +``config.yml`` file exists and take you through the setup process. + +There is currently no way to edit the ``config.yml`` file from within the bot +after it has been created, but YAML is a very straightforward format, so you +should be able to make any necessary changes yourself. Check out the +`explanation of YAML`_ on Wikipedia for help. + +After setup, the bot will start. This means it will connect to the IRC servers +it has been configured for, schedule bot tasks to run at specific times, and +then wait for instructions (as commands on IRC). For a list of commands, say +"``!help``" (commands are messages prefixed with an exclamation mark). + +You can stop the bot at any time with Control+C, same as you stop a normal +Python program, and it will try to exit safely. You can also use the +"``!quit``" command on IRC. + +Customizing +----------- + +The bot's working directory contains a ``commands`` subdirectory and a +``tasks`` subdirectory. Custom IRC commands can be placed in the former, +whereas custom wiki bot tasks go into the latter. Developing custom modules is +explained below, and in more detail through the bot's documentation on PyPI_ +(or in the ``docs/`` dir). + +Note that custom commands will override built-in commands and tasks with the +same name. + +``Bot`` and ``BotConfig`` +~~~~~~~~~~~~~~~~~~~~~~~~~ + +`earwigbot.bot.Bot`_ is EarwigBot's main class. You don't have to instantiate +this yourself, but it's good to be familiar with its attributes and methods, +because it is the main way to communicate with other parts of the bot. A +``Bot`` object is accessible as an attribute of commands and tasks (i.e., +``self.bot``). + +`earwigbot.config.BotConfig`_ stores configuration information for the bot. Its +docstring explains what each attribute is used for, but essentially each "node" +(one of ``config.components``, ``wiki``, ``irc``, ``commands``, ``tasks``, and +``metadata``) maps to a section of the bot's ``config.yml`` file. For example, +if ``config.yml`` includes something like:: + + irc: + frontend: + nick: MyAwesomeBot + channels: + - "##earwigbot" + - "#channel" + - "#other-channel" + +...then ``config.irc["frontend"]["nick"]`` will be ``"MyAwesomeBot"`` and +``config.irc["frontend"]["channels"]`` will be ``["##earwigbot", "#channel", +"#other-channel"]``. + +Custom IRC commands +~~~~~~~~~~~~~~~~~~~ + +Custom commands are subclasses of `earwigbot.commands.Command`_ that override +``Command``'s ``process()`` (and optionally ``check()`` or ``setup()``) +methods. + +The bot has a wide selection of built-in commands and plugins to act as sample +code and/or to give ideas. Start with test_, and then check out chanops_ and +afc_status_ for some more complicated scripts. + +Custom bot tasks +~~~~~~~~~~~~~~~~ + +Custom tasks are subclasses of `earwigbot.tasks.Task`_ that override ``Task``'s +``run()`` (and optionally ``setup()``) methods. + +See the built-in wikiproject_tagger_ task for a relatively straightforward +task, or the afc_statistics_ plugin for a more complicated one. + +The Wiki Toolset +---------------- + +EarwigBot's answer to the `Pywikipedia framework`_ is the Wiki Toolset +(``earwigbot.wiki``), which you will mainly access through ``bot.wiki``. + +``bot.wiki`` provides three methods for the management of Sites - +``get_site()``, ``add_site()``, and ``remove_site()``. Sites are objects that +simply represent a MediaWiki site. A single instance of EarwigBot (i.e. a +single *working directory*) is expected to relate to a single site or group of +sites using the same login info (like all WMF wikis with CentralAuth). + +Load your default site (the one that you picked during setup) with +``site = bot.wiki.get_site()``. + +Not all aspects of the toolset are covered in the docs. Explore `its code and +docstrings`_ to learn how to use it in a more hands-on fashion. For reference, +``bot.wiki`` is an instance of ``earwigbot.wiki.SitesDB`` tied to the +``sites.db`` file in the bot's working directory. + +Footnotes +--------- + +- Questions, comments, or suggestions about the documentation? `Let me know`_ + so I can improve it for other people. + +.. [1] ``python setup.py install``/``develop`` may require root, or use the + ``--user`` switch to install for the current user only. + +.. _EarwigBot: http://en.wikipedia.org/wiki/User:EarwigBot +.. _Python: http://python.org/ +.. _Wikipedia: http://en.wikipedia.org/ +.. _IRC: http://en.wikipedia.org/wiki/Internet_Relay_Chat +.. _PyPI: http://packages.python.org/earwigbot +.. _Pywikipedia framework: http://pywikipediabot.sourceforge.net/ +.. _copyright violation detector: http://en.wikipedia.org/wiki/Wikipedia:Bots/Requests_for_approval/EarwigBot_1 +.. _several ongoing tasks: http://en.wikipedia.org/wiki/User:EarwigBot#Tasks +.. _my instance of EarwigBot: http://en.wikipedia.org/wiki/User:EarwigBot +.. _earwigbot-plugins: https://github.com/earwig/earwigbot-plugins +.. _Python Package Index: http://pypi.python.org +.. _get pip: http://pypi.python.org/pypi/pip +.. _git flow: http://nvie.com/posts/a-successful-git-branching-model/ +.. _explanation of YAML: http://en.wikipedia.org/wiki/YAML +.. _earwigbot.bot.Bot: https://github.com/earwig/earwigbot/blob/develop/earwigbot/bot.py +.. _earwigbot.config.BotConfig: https://github.com/earwig/earwigbot/blob/develop/earwigbot/config.py +.. _earwigbot.commands.Command: https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/__init__.py +.. _test: https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/test.py +.. _chanops: https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/chanops.py +.. _afc_status: https://github.com/earwig/earwigbot-plugins/blob/develop/commands/afc_status.py +.. _earwigbot.tasks.Task: https://github.com/earwig/earwigbot/blob/develop/earwigbot/tasks/__init__.py +.. _wikiproject_tagger: https://github.com/earwig/earwigbot/blob/develop/earwigbot/tasks/wikiproject_tagger.py +.. _afc_statistics: https://github.com/earwig/earwigbot-plugins/blob/develop/tasks/afc_statistics.py +.. _its code and docstrings: https://github.com/earwig/earwigbot/tree/develop/earwigbot/wiki +.. _Let me know: ben.kurtovic@verizon.net diff --git a/bot.py b/bot.py deleted file mode 100755 index d8f2d21..0000000 --- a/bot.py +++ /dev/null @@ -1,70 +0,0 @@ -#! /usr/bin/env python -# -*- coding: utf-8 -*- -# -# Copyright (C) 2009-2012 by Ben Kurtovic -# -# Permission is hereby granted, free of charge, to any person obtaining a copy -# of this software and associated documentation files (the "Software"), to deal -# in the Software without restriction, including without limitation the rights -# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -# copies of the Software, and to permit persons to whom the Software is -# furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included in -# all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -# SOFTWARE. - -""" -EarwigBot - -This is a thin wrapper for EarwigBot's main bot code, specified by bot_script. -The wrapper will automatically restart the bot when it shuts down (from -!restart, for example). It requests the bot's password at startup and reuses it -every time the bot restarts internally, so you do not need to re-enter the -password after using !restart. - -For information about the bot as a whole, see the attached README.md file (in -markdown format!), the docs/ directory, and the LICENSE file for licensing -information. EarwigBot is released under the MIT license. -""" -from getpass import getpass -from subprocess import Popen, PIPE -from os import path -from sys import executable -from time import sleep - -import earwigbot - -bot_script = path.join(earwigbot.__path__[0], "runner.py") - -def main(): - print "EarwigBot v{0}\n".format(earwigbot.__version__) - - is_encrypted = earwigbot.config.config.load() - if is_encrypted: # Passwords in the config file are encrypted - key = getpass("Enter key to unencrypt bot passwords: ") - else: - key = None - - while 1: - bot = Popen([executable, bot_script], stdin=PIPE) - print >> bot.stdin, path.dirname(path.abspath(__file__)) - if is_encrypted: - print >> bot.stdin, key - return_code = bot.wait() - if return_code == 1: - exit() # Let critical exceptions in the subprocess cause us to - # exit as well - else: - sleep(5) # Sleep between bot runs following a non-critical - # subprocess exit - -if __name__ == "__main__": - main() diff --git a/docs/Makefile b/docs/Makefile new file mode 100644 index 0000000..80e5ec7 --- /dev/null +++ b/docs/Makefile @@ -0,0 +1,153 @@ +# Makefile for Sphinx documentation +# + +# You can set these variables from the command line. +SPHINXOPTS = +SPHINXBUILD = sphinx-build +PAPER = +BUILDDIR = _build + +# Internal variables. +PAPEROPT_a4 = -D latex_paper_size=a4 +PAPEROPT_letter = -D latex_paper_size=letter +ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . +# the i18n builder cannot share the environment and doctrees with the others +I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . + +.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext + +help: + @echo "Please use \`make ' where is one of" + @echo " html to make standalone HTML files" + @echo " dirhtml to make HTML files named index.html in directories" + @echo " singlehtml to make a single large HTML file" + @echo " pickle to make pickle files" + @echo " json to make JSON files" + @echo " htmlhelp to make HTML files and a HTML help project" + @echo " qthelp to make HTML files and a qthelp project" + @echo " devhelp to make HTML files and a Devhelp project" + @echo " epub to make an epub" + @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" + @echo " latexpdf to make LaTeX files and run them through pdflatex" + @echo " text to make text files" + @echo " man to make manual pages" + @echo " texinfo to make Texinfo files" + @echo " info to make Texinfo files and run them through makeinfo" + @echo " gettext to make PO message catalogs" + @echo " changes to make an overview of all changed/added/deprecated items" + @echo " linkcheck to check all external links for integrity" + @echo " doctest to run all doctests embedded in the documentation (if enabled)" + +clean: + -rm -rf $(BUILDDIR)/* + +html: + $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html + @echo + @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." + +dirhtml: + $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml + @echo + @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." + +singlehtml: + $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml + @echo + @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." + +pickle: + $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle + @echo + @echo "Build finished; now you can process the pickle files." + +json: + $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json + @echo + @echo "Build finished; now you can process the JSON files." + +htmlhelp: + $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp + @echo + @echo "Build finished; now you can run HTML Help Workshop with the" \ + ".hhp project file in $(BUILDDIR)/htmlhelp." + +qthelp: + $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp + @echo + @echo "Build finished; now you can run "qcollectiongenerator" with the" \ + ".qhcp project file in $(BUILDDIR)/qthelp, like this:" + @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/EarwigBot.qhcp" + @echo "To view the help file:" + @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/EarwigBot.qhc" + +devhelp: + $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp + @echo + @echo "Build finished." + @echo "To view the help file:" + @echo "# mkdir -p $$HOME/.local/share/devhelp/EarwigBot" + @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/EarwigBot" + @echo "# devhelp" + +epub: + $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub + @echo + @echo "Build finished. The epub file is in $(BUILDDIR)/epub." + +latex: + $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex + @echo + @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." + @echo "Run \`make' in that directory to run these through (pdf)latex" \ + "(use \`make latexpdf' here to do that automatically)." + +latexpdf: + $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex + @echo "Running LaTeX files through pdflatex..." + $(MAKE) -C $(BUILDDIR)/latex all-pdf + @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." + +text: + $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text + @echo + @echo "Build finished. The text files are in $(BUILDDIR)/text." + +man: + $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man + @echo + @echo "Build finished. The manual pages are in $(BUILDDIR)/man." + +texinfo: + $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo + @echo + @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." + @echo "Run \`make' in that directory to run these through makeinfo" \ + "(use \`make info' here to do that automatically)." + +info: + $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo + @echo "Running Texinfo files through makeinfo..." + make -C $(BUILDDIR)/texinfo info + @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." + +gettext: + $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale + @echo + @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." + +changes: + $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes + @echo + @echo "The overview file is in $(BUILDDIR)/changes." + +linkcheck: + $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck + @echo + @echo "Link check complete; look for any errors in the above output " \ + "or in $(BUILDDIR)/linkcheck/output.txt." + +doctest: + $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest + @echo "Testing of doctests in the sources finished, look at the " \ + "results in $(BUILDDIR)/doctest/output.txt." diff --git a/docs/api/earwigbot.commands.rst b/docs/api/earwigbot.commands.rst new file mode 100644 index 0000000..e2af424 --- /dev/null +++ b/docs/api/earwigbot.commands.rst @@ -0,0 +1,9 @@ +commands Package +================ + +:mod:`commands` Package +----------------------- + +.. automodule:: earwigbot.commands + :members: + :undoc-members: diff --git a/docs/api/earwigbot.irc.rst b/docs/api/earwigbot.irc.rst new file mode 100644 index 0000000..7503f18 --- /dev/null +++ b/docs/api/earwigbot.irc.rst @@ -0,0 +1,46 @@ +irc Package +=========== + +:mod:`irc` Package +------------------ + +.. automodule:: earwigbot.irc + :members: + :undoc-members: + +:mod:`connection` Module +------------------------ + +.. automodule:: earwigbot.irc.connection + :members: + :undoc-members: + +:mod:`data` Module +------------------ + +.. automodule:: earwigbot.irc.data + :members: + :undoc-members: + +:mod:`frontend` Module +---------------------- + +.. automodule:: earwigbot.irc.frontend + :members: + :undoc-members: + :show-inheritance: + +:mod:`rc` Module +---------------- + +.. automodule:: earwigbot.irc.rc + :members: + :undoc-members: + +:mod:`watcher` Module +--------------------- + +.. automodule:: earwigbot.irc.watcher + :members: + :undoc-members: + :show-inheritance: diff --git a/docs/api/earwigbot.rst b/docs/api/earwigbot.rst new file mode 100644 index 0000000..118a951 --- /dev/null +++ b/docs/api/earwigbot.rst @@ -0,0 +1,56 @@ +earwigbot Package +================= + +:mod:`earwigbot` Package +------------------------ + +.. automodule:: earwigbot.__init__ + :members: + :undoc-members: + +:mod:`bot` Module +----------------- + +.. automodule:: earwigbot.bot + :members: + :undoc-members: + +:mod:`config` Module +-------------------- + +.. automodule:: earwigbot.config + :members: + :undoc-members: + +:mod:`exceptions` Module +------------------------ + +.. automodule:: earwigbot.exceptions + :members: + :undoc-members: + :show-inheritance: + +:mod:`managers` Module +---------------------- + +.. automodule:: earwigbot.managers + :members: _ResourceManager, CommandManager, TaskManager + :undoc-members: + :show-inheritance: + +:mod:`util` Module +------------------ + +.. automodule:: earwigbot.util + :members: + :undoc-members: + +Subpackages +----------- + +.. toctree:: + + earwigbot.commands + earwigbot.irc + earwigbot.tasks + earwigbot.wiki diff --git a/docs/api/earwigbot.tasks.rst b/docs/api/earwigbot.tasks.rst new file mode 100644 index 0000000..58ac953 --- /dev/null +++ b/docs/api/earwigbot.tasks.rst @@ -0,0 +1,9 @@ +tasks Package +============= + +:mod:`tasks` Package +-------------------- + +.. automodule:: earwigbot.tasks + :members: + :undoc-members: diff --git a/docs/api/earwigbot.wiki.rst b/docs/api/earwigbot.wiki.rst new file mode 100644 index 0000000..806b3eb --- /dev/null +++ b/docs/api/earwigbot.wiki.rst @@ -0,0 +1,59 @@ +wiki Package +============ + +:mod:`wiki` Package +------------------- + +.. automodule:: earwigbot.wiki + :members: + :undoc-members: + +:mod:`category` Module +---------------------- + +.. automodule:: earwigbot.wiki.category + :members: + :undoc-members: + +:mod:`constants` Module +----------------------- + +.. automodule:: earwigbot.wiki.constants + :members: + :undoc-members: + +:mod:`copyright` Module +----------------------- + +.. automodule:: earwigbot.wiki.copyright + :members: + :undoc-members: + +:mod:`page` Module +------------------ + +.. automodule:: earwigbot.wiki.page + :members: + :undoc-members: + :show-inheritance: + +:mod:`site` Module +------------------ + +.. automodule:: earwigbot.wiki.site + :members: + :undoc-members: + +:mod:`sitesdb` Module +--------------------- + +.. automodule:: earwigbot.wiki.sitesdb + :members: + :undoc-members: + +:mod:`user` Module +------------------ + +.. automodule:: earwigbot.wiki.user + :members: + :undoc-members: diff --git a/docs/api/modules.rst b/docs/api/modules.rst new file mode 100644 index 0000000..7c4c110 --- /dev/null +++ b/docs/api/modules.rst @@ -0,0 +1,7 @@ +earwigbot +========= + +.. toctree:: + :maxdepth: 4 + + earwigbot diff --git a/docs/conf.py b/docs/conf.py new file mode 100644 index 0000000..92b2d74 --- /dev/null +++ b/docs/conf.py @@ -0,0 +1,242 @@ +# -*- coding: utf-8 -*- +# +# EarwigBot documentation build configuration file, created by +# sphinx-quickstart on Sun Apr 29 01:42:25 2012. +# +# This file is execfile()d with the current directory set to its containing dir. +# +# Note that not all possible configuration values are present in this +# autogenerated file. +# +# All configuration values have a default; values that are commented out +# serve to show the default. + +import sys, os + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +sys.path.insert(0, os.path.abspath('..')) + +# -- General configuration ----------------------------------------------------- + +# If your documentation needs a minimal Sphinx version, state it here. +#needs_sphinx = '1.0' + +# Add any Sphinx extension module names here, as strings. They can be extensions +# coming with Sphinx (named 'sphinx.ext.*') or your custom ones. +extensions = ['sphinx.ext.autodoc', 'sphinx.ext.coverage', 'sphinx.ext.viewcode'] + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# The suffix of source filenames. +source_suffix = '.rst' + +# The encoding of source files. +#source_encoding = 'utf-8-sig' + +# The master toctree document. +master_doc = 'index' + +# General information about the project. +project = u'EarwigBot' +copyright = u'2009, 2010, 2011, 2012 by Ben Kurtovic' + +# The version info for the project you're documenting, acts as replacement for +# |version| and |release|, also used in various other places throughout the +# built documents. +# +# The short X.Y version. +version = '0.1' +# The full version, including alpha/beta/rc tags. +release = '0.1.dev' + +# The language for content autogenerated by Sphinx. Refer to documentation +# for a list of supported languages. +#language = None + +# There are two options for replacing |today|: either, you set today to some +# non-false value, then it is used: +#today = '' +# Else, today_fmt is used as the format for a strftime call. +#today_fmt = '%B %d, %Y' + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +exclude_patterns = ['_build'] + +# The reST default role (used for this markup: `text`) to use for all documents. +#default_role = None + +# If true, '()' will be appended to :func: etc. cross-reference text. +#add_function_parentheses = True + +# If true, the current module name will be prepended to all description +# unit titles (such as .. function::). +#add_module_names = True + +# If true, sectionauthor and moduleauthor directives will be shown in the +# output. They are ignored by default. +#show_authors = False + +# The name of the Pygments (syntax highlighting) style to use. +pygments_style = 'sphinx' + +# A list of ignored prefixes for module index sorting. +#modindex_common_prefix = [] + + +# -- Options for HTML output --------------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +html_theme = 'nature' + +# Theme options are theme-specific and customize the look and feel of a theme +# further. For a list of options available for each theme, see the +# documentation. +#html_theme_options = {} + +# Add any paths that contain custom themes here, relative to this directory. +#html_theme_path = [] + +# The name for this set of Sphinx documents. If None, it defaults to +# " v documentation". +#html_title = None + +# A shorter title for the navigation bar. Default is the same as html_title. +#html_short_title = None + +# The name of an image file (relative to this directory) to place at the top +# of the sidebar. +#html_logo = None + +# The name of an image file (within the static path) to use as favicon of the +# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 +# pixels large. +#html_favicon = None + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +html_static_path = ['_static'] + +# If not '', a 'Last updated on:' timestamp is inserted at every page bottom, +# using the given strftime format. +#html_last_updated_fmt = '%b %d, %Y' + +# If true, SmartyPants will be used to convert quotes and dashes to +# typographically correct entities. +#html_use_smartypants = True + +# Custom sidebar templates, maps document names to template names. +#html_sidebars = {} + +# Additional templates that should be rendered to pages, maps page names to +# template names. +#html_additional_pages = {} + +# If false, no module index is generated. +#html_domain_indices = True + +# If false, no index is generated. +#html_use_index = True + +# If true, the index is split into individual pages for each letter. +#html_split_index = False + +# If true, links to the reST sources are added to the pages. +#html_show_sourcelink = True + +# If true, "Created using Sphinx" is shown in the HTML footer. Default is True. +#html_show_sphinx = True + +# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. +#html_show_copyright = True + +# If true, an OpenSearch description file will be output, and all pages will +# contain a tag referring to it. The value of this option must be the +# base URL from which the finished HTML is served. +#html_use_opensearch = '' + +# This is the file name suffix for HTML files (e.g. ".xhtml"). +#html_file_suffix = None + +# Output file base name for HTML help builder. +htmlhelp_basename = 'EarwigBotdoc' + + +# -- Options for LaTeX output -------------------------------------------------- + +latex_elements = { +# The paper size ('letterpaper' or 'a4paper'). +#'papersize': 'letterpaper', + +# The font size ('10pt', '11pt' or '12pt'). +#'pointsize': '10pt', + +# Additional stuff for the LaTeX preamble. +#'preamble': '', +} + +# Grouping the document tree into LaTeX files. List of tuples +# (source start file, target name, title, author, documentclass [howto/manual]). +latex_documents = [ + ('index', 'EarwigBot.tex', u'EarwigBot Documentation', + u'Ben Kurtovic', 'manual'), +] + +# The name of an image file (relative to this directory) to place at the top of +# the title page. +#latex_logo = None + +# For "manual" documents, if this is true, then toplevel headings are parts, +# not chapters. +#latex_use_parts = False + +# If true, show page references after internal links. +#latex_show_pagerefs = False + +# If true, show URL addresses after external links. +#latex_show_urls = False + +# Documents to append as an appendix to all manuals. +#latex_appendices = [] + +# If false, no module index is generated. +#latex_domain_indices = True + + +# -- Options for manual page output -------------------------------------------- + +# One entry per manual page. List of tuples +# (source start file, name, description, authors, manual section). +man_pages = [ + ('index', 'earwigbot', u'EarwigBot Documentation', + [u'Ben Kurtovic'], 1) +] + +# If true, show URL addresses after external links. +#man_show_urls = False + + +# -- Options for Texinfo output ------------------------------------------------ + +# Grouping the document tree into Texinfo files. List of tuples +# (source start file, target name, title, author, +# dir menu entry, description, category) +texinfo_documents = [ + ('index', 'EarwigBot', u'EarwigBot Documentation', + u'Ben Kurtovic', 'EarwigBot', 'One line description of project.', + 'Miscellaneous'), +] + +# Documents to append as an appendix to all manuals. +#texinfo_appendices = [] + +# If false, no module index is generated. +#texinfo_domain_indices = True + +# How to display URL addresses: 'footnote', 'no', or 'inline'. +#texinfo_show_urls = 'footnote' diff --git a/docs/customizing.rst b/docs/customizing.rst new file mode 100644 index 0000000..3336353 --- /dev/null +++ b/docs/customizing.rst @@ -0,0 +1,240 @@ +Customizing +=========== + +The bot's working directory contains a :file:`commands` subdirectory and a +:file:`tasks` subdirectory. Custom IRC commands can be placed in the former, +whereas custom wiki bot tasks go into the latter. Developing custom modules is +explained in detail in this documentation. + +Note that custom commands will override built-in commands and tasks with the +same name. + +:py:class:`~earwigbot.bot.Bot` and :py:class:`~earwigbot.bot.BotConfig` +----------------------------------------------------------------------- + +:py:class:`earwigbot.bot.Bot` is EarwigBot's main class. You don't have to +instantiate this yourself, but it's good to be familiar with its attributes and +methods, because it is the main way to communicate with other parts of the bot. +A :py:class:`~earwigbot.bot.Bot` object is accessible as an attribute of +commands and tasks (i.e., :py:attr:`self.bot`). + +The most useful attributes are: + +- :py:attr:`~earwigbot.bot.Bot.config`: an instance of + :py:class:`~earwigbot.config.BotConfig`, for accessing the bot's + configuration data (see below). + +- :py:attr:`~earwigbot.bot.Bot.commands`: the bot's + :py:class:`~earwigbot.managers.CommandManager`, which is used internally to + run IRC commands (through + :py:meth:`commands.call() `, which + you shouldn't have to use); you can safely reload all commands with + :py:meth:`commands.load() `. + +- :py:attr:`~earwigbot.bot.Bot.tasks`: the bot's + :py:class:`~earwigbot.managers.TaskManager`, which can be used to start tasks + with :py:meth:`tasks.start(task_name, **kwargs) + `. :py:meth:`tasks.load() + ` can be used to safely reload all + tasks. + +- :py:attr:`~earwigbot.bot.Bot.frontend` / + :py:attr:`~earwigbot.bot.Bot.watcher`: instances of + :py:class:`earwigbot.irc.Frontend ` and + :py:class:`earwigbot.irc.Watcher `, + respectively, which represent the bot's connections to these two servers; you + can, for example, send a message to the frontend with + :py:meth:`frontend.say(chan, msg) + ` (more on communicating with IRC + below). + +- :py:attr:`~earwigbot.bot.Bot.wiki`: interface with the + :doc:`Wiki Toolset `. + +- Finally, :py:meth:`~earwigbot.bot.Bot.restart` (restarts IRC components and + reloads config, commands, and tasks) and :py:meth:`~earwigbot.bot.Bot.stop` + can be used almost anywhere. Both take an optional "reason" that will be + logged and used as the quit message when disconnecting from IRC. + +:py:class:`earwigbot.config.BotConfig` stores configuration information for the +bot. Its docstrings explains what each attribute is used for, but essentially +each "node" (one of :py:attr:`config.components +`, +:py:attr:`~earwigbot.config.BotConfig.wiki`, +:py:attr:`~earwigbot.config.BotConfig.irc`, +:py:attr:`~earwigbot.config.BotConfig.commands`, +:py:attr:`~earwigbot.config.BotConfig.tasks`, or +:py:attr:`~earwigbot.config.BotConfig.metadata`) maps to a section +of the bot's :file:`config.yml` file. For example, if :file:`config.yml` +includes something like:: + + irc: + frontend: + nick: MyAwesomeBot + channels: + - "##earwigbot" + - "#channel" + - "#other-channel" + +...then :py:attr:`config.irc["frontend"]["nick"]` will be ``"MyAwesomeBot"`` +and :py:attr:`config.irc["frontend"]["channels"]` will be +``["##earwigbot", "#channel", "#other-channel"]``. + +Custom IRC commands +------------------- + +Custom commands are subclasses of :py:class:`earwigbot.commands.Command` that +override :py:class:`~earwigbot.commands.Command`'s +:py:meth:`~earwigbot.commands.Command.process` (and optionally +:py:meth:`~earwigbot.commands.Command.check` or +:py:meth:`~earwigbot.commands.Command.setup`) methods. + +:py:class:`~earwigbot.commands.Command`'s docstrings should explain what each +attribute and method is for and what they should be overridden with, but these +are the basics: + +- Class attribute :py:attr:`~earwigbot.commands.Command.name` is the name of + the command. This must be specified. + +- Class attribute :py:attr:`~earwigbot.commands.Command.commands` is a list of + names that will trigger this command. It defaults to the command's + :py:attr:`~earwigbot.commands.Command.name`, but you can override it with + multiple names to serve as aliases. This is handled by the default + :py:meth:`~earwigbot.commands.Command.check` implementation (see below), so + if :py:meth:`~earwigbot.commands.Command.check` is overridden, this is + ignored by everything except the help_ command (so ``!help alias`` will + trigger help for the actual command). + +- Class attribute :py:attr:`~earwigbot.commands.Command.hooks` is a list of the + "IRC events" that this command might respond to. It defaults to ``["msg"]``, + but options include ``"msg_private"`` (for private messages only), + ``"msg_public"`` (for channel messages only), and ``"join"`` (for when a user + joins a channel). See the afc_status_ plugin for a command that responds to + other hook types. + +- Method :py:meth:`~earwigbot.commands.Command.setup` is called *once* with no + arguments immediately after the command is first loaded. Does nothing by + default; treat it like an :py:meth:`__init__` if you want + (:py:meth:`~earwigbot.tasks.Command.__init__` does things by default and a + dedicated setup method is often easier than overriding + :py:meth:`~earwigbot.tasks.Command.__init__` and using :py:obj:`super`). + +- Method :py:meth:`~earwigbot.commands.Command.check` is passed a + :py:class:`~earwigbot.irc.data.Data` object, and should return ``True`` if + you want to respond to this message, or ``False`` otherwise. The default + behavior is to return ``True`` only if :py:attr:`data.is_command` is ``True`` + and :py:attr:`data.command` ``==`` + :py:attr:`~earwigbot.commands.Command.name` (or :py:attr:`data.command + ` is in + :py:attr:`~earwigbot.commands.Command.commands` if that list is overriden; + see above), which is suitable for most cases. A possible reason for + overriding is if you want to do something in response to events from a + specific channel only. Note that by returning ``True``, you prevent any other + commands from responding to this message. + +- Method :py:meth:`~earwigbot.commands.Command.process` is passed the same + :py:class:`~earwigbot.irc.data.Data` object as + :py:meth:`~earwigbot.commands.Command.check`, but only if + :py:meth:`~earwigbot.commands.Command.check` returned ``True``. This is where + the bulk of your command goes. To respond to IRC messages, there are a number + of methods of :py:class:`~earwigbot.commands.Command` at your disposal. See + the test_ command for a simple example, or look in + :py:class:`~earwigbot.commands.Command`'s + :py:meth:`~earwigbot.commands.Command.__init__` method for the full list. + + The most common ones are :py:meth:`say(chan_or_user, msg) + `, :py:meth:`reply(data, msg) + ` (convenience function; sends + a reply to the issuer of the command in the channel it was received), + :py:meth:`action(chan_or_user, msg) + `, + :py:meth:`notice(chan_or_user, msg) + `, :py:meth:`join(chan) + `, and + :py:meth:`part(chan) `. + +Commands have access to :py:attr:`config.commands[command_name]` for config +information, which is a node in :file:`config.yml` like every other attribute +of :py:attr:`bot.config`. This can be used to store, for example, API keys or +SQL connection info, so that these can be easily changed without modifying the +command itself. + +The command *class* doesn't need a specific name, but it should logically +follow the command's name. The filename doesn't matter, but it is recommended +to match the command name for readability. Multiple command classes are allowed +in one file. + +The bot has a wide selection of built-in commands and plugins to act as sample +code and/or to give ideas. Start with test_, and then check out chanops_ and +afc_status_ for some more complicated scripts. + +Custom bot tasks +---------------- + +Custom tasks are subclasses of :py:class:`earwigbot.tasks.Task` that +override :py:class:`~earwigbot.tasks.Task`'s +:py:meth:`~earwigbot.tasks.Task.run` (and optionally +:py:meth:`~earwigbot.tasks.Task.setup`) methods. + +:py:class:`~earwigbot.tasks.Task`'s docstrings should explain what each +attribute and method is for and what they should be overridden with, but these +are the basics: + +- Class attribute :py:attr:`~earwigbot.tasks.Task.name` is the name of the + task. This must be specified. + +- Class attribute :py:attr:`~earwigbot.tasks.Task.number` can be used to store + an optional "task number", possibly for use in edit summaries (to be + generated with :py:meth:`~earwigbot.tasks.Task.make_summary`). For + example, EarwigBot's :py:attr:`config.wiki["summary"]` is + ``"([[WP:BOT|Bot]]; [[User:EarwigBot#Task $1|Task $1]]): $2"``, which the + task class's :py:meth:`make_summary(comment) + ` method will take and replace + ``$1`` with the task number and ``$2`` with the details of the edit. + + Additionally, :py:meth:`~earwigbot.tasks.Task.shutoff_enabled` (which checks + whether the bot has been told to stop on-wiki by checking the content of a + particular page) can check a different page for each task using similar + variables. EarwigBot's :py:attr:`config.wiki["shutoff"]["page"]` is + ``"User:$1/Shutoff/Task $2"``; ``$1`` is substituted with the bot's username, + and ``$2`` is substituted with the task number, so, e.g., task #14 checks the + page ``[[User:EarwigBot/Shutoff/Task 14]].`` If the page's content does *not* + match :py:attr:`config.wiki["shutoff"]["disabled"]` (``"run"`` by default), + then shutoff is considered to be *enabled* and + :py:meth:`~earwigbot.tasks.Task.shutoff_enabled` will return ``True``, + indicating the task should not run. If you don't intend to use either of + these methods, feel free to leave this attribute blank. + +- Method :py:meth:`~earwigbot.tasks.Task.setup` is called *once* with no + arguments immediately after the task is first loaded. Does nothing by + default; treat it like an :py:meth:`__init__` if you want + (:py:meth:`~earwigbot.tasks.Task.__init__` does things by default and a + dedicated setup method is often easier than overriding + :py:meth:`~earwigbot.tasks.Task.__init__` and using :py:obj:`super`). + +- Method :py:meth:`~earwigbot.tasks.Task.run` is called with any number of + keyword arguments every time the task is executed (by + :py:meth:`tasks.start(task_name, **kwargs) + `, usually). This is where the bulk of + the task's code goes. For interfacing with MediaWiki sites, read up on the + :doc:`Wiki Toolset `. + +Tasks have access to :py:attr:`config.tasks[task_name]` for config information, +which is a node in :file:`config.yml` like every other attribute of +:py:attr:`bot.config`. This can be used to store, for example, edit summaries +or templates to append to user talk pages, so that these can be easily changed +without modifying the task itself. + +The task *class* doesn't need a specific name, but it should logically follow +the task's name. The filename doesn't matter, but it is recommended to match +the task name for readability. Multiple tasks classes are allowed in one file. + +See the built-in wikiproject_tagger_ task for a relatively straightforward +task, or the afc_statistics_ plugin for a more complicated one. + +.. _help: https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/help.py +.. _afc_status: https://github.com/earwig/earwigbot-plugins/blob/develop/commands/afc_status.py +.. _test: https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/test.py +.. _chanops: https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/chanops.py +.. _wikiproject_tagger: https://github.com/earwig/earwigbot/blob/develop/earwigbot/tasks/wikiproject_tagger.py +.. _afc_statistics: https://github.com/earwig/earwigbot-plugins/blob/develop/tasks/afc_statistics.py diff --git a/docs/index.rst b/docs/index.rst new file mode 100644 index 0000000..8d446dc --- /dev/null +++ b/docs/index.rst @@ -0,0 +1,48 @@ +EarwigBot v0.1 Documentation +============================ + +EarwigBot_ is a Python_ robot that edits Wikipedia_ and interacts with people +over IRC_. + +History +------- + +Development began, based on the `Pywikipedia framework`_, in early 2009. +Approval for its fist task, a `copyright violation detector`_, was carried out +in May, and the bot has been running consistently ever since (with the +exception of Jan/Feb 2011). It currently handles `several ongoing tasks`_ +ranging from statistics generation to category cleanup, and on-demand tasks +such as WikiProject template tagging. Since it started running, the bot has +made over 50,000 edits. + +A project to rewrite it from scratch began in early April 2011, thus moving +away from the Pywikipedia framework and allowing for less overall code, better +integration between bot parts, and easier maintenance. + +.. _EarwigBot: http://en.wikipedia.org/wiki/User:EarwigBot +.. _Python: http://python.org/ +.. _Wikipedia: http://en.wikipedia.org/ +.. _IRC: http://en.wikipedia.org/wiki/Internet_Relay_Chat +.. _Pywikipedia framework: http://pywikipediabot.sourceforge.net/ +.. _copyright violation detector: http://en.wikipedia.org/wiki/Wikipedia:Bots/Requests_for_approval/EarwigBot_1 +.. _several ongoing tasks: http://en.wikipedia.org/wiki/User:EarwigBot#Tasks + +Contents +-------- + +.. toctree:: + :maxdepth: 2 + + installation + setup + customizing + toolset + tips + API Reference + +Indices and tables +------------------ + +* :ref:`genindex` +* :ref:`modindex` +* :ref:`search` diff --git a/docs/installation.rst b/docs/installation.rst new file mode 100644 index 0000000..12fc907 --- /dev/null +++ b/docs/installation.rst @@ -0,0 +1,55 @@ +Installation +============ + +This package contains the core :py:mod:`earwigbot`, abstracted enough that it +should be usable and customizable by anyone running a bot on a MediaWiki site. +Since it is component-based, the IRC components can be disabled if desired. IRC +commands and bot tasks specific to `my instance of EarwigBot`_ that I don't +feel the average user will need are available from the repository +`earwigbot-plugins`_. + +It's recommended to run the bot's unit tests before installing. Run +:command:`python setup.py test` from the project's root directory. Note that +some tests require an internet connection, and others may take a while to run. +Coverage is currently rather incomplete. + +Latest release (v0.1) +--------------------- + +EarwigBot is available from the `Python Package Index`_, so you can install the +latest release with :command:`pip install earwigbot` (`get pip`_). + +You can also install it from source [1]_ directly:: + + curl -Lo earwigbot.tgz https://github.com/earwig/earwigbot/tarball/v0.1 + tar -xf earwigbot.tgz + cd earwig-earwigbot-* + python setup.py install + cd .. + rm -r earwigbot.tgz earwig-earwigbot-* + +Development version +------------------- + +You can install the development version of the bot from :command:`git` by using +setuptools/`distribute`_'s :command:`develop` command [1]_, probably on the +``develop`` branch which contains (usually) working code. ``master`` contains +the latest release. EarwigBot uses `git flow`_, so you're free to browse by +tags or by new features (``feature/*`` branches):: + + git clone git://github.com/earwig/earwigbot.git earwigbot + cd earwigbot + python setup.py develop + +.. rubric:: Footnotes + +.. [1] :command:`python setup.py install`/:command:`develop` may require root, + or use the :command:`--user` switch to install for the current user + only. + +.. _my instance of EarwigBot: http://en.wikipedia.org/wiki/User:EarwigBot +.. _earwigbot-plugins: https://github.com/earwig/earwigbot-plugins +.. _Python Package Index: http://pypi.python.org +.. _get pip: http://pypi.python.org/pypi/pip +.. _distribute: http://pypi.python.org/pypi/distribute +.. _git flow: http://nvie.com/posts/a-successful-git-branching-model/ diff --git a/docs/setup.rst b/docs/setup.rst new file mode 100644 index 0000000..81523df --- /dev/null +++ b/docs/setup.rst @@ -0,0 +1,28 @@ +Setup +===== + +The bot stores its data in a "working directory", including its config file and +databases. This is also the location where you will place custom IRC commands +and bot tasks, which will be explained later. It doesn't matter where this +directory is, as long as the bot can write to it. + +Start the bot with :command:`earwigbot path/to/working/dir`, or just +:command:`earwigbot` if the working directory is the current directory. It will +notice that no :file:`config.yml` file exists and take you through the setup +process. + +There is currently no way to edit the :file:`config.yml` file from within the +bot after it has been created, but YAML is a very straightforward format, so +you should be able to make any necessary changes yourself. Check out the +`explanation of YAML`_ on Wikipedia for help. + +After setup, the bot will start. This means it will connect to the IRC servers +it has been configured for, schedule bot tasks to run at specific times, and +then wait for instructions (as commands on IRC). For a list of commands, say +"``!help``" (commands are messages prefixed with an exclamation mark). + +You can stop the bot at any time with :kbd:`Control-c`, same as you stop a +normal Python program, and it will try to exit safely. You can also use the +"``!quit``" command on IRC. + +.. _explanation of YAML: http://en.wikipedia.org/wiki/YAML diff --git a/docs/tips.rst b/docs/tips.rst new file mode 100644 index 0000000..4f4052e --- /dev/null +++ b/docs/tips.rst @@ -0,0 +1,46 @@ +Tips +==== + +- Logging_ is a fantastic way to monitor the bot's progress as it runs. It has + a slew of built-in loggers, and enabling log retention (so logs are saved to + :file:`logs/` in the working directory) is highly recommended. In the normal + setup, there are three log files, each of which "rotate" at a specific time + (:file:`filename.log` becomes :file:`filename.log.2012-04-10`, for example). + The :file:`debug.log` file rotates every hour, and maintains six hours of + logs of every level (``DEBUG`` and up). :file:`bot.log` rotates every day at + midnight, and maintains seven days of non-debug logs (``INFO`` and up). + Finally, :file:`error.log` rotates every Sunday night, and maintains four + weeks of logs indicating unexpected events (``WARNING`` and up). + + To use logging in your commands or tasks (recommended), + :py:class:~earwigbot.commands.BaseCommand` and + :py:class:~earwigbot.tasks.BaseTask` provide :py:attr:`logger` attributes + configured for the specific command or task. If you're working with other + classes, :py:attr:`bot.logger` is the root logger + (:py:obj:`logging.getLogger("earwigbot")` by default), so you can use + :py:func:`~logging.Logger.getChild` to make your logger. For example, task + loggers are essentially + :py:attr:`bot.logger.getChild("tasks").getChild(task.name) `. + +- A very useful IRC command is "``!reload``", which reloads all commands and + tasks without restarting the bot. [1]_ Combined with using the `!git plugin`_ + for pulling repositories from IRC, this can provide a seamless command/task + development workflow if the bot runs on an external server and you set up + its working directory as a git repo. + +- You can run a task by itself instead of the entire bot with + :command:`earwigbot path/to/working/dir --task task_name`. + +- Questions, comments, or suggestions about the documentation? `Let me know`_, + or `create an issue`_ so I can improve it for other people. + +.. rubric:: Footnotes + +.. [1] In reality, all this does is call :py:meth:`bot.commands.load() + ` and + :py:meth:`bot.tasks.load() `! + +.. _logging: http://docs.python.org/library/logging.html +.. _!git plugin: https://github.com/earwig/earwigbot-plugins/blob/develop/commands/git.py +.. _Let me know: ben.kurtovic@verizon.net +.. _create an issue: https://github.com/earwig/earwigbot/issues diff --git a/docs/toolset.rst b/docs/toolset.rst new file mode 100644 index 0000000..c7808d2 --- /dev/null +++ b/docs/toolset.rst @@ -0,0 +1,244 @@ +The Wiki Toolset +================ + +EarwigBot's answer to the `Pywikipedia framework`_ is the Wiki Toolset +(:py:mod:`earwigbot.wiki`), which you will mainly access through +:py:attr:`bot.wiki `. + +:py:attr:`bot.wiki ` provides three methods for the +management of Sites - :py:meth:`~earwigbot.wiki.sitesdb.SitesDB.get_site`, +:py:meth:`~earwigbot.wiki.sitesdb.SitesDB.add_site`, and +:py:meth:`~earwigbot.wiki.sitesdb.SitesDB.remove_site`. Sites are objects that +simply represent a MediaWiki site. A single instance of EarwigBot (i.e. a +single *working directory*) is expected to relate to a single site or group of +sites using the same login info (like all WMF wikis with `CentralAuth`_). + +Load your default site (the one that you picked during setup) with +``site = bot.wiki.get_site()``. + +Dealing with other sites +~~~~~~~~~~~~~~~~~~~~~~~~ + +*Skip this section if you're only working with one site.* + +If a site is *already known to the bot* (meaning that it is stored in the +:file:`sites.db` file, which includes just your default wiki at first), you can +load a site with ``site = bot.wiki.get_site(name)``, where ``name`` might be +``"enwiki"`` or ``"frwiktionary"`` (you can also do +``site = bot.wiki.get_site(project="wikipedia", lang="en")``). Recall that not +giving any arguments to ``get_site()`` will return the default site. + +:py:meth:`~earwigbot.wiki.sitesdb.SitesDB.add_site` is used to add new sites to +the sites database. It may be called with similar arguments as +:py:meth:`~earwigbot.wiki.sitesdb.SitesDB.get_site`, but the difference is +important. :py:meth:`~earwigbot.wiki.sitesdb.SitesDB.get_site` only needs +enough information to identify the site in its database, which is usually just +its name; the database stores all other necessary connection info. With +:py:meth:`~earwigbot.wiki.sitesdb.SitesDB.add_site`, you need to provide enough +connection info so the toolset can successfully access the site's API/SQL +databases and store that information for later. That might not be much; for WMF +wikis, you can usually use code like this:: + + project, lang = "wikipedia", "es" + try: + site = bot.wiki.get_site(project=project, lang=lang) + except earwigbot.SiteNotFoundError: + # Load site info from http://es.wikipedia.org/w/api.php: + site = bot.wiki.add_site(project=project, lang=lang) + +This works because EarwigBot assumes that the URL for the site is +``"//{lang}.{project}.org"`` and the API is at ``/w/api.php``; this might +change if you're dealing with non-WMF wikis, where the code might look +something more like:: + + project, lang = "mywiki", "it" + try: + site = bot.wiki.get_site(project=project, lang=lang) + except earwigbot.SiteNotFoundError: + # Load site info from http://mysite.net/mywiki/it/s/api.php: + base_url = "http://mysite.net/" + project + "/" + lang + db_name = lang + project + "_p" + sql = {host: "sql.mysite.net", db: db_name} + site = bot.wiki.add_site(base_url=base_url, script_path="/s", sql=sql) + +:py:meth:`~earwigbot.wiki.sitesdb.SitesDB.remove_site` does the opposite of +:py:meth:`~earwigbot.wiki.sitesdb.SitesDB.add_site`: give it a site's name or a +project/lang pair like :py:meth:`~earwigbot.wiki.sitesdb.SitesDB.get_site` +takes, and it'll remove that site from the sites database. + +Sites +~~~~~ + +:py:class:`earwigbot.wiki.Site ` objects provide the +following attributes: + +- :py:attr:`~earwigbot.wiki.site.Site.name`: the site's name (or "wikiid"), + like ``"enwiki"`` +- :py:attr:`~earwigbot.wiki.site.Site.project`: the site's project name, like + ``"wikipedia"`` +- :py:attr:`~earwigbot.wiki.site.Site.lang`: the site's language code, like + ``"en"`` +- :py:attr:`~earwigbot.wiki.site.Site.domain`: the site's web domain, like + ``"en.wikipedia.org"`` +- :py:attr:`~earwigbot.wiki.site.Site.url`: the site's full base URL, like + ``"https://en.wikipedia.org"`` + +and the following methods: + +- :py:meth:`api_query(**kwargs) `: does an + API query with the given keyword arguments as params +- :py:meth:`sql_query(query, params=(), ...) + `: does an SQL query and yields its + results (as a generator) +- :py:meth:`~earwigbot.wiki.site.Site.get_replag`: returns the estimated + database replication lag (if we have the site's SQL connection info) +- :py:meth:`namespace_id_to_name(id, all=False) + `: given a namespace ID, + returns the primary associated namespace name (or a list of all names when + ``all`` is ``True``) +- :py:meth:`namespace_name_to_id(name) + `: given a namespace name, + returns the associated namespace ID +- :py:meth:`get_page(title, follow_redirects=False, ...) + `: returns a ``Page`` object for the given + title (or a :py:class:`~earwigbot.wiki.category.Category` object if the + page's namespace is "``Category:``") +- :py:meth:`get_category(catname, follow_redirects=False, ...) + `: returns a ``Category`` object for + the given title (sans namespace) +- :py:meth:`get_user(username) `: returns a + :py:class:`~earwigbot.wiki.user.User` object for the given username +- :py:meth:`delegate(services, ...) `: + delegates a task to either the API or SQL depending on various conditions, + such as server lag + +Pages and categories +~~~~~~~~~~~~~~~~~~~~ + +Create :py:class:`earwigbot.wiki.Page ` objects with +:py:meth:`site.get_page(title) `, +:py:meth:`page.toggle_talk() `, +:py:meth:`user.get_userpage() `, or +:py:meth:`user.get_talkpage() `. They +provide the following attributes: + +- :py:attr:`~earwigbot.wiki.page.Page.site`: the page's corresponding + :py:class:`~earwigbot.wiki.site.Site` object +- :py:attr:`~earwigbot.wiki.page.Page.title`: the page's title, or pagename +- :py:attr:`~earwigbot.wiki.page.Page.exists`: whether or not the page exists +- :py:attr:`~earwigbot.wiki.page.Page.pageid`: an integer ID representing the + page +- :py:attr:`~earwigbot.wiki.page.Page.url`: the page's URL +- :py:attr:`~earwigbot.wiki.page.Page.namespace`: the page's namespace as an + integer +- :py:attr:`~earwigbot.wiki.page.Page.protection`: the page's current + protection status +- :py:attr:`~earwigbot.wiki.page.Page.is_talkpage`: ``True`` if the page is a + talkpage, else ``False`` +- :py:attr:`~earwigbot.wiki.page.Page.is_redirect`: ``True`` if the page is a + redirect, else ``False`` + +and the following methods: + +- :py:meth:`~earwigbot.wiki.page.Page.reload`: forcibly reloads the page's + attributes (emphasis on *reload* - this is only necessary if there is reason + to believe they have changed) +- :py:meth:`toggle_talk(...) `: returns a + content page's talk page, or vice versa +- :py:meth:`~earwigbot.wiki.page.Page.get`: returns page content +- :py:meth:`~earwigbot.wiki.page.Page.get_redirect_target`: if the page is a + redirect, returns its destination +- :py:meth:`~earwigbot.wiki.page.Page.get_creator`: returns a + :py:class:`~earwigbot.wiki.user.User` object representing the first user to + edit the page +- :py:meth:`edit(text, summary, minor=False, bot=True, force=False) + `: replaces the page's content with ``text`` + or creates a new page +- :py:meth:`add_section(text, title, minor=False, bot=True, force=False) + `: adds a new section named ``title`` + at the bottom of the page +- :py:meth:`copyvio_check(...) + `: checks the page for + copyright violations +- :py:meth:`copyvio_compare(url, ...) + `: checks the page like + :py:meth:`~earwigbot.wiki.copyvios.CopyvioMixIn.copyvio_check`, but + against a specific URL +- :py:meth:`check_exclusion(username=None, optouts=None) + `: checks whether or not we are + allowed to edit the page per ``{{bots}}``/``{{nobots}}`` + +Additionally, :py:class:`~earwigbot.wiki.category.Category` objects (created +with :py:meth:`site.get_category(name) ` +or :py:meth:`site.get_page(title) ` where +``title`` is in the ``Category:`` namespace) provide the following additional +attributes: + +- :py:attr:`~earwigbot.wiki.category.Category.size`: the total number of + members in the category +- :py:attr:`~earwigbot.wiki.category.Category.pages`: the number of pages in + the category +- :py:attr:`~earwigbot.wiki.category.Category.files`: the number of files in + the category +- :py:attr:`~earwigbot.wiki.category.Category.subcats`: the number of + subcategories in the category + +And the following additional method: + +- :py:meth:`get_members(limit=None, ...) + `: iterates over + :py:class:`~earwigbot.wiki.page.Page`\ s in the category, until either the + category is exhausted or (if given) ``limit`` is reached + +Users +~~~~~ + +Create :py:class:`earwigbot.wiki.User ` objects with +:py:meth:`site.get_user(name) ` or +:py:meth:`page.get_creator() `. They +provide the following attributes: + +- :py:attr:`~earwigbot.wiki.user.User.site`: the user's corresponding + :py:class:`~earwigbot.wiki.site.Site` object +- :py:attr:`~earwigbot.wiki.user.User.name`: the user's username +- :py:attr:`~earwigbot.wiki.user.User.exists`: ``True`` if the user exists, or + ``False`` if they do not +- :py:attr:`~earwigbot.wiki.user.User.userid`: an integer ID representing the + user +- :py:attr:`~earwigbot.wiki.user.User.blockinfo`: information about any current + blocks on the user (``False`` if no block, or a dict of + ``{"by": blocking_user, "reason": block_reason, + "expiry": block_expire_time}``) +- :py:attr:`~earwigbot.wiki.user.User.groups`: a list of the user's groups +- :py:attr:`~earwigbot.wiki.user.User.rights`: a list of the user's rights +- :py:attr:`~earwigbot.wiki.user.User.editcount`: the number of edits made by + the user +- :py:attr:`~earwigbot.wiki.user.User.registration`: the time the user + registered as a :py:obj:`time.struct_time` +- :py:attr:`~earwigbot.wiki.user.User.emailable`: ``True`` if you can email the + user, ``False`` if you cannot +- :py:attr:`~earwigbot.wiki.user.User.gender`: the user's gender (``"male"``, + ``"female"``, or ``"unknown"``) + +and the following methods: + +- :py:meth:`~earwigbot.wiki.user.User.reload`: forcibly reloads the user's + attributes (emphasis on *reload* - this is only necessary if there is reason + to believe they have changed) +- :py:meth:`~earwigbot.wiki.user.User.get_userpage`: returns a + :py:class:`~earwigbot.wiki.page.Page` object representing the user's userpage +- :py:meth:`~earwigbot.wiki.user.User.get_talkpage`: returns a + :py:class:`~earwigbot.wiki.page.Page` object representing the user's talkpage + +Additional features +~~~~~~~~~~~~~~~~~~~ + +Not all aspects of the toolset are covered here. Explore `its code and +docstrings`_ to learn how to use it in a more hands-on fashion. For reference, +:py:attr:`bot.wiki ` is an instance of +:py:class:`earwigbot.wiki.SitesDB ` tied to the +:file:`sites.db` file in the bot's working directory. + +.. _Pywikipedia framework: http://pywikipediabot.sourceforge.net/ +.. _CentralAuth: http://www.mediawiki.org/wiki/Extension:CentralAuth +.. _its code and docstrings: https://github.com/earwig/earwigbot/tree/develop/earwigbot/wiki diff --git a/earwigbot/__init__.py b/earwigbot/__init__.py index 0da7679..43f4daf 100644 --- a/earwigbot/__init__.py +++ b/earwigbot/__init__.py @@ -1,17 +1,17 @@ # -*- coding: utf-8 -*- -# +# # Copyright (C) 2009-2012 by Ben Kurtovic -# +# # Permission is hereby granted, free of charge, to any person obtaining a copy # of this software and associated documentation files (the "Software"), to deal # in the Software without restriction, including without limitation the rights # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -# copies of the Software, and to permit persons to whom the Software is +# copies of the Software, and to permit persons to whom the Software is # furnished to do so, subject to the following conditions: -# +# # The above copyright notice and this permission notice shall be included in # all copies or substantial portions of the Software. -# +# # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE @@ -21,17 +21,42 @@ # SOFTWARE. """ -EarwigBot - http://earwig.github.com/earwig/earwigbot -See README.md for a basic overview, or the docs/ directory for details. +`EarwigBot `_ is a Python robot that edits +Wikipedia and interacts with people over IRC. + +See :file:`README.rst` for an overview, or the :file:`docs/` directory for +details. This documentation is also available `online +`_. """ __author__ = "Ben Kurtovic" -__copyright__ = "Copyright (C) 2009, 2010, 2011 by Ben Kurtovic" +__copyright__ = "Copyright (C) 2009, 2010, 2011, 2012 by Ben Kurtovic" __license__ = "MIT License" __version__ = "0.1.dev" __email__ = "ben.kurtovic@verizon.net" +__release__ = False + +if not __release__: + def _get_git_commit_id(): + """Return the ID of the git HEAD commit.""" + from git import Repo + from os.path import split, dirname + path = split(dirname(__file__))[0] + commit_id = Repo(path).head.object.hexsha + return commit_id[:8] + try: + __version__ += ".git+" + _get_git_commit_id() + except Exception: + pass + finally: + del _get_git_commit_id -from earwigbot import ( - blowfish, config, classes, commands, config, frontend, main, rules, tasks, - tests, watcher, wiki -) +from earwigbot import bot +from earwigbot import commands +from earwigbot import config +from earwigbot import exceptions +from earwigbot import irc +from earwigbot import managers +from earwigbot import tasks +from earwigbot import util +from earwigbot import wiki diff --git a/earwigbot/blowfish.py b/earwigbot/blowfish.py deleted file mode 100755 index 4df8c75..0000000 --- a/earwigbot/blowfish.py +++ /dev/null @@ -1,556 +0,0 @@ -#! /usr/bin/env python -# -*- coding: utf-8 -*- -# -# blowfish.py -# Copyright (C) 2002 Michael Gilfix -# Copyright (C) 2011, 2012 Ben Kurtovic -# -# This module is open source; you can redistribute it and/or -# modify it under the terms of the GPL or Artistic License. -# These licenses are available at http://www.opensource.org -# -# This software must be used and distributed in accordance -# with the law. The author claims no liability for its -# misuse. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. -# - -""" -Blowfish Encryption - -This module is a pure python implementation of Bruce Schneier's encryption -scheme 'Blowfish'. Blowish is a 16-round Feistel Network cipher and offers -substantial speed gains over DES. - -The key is a string of length anywhere between 64 and 448 bits, or equivalently -8 and 56 bytes. The encryption and decryption functions operate on 64-bit -blocks, or 8 byte strings. - -The entire Blowfish() class (excluding verify_key()) is by Michael Gilfix -. - -Blowfish.verify_key(), exception classes, encrypt() and decrypt() wrappers, and -interactive mode are by Ben Kurtovic . -""" - -class BlowfishError(Exception): - """Base exception class for errors involving blowfish - encryption/decryption.""" - -class BlockSizeError(BlowfishError): - """Attempted to handle a block not 8 bytes in length.""" - -class KeyLengthError(BlowfishError): - """Attempted to use a key that is either less than 8 bytes or more than 56 - bytes in length.""" - -class DecryptionError(BlowfishError): - """Attempted to decrypt malformed cyphertext (e.g., not evenly divisible - into 8-byte blocks) or attempted to decrypt using a bad key.""" - -class Blowfish(object): - """Blowfish encryption Scheme - - This class implements the encryption and decryption - functionality of the Blowfish cipher. - - Public functions: - - def __init__ (self, key) - Creates an instance of blowfish using 'key' - as the encryption key. Key is a string of - length ranging from 8 to 56 bytes (64 to 448 - bits). Once the instance of the object is - created, the key is no longer necessary. - - def encrypt (self, data): - Encrypt an 8 byte (64-bit) block of text - where 'data' is an 8 byte string. Returns an - 8-byte encrypted string. - - def decrypt (self, data): - Decrypt an 8 byte (64-bit) encrypted block - of text, where 'data' is the 8 byte encrypted - string. Returns an 8-byte string of plaintext. - - def cipher (self, xl, xr, direction): - Encrypts a 64-bit block of data where xl is - the upper 32-bits and xr is the lower 32-bits. - 'direction' is the direction to apply the - cipher, either ENCRYPT or DECRYPT constants. - returns a tuple of either encrypted or decrypted - data of the left half and right half of the - 64-bit block. - - Private members: - - def __round_func (self, xl) - Performs an obscuring function on the 32-bit - block of data 'xl', which is the left half of - the 64-bit block of data. Returns the 32-bit - result as a long integer. - """ - - # Cipher directions - ENCRYPT = 0 - DECRYPT = 1 - - # For the __round_func - modulus = long (2) ** 32 - - def __init__ (self, key): - self.verify_key(key) - - self.p_boxes = [ - 0x243F6A88, 0x85A308D3, 0x13198A2E, 0x03707344, - 0xA4093822, 0x299F31D0, 0x082EFA98, 0xEC4E6C89, - 0x452821E6, 0x38D01377, 0xBE5466CF, 0x34E90C6C, - 0xC0AC29B7, 0xC97C50DD, 0x3F84D5B5, 0xB5470917, - 0x9216D5D9, 0x8979FB1B - ] - - self.s_boxes = [ - [ - 0xD1310BA6, 0x98DFB5AC, 0x2FFD72DB, 0xD01ADFB7, - 0xB8E1AFED, 0x6A267E96, 0xBA7C9045, 0xF12C7F99, - 0x24A19947, 0xB3916CF7, 0x0801F2E2, 0x858EFC16, - 0x636920D8, 0x71574E69, 0xA458FEA3, 0xF4933D7E, - 0x0D95748F, 0x728EB658, 0x718BCD58, 0x82154AEE, - 0x7B54A41D, 0xC25A59B5, 0x9C30D539, 0x2AF26013, - 0xC5D1B023, 0x286085F0, 0xCA417918, 0xB8DB38EF, - 0x8E79DCB0, 0x603A180E, 0x6C9E0E8B, 0xB01E8A3E, - 0xD71577C1, 0xBD314B27, 0x78AF2FDA, 0x55605C60, - 0xE65525F3, 0xAA55AB94, 0x57489862, 0x63E81440, - 0x55CA396A, 0x2AAB10B6, 0xB4CC5C34, 0x1141E8CE, - 0xA15486AF, 0x7C72E993, 0xB3EE1411, 0x636FBC2A, - 0x2BA9C55D, 0x741831F6, 0xCE5C3E16, 0x9B87931E, - 0xAFD6BA33, 0x6C24CF5C, 0x7A325381, 0x28958677, - 0x3B8F4898, 0x6B4BB9AF, 0xC4BFE81B, 0x66282193, - 0x61D809CC, 0xFB21A991, 0x487CAC60, 0x5DEC8032, - 0xEF845D5D, 0xE98575B1, 0xDC262302, 0xEB651B88, - 0x23893E81, 0xD396ACC5, 0x0F6D6FF3, 0x83F44239, - 0x2E0B4482, 0xA4842004, 0x69C8F04A, 0x9E1F9B5E, - 0x21C66842, 0xF6E96C9A, 0x670C9C61, 0xABD388F0, - 0x6A51A0D2, 0xD8542F68, 0x960FA728, 0xAB5133A3, - 0x6EEF0B6C, 0x137A3BE4, 0xBA3BF050, 0x7EFB2A98, - 0xA1F1651D, 0x39AF0176, 0x66CA593E, 0x82430E88, - 0x8CEE8619, 0x456F9FB4, 0x7D84A5C3, 0x3B8B5EBE, - 0xE06F75D8, 0x85C12073, 0x401A449F, 0x56C16AA6, - 0x4ED3AA62, 0x363F7706, 0x1BFEDF72, 0x429B023D, - 0x37D0D724, 0xD00A1248, 0xDB0FEAD3, 0x49F1C09B, - 0x075372C9, 0x80991B7B, 0x25D479D8, 0xF6E8DEF7, - 0xE3FE501A, 0xB6794C3B, 0x976CE0BD, 0x04C006BA, - 0xC1A94FB6, 0x409F60C4, 0x5E5C9EC2, 0x196A2463, - 0x68FB6FAF, 0x3E6C53B5, 0x1339B2EB, 0x3B52EC6F, - 0x6DFC511F, 0x9B30952C, 0xCC814544, 0xAF5EBD09, - 0xBEE3D004, 0xDE334AFD, 0x660F2807, 0x192E4BB3, - 0xC0CBA857, 0x45C8740F, 0xD20B5F39, 0xB9D3FBDB, - 0x5579C0BD, 0x1A60320A, 0xD6A100C6, 0x402C7279, - 0x679F25FE, 0xFB1FA3CC, 0x8EA5E9F8, 0xDB3222F8, - 0x3C7516DF, 0xFD616B15, 0x2F501EC8, 0xAD0552AB, - 0x323DB5FA, 0xFD238760, 0x53317B48, 0x3E00DF82, - 0x9E5C57BB, 0xCA6F8CA0, 0x1A87562E, 0xDF1769DB, - 0xD542A8F6, 0x287EFFC3, 0xAC6732C6, 0x8C4F5573, - 0x695B27B0, 0xBBCA58C8, 0xE1FFA35D, 0xB8F011A0, - 0x10FA3D98, 0xFD2183B8, 0x4AFCB56C, 0x2DD1D35B, - 0x9A53E479, 0xB6F84565, 0xD28E49BC, 0x4BFB9790, - 0xE1DDF2DA, 0xA4CB7E33, 0x62FB1341, 0xCEE4C6E8, - 0xEF20CADA, 0x36774C01, 0xD07E9EFE, 0x2BF11FB4, - 0x95DBDA4D, 0xAE909198, 0xEAAD8E71, 0x6B93D5A0, - 0xD08ED1D0, 0xAFC725E0, 0x8E3C5B2F, 0x8E7594B7, - 0x8FF6E2FB, 0xF2122B64, 0x8888B812, 0x900DF01C, - 0x4FAD5EA0, 0x688FC31C, 0xD1CFF191, 0xB3A8C1AD, - 0x2F2F2218, 0xBE0E1777, 0xEA752DFE, 0x8B021FA1, - 0xE5A0CC0F, 0xB56F74E8, 0x18ACF3D6, 0xCE89E299, - 0xB4A84FE0, 0xFD13E0B7, 0x7CC43B81, 0xD2ADA8D9, - 0x165FA266, 0x80957705, 0x93CC7314, 0x211A1477, - 0xE6AD2065, 0x77B5FA86, 0xC75442F5, 0xFB9D35CF, - 0xEBCDAF0C, 0x7B3E89A0, 0xD6411BD3, 0xAE1E7E49, - 0x00250E2D, 0x2071B35E, 0x226800BB, 0x57B8E0AF, - 0x2464369B, 0xF009B91E, 0x5563911D, 0x59DFA6AA, - 0x78C14389, 0xD95A537F, 0x207D5BA2, 0x02E5B9C5, - 0x83260376, 0x6295CFA9, 0x11C81968, 0x4E734A41, - 0xB3472DCA, 0x7B14A94A, 0x1B510052, 0x9A532915, - 0xD60F573F, 0xBC9BC6E4, 0x2B60A476, 0x81E67400, - 0x08BA6FB5, 0x571BE91F, 0xF296EC6B, 0x2A0DD915, - 0xB6636521, 0xE7B9F9B6, 0xFF34052E, 0xC5855664, - 0x53B02D5D, 0xA99F8FA1, 0x08BA4799, 0x6E85076A - ], - [ - 0x4B7A70E9, 0xB5B32944, 0xDB75092E, 0xC4192623, - 0xAD6EA6B0, 0x49A7DF7D, 0x9CEE60B8, 0x8FEDB266, - 0xECAA8C71, 0x699A17FF, 0x5664526C, 0xC2B19EE1, - 0x193602A5, 0x75094C29, 0xA0591340, 0xE4183A3E, - 0x3F54989A, 0x5B429D65, 0x6B8FE4D6, 0x99F73FD6, - 0xA1D29C07, 0xEFE830F5, 0x4D2D38E6, 0xF0255DC1, - 0x4CDD2086, 0x8470EB26, 0x6382E9C6, 0x021ECC5E, - 0x09686B3F, 0x3EBAEFC9, 0x3C971814, 0x6B6A70A1, - 0x687F3584, 0x52A0E286, 0xB79C5305, 0xAA500737, - 0x3E07841C, 0x7FDEAE5C, 0x8E7D44EC, 0x5716F2B8, - 0xB03ADA37, 0xF0500C0D, 0xF01C1F04, 0x0200B3FF, - 0xAE0CF51A, 0x3CB574B2, 0x25837A58, 0xDC0921BD, - 0xD19113F9, 0x7CA92FF6, 0x94324773, 0x22F54701, - 0x3AE5E581, 0x37C2DADC, 0xC8B57634, 0x9AF3DDA7, - 0xA9446146, 0x0FD0030E, 0xECC8C73E, 0xA4751E41, - 0xE238CD99, 0x3BEA0E2F, 0x3280BBA1, 0x183EB331, - 0x4E548B38, 0x4F6DB908, 0x6F420D03, 0xF60A04BF, - 0x2CB81290, 0x24977C79, 0x5679B072, 0xBCAF89AF, - 0xDE9A771F, 0xD9930810, 0xB38BAE12, 0xDCCF3F2E, - 0x5512721F, 0x2E6B7124, 0x501ADDE6, 0x9F84CD87, - 0x7A584718, 0x7408DA17, 0xBC9F9ABC, 0xE94B7D8C, - 0xEC7AEC3A, 0xDB851DFA, 0x63094366, 0xC464C3D2, - 0xEF1C1847, 0x3215D908, 0xDD433B37, 0x24C2BA16, - 0x12A14D43, 0x2A65C451, 0x50940002, 0x133AE4DD, - 0x71DFF89E, 0x10314E55, 0x81AC77D6, 0x5F11199B, - 0x043556F1, 0xD7A3C76B, 0x3C11183B, 0x5924A509, - 0xF28FE6ED, 0x97F1FBFA, 0x9EBABF2C, 0x1E153C6E, - 0x86E34570, 0xEAE96FB1, 0x860E5E0A, 0x5A3E2AB3, - 0x771FE71C, 0x4E3D06FA, 0x2965DCB9, 0x99E71D0F, - 0x803E89D6, 0x5266C825, 0x2E4CC978, 0x9C10B36A, - 0xC6150EBA, 0x94E2EA78, 0xA5FC3C53, 0x1E0A2DF4, - 0xF2F74EA7, 0x361D2B3D, 0x1939260F, 0x19C27960, - 0x5223A708, 0xF71312B6, 0xEBADFE6E, 0xEAC31F66, - 0xE3BC4595, 0xA67BC883, 0xB17F37D1, 0x018CFF28, - 0xC332DDEF, 0xBE6C5AA5, 0x65582185, 0x68AB9802, - 0xEECEA50F, 0xDB2F953B, 0x2AEF7DAD, 0x5B6E2F84, - 0x1521B628, 0x29076170, 0xECDD4775, 0x619F1510, - 0x13CCA830, 0xEB61BD96, 0x0334FE1E, 0xAA0363CF, - 0xB5735C90, 0x4C70A239, 0xD59E9E0B, 0xCBAADE14, - 0xEECC86BC, 0x60622CA7, 0x9CAB5CAB, 0xB2F3846E, - 0x648B1EAF, 0x19BDF0CA, 0xA02369B9, 0x655ABB50, - 0x40685A32, 0x3C2AB4B3, 0x319EE9D5, 0xC021B8F7, - 0x9B540B19, 0x875FA099, 0x95F7997E, 0x623D7DA8, - 0xF837889A, 0x97E32D77, 0x11ED935F, 0x16681281, - 0x0E358829, 0xC7E61FD6, 0x96DEDFA1, 0x7858BA99, - 0x57F584A5, 0x1B227263, 0x9B83C3FF, 0x1AC24696, - 0xCDB30AEB, 0x532E3054, 0x8FD948E4, 0x6DBC3128, - 0x58EBF2EF, 0x34C6FFEA, 0xFE28ED61, 0xEE7C3C73, - 0x5D4A14D9, 0xE864B7E3, 0x42105D14, 0x203E13E0, - 0x45EEE2B6, 0xA3AAABEA, 0xDB6C4F15, 0xFACB4FD0, - 0xC742F442, 0xEF6ABBB5, 0x654F3B1D, 0x41CD2105, - 0xD81E799E, 0x86854DC7, 0xE44B476A, 0x3D816250, - 0xCF62A1F2, 0x5B8D2646, 0xFC8883A0, 0xC1C7B6A3, - 0x7F1524C3, 0x69CB7492, 0x47848A0B, 0x5692B285, - 0x095BBF00, 0xAD19489D, 0x1462B174, 0x23820E00, - 0x58428D2A, 0x0C55F5EA, 0x1DADF43E, 0x233F7061, - 0x3372F092, 0x8D937E41, 0xD65FECF1, 0x6C223BDB, - 0x7CDE3759, 0xCBEE7460, 0x4085F2A7, 0xCE77326E, - 0xA6078084, 0x19F8509E, 0xE8EFD855, 0x61D99735, - 0xA969A7AA, 0xC50C06C2, 0x5A04ABFC, 0x800BCADC, - 0x9E447A2E, 0xC3453484, 0xFDD56705, 0x0E1E9EC9, - 0xDB73DBD3, 0x105588CD, 0x675FDA79, 0xE3674340, - 0xC5C43465, 0x713E38D8, 0x3D28F89E, 0xF16DFF20, - 0x153E21E7, 0x8FB03D4A, 0xE6E39F2B, 0xDB83ADF7 - ], - [ - 0xE93D5A68, 0x948140F7, 0xF64C261C, 0x94692934, - 0x411520F7, 0x7602D4F7, 0xBCF46B2E, 0xD4A20068, - 0xD4082471, 0x3320F46A, 0x43B7D4B7, 0x500061AF, - 0x1E39F62E, 0x97244546, 0x14214F74, 0xBF8B8840, - 0x4D95FC1D, 0x96B591AF, 0x70F4DDD3, 0x66A02F45, - 0xBFBC09EC, 0x03BD9785, 0x7FAC6DD0, 0x31CB8504, - 0x96EB27B3, 0x55FD3941, 0xDA2547E6, 0xABCA0A9A, - 0x28507825, 0x530429F4, 0x0A2C86DA, 0xE9B66DFB, - 0x68DC1462, 0xD7486900, 0x680EC0A4, 0x27A18DEE, - 0x4F3FFEA2, 0xE887AD8C, 0xB58CE006, 0x7AF4D6B6, - 0xAACE1E7C, 0xD3375FEC, 0xCE78A399, 0x406B2A42, - 0x20FE9E35, 0xD9F385B9, 0xEE39D7AB, 0x3B124E8B, - 0x1DC9FAF7, 0x4B6D1856, 0x26A36631, 0xEAE397B2, - 0x3A6EFA74, 0xDD5B4332, 0x6841E7F7, 0xCA7820FB, - 0xFB0AF54E, 0xD8FEB397, 0x454056AC, 0xBA489527, - 0x55533A3A, 0x20838D87, 0xFE6BA9B7, 0xD096954B, - 0x55A867BC, 0xA1159A58, 0xCCA92963, 0x99E1DB33, - 0xA62A4A56, 0x3F3125F9, 0x5EF47E1C, 0x9029317C, - 0xFDF8E802, 0x04272F70, 0x80BB155C, 0x05282CE3, - 0x95C11548, 0xE4C66D22, 0x48C1133F, 0xC70F86DC, - 0x07F9C9EE, 0x41041F0F, 0x404779A4, 0x5D886E17, - 0x325F51EB, 0xD59BC0D1, 0xF2BCC18F, 0x41113564, - 0x257B7834, 0x602A9C60, 0xDFF8E8A3, 0x1F636C1B, - 0x0E12B4C2, 0x02E1329E, 0xAF664FD1, 0xCAD18115, - 0x6B2395E0, 0x333E92E1, 0x3B240B62, 0xEEBEB922, - 0x85B2A20E, 0xE6BA0D99, 0xDE720C8C, 0x2DA2F728, - 0xD0127845, 0x95B794FD, 0x647D0862, 0xE7CCF5F0, - 0x5449A36F, 0x877D48FA, 0xC39DFD27, 0xF33E8D1E, - 0x0A476341, 0x992EFF74, 0x3A6F6EAB, 0xF4F8FD37, - 0xA812DC60, 0xA1EBDDF8, 0x991BE14C, 0xDB6E6B0D, - 0xC67B5510, 0x6D672C37, 0x2765D43B, 0xDCD0E804, - 0xF1290DC7, 0xCC00FFA3, 0xB5390F92, 0x690FED0B, - 0x667B9FFB, 0xCEDB7D9C, 0xA091CF0B, 0xD9155EA3, - 0xBB132F88, 0x515BAD24, 0x7B9479BF, 0x763BD6EB, - 0x37392EB3, 0xCC115979, 0x8026E297, 0xF42E312D, - 0x6842ADA7, 0xC66A2B3B, 0x12754CCC, 0x782EF11C, - 0x6A124237, 0xB79251E7, 0x06A1BBE6, 0x4BFB6350, - 0x1A6B1018, 0x11CAEDFA, 0x3D25BDD8, 0xE2E1C3C9, - 0x44421659, 0x0A121386, 0xD90CEC6E, 0xD5ABEA2A, - 0x64AF674E, 0xDA86A85F, 0xBEBFE988, 0x64E4C3FE, - 0x9DBC8057, 0xF0F7C086, 0x60787BF8, 0x6003604D, - 0xD1FD8346, 0xF6381FB0, 0x7745AE04, 0xD736FCCC, - 0x83426B33, 0xF01EAB71, 0xB0804187, 0x3C005E5F, - 0x77A057BE, 0xBDE8AE24, 0x55464299, 0xBF582E61, - 0x4E58F48F, 0xF2DDFDA2, 0xF474EF38, 0x8789BDC2, - 0x5366F9C3, 0xC8B38E74, 0xB475F255, 0x46FCD9B9, - 0x7AEB2661, 0x8B1DDF84, 0x846A0E79, 0x915F95E2, - 0x466E598E, 0x20B45770, 0x8CD55591, 0xC902DE4C, - 0xB90BACE1, 0xBB8205D0, 0x11A86248, 0x7574A99E, - 0xB77F19B6, 0xE0A9DC09, 0x662D09A1, 0xC4324633, - 0xE85A1F02, 0x09F0BE8C, 0x4A99A025, 0x1D6EFE10, - 0x1AB93D1D, 0x0BA5A4DF, 0xA186F20F, 0x2868F169, - 0xDCB7DA83, 0x573906FE, 0xA1E2CE9B, 0x4FCD7F52, - 0x50115E01, 0xA70683FA, 0xA002B5C4, 0x0DE6D027, - 0x9AF88C27, 0x773F8641, 0xC3604C06, 0x61A806B5, - 0xF0177A28, 0xC0F586E0, 0x006058AA, 0x30DC7D62, - 0x11E69ED7, 0x2338EA63, 0x53C2DD94, 0xC2C21634, - 0xBBCBEE56, 0x90BCB6DE, 0xEBFC7DA1, 0xCE591D76, - 0x6F05E409, 0x4B7C0188, 0x39720A3D, 0x7C927C24, - 0x86E3725F, 0x724D9DB9, 0x1AC15BB4, 0xD39EB8FC, - 0xED545578, 0x08FCA5B5, 0xD83D7CD3, 0x4DAD0FC4, - 0x1E50EF5E, 0xB161E6F8, 0xA28514D9, 0x6C51133C, - 0x6FD5C7E7, 0x56E14EC4, 0x362ABFCE, 0xDDC6C837, - 0xD79A3234, 0x92638212, 0x670EFA8E, 0x406000E0 - ], - [ - 0x3A39CE37, 0xD3FAF5CF, 0xABC27737, 0x5AC52D1B, - 0x5CB0679E, 0x4FA33742, 0xD3822740, 0x99BC9BBE, - 0xD5118E9D, 0xBF0F7315, 0xD62D1C7E, 0xC700C47B, - 0xB78C1B6B, 0x21A19045, 0xB26EB1BE, 0x6A366EB4, - 0x5748AB2F, 0xBC946E79, 0xC6A376D2, 0x6549C2C8, - 0x530FF8EE, 0x468DDE7D, 0xD5730A1D, 0x4CD04DC6, - 0x2939BBDB, 0xA9BA4650, 0xAC9526E8, 0xBE5EE304, - 0xA1FAD5F0, 0x6A2D519A, 0x63EF8CE2, 0x9A86EE22, - 0xC089C2B8, 0x43242EF6, 0xA51E03AA, 0x9CF2D0A4, - 0x83C061BA, 0x9BE96A4D, 0x8FE51550, 0xBA645BD6, - 0x2826A2F9, 0xA73A3AE1, 0x4BA99586, 0xEF5562E9, - 0xC72FEFD3, 0xF752F7DA, 0x3F046F69, 0x77FA0A59, - 0x80E4A915, 0x87B08601, 0x9B09E6AD, 0x3B3EE593, - 0xE990FD5A, 0x9E34D797, 0x2CF0B7D9, 0x022B8B51, - 0x96D5AC3A, 0x017DA67D, 0xD1CF3ED6, 0x7C7D2D28, - 0x1F9F25CF, 0xADF2B89B, 0x5AD6B472, 0x5A88F54C, - 0xE029AC71, 0xE019A5E6, 0x47B0ACFD, 0xED93FA9B, - 0xE8D3C48D, 0x283B57CC, 0xF8D56629, 0x79132E28, - 0x785F0191, 0xED756055, 0xF7960E44, 0xE3D35E8C, - 0x15056DD4, 0x88F46DBA, 0x03A16125, 0x0564F0BD, - 0xC3EB9E15, 0x3C9057A2, 0x97271AEC, 0xA93A072A, - 0x1B3F6D9B, 0x1E6321F5, 0xF59C66FB, 0x26DCF319, - 0x7533D928, 0xB155FDF5, 0x03563482, 0x8ABA3CBB, - 0x28517711, 0xC20AD9F8, 0xABCC5167, 0xCCAD925F, - 0x4DE81751, 0x3830DC8E, 0x379D5862, 0x9320F991, - 0xEA7A90C2, 0xFB3E7BCE, 0x5121CE64, 0x774FBE32, - 0xA8B6E37E, 0xC3293D46, 0x48DE5369, 0x6413E680, - 0xA2AE0810, 0xDD6DB224, 0x69852DFD, 0x09072166, - 0xB39A460A, 0x6445C0DD, 0x586CDECF, 0x1C20C8AE, - 0x5BBEF7DD, 0x1B588D40, 0xCCD2017F, 0x6BB4E3BB, - 0xDDA26A7E, 0x3A59FF45, 0x3E350A44, 0xBCB4CDD5, - 0x72EACEA8, 0xFA6484BB, 0x8D6612AE, 0xBF3C6F47, - 0xD29BE463, 0x542F5D9E, 0xAEC2771B, 0xF64E6370, - 0x740E0D8D, 0xE75B1357, 0xF8721671, 0xAF537D5D, - 0x4040CB08, 0x4EB4E2CC, 0x34D2466A, 0x0115AF84, - 0xE1B00428, 0x95983A1D, 0x06B89FB4, 0xCE6EA048, - 0x6F3F3B82, 0x3520AB82, 0x011A1D4B, 0x277227F8, - 0x611560B1, 0xE7933FDC, 0xBB3A792B, 0x344525BD, - 0xA08839E1, 0x51CE794B, 0x2F32C9B7, 0xA01FBAC9, - 0xE01CC87E, 0xBCC7D1F6, 0xCF0111C3, 0xA1E8AAC7, - 0x1A908749, 0xD44FBD9A, 0xD0DADECB, 0xD50ADA38, - 0x0339C32A, 0xC6913667, 0x8DF9317C, 0xE0B12B4F, - 0xF79E59B7, 0x43F5BB3A, 0xF2D519FF, 0x27D9459C, - 0xBF97222C, 0x15E6FC2A, 0x0F91FC71, 0x9B941525, - 0xFAE59361, 0xCEB69CEB, 0xC2A86459, 0x12BAA8D1, - 0xB6C1075E, 0xE3056A0C, 0x10D25065, 0xCB03A442, - 0xE0EC6E0E, 0x1698DB3B, 0x4C98A0BE, 0x3278E964, - 0x9F1F9532, 0xE0D392DF, 0xD3A0342B, 0x8971F21E, - 0x1B0A7441, 0x4BA3348C, 0xC5BE7120, 0xC37632D8, - 0xDF359F8D, 0x9B992F2E, 0xE60B6F47, 0x0FE3F11D, - 0xE54CDA54, 0x1EDAD891, 0xCE6279CF, 0xCD3E7E6F, - 0x1618B166, 0xFD2C1D05, 0x848FD2C5, 0xF6FB2299, - 0xF523F357, 0xA6327623, 0x93A83531, 0x56CCCD02, - 0xACF08162, 0x5A75EBB5, 0x6E163697, 0x88D273CC, - 0xDE966292, 0x81B949D0, 0x4C50901B, 0x71C65614, - 0xE6C6C7BD, 0x327A140A, 0x45E1D006, 0xC3F27B9A, - 0xC9AA53FD, 0x62A80F00, 0xBB25BFE2, 0x35BDD2F6, - 0x71126905, 0xB2040222, 0xB6CBCF7C, 0xCD769C2B, - 0x53113EC0, 0x1640E3D3, 0x38ABBD60, 0x2547ADF0, - 0xBA38209C, 0xF746CE76, 0x77AFA1C5, 0x20756060, - 0x85CBFE4E, 0x8AE88DD8, 0x7AAAF9B0, 0x4CF9AA7E, - 0x1948C25C, 0x02FB8A8C, 0x01C36AE4, 0xD6EBE1F9, - 0x90D4F869, 0xA65CDEA0, 0x3F09252D, 0xC208E69F, - 0xB74E6132, 0xCE77E25B, 0x578FDFE3, 0x3AC372E6 - ] - ] - - # Cycle through the p-boxes and round-robin XOR the - # key with the p-boxes - key_len = len (key) - index = 0 - for i in range (len (self.p_boxes)): - val = (ord (key[index % key_len]) << 24) + \ - (ord (key[(index + 1) % key_len]) << 16) + \ - (ord (key[(index + 2) % key_len]) << 8) + \ - ord (key[(index + 3) % key_len]) - self.p_boxes[i] = self.p_boxes[i] ^ val - index = index + 4 - - # For the chaining process - l, r = 0, 0 - - # Begin chain replacing the p-boxes - for i in range (0, len (self.p_boxes), 2): - l, r = self.cipher (l, r, self.ENCRYPT) - self.p_boxes[i] = l - self.p_boxes[i + 1] = r - - # Chain replace the s-boxes - for i in range (len (self.s_boxes)): - for j in range (0, len (self.s_boxes[i]), 2): - l, r = self.cipher (l, r, self.ENCRYPT) - self.s_boxes[i][j] = l - self.s_boxes[i][j + 1] = r - - def cipher(self, xl, xr, direction): - if direction == self.ENCRYPT: - for i in range (16): - xl = xl ^ self.p_boxes[i] - xr = self.__round_func (xl) ^ xr - xl, xr = xr, xl - xl, xr = xr, xl - xr = xr ^ self.p_boxes[16] - xl = xl ^ self.p_boxes[17] - else: - for i in range (17, 1, -1): - xl = xl ^ self.p_boxes[i] - xr = self.__round_func (xl) ^ xr - xl, xr = xr, xl - xl, xr = xr, xl - xr = xr ^ self.p_boxes[1] - xl = xl ^ self.p_boxes[0] - return xl, xr - - def __round_func(self, xl): - a = (xl & 0xFF000000) >> 24 - b = (xl & 0x00FF0000) >> 16 - c = (xl & 0x0000FF00) >> 8 - d = xl & 0x000000FF - - # Perform all ops as longs then and out the last 32-bits to - # obtain the integer - f = (long (self.s_boxes[0][a]) + long (self.s_boxes[1][b])) % self.modulus - f = f ^ long (self.s_boxes[2][c]) - f = f + long (self.s_boxes[3][d]) - f = (f % self.modulus) & 0xFFFFFFFF - - return f - - def encrypt(self, data): - if not len(data) == 8: - e = "blocks must be 8 bytes long, but tried to encrypt one {0} bytes long" - raise BlockSizeError(e.format(len(data))) - - # Use big endianess since that's what everyone else uses - xl = ord (data[3]) | (ord (data[2]) << 8) | (ord (data[1]) << 16) | (ord (data[0]) << 24) - xr = ord (data[7]) | (ord (data[6]) << 8) | (ord (data[5]) << 16) | (ord (data[4]) << 24) - - cl, cr = self.cipher (xl, xr, self.ENCRYPT) - chars = ''.join ([ - chr ((cl >> 24) & 0xFF), chr ((cl >> 16) & 0xFF), chr ((cl >> 8) & 0xFF), chr (cl & 0xFF), - chr ((cr >> 24) & 0xFF), chr ((cr >> 16) & 0xFF), chr ((cr >> 8) & 0xFF), chr (cr & 0xFF) - ]) - return chars - - def decrypt(self, data): - if not len(data) == 8: - e = "blocks must be 8 bytes long, but tried to decrypt one {0} bytes long" - raise BlockSizeError(e.format(len(data))) - - # Use big endianess since that's what everyone else uses - cl = ord (data[3]) | (ord (data[2]) << 8) | (ord (data[1]) << 16) | (ord (data[0]) << 24) - cr = ord (data[7]) | (ord (data[6]) << 8) | (ord (data[5]) << 16) | (ord (data[4]) << 24) - - xl, xr = self.cipher (cl, cr, self.DECRYPT) - chars = ''.join ([ - chr ((xl >> 24) & 0xFF), chr ((xl >> 16) & 0xFF), chr ((xl >> 8) & 0xFF), chr (xl & 0xFF), - chr ((xr >> 24) & 0xFF), chr ((xr >> 16) & 0xFF), chr ((xr >> 8) & 0xFF), chr (xr & 0xFF) - ]) - return chars - - def blocksize(self): - return 8 - - def key_length(self): - return 56 - - def key_bits(self): - return 56 * 8 - - def verify_key(self, key): - """Make sure our key is not too short or too long. - - If there's a problem, raise KeyTooShortError() or KeyTooLongError(). - """ - if not key: - raise KeyLengthError("no key given") - if len(key) < 8: - e = "key is {0} bytes long, but it must be at least 8" - raise KeyLengthError(e.format(len(key))) - if len(key) > 56: - e = "key is {0} bytes long, but it must be less than 56" - raise KeyLengthError(e.format(len(key))) - -def encrypt(key, plaintext): - """Encrypt any length of plaintext using a given key that must be between - 8 and 56 bytes in length. This is a convienence function that can handle - plaintext that is not a single block in length. It will auto-pad blocks - that are less than 8 bytes with spaces that are automatically removed by - decrypt(). Actual spaces in plaintext are preserved.""" - cypher = Blowfish(key) - - msg = "TRUE{0}|{1}".format(len(plaintext), plaintext) - while len(msg) % 8 > 0: - msg += " " # pad message to form complete 8-byte blocks - - blocks = [msg[f:f+8] for f in range(0, len(msg), 8)] - cyphertext = ''.join(map(cypher.encrypt, blocks)) - - return cyphertext.encode('hex') - -def decrypt(key, cyphertext): - """Decrypt the result of encrypt() using the original key, or raise - DecryptionError().""" - cypher = Blowfish(key) - - try: - cyphertext = cyphertext.decode("hex") - except (TypeError, AttributeError) as error: - e = error.message - raise DecryptionError("cyphertext could not be decoded: " + e.lower()) - - if len(cyphertext) % 8 > 0: - e = "cyphertext cannot be broken into 8-byte blocks evenly" - raise DecryptionError(e) - - blocks = [cyphertext[f:f+8] for f in range(0, len(cyphertext), 8)] - msg = ''.join(map(cypher.decrypt, blocks)) - - # Sanity check to ensure valid decryption: - if not msg.startswith("TRUE"): - e = "the given key is incorrect, or part of the cyphertext is malformed" - raise DecryptionError(e) - - size, msg = msg[4:].split("|", 1) - while len(msg) > int(size): - msg = msg[:-1] # Remove the padding that we applied earlier - - return msg - -if __name__ == '__main__': - action = raw_input("Would you like to [e]ncrypt or [d]ecrypt? ") - if action.lower().startswith("e"): - key = raw_input("Enter a key: ") - plaintext = raw_input("Enter a message to encrypt: ") - print "\n" + encrypt(key, plaintext) - elif action.lower().startswith("d"): - key = raw_input("Enter a key: ") - cyphertext = raw_input("Enter a message to decrypt: ") - print "\n" + decrypt(key, cyphertext) - else: - print "Unknown action: '{0}'".format(action) diff --git a/earwigbot/bot.py b/earwigbot/bot.py new file mode 100644 index 0000000..8607dc1 --- /dev/null +++ b/earwigbot/bot.py @@ -0,0 +1,212 @@ +# -*- coding: utf-8 -*- +# +# Copyright (C) 2009-2012 by Ben Kurtovic +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +import logging +from threading import Lock, Thread, enumerate as enumerate_threads +from time import sleep, time + +from earwigbot import __version__ +from earwigbot.config import BotConfig +from earwigbot.irc import Frontend, Watcher +from earwigbot.managers import CommandManager, TaskManager +from earwigbot.wiki import SitesDB + +__all__ = ["Bot"] + +class Bot(object): + """ + **EarwigBot: Main Bot Class** + + The :py:class:`Bot` class is the core of EarwigBot, essentially responsible + for starting the various bot components and making sure they are all happy. + + EarwigBot has three components that can run independently of each other: an + IRC front-end, an IRC watcher, and a wiki scheduler. + + - The IRC front-end runs on a normal IRC server and expects users to + interact with it/give it commands. + - The IRC watcher runs on a wiki recent-changes server and listens for + edits. Users cannot interact with this part of the bot. + - The wiki scheduler runs wiki-editing bot tasks in separate threads at + user-defined times through a cron-like interface. + + The :py:class:`Bot` object is accessible from within commands and tasks as + :py:attr:`self.bot`. This is the primary way to access data from other + components of the bot. For example, our + :py:class:`~earwigbot.config.BotConfig` object is accessable from + :py:attr:`bot.config`, tasks can be started with + :py:meth:`bot.tasks.start() `, and + sites can be loaded from the wiki toolset with + :py:meth:`bot.wiki.get_site() `. + """ + + def __init__(self, root_dir, level=logging.INFO): + self.config = BotConfig(root_dir, level) + self.logger = logging.getLogger("earwigbot") + self.commands = CommandManager(self) + self.tasks = TaskManager(self) + self.wiki = SitesDB(self) + self.frontend = None + self.watcher = None + + self.component_lock = Lock() + self._keep_looping = True + + self.config.load() + self.commands.load() + self.tasks.load() + + def __repr__(self): + """Return the canonical string representation of the Bot.""" + return "Bot(config={0!r})".format(self.config) + + def __str__(self): + """Return a nice string representation of the Bot.""" + return "".format(self.config.root_dir) + + def _dispatch_irc_component(self, name, klass): + """Create a new IRC component, record it internally, and start it.""" + component = klass(self) + setattr(self, name, component) + Thread(name="irc_" + name, target=component.loop).start() + + def _start_irc_components(self): + """Start the IRC frontend/watcher in separate threads if enabled.""" + if self.config.components.get("irc_frontend"): + self.logger.info("Starting IRC frontend") + self._dispatch_irc_component("frontend", Frontend) + if self.config.components.get("irc_watcher"): + self.logger.info("Starting IRC watcher") + self._dispatch_irc_component("watcher", Watcher) + + def _start_wiki_scheduler(self): + """Start the wiki scheduler in a separate thread if enabled.""" + def wiki_scheduler(): + while self._keep_looping: + time_start = time() + self.tasks.schedule() + time_end = time() + time_diff = time_start - time_end + if time_diff < 60: # Sleep until the next minute + sleep(60 - time_diff) + + if self.config.components.get("wiki_scheduler"): + self.logger.info("Starting wiki scheduler") + thread = Thread(name="wiki_scheduler", target=wiki_scheduler) + thread.daemon = True # Stop if other threads stop + thread.start() + + def _keep_irc_component_alive(self, name, klass): + """Ensure that IRC components stay connected, else restart them.""" + component = getattr(self, name) + if component: + component.keep_alive() + if component.is_stopped(): + log = "IRC {0} has stopped; restarting".format(name) + self.logger.warn(log) + self._dispatch_irc_component(name, klass) + + def _stop_irc_components(self, msg): + """Request the IRC frontend and watcher to stop if enabled.""" + if self.frontend: + self.frontend.stop(msg) + if self.watcher: + self.watcher.stop(msg) + + def _stop_daemon_threads(self): + """Notify the user of which threads are going to be killed. + + Unfortunately, there is no method right now of stopping command and + task threads safely. This is because there is no way to tell them to + stop like the IRC components can be told; furthermore, they are run as + daemons, and daemon threads automatically stop without calling any + __exit__ or try/finally code when all non-daemon threads stop. They + were originally implemented as regular non-daemon threads, but this + meant there was no way to completely stop the bot if tasks were + running, because all other threads would exit and threading would + absorb KeyboardInterrupts. + + The advantage of this is that stopping the bot is truly guarenteed to + *stop* the bot, while the disadvantage is that the threads are given no + advance warning of their forced shutdown. + """ + tasks = [] + non_tasks = self.config.components.keys() + ["MainThread", "reminder"] + for thread in enumerate_threads(): + if thread.name not in non_tasks and thread.is_alive(): + tasks.append(thread.name) + if tasks: + log = "The following commands or tasks will be killed: {0}" + self.logger.warn(log.format(" ".join(tasks))) + + def run(self): + """Main entry point into running the bot. + + Starts all config-enabled components and then enters an idle loop, + ensuring that all components remain online and restarting components + that get disconnected from their servers. + """ + self.logger.info("Starting bot (EarwigBot {0})".format(__version__)) + self._start_irc_components() + self._start_wiki_scheduler() + while self._keep_looping: + with self.component_lock: + self._keep_irc_component_alive("frontend", Frontend) + self._keep_irc_component_alive("watcher", Watcher) + sleep(2) + + def restart(self, msg=None): + """Reload config, commands, tasks, and safely restart IRC components. + + This is thread-safe, and it will gracefully stop IRC components before + reloading anything. Note that you can safely reload commands or tasks + without restarting the bot with :py:meth:`bot.commands.load() + ` or + :py:meth:`bot.tasks.load() `. + These should not interfere with running components or tasks. + + If given, *msg* will be used as our quit message. + """ + if msg: + self.logger.info('Restarting bot ("{0}")'.format(msg)) + else: + self.logger.info("Restarting bot") + with self.component_lock: + self._stop_irc_components(msg) + self.config.load() + self.commands.load() + self.tasks.load() + self._start_irc_components() + + def stop(self, msg=None): + """Gracefully stop all bot components. + + If given, *msg* will be used as our quit message. + """ + if msg: + self.logger.info('Stopping bot ("{0}")'.format(msg)) + else: + self.logger.info("Stopping bot") + with self.component_lock: + self._stop_irc_components(msg) + self._keep_looping = False + self._stop_daemon_threads() diff --git a/earwigbot/classes/base_command.py b/earwigbot/classes/base_command.py deleted file mode 100644 index 10ccbc8..0000000 --- a/earwigbot/classes/base_command.py +++ /dev/null @@ -1,75 +0,0 @@ -# -*- coding: utf-8 -*- -# -# Copyright (C) 2009-2012 by Ben Kurtovic -# -# Permission is hereby granted, free of charge, to any person obtaining a copy -# of this software and associated documentation files (the "Software"), to deal -# in the Software without restriction, including without limitation the rights -# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -# copies of the Software, and to permit persons to whom the Software is -# furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included in -# all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -# SOFTWARE. - -import logging - -__all__ = ["BaseCommand"] - -class BaseCommand(object): - """A base class for commands on IRC. - - This docstring is reported to the user when they use !help . - """ - # This is the command's name, as reported to the user when they use !help: - name = None - - # Hooks are "msg", "msg_private", "msg_public", and "join". "msg" is the - # default behavior; if you wish to override that, change the value in your - # command subclass: - hooks = ["msg"] - - def __init__(self, connection): - """Constructor for new commands. - - This is called once when the command is loaded (from - commands._load_command()). `connection` is a Connection object, - allowing us to do self.connection.say(), self.connection.send(), etc, - from within a method. - """ - self.connection = connection - logger_name = ".".join(("earwigbot", "commands", self.name)) - self.logger = logging.getLogger(logger_name) - self.logger.setLevel(logging.DEBUG) - - def check(self, data): - """Returns whether this command should be called in response to 'data'. - - Given a Data() instance, return True if we should respond to this - activity, or False if we should ignore it or it doesn't apply to us. - - Most commands return True if data.command == self.name, otherwise they - return False. This is the default behavior of check(); you need only - override it if you wish to change that. - """ - if data.is_command and data.command == self.name: - return True - return False - - def process(self, data): - """Main entry point for doing a command. - - Handle an activity (usually a message) on IRC. At this point, thanks - to self.check() which is called automatically by the command handler, - we know this is something we should respond to, so (usually) something - like 'if data.command != "command_name": return' is unnecessary. - """ - pass diff --git a/earwigbot/classes/base_task.py b/earwigbot/classes/base_task.py deleted file mode 100644 index 85229f2..0000000 --- a/earwigbot/classes/base_task.py +++ /dev/null @@ -1,117 +0,0 @@ -# -*- coding: utf-8 -*- -# -# Copyright (C) 2009-2012 by Ben Kurtovic -# -# Permission is hereby granted, free of charge, to any person obtaining a copy -# of this software and associated documentation files (the "Software"), to deal -# in the Software without restriction, including without limitation the rights -# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -# copies of the Software, and to permit persons to whom the Software is -# furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included in -# all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -# SOFTWARE. - -import logging - -from earwigbot.config import config -from earwigbot import wiki - -__all__ = ["BaseTask"] - -class BaseTask(object): - """A base class for bot tasks that edit Wikipedia.""" - name = None - number = 0 - - def __init__(self): - """Constructor for new tasks. - - This is called once immediately after the task class is loaded by - the task manager (in tasks._load_task()). - """ - pass - - def _setup_logger(self): - """Set up a basic module-level logger.""" - logger_name = ".".join(("earwigbot", "tasks", self.name)) - self.logger = logging.getLogger(logger_name) - self.logger.setLevel(logging.DEBUG) - - def run(self, **kwargs): - """Main entry point to run a given task. - - This is called directly by tasks.start() and is the main way to make a - task do stuff. kwargs will be any keyword arguments passed to start() - which are entirely optional. - - The same task instance is preserved between runs, so you can - theoretically store data in self (e.g. - start('mytask', action='store', data='foo')) and then use it later - (e.g. start('mytask', action='save')). - """ - pass - - def make_summary(self, comment): - """Makes an edit summary by filling in variables in a config value. - - config.wiki["summary"] is used, where $2 is replaced by the main - summary body, given as a method arg, and $1 is replaced by the task - number. - - If the config value is not found, we just return the arg as-is. - """ - try: - summary = config.wiki["summary"] - except KeyError: - return comment - return summary.replace("$1", str(self.number)).replace("$2", comment) - - def shutoff_enabled(self, site=None): - """Returns whether on-wiki shutoff is enabled for this task. - - We check a certain page for certain content. This is determined by - our config file: config.wiki["shutoff"]["page"] is used as the title, - with $1 replaced by our username and $2 replaced by the task number, - and config.wiki["shutoff"]["disabled"] is used as the content. - - If the page has that content or the page does not exist, then shutoff - is "disabled", meaning the bot is supposed to run normally, and we - return False. If the page's content is something other than what we - expect, shutoff is enabled, and we return True. - - If a site is not provided, we'll try to use self.site if it's set. - Otherwise, we'll use our default site. - """ - if not site: - try: - site = self.site - except AttributeError: - site = wiki.get_site() - - try: - cfg = config.wiki["shutoff"] - except KeyError: - return False - title = cfg.get("page", "User:$1/Shutoff/Task $2") - username = site.get_user().name() - title = title.replace("$1", username).replace("$2", str(self.number)) - page = site.get_page(title) - - try: - content = page.get() - except wiki.PageNotFoundError: - return False - if content == cfg.get("disabled", "run"): - return False - - self.logger.warn("Emergency task shutoff has been enabled!") - return True diff --git a/earwigbot/classes/connection.py b/earwigbot/classes/connection.py deleted file mode 100644 index 5a45145..0000000 --- a/earwigbot/classes/connection.py +++ /dev/null @@ -1,115 +0,0 @@ -# -*- coding: utf-8 -*- -# -# Copyright (C) 2009-2012 by Ben Kurtovic -# -# Permission is hereby granted, free of charge, to any person obtaining a copy -# of this software and associated documentation files (the "Software"), to deal -# in the Software without restriction, including without limitation the rights -# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -# copies of the Software, and to permit persons to whom the Software is -# furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included in -# all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -# SOFTWARE. - -import socket -import threading - -__all__ = ["BrokenSocketException", "Connection"] - -class BrokenSocketException(Exception): - """A socket has broken, because it is not sending data. Raised by - Connection.get().""" - pass - -class Connection(object): - """A class to interface with IRC.""" - - def __init__(self, host=None, port=None, nick=None, ident=None, - realname=None, logger=None): - self.host = host - self.port = port - self.nick = nick - self.ident = ident - self.realname = realname - self.logger = logger - - # A lock to prevent us from sending two messages at once: - self.lock = threading.Lock() - - def connect(self): - """Connect to our IRC server.""" - self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - try: - self.sock.connect((self.host, self.port)) - except socket.error: - self.logger.critical("Couldn't connect to IRC server", exc_info=1) - exit(1) - self.send("NICK %s" % self.nick) - self.send("USER %s %s * :%s" % (self.ident, self.host, self.realname)) - - def close(self): - """Close our connection with the IRC server.""" - try: - self.sock.shutdown(socket.SHUT_RDWR) # shut down connection first - except socket.error: - pass # ignore if the socket is already down - self.sock.close() - - def get(self, size=4096): - """Receive (i.e. get) data from the server.""" - data = self.sock.recv(4096) - if not data: - # Socket isn't giving us any data, so it is dead or broken: - raise BrokenSocketException() - return data - - def send(self, msg): - """Send data to the server.""" - # Ensure that we only send one message at a time with a blocking lock: - with self.lock: - self.sock.sendall(msg + "\r\n") - self.logger.debug(msg) - - def say(self, target, msg): - """Send a private message to a target on the server.""" - message = "".join(("PRIVMSG ", target, " :", msg)) - self.send(message) - - def reply(self, data, msg): - """Send a private message as a reply to a user on the server.""" - message = "".join((chr(2), data.nick, chr(0x0f), ": ", msg)) - self.say(data.chan, message) - - def action(self, target, msg): - """Send a private message to a target on the server as an action.""" - message = "".join((chr(1), "ACTION ", msg, chr(1))) - self.say(target, message) - - def notice(self, target, msg): - """Send a notice to a target on the server.""" - message = "".join(("NOTICE ", target, " :", msg)) - self.send(message) - - def join(self, chan): - """Join a channel on the server.""" - message = " ".join(("JOIN", chan)) - self.send(message) - - def part(self, chan): - """Part from a channel on the server.""" - message = " ".join(("PART", chan)) - self.send(message) - - def mode(self, chan, level, msg): - """Send a mode message to the server.""" - message = " ".join(("MODE", chan, level, msg)) - self.send(message) diff --git a/earwigbot/classes/data.py b/earwigbot/classes/data.py deleted file mode 100644 index fae6a52..0000000 --- a/earwigbot/classes/data.py +++ /dev/null @@ -1,79 +0,0 @@ -# -*- coding: utf-8 -*- -# -# Copyright (C) 2009-2012 by Ben Kurtovic -# -# Permission is hereby granted, free of charge, to any person obtaining a copy -# of this software and associated documentation files (the "Software"), to deal -# in the Software without restriction, including without limitation the rights -# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -# copies of the Software, and to permit persons to whom the Software is -# furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included in -# all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -# SOFTWARE. - -import re - -__all__ = ["KwargParseException", "Data"] - -class KwargParseException(Exception): - """Couldn't parse a certain keyword argument in self.args, probably because - it was given incorrectly: e.g., no value (abc), just a value (=xyz), just - an equal sign (=), instead of the correct (abc=xyz).""" - pass - -class Data(object): - """Store data from an individual line received on IRC.""" - - def __init__(self, line): - self.line = line - self.chan = self.nick = self.ident = self.host = self.msg = "" - - def parse_args(self): - """Parse command args from self.msg into self.command and self.args.""" - args = self.msg.strip().split() - - while "" in args: - args.remove("") - - # Isolate command arguments: - self.args = args[1:] - self.is_command = False # is this message a command? - - try: - self.command = args[0] - except IndexError: - self.command = None - - try: - if self.command.startswith('!') or self.command.startswith('.'): - self.is_command = True - self.command = self.command[1:] # Strip the '!' or '.' - self.command = self.command.lower() - except AttributeError: - pass - - def parse_kwargs(self): - """Parse keyword arguments embedded in self.args. - - Parse a command given as "!command key1=value1 key2=value2..." into a - dict, self.kwargs, like {'key1': 'value2', 'key2': 'value2'...}. - """ - self.kwargs = {} - for arg in self.args[2:]: - try: - key, value = re.findall("^(.*?)\=(.*?)$", arg)[0] - except IndexError: - raise KwargParseException(arg) - if key and value: - self.kwargs[key] = value - else: - raise KwargParseException(arg) diff --git a/earwigbot/commands/__init__.py b/earwigbot/commands/__init__.py index fda7072..e213cfe 100644 --- a/earwigbot/commands/__init__.py +++ b/earwigbot/commands/__init__.py @@ -1,17 +1,17 @@ # -*- coding: utf-8 -*- # # Copyright (C) 2009-2012 by Ben Kurtovic -# +# # Permission is hereby granted, free of charge, to any person obtaining a copy # of this software and associated documentation files (the "Software"), to deal # in the Software without restriction, including without limitation the rights # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -# copies of the Software, and to permit persons to whom the Software is +# copies of the Software, and to permit persons to whom the Software is # furnished to do so, subject to the following conditions: -# +# # The above copyright notice and this permission notice shall be included in # all copies or substantial portions of the Software. -# +# # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE @@ -20,92 +20,103 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. -""" -EarwigBot's IRC Command Manager - -This package provides the IRC "commands" used by the bot's front-end component. -In __init__, you can find some functions used to load and run these commands. -""" - -import logging -import os -import sys - -from earwigbot.classes import BaseCommand -from earwigbot.config import config - -__all__ = ["load", "get_all", "check"] +__all__ = ["Command"] -# Base directory when searching for commands: -base_dir = os.path.dirname(os.path.abspath(__file__)) - -# Store commands in a dict, where the key is the command's name and the value -# is an instance of the command's class: -_commands = {} - -# Logger for this module: -logger = logging.getLogger("earwigbot.tasks") - -def _load_command(connection, filename): - """Try to load a specific command from a module, identified by file name. - - Given a Connection object and a filename, we'll first try to import it, - and if that works, make an instance of the 'Command' class inside (assuming - it is an instance of BaseCommand), add it to _commands, and report the - addition to the user. Any problems along the way will either be ignored or - reported. +class Command(object): """ - global _commands - - # Strip .py from the end of the filename and join with our package name: - name = ".".join(("commands", filename[:-3])) - try: - __import__(name) - except: - logger.exception("Couldn't load file {0}".format(filename)) - return - - command = sys.modules[name].Command(connection) - if not isinstance(command, BaseCommand): - return + **EarwigBot: Base IRC Command** - _commands[command.name] = command - logger.debug("Added command {0}".format(command.name)) + This package provides built-in IRC "commands" used by the bot's front-end + component. Additional commands can be installed as plugins in the bot's + working directory. -def load(connection): - """Load all valid commands into the _commands global variable. + This class (import with ``from earwigbot.commands import Command``), can be + subclassed to create custom IRC commands. - `connection` is a Connection object that is given to each command's - constructor. + This docstring is reported to the user when they type ``"!help + "``. """ - files = os.listdir(base_dir) - files.sort() - - for filename in files: - if filename.startswith("_") or not filename.endswith(".py"): - continue - try: - _load_command(connection, filename) - except AttributeError: - pass # The file is doesn't contain a command, so just move on - - msg = "Found {0} commands: {1}" - logger.info(msg.format(len(_commands), ", ".join(_commands.keys()))) - -def get_all(): - """Return our dict of all loaded commands.""" - return _commands - -def check(hook, data): - """Given an event on IRC, check if there's anything we can respond to.""" - # Parse command arguments into data.command and data.args: - data.parse_args() - - for command in _commands.values(): - if hook in command.hooks: - if command.check(data): - try: - command.process(data) - except: - logger.exception("Error executing command '{0}'".format(data.command)) - break + # The command's name, as reported to the user when they use !help: + name = None + + # A list of names that will trigger this command. If left empty, it will + # be triggered by the command's name and its name only: + commands = [] + + # Hooks are "msg", "msg_private", "msg_public", and "join". "msg" is the + # default behavior; if you wish to override that, change the value in your + # command subclass: + hooks = ["msg"] + + def __init__(self, bot): + """Constructor for new commands. + + This is called once when the command is loaded (from + :py:meth:`commands.load() `). + *bot* is out base :py:class:`~earwigbot.bot.Bot` object. Don't override + this directly; if you do, remember to place + ``super(Command, self).__init()`` first. Use :py:meth:`setup` for + typical command-init/setup needs. + """ + self.bot = bot + self.config = bot.config + self.logger = bot.commands.logger.getChild(self.name) + + # Convenience functions: + self.say = lambda target, msg, hidelog=False: self.bot.frontend.say(target, msg, hidelog) + self.reply = lambda data, msg, hidelog=False: self.bot.frontend.reply(data, msg, hidelog) + self.action = lambda target, msg, hidelog=False: self.bot.frontend.action(target, msg, hidelog) + self.notice = lambda target, msg, hidelog=False: self.bot.frontend.notice(target, msg, hidelog) + self.join = lambda chan, hidelog=False: self.bot.frontend.join(chan, hidelog) + self.part = lambda chan, msg=None, hidelog=False: self.bot.frontend.part(chan, msg, hidelog) + self.mode = lambda t, level, msg, hidelog=False: self.bot.frontend.mode(t, level, msg, hidelog) + self.ping = lambda target, hidelog=False: self.bot.frontend.ping(target, hidelog) + self.pong = lambda target, hidelog=False: self.bot.frontend.pong(target, hidelog) + + self.setup() + + def __repr__(self): + """Return the canonical string representation of the Command.""" + res = "Command(name={0!r}, commands={1!r}, hooks={2!r}, bot={3!r})" + return res.format(self.name, self.commands, self.hooks, self.bot) + + def __str__(self): + """Return a nice string representation of the Command.""" + return "".format(self.name, self.bot) + + def setup(self): + """Hook called immediately after the command is loaded. + + Does nothing by default; feel free to override. + """ + pass + + def check(self, data): + """Return whether this command should be called in response to *data*. + + Given a :py:class:`~earwigbot.irc.data.Data` instance, return ``True`` + if we should respond to this activity, or ``False`` if we should ignore + it and move on. Be aware that since this is called for each message + sent on IRC, it should be cheap to execute and unlikely to throw + exceptions. + + Most commands return ``True`` only if :py:attr:`data.command + ` ``==`` :py:attr:`self.name `, + or :py:attr:`data.command ` is in + :py:attr:`self.commands ` if that list is overriden. This is + the default behavior; you should only override it if you wish to change + that. + """ + if self.commands: + return data.is_command and data.command in self.commands + return data.is_command and data.command == self.name + + def process(self, data): + """Main entry point for doing a command. + + Handle an activity (usually a message) on IRC. At this point, thanks + to :py:meth:`check` which is called automatically by the command + handler, we know this is something we should respond to. Place your + command's body here. + """ + pass diff --git a/earwigbot/commands/_old.py b/earwigbot/commands/_old.py deleted file mode 100644 index af16e43..0000000 --- a/earwigbot/commands/_old.py +++ /dev/null @@ -1,979 +0,0 @@ -# -*- coding: utf-8 -*- -###### -###### NOTE: -###### This is an old commands file from the previous version of EarwigBot. -###### It is not used by the new EarwigBot and is simply here for reference -###### when developing new commands. -###### -### EarwigBot - -## Import basics. -import sys, socket, string, time, codecs, os, traceback, thread, re, urllib, web, math, unicodedata - -## Import our functions. -import config - -## Set up constants. -HOST, PORT, NICK, IDENT, REALNAME, CHANS, REPORT_CHAN, WELCOME_CHAN, HOST2, CHAN2, OWNER, ADMINS, ADMINS_R, PASS = config.host, config.port, config.nick, config.ident, config.realname, config.chans, config.report_chan, config.welcome_chan, config.host2, config.chan2, config.owner, config.admins, config.admin_readable, config.password - -def get_commandList(): - return {'quiet': 'quiet', - 'welcome': 'welcome', - 'greet': 'welcome', - 'linker': 'linker', - 'auth': 'auth', - 'access': 'access', - 'join': 'join', - 'part': 'part', - 'restart': 'restart', - 'quit': 'quit', - 'die': 'quit', - 'msg': 'msg', - 'me': 'me', - 'calc': 'calc', - 'dice': 'dice', - 'tock': 'tock', - 'beats': 'beats', - 'copyvio': 'copyvio', - 'copy': 'copyvio', - 'copyright': 'copyvio', - 'dict': 'dictionary', - 'dictionary': 'dictionary', - 'ety': 'etymology', - 'etymology': 'etymology', - 'lang': 'langcode', - 'langcode': 'langcode', - 'num': 'number', - 'number': 'number', - 'count': 'number', - 'c': 'number', - 'nick': 'nick', - 'op': 'op', - 'deop': 'deop', - 'voice': 'voice', - 'devoice': 'devoice', - 'pend': 'pending', - 'pending': 'pending', - 'sub': 'submissions', - 'submissions': 'submissions', - 'praise': 'praise', - 'leonard': 'leonard', - 'groovedog': 'groovedog', - 'earwig': 'earwig', - 'macmed': 'macmed', - 'cubs197': 'cubs197', - 'sparksboy': 'sparksboy', - 'tim_song': 'tim_song', - 'tim': 'tim_song', - 'blurpeace': 'blurpeace', - 'sausage': 'sausage', - 'mindstormskid': 'mindstormskid', - 'mcjohn': 'mcjohn', - 'fetchcomms': 'fetchcomms', - 'trout': 'trout', - 'kill': 'kill', - 'destroy': 'kill', - 'murder': 'kill', - 'fish': 'fish', - 'report': 'report', - 'commands': 'commands', - 'help': 'help', - 'doc': 'help', - 'documentation': 'help', - 'mysql': 'mysql', - 'remind': 'reminder', - 'reminder': 'reminder', - 'notes': 'notes', - 'note': 'notes', - 'about': 'notes', - 'data': 'notes', - 'database': 'notes', - 'hash': 'hash', - 'lookup': 'lookup', - 'ip': 'lookup' - } - -def main(command, line, line2, nick, chan, host, auth, notice, say, reply, s): - try: - parse(command, line, line2, nick, chan, host, auth, notice, say, reply, s) - except Exception: - trace = traceback.format_exc() # Traceback. - print trace # Print. - lines = list(reversed(trace.splitlines())) # Convert lines to process traceback.... - report2 = [lines[0].strip()] - for line in lines: - line = line.strip() - if line.startswith('File "/'): - report2.append(line[0].lower() + line[1:]) - break - else: report2.append('source unknown') - say(report2[0] + ' (' + report2[1] + ')', chan) - -def parse(command, line, line2, nick, chan, host, auth, notice, say, reply, s): - authy = auth(host) - if command == "access": - a = 'The bot\'s owner is "%s".' % OWNER - b = 'The bot\'s admins are "%s".' % ', '.join(ADMINS_R) - reply(a, chan, nick) - reply(b, chan, nick) - return - if command == "join": - if authy == "owner" or authy == "admin": - try: - channel = line2[4] - except Exception: - channel = chan - s.send("JOIN %s\r\n" % channel) - else: - reply("You aren't authorized to use that command.", chan, nick) - return - if command == "part": - if authy == "owner" or authy == "admin": - try: - channel = line2[4] - except Exception: - channel = chan - s.send("PART %s\r\n" % channel) - else: - reply("You aren't authorized to use that command.", chan, nick) - return - if command == "restart": - import thread - if authy == "owner": - s.send("QUIT\r\n") - time.sleep(5) - os.system("nice -15 python main.py") - exit() - else: - reply("Only the owner, %s, can stop the bot. This incident will be reported." % OWNER, chan, nick) - return - if command == "quit" or command == "die": - if authy != "owner": - if command != "suicide": - reply("Only the owner, %s, can stop the bot. This incident will be reported." % OWNER, chan, nick) - else: - say("\x01ACTION hands %s a gun... have fun :D\x01" % nick, nick) - else: - if command == "suicide": - say("\x01ACTION stabs himself with a knife.\x01", chan) - time.sleep(0.2) - try: - s.send("QUIT :%s\r\n" % ' '.join(line2[4:])) - except Exception: - s.send("QUIT\r\n") - __import__('os')._exit(0) - return - if command == "msg": - if authy == "owner" or authy == "admin": - say(' '.join(line2[5:]), line2[4]) - else: - reply("You aren't authorized to use that command.", chan, nick) - return - if command == "me": - if authy == "owner" or authy == "admin": - say("\x01ACTION %s\x01" % ' '.join(line2[5:]), line2[4]) - else: - reply("You aren't authorized to use that command.", chan, nick) - return - if command == "calc": - r_result = re.compile(r'(?i)(.*?)') - r_tag = re.compile(r'<\S+.*?>') - subs = [ - (' in ', ' -> '), - (' over ', ' / '), - (u'£', 'GBP '), - (u'€', 'EUR '), - ('\$', 'USD '), - (r'\bKB\b', 'kilobytes'), - (r'\bMB\b', 'megabytes'), - (r'\bGB\b', 'kilobytes'), - ('kbps', '(kilobits / second)'), - ('mbps', '(megabits / second)') - ] - try: - q = ' '.join(line2[4:]) - except Exception: - say("0?", chan) - return - query = q[:] - for a, b in subs: - query = re.sub(a, b, query) - query = query.rstrip(' \t') - - precision = 5 - if query[-3:] in ('GBP', 'USD', 'EUR', 'NOK'): - precision = 2 - query = web.urllib.quote(query.encode('utf-8')) - - uri = 'http://futureboy.us/fsp/frink.fsp?fromVal=' - bytes = web.get(uri + query) - m = r_result.search(bytes) - if m: - result = m.group(1) - result = r_tag.sub('', result) # strip span.warning tags - result = result.replace('>', '>') - result = result.replace('(undefined symbol)', '(?) ') - - if '.' in result: - try: result = str(round(float(result), precision)) - except ValueError: pass - - if not result.strip(): - result = '?' - elif ' in ' in q: - result += ' ' + q.split(' in ', 1)[1] - - say(q + ' = ' + result[:350], chan) - else: reply("Sorry, can't calculate that.", chan, nick) - return - if command == "dice": - import random - try: - set = range(int(line2[4]), int(line2[5]) + 1) - except Exception: - set = range(1, 7) - num = random.choice(set) - reply("You rolled a %s." % num, chan, nick) - if len(set) < 30: - say("Set consisted of %s." % set, nick) - else: - say("Set consisted of %s... and %s others." % (set[:30], len(set) - 30), nick) - return - if command == "tock": - u = urllib.urlopen('http://tycho.usno.navy.mil/cgi-bin/timer.pl') - info = u.info() - u.close() - say('"' + info['Date'] + '" - tycho.usno.navy.mil', chan) - return - if command == "beats": - beats = ((time.time() + 3600) % 86400) / 86.4 - beats = int(math.floor(beats)) - say('@%03i' % beats, chan) - return - if command == "copyvio" or command == "copy" or command == "copyright": - url = "http://en.wikipedia.org/wiki/User:EarwigBot/AfC copyvios" - query = urllib.urlopen(url) - data = query.read() - url = "http://toolserver.org/~earwig/earwigbot/pywikipedia/error.txt" - query = urllib.urlopen(url) - data2 = query.read() - if "critical" in data2: - text = "AfC copyvio situation is CRITICAL: Major disaster." - elif "exceed" in data2: - text = "AfC copyvio situation is CRITICAL: Queries exceeded error." - elif "spam" in data2: - text = "AfC copyvio situation is CRITICAL: Spamfilter error." - elif "

" in data: - text = "AfC copyvio situation is BAD: Unsolved copyvios at [[User:EarwigBot/AfC copyvios]]" - else: - text = "AfC copyvio situation is OK: OK." - reply(text, chan, nick) - return - if command == "dict" or command == "dictionary": - def trim(thing): - if thing.endswith(' '): - thing = thing[:-6] - return thing.strip(' :.') - r_li = re.compile(r'(?ims)
  • .*?
    • ') - r_tag = re.compile(r'<[^>]+>') - r_parens = re.compile(r'(?<=\()(?:[^()]+|\([^)]+\))*(?=\))') - r_word = re.compile(r'^[A-Za-z0-9\' -]+$') - uri = 'http://encarta.msn.com/dictionary_/%s.html' - r_info = re.compile(r'(?:ResultBody">

    (.*?) )|(?:(.*?))') - try: - word = line2[4] - except Exception: - reply("Please enter a word.", chan, nick) - return - word = urllib.quote(word.encode('utf-8')) - bytes = web.get(uri % word) - results = {} - wordkind = None - for kind, sense in r_info.findall(bytes): - kind, sense = trim(kind), trim(sense) - if kind: wordkind = kind - elif sense: - results.setdefault(wordkind, []).append(sense) - result = word.encode('utf-8') + ' - ' - for key in sorted(results.keys()): - if results[key]: - result += (key or '') + ' 1. ' + results[key][0] - if len(results[key]) > 1: - result += ', 2. ' + results[key][1] - result += '; ' - result = result.rstrip('; ') - if result.endswith('-') and (len(result) < 30): - reply('Sorry, no definition found.', chan, nick) - else: say(result, chan) - return - if command == "ety" or command == "etymology": - etyuri = 'http://etymonline.com/?term=%s' - etysearch = 'http://etymonline.com/?search=%s' - r_definition = re.compile(r'(?ims)]*>.*?') - r_tag = re.compile(r'<(?!!)[^>]+>') - r_whitespace = re.compile(r'[\t\r\n ]+') - abbrs = [ - 'cf', 'lit', 'etc', 'Ger', 'Du', 'Skt', 'Rus', 'Eng', 'Amer.Eng', 'Sp', - 'Fr', 'N', 'E', 'S', 'W', 'L', 'Gen', 'J.C', 'dial', 'Gk', - '19c', '18c', '17c', '16c', 'St', 'Capt', 'obs', 'Jan', 'Feb', 'Mar', - 'Apr', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec', 'c', 'tr', 'e', 'g' - ] - t_sentence = r'^.*?(?') - s = s.replace('<', '<') - s = s.replace('&', '&') - return s - def text(html): - html = r_tag.sub('', html) - html = r_whitespace.sub(' ', html) - return unescape(html).strip() - try: - word = line2[4] - except Exception: - reply("Please enter a word.", chan, nick) - return - def ety(word): - if len(word) > 25: - raise ValueError("Word too long: %s[...]" % word[:10]) - word = {'axe': 'ax/axe'}.get(word, word) - bytes = web.get(etyuri % word) - definitions = r_definition.findall(bytes) - if not definitions: - return None - defn = text(definitions[0]) - m = r_sentence.match(defn) - if not m: - return None - sentence = m.group(0) - try: - sentence = unicode(sentence, 'iso-8859-1') - sentence = sentence.encode('utf-8') - except: pass - maxlength = 275 - if len(sentence) > maxlength: - sentence = sentence[:maxlength] - words = sentence[:-5].split(' ') - words.pop() - sentence = ' '.join(words) + ' [...]' - sentence = '"' + sentence.replace('"', "'") + '"' - return sentence + ' - ' + (etyuri % word) - try: - result = ety(word.encode('utf-8')) - except IOError: - msg = "Can't connect to etymonline.com (%s)" % (etyuri % word) - reply(msg, chan, nick) - return - except AttributeError: - result = None - if result is not None: - reply(result, chan, nick) - else: - uri = etysearch % word - msg = 'Can\'t find the etymology for "%s". Try %s' % (word, uri) - reply(msg, chan, nick) - return - if command == "num" or command == "number" or command == "count" or command == "c": - try: - params = string.lower(line2[4]) - except Exception: - params = False - if params == "old" or params == "afc" or params == "a": - number = unicode(int(len(re.findall("title=", urllib.urlopen("http://en.wikipedia.org/w/api.php?action=query&list=categorymembers&cmtitle=Category:Pending_AfC_submissions&cmlimit=500").read()))) - 2) - reply("There are currently %s pending AfC submissions." % number, chan, nick) - elif params == "redirect" or params == "redir" or params == "redirs" or params == "redirects" or params == "r": - redir_data = urllib.urlopen("http://en.wikipedia.org/w/index.php?title=Wikipedia:Articles_for_creation/Redirects").read() - redirs = (string.count(redir_data, "

    ") - 1) - (string.count(redir_data, '')) - reply("There are currently %s open redirect requests." % redirs, chan, nick) - elif params == "files" or params == "ffu" or params == "file" or params == "image" or params == "images" or params == "ifu" or params == "f": - file_data = re.sub("

    Contents

    ", "", urllib.urlopen("http://en.wikipedia.org/w/index.php?title=Wikipedia:Files_for_upload").read()) - files = (string.count(file_data, "

    ") - 1) - (string.count(file_data, '

    ')) - reply("There are currently %s open file upload requests." % files, chan, nick) - elif params == "aggregate" or params == "agg": - subs = unicode(int(len(re.findall("title=", urllib.urlopen("http://en.wikipedia.org/w/api.php?action=query&list=categorymembers&cmtitle=Category:Pending_AfC_submissions&cmlimit=500").read()))) - 2) - redir_data = urllib.urlopen("http://en.wikipedia.org/w/index.php?title=Wikipedia:Articles_for_creation/Redirects").read() - file_data = re.sub("

    Contents

    ", "", urllib.urlopen("http://en.wikipedia.org/w/index.php?title=Wikipedia:Files_for_upload").read()) - redirs = (string.count(redir_data, "

    ")) - (string.count(redir_data, '

    ')) - files = (string.count(file_data, "

    ") - 1) - (string.count(file_data, '

    ')) - aggregate = (int(subs) * 5) + (int(redirs) * 2) + (int(files) * 2) - if aggregate == 0: - stat = "clear" - elif aggregate < 60: - stat = "almost clear" - elif aggregate < 125: - stat = "small backlog" - elif aggregate < 175: - stat = "average backlog" - elif aggregate < 250: - stat = "backlogged" - elif aggregate < 300: - stat = "heavily backlogged" - else: - stat = "severely backlogged" - reply("Aggregate is currently %s (%s)." % (aggregate, stat), chan, nick) - else: - subs = unicode(int(len(re.findall("title=", urllib.urlopen("http://en.wikipedia.org/w/api.php?action=query&list=categorymembers&cmtitle=Category:Pending_AfC_submissions&cmlimit=500").read()))) - 2) - redir_data = urllib.urlopen("http://en.wikipedia.org/w/index.php?title=Wikipedia:Articles_for_creation/Redirects").read() - file_data = re.sub("

    Contents

    ", "", urllib.urlopen("http://en.wikipedia.org/w/index.php?title=Wikipedia:Files_for_upload").read()) - redirs = (string.count(redir_data, "

    ")) - (string.count(redir_data, '

    ')) - files = (string.count(file_data, "

    ") - 1) - (string.count(file_data, '

    ')) - reply("There are currently %s pending submissions, %s open redirect requests, and %s open file upload requests." % (subs, redirs, files), chan, nick) - return - if command == "nick": - if authy == "owner": - try: - new_nick = line2[4] - except Exception: - reply("Please specify a nick to change to.", chan, nick) - return - s.send("NICK %s\r\n" % new_nick) - else: - reply("You aren't authorized to use that command.", chan, nick) - return - if command == "op" or command == "deop" or command == "voice" or command == "devoice": - if authy == "owner" or authy == "admin": - try: - user = line2[4] - except Exception: - user = nick - say("%s %s %s" % (command, chan, user), "ChanServ") - else: - reply("You aren't authorized to use that command.", chan, nick) - return - if command == "pend" or command == "pending": - say("Pending submissions status page: .", chan) - say("Pending submissions category: .", chan) - return - if command == "sub" or command == "submissions": - try: - number = int(line2[4]) - except Exception: - reply("Please enter a number.", chan, nick) - return - do_url = False - try: - if "url" in line2[5:]: do_url = True - except Exception: - pass - url = "http://en.wikipedia.org/w/api.php?action=query&list=categorymembers&cmtitle=Category:Pending_AfC_submissions&cmlimit=500&cmsort=timestamp" - query = urllib.urlopen(url) - data = query.read() - pages = re.findall("title="(.*?)"", data) - try: - pages.remove("Wikipedia:Articles for creation/Redirects") - except Exception: - pass - try: - pages.remove("Wikipedia:Files for upload") - except Exception: - pass - pages.reverse() - pages = pages[:number] - if not do_url: - s = string.join(pages, "]], [[") - s = "[[%s]]" % s - else: - s = string.join(pages, ">, ,_<", ">, <", s) - report = "\x02First %s pending AfC submissions:\x0F %s" % (number, s) - say(report, chan) - return - if command == "praise" or command == "leonard" or command == "groovedog" or command == "earwig" or command == "macmed" or command == "cubs197" or command == "sparksboy" or command == "tim_song" or command == "tim" or command == "sausage" or command == "mindstormskid" or command == "mcjohn" or command == "fetchcomms" or command == "blurpeace": - bad = False - if command == "leonard": - special = "AfC redirect reviewer" - user = "Leonard^Bloom" - elif command == "groovedog": - special = "heh" - user = "GrooveDog" - elif command == "earwig": - special = "Python programmer" - user = "Earwig" - elif command == "macmed": - special = "CSD tagger" - user = "MacMed" - elif command == "mindstormskid": - special = "Lego fanatic" - user = "MindstormsKid" - elif command == "cubs197": - special = "IRC dude" - user = "Cubs197" - elif command == "sparksboy": - special = "pet owner" - user = "SparksBoy" - elif command == "tim_song" or command == "tim": - special = "JavaScript programmer" - user = "Tim_Song" - elif command == "sausage": - special = "helper" - user = "chzz" - elif command == "mcjohn": - special = "edit summary writer" - user = "McJohn" - elif command == "fetchcomms": - special = "n00b" - user = "Fetchcomms" - elif command == "blurpeace": - special = "Commons admin" - user = "Blurpeace" - else: - say("Only a true fool would use that command, %s." % nick, chan) - # say("The users who you can praise are: Leonard^Bloom, GrooveDog, Earwig, MacMed, Cubs197, SparksBoy, MindstormsKid, Chzz, McJohn, Tim_Song, Fetchcomms, and Blurpeace.", chan) - return - if not bad: - say("\x02%s\x0F is the bestest %s evah!" % (user, special), chan) - if bad: - say("\x02%s\x0F is worstest %s evah!" % (user, special), chan) - return - if command == "trout": - try: - user = line2[4] - user = ' '.join(line2[4:]) - except Exception: - reply("Hahahahahahahaha...", chan, nick) - return - normal = unicodedata.normalize('NFKD', unicode(string.lower(user))) - if "itself" in normal: - reply("I'm not that stupid ;)", chan, nick) - return - elif "earwigbot" in normal: - reply("I'm not that stupid ;)", chan, nick) - elif "earwig" not in normal and "ear wig" not in normal: - text = 'slaps %s around a bit with a large trout.' % user - msg = '\x01ACTION %s\x01' % text - say(msg, chan) - else: - reply("I refuse to hurt anything with \"Earwig\" in its name :P", chan, nick) - return - if command == "kill" or command == "destroy" or command == "murder": - reply("Who do you think I am? The Mafia?", chan, nick) - return - if command == "fish": - try: - user = line2[4] - fish = ' '.join(line2[5:]) - except Exception: - reply("Hahahahahahahaha...", chan, nick) - return - normal = unicodedata.normalize('NFKD', unicode(string.lower(user))) - if "itself" in normal: - reply("I'm not that stupid ;)", chan, nick) - return - elif "earwigbot" in normal: - reply("I'm not that stupid ;)", chan, nick) - elif "earwig" not in normal and "ear wig" not in normal: - text = 'slaps %s around a bit with a %s.' % (user, fish) - msg = '\x01ACTION %s\x01' % text - say(msg, chan) - else: - reply("I refuse to hurt anything with \"Earwig\" in its name :P", chan, nick) - return - if command == "report": - def find_status(name="", talk=False): - enname = re.sub(" ", "_", name) - if talk == True: - enname = "Wikipedia_talk:Articles_for_creation/%s" % enname - if talk == False: - enname = "Wikipedia:Articles_for_creation/%s" % enname - url = "http://en.wikipedia.org/w/api.php?action=query&titles=%s&prop=revisions&rvprop=content" % enname - query = urllib.urlopen(url) - data = query.read() - status = "" - if "{{AFC submission|D" in data or "{{AFC submission|d" in data: - reason = re.findall("(D|d)\|(.*?)\|", data) - if reason[0][1] != "reason": - status = "Declined, reason is '%s'" % reason[0][1] - if reason[0][1] == "reason": - status = "Declined, reason is a custom reason" - if "{{AFC submission|H" in data or "{{AFC submission|h" in data: - reason = re.findall("(H|h)\|(.*?)\|", data) - if reason[0][1] != "reason": - status = "Held, reason is '%s'" % reason[0][1] - if reason[0][1] == "reason": - status = "Held, reason is a custom reason" - if "{{AFC submission||" in data: - status = "Pending" - if "{{AFC submission|R" in data or "{{AFC submission|r" in data: - status = "Reviewing" - if not status: - exist = exists(name=enname) - if exist == True: - status = "Accepted" - if exist == False: - status = "Not found" - return status - def exists(name=""): - url = "http://en.wikipedia.org/wiki/%s" % name - query = urllib.urlopen(url) - data = query.read() - if "Wikipedia does not have a" in data: - return False - return True - def get_submitter(name="", talk=False): - enname = re.sub(" ", "_", name) - if talk == True: - enname = "Wikipedia_talk:Articles_for_creation/%s" % enname - if talk == False: - enname = "Wikipedia:Articles_for_creation/%s" % enname - url = "http://en.wikipedia.org/w/api.php?action=query&titles=%s&prop=revisions&rvprop=user&rvdir=newer&rvlimit=1" % enname - query = urllib.urlopen(url) - data = query.read() - extract = re.findall("user="(.*?)"", data) - if "anon=" in data: - anon = True - else: - anon = False - try: - return extract[0], anon - except BaseException: - print extract - return "", anon - try: - rawSub = line2[4] - rawSub = ' '.join(line2[4:]) - except Exception: - reply("You need to specify a submission name in order to use %s!" % command, chan, nick) - return - talk = False - if "[[" in rawSub and "]]" in rawSub: - name = re.sub("\[\[(.*)\]\]", "\\1", rawSub) - name = re.sub(" ", "_", name) - name = urllib.quote(name, ":/") - name = "http://en.wikipedia.org/wiki/%s" % name - if "talk:" in name: - talk = True - elif "http://" in rawSub: - name = rawSub - if "talk:" in name: - talk = True - elif "en.wikipedia.org" in rawSub: - name = "http://%s" % rawSub - if "talk:" in name: - talk = True - elif "Wikipedia:" in rawSub or "Wikipedia_talk:" in rawSub or "Wikipedia talk:" in rawSub: - name = re.sub(" ", "_", rawSub) - name = urllib.quote(name, ":/") - name = "http://en.wikipedia.org/wiki/%s" % name - if "talk:" in name: - talk = True - else: - url = "http://en.wikipedia.org/wiki/" - pagename = re.sub(" ", "_", rawSub) - pagename = urllib.quote(pagename, ":/") - pagename = "Wikipedia:Articles_for_creation/%s" % pagename - page = urllib.urlopen("%s%s" % (url, pagename)) - text = page.read() - name = "http://en.wikipedia.org/wiki/%s" % pagename - if "Wikipedia does not have a" in text: - pagename = re.sub(" ", "_", rawSub) - pagename = urllib.quote(pagename, ":/") - pagename = "Wikipedia_talk:Articles_for_creation/%s" % pagename - page = urllib.urlopen("%s%s" % (url, pagename)) - name = "http://en.wikipedia.org/wiki/%s" % pagename - talk = True - unname = re.sub("http://en.wikipedia.org/wiki/Wikipedia:Articles_for_creation/", "", name) - unname = re.sub("http://en.wikipedia.org/wiki/Wikipedia_talk:Articles_for_creation/", "", unname) - unname = re.sub("_", " ", unname) - if "talk" in unname: - talk = True - submitter, anon = get_submitter(name=unname, talk=talk) - status = find_status(name=unname, talk=talk) - if submitter != "": - if anon == True: - submitter_page = "Special:Contributions/%s" % submitter - if anon == False: - unsubmit = re.sub(" ", "_", submitter) - unsubmit = urllib.quote(unsubmit, ":/") - submitter_page = "User:%s" % unsubmit - if status == "Accepted": - submitterm = "Reviewer" - else: - submitterm = "Submitter" - line1 = "\x02AfC submission report for %s:" % unname - line2 = "\x02URL: \x0301\x0F%s" % name - if submitter != "": - line3 = "\x02%s: \x0F\x0302%s (\x0301\x0Fhttp://en.wikipedia.org/wiki/%s)." % (submitterm, submitter, submitter_page) - line4 = "\x02Status: \x0F\x0302%s." % status - say(line1, chan) - time.sleep(0.1) - say(line2, chan) - time.sleep(0.1) - if submitter != "": - say(line3, chan) - time.sleep(0.1) - say(line4, chan) - return - if command == "commands": - if chan.startswith("#"): - reply("Please use that command in a private message.", chan, nick) - return - others2 = get_commandList().values() - others = [] - for com in others2: - if com == "copyvio" or com == "number" or com == "pending" or com == "report" or com == "submissions" or com == "access" or com == "help" or com == "join" or com == "linker" or com == "nick" or com == "op" or com == "part" or com == "quiet" or com == "quit" or com == "restart" or com == "voice" or com == "welcome" or com == "fish" or com == "praise" or com == "trout" or com == "notes": - continue - if com in others: continue - others.append(com) - others.sort() - say("\x02AFC commands:\x0F copyvio, number, pending, report, submissions.", chan) - time.sleep(0.1) - say("\x02Bot operation and channel maintaince commands:\x0F access, help, join, linker, nick, op, part, quiet, quit, restart, voice, welcome.", chan) - time.sleep(0.1) - say("\x02Fun commands:\x0F fish, praise, trout, and numerous easter eggs", chan) - time.sleep(0.1) - say("\x02Other commands:\x0F %s" % ', '.join(others), chan) - time.sleep(0.1) - say("The bot maintains a mini-wiki. Type \"!notes help\" for more information.", chan) - time.sleep(0.1) - say("See http://enwp.org/User:The_Earwig/Bots/IRC for details. For help on a specific command, type '!help command'.", chan) - return - if command == "help" or command == "doc" or command == "documentation": - try: - com = line2[4] - except Exception: - reply("Hi, I'm a bot that does work for Articles for Creation. You can find information about me at http://enwp.org/User:The_Earwig/Bots/IRC. Say \"!commands\" to me in a private message for some of my abilities. Earwig is my owner and creator, and you can contact him at http://enwp.org/User_talk:The_Earwig.", chan, nick) - return - say("Sorry, command documentation has not been implemented yet.", chan) - return - if command == "mysql": - if authy != "owner": - reply("You aren't authorized to use this command.", chan, nick) - return - import MySQLdb - try: - strings = line2[4] - strings = ' '.join(line2[4:]) - if "db:" in strings: - database = re.findall("db\:(.*?)\s", strings)[0] - else: - database = "enwiki_p" - if "time:" in strings: - times = int(re.findall("time\:(.*?)\s", strings)[0]) - else: - times = 60 - file = re.findall("file\:(.*?)\s", strings)[0] - sqlquery = re.findall("query\:(.*?)\Z", strings)[0] - except Exception: - reply("You did not specify enough data for the bot to continue.", chan, nick) - return - database2 = database[:-2] + "-p" - db = MySQLdb.connect(db=database, host="%s.rrdb.toolserver.org" % database2, read_default_file="/home/earwig/.my.cnf") - db.query(sqlquery) - r = db.use_result() - data = r.fetch_row(0) - try: - f = codecs.open("/home/earwig/public_html/reports/%s/%s" % (database[:-2], file), 'r') - reply("A file already exists with that name.", chan, nick) - return - except Exception: - pass - f = codecs.open("/home/earwig/public_html/reports/%s/%s" % (database[:-2], file), 'a', 'utf-8') - for line in data: - new_line = [] - for l in line: - new_line.append(str(l)) - f.write(' '.join(new_line) + "\n") - f.close() - reply("Query completed successfully. See http://toolserver.org/~earwig/reports/%s/%s. I will delete the report in %s seconds." % (database[:-2], file, times), chan, nick) - time.sleep(times) - os.remove("/home/earwig/public_html/reports/%s/%s" % (database[:-2], file)) - return - if command == "remind" or command == "reminder": - try: - times = int(line2[4]) - content = ' '.join(line2[5:]) - except Exception: - reply("Please specify a time and a note in the following format: !remind