diff --git a/.gitignore b/.gitignore index c963c06..4984243 100644 --- a/.gitignore +++ b/.gitignore @@ -1,4 +1,6 @@ *.pyc +*.egg *.egg-info .DS_Store -build/ +build +docs/_build diff --git a/README.md b/README.md deleted file mode 100644 index 3ecc02b..0000000 --- a/README.md +++ /dev/null @@ -1,39 +0,0 @@ -[EarwigBot](http://en.wikipedia.org/wiki/User:EarwigBot) is a -[Python](http://python.org/) robot that edits -[Wikipedia](http://en.wikipedia.org/) and interacts with people over -[IRC](http://en.wikipedia.org/wiki/Internet_Relay_Chat). - -# History - -Development began, based on the -[Pywikipedia framework](http://pywikipediabot.sourceforge.net/), in early 2009. -Approval for its fist task, a -[copyright violation detector](http://en.wikipedia.org/wiki/Wikipedia:Bots/Requests_for_approval/EarwigBot_1), -was carried out in May, and the bot has been running consistently ever since -(with the exception of Jan/Feb 2011). It currently handles -[several ongoing tasks](http://en.wikipedia.org/wiki/User:EarwigBot#Tasks), -ranging from statistics generation to category cleanup, and on-demand tasks -such as WikiProject template tagging. Since it started running, the bot has -made over 45,000 edits. - -A project to rewrite it from scratch began in early April 2011, thus moving -away from the Pywikipedia framework and allowing for less overall code, better -integration between bot parts, and easier maintenance. - -# Installation - -## Dependencies - -EarwigBot uses the MySQL library -[oursql](http://packages.python.org/oursql/) (>= 0.9.2) for communicating with -MediaWiki databases, and some tasks use their own tables for storage. -Additionally, the afc_history task uses -[matplotlib](http://matplotlib.sourceforge.net/) and -[numpy](http://numpy.scipy.org/) for graphing AfC statistics. Neither of these -modules are required for the main bot itself. - -`earwigbot.wiki.copyright` requires access to a search engine for detecting -copyright violations. Currently, -[Yahoo! BOSS](http://developer.yahoo.com/search/boss/) is the only engine -supported, and this requires -[oauth2](https://github.com/simplegeo/python-oauth2). diff --git a/README.rst b/README.rst new file mode 100644 index 0000000..cafdc9f --- /dev/null +++ b/README.rst @@ -0,0 +1,510 @@ +EarwigBot +========= + +EarwigBot_ is a Python_ robot that edits Wikipedia_ and interacts with people +over IRC_. This file provides a basic overview of how to install and setup the +bot; more detailed information is located in the ``docs/`` directory (available +online at PyPI_). + +History +------- + +Development began, based on the `Pywikipedia framework`_, in early 2009. +Approval for its fist task, a `copyright violation detector`_, was carried out +in May, and the bot has been running consistently ever since (with the +exception of Jan/Feb 2011). It currently handles `several ongoing tasks`_ +ranging from statistics generation to category cleanup, and on-demand tasks +such as WikiProject template tagging. Since it started running, the bot has +made over 50,000 edits. + +A project to rewrite it from scratch began in early April 2011, thus moving +away from the Pywikipedia framework and allowing for less overall code, better +integration between bot parts, and easier maintenance. + +Installation +------------ + +This package contains the core ``earwigbot``, abstracted enough that it should +be usable and customizable by anyone running a bot on a MediaWiki site. Since +it is component-based, the IRC components can be disabled if desired. IRC +commands and bot tasks specific to `my instance of EarwigBot`_ that I don't +feel the average user will need are available from the repository +`earwigbot-plugins`_. + +It's recommended to run the bot's unit tests before installing. Run ``python +setup.py test`` from the project's root directory. Note that some +tests require an internet connection, and others may take a while to run. +Coverage is currently rather incomplete. + +Latest release (v0.1) +~~~~~~~~~~~~~~~~~~~~~ + +EarwigBot is available from the `Python Package Index`_, so you can install the +latest release with ``pip install earwigbot`` (`get pip`_). + +You can also install it from source [1]_ directly:: + + curl -Lo earwigbot.tgz https://github.com/earwig/earwigbot/tarball/v0.1 + tar -xf earwigbot.tgz + cd earwig-earwigbot-* + python setup.py install + cd .. + rm -r earwigbot.tgz earwig-earwigbot-* + +Development version +~~~~~~~~~~~~~~~~~~~ + +You can install the development version of the bot from ``git`` by using +setuptools/distribute's ``develop`` command [1]_, probably on the ``develop`` +branch which contains (usually) working code. ``master`` contains the latest +release. EarwigBot uses `git flow`_, so you're free to +browse by tags or by new features (``feature/*`` branches):: + + git clone git://github.com/earwig/earwigbot.git earwigbot + cd earwigbot + python setup.py develop + +Setup +----- + +The bot stores its data in a "working directory", including its config file and +databases. This is also the location where you will place custom IRC commands +and bot tasks, which will be explained later. It doesn't matter where this +directory is, as long as the bot can write to it. + +Start the bot with ``earwigbot path/to/working/dir``, or just ``earwigbot`` if +the working directory is the current directory. It will notice that no +``config.yml`` file exists and take you through the setup process. + +There is currently no way to edit the ``config.yml`` file from within the bot +after it has been created, but YAML is a very straightforward format, so you +should be able to make any necessary changes yourself. Check out the +`explanation of YAML`_ on Wikipedia for help. + +After setup, the bot will start. This means it will connect to the IRC servers +it has been configured for, schedule bot tasks to run at specific times, and +then wait for instructions (as commands on IRC). For a list of commands, say +"``!help``" (commands are messages prefixed with an exclamation mark). + +You can stop the bot at any time with Control+C, same as you stop a normal +Python program, and it will try to exit safely. You can also use the +"``!quit``" command on IRC. + +Customizing +----------- + +The bot's working directory contains a ``commands`` subdirectory and a +``tasks`` subdirectory. Custom IRC commands can be placed in the former, +whereas custom wiki bot tasks go into the latter. Developing custom modules is +explained below, and in more detail through the bot's documentation on PyPI_. + +Note that custom commands will override built-in commands and tasks with the +same name. + +``Bot`` and ``BotConfig`` +~~~~~~~~~~~~~~~~~~~~~~~~~ + +`earwigbot.bot.Bot`_ is EarwigBot's main class. You don't have to instantiate +this yourself, but it's good to be familiar with its attributes and methods, +because it is the main way to communicate with other parts of the bot. A +``Bot`` object is accessible as an attribute of commands and tasks (i.e., +``self.bot``). + +The most useful attributes are: + +- ``bot.config``: an instance of ``BotConfig``, for accessing the bot's + configuration data (see below). + +- ``bot.commands``: the bot's ``CommandManager``, which is used internally to + run IRC commands (through ``bot.commands.call()``, which you shouldn't have + to use); you can safely reload all commands with ``bot.commands.load()``. + +- ``bot.tasks``: the bot's ``TaskManager``, which can be used to start tasks + with ``bot.tasks.start(task_name, **kwargs)``. ``bot.tasks.load()`` can be + used to safely reload all tasks. + +- ``bot.frontend`` / ``bot.watcher``: instances of ``earwigbot.irc.Frontend`` + and ``earwigbot.irc.Watcher``, respectively, which represent the bot's + connections to these two servers; you can, for example, send a message to the + frontend with ``bot.frontend.say(chan, msg)`` (more on communicating with IRC + below). + +- ``bot.wiki``: interface with `the Wiki Toolset`_ (see below). + +- Finally, ``bot.restart()`` (restarts IRC components and reloads config, + commands, and tasks) and ``bot.stop()`` can be used almost anywhere. Both + take an optional "reason" that will be logged and used as the quit message + when disconnecting from IRC. + +`earwigbot.config.BotConfig`_ stores configuration information for the bot. Its +docstring explains what each attribute is used for, but essentially each "node" +(one of ``config.components``, ``wiki``, ``tasks``, ``irc``, and ``metadata``) +maps to a section of the bot's ``config.yml`` file. For example, if +``config.yml`` includes something like:: + + irc: + frontend: + nick: MyAwesomeBot + channels: + - "##earwigbot" + - "#channel" + - "#other-channel" + +...then ``config.irc["frontend"]["nick"]`` will be ``"MyAwesomeBot"`` and +``config.irc["frontend"]["channels"]`` will be ``["##earwigbot", "#channel", +"#other-channel"]``. + +Custom IRC commands +~~~~~~~~~~~~~~~~~~~ + +Custom commands are subclasses of `earwigbot.commands.BaseCommand`_ that +override ``BaseCommand``'s ``process()`` (and optionally ``check()``) methods. + +``BaseCommand``'s docstrings should explain what each attribute and method is +for and what they should be overridden with, but these are the basics: + +- Class attribute ``name`` is the name of the command. This must be specified. + +- Class attribute ``hooks`` is a list of the "IRC events" that this command + might respond to. It defaults to ``["msg"]``, but options include + ``"msg_private"`` (for private messages only), ``"msg_public"`` (for channel + messages only), and ``"join"`` (for when a user joins a channel). See the + afc_status_ plugin for a command that responds to other hook types. + +- Method ``check()`` is passed a ``Data`` [2]_ object, and should return + ``True`` if you want to respond to this message, or ``False`` otherwise. The + default behavior is to return ``True`` only if ``data.is_command`` is + ``True`` and ``data.command == self.name``, which is suitable for most cases. + A common, straightforward reason for overriding is if a command has aliases + (see chanops_ for an example). Note that by returning ``True``, you prevent + any other commands from responding to this message. + +- Method ``process()`` is passed the same ``Data`` object as ``check()``, but + only if ``check()`` returned ``True``. This is where the bulk of your command + goes. To respond to IRC messages, there are a number of methods of + ``BaseCommand`` at your disposal. See the the test_ command for a simple + example, or look in BaseCommand's ``__init__`` method for the full list. + + The most common ones are ``self.say(chan_or_user, msg)``, + ``self.reply(data, msg)`` (convenience function; sends a reply to the + issuer of the command in the channel it was received), + ``self.action(chan_or_user, msg)``, ``self.notice(chan_or_user, msg)``, + ``self.join(chan)``, and ``self.part(chan)``. + +It's important to name the command class ``Command`` within the file, or else +the bot might not recognize it as a command. The name of the file doesn't +really matter and need not match the command's name, but this is recommended +for readability. + +The bot has a wide selection of built-in commands and plugins to act as sample +code and/or to give ideas. Start with test_, and then check out chanops_ and +afc_status_ for some more complicated scripts. + +Custom bot tasks +~~~~~~~~~~~~~~~~ + +Custom tasks are subclasses of `earwigbot.tasks.BaseTask`_ that override +``BaseTask``'s ``run()`` (and optionally ``setup()``) methods. + +``BaseTask``'s docstrings should explain what each attribute and method is for +and what they should be overridden with, but these are the basics: + +- Class attribute ``name`` is the name of the task. This must be specified. + +- Class attribute ``number`` can be used to store an optional "task number", + possibly for use in edit summaries (to be generated with ``make_summary()``). + For example, EarwigBot's ``config.wiki["summary"]`` is + ``"([[WP:BOT|Bot]]; [[User:EarwigBot#Task $1|Task $1]]): $2"``, which the + task class's ``make_summary(comment)`` method will take and replace ``$1`` + with the task number and ``$2`` with the details of the edit. + + Additionally, ``shutoff_enabled()`` (which checks whether the bot has been + told to stop on-wiki by checking the content of a particular page) can check + a different page for each task using similar variables. EarwigBot's + ``config.wiki["shutoff"]["page"]`` is ``"User:$1/Shutoff/Task $2"``; ``$1`` + is substituted with the bot's username, and ``$2`` is substituted with the + task number, so, e.g., task #14 checks the page + ``[[User:EarwigBot/Shutoff/Task 14]].`` If the page's content does *not* + match ``config.wiki["shutoff"]["disabled"]`` (``"run"`` by default), then + shutoff is considered to be *enabled* and ``shutoff_enabled()`` will return + ``True``, indicating the task should not run. If you don't intend to use + either of these methods, feel free to leave this attribute blank. + +- Method ``setup()`` is called *once* with no arguments immediately after the + task is first loaded. Does nothing by default; treat it like an + ``__init__()`` if you want (``__init__()`` does things by default and a + dedicated setup method is often easier than overriding ``__init__()`` and + using ``super``). + +- Method ``run()`` is called with any number of keyword arguments every time + the task is executed (by ``bot.tasks.start(task_name, **kwargs)``, usually). + This is where the bulk of the task's code goes. For interfacing with + MediaWiki sites, read up on `the Wiki Toolset`_ below. + +Tasks have access to ``config.tasks[task_name]`` for config information, which +is a node in ``config.yml`` like every other attribute of ``bot.config``. This +can be used to store, for example, edit summaries, or templates to append to +user talk pages, so that these can be easily changed without modifying the task +itself. + +It's important to name the task class ``Task`` within the file, or else the bot +might not recognize it as a task. The name of the file doesn't really matter +and need not match the task's name, but this is recommended for readability. + +See the built-in wikiproject_tagger_ task for a relatively straightforward +task, or the afc_statistics_ plugin for a more complicated one. + +The Wiki Toolset +---------------- + +EarwigBot's answer to the `Pywikipedia framework`_ is the Wiki Toolset +(``earwigbot.wiki``), which you will mainly access through ``bot.wiki``. + +``bot.wiki`` provides three methods for the management of Sites - +``get_site()``, ``add_site()``, and ``remove_site()``. Sites are objects that +simply represent a MediaWiki site. A single instance of EarwigBot (i.e. a +single *working directory*) is expected to relate to a single site or group of +sites using the same login info (like all WMF wikis with CentralAuth). + +Load your default site (the one that you picked during setup) with +``site = bot.wiki.get_site()``. + +Dealing with other sites +~~~~~~~~~~~~~~~~~~~~~~~~ + +*Skip this section if you're only working with one site.* + +If a site is *already known to the bot* (meaning that it is stored in the +``sites.db`` file, which includes just your default wiki at first), you can +load a site with ``site = bot.wiki.get_site(name)``, where ``name`` might be +``"enwiki"`` or ``"frwiktionary"`` (you can also do +``site = bot.wiki.get_site(project="wikipedia", lang="en")``). Recall that not +giving any arguments to ``get_site()`` will return the default site. + +``add_site()`` is used to add new sites to the sites database. It may be called +with similar arguments as ``get_site()``, but the difference is important. +``get_site()`` only needs enough information to identify the site in its +database, which is usually just its name; the database stores all other +necessary connection info. With ``add_site()``, you need to provide enough +connection info so the toolset can successfully access the site's API/SQL +databases and store that information for later. That might not be much; for +WMF wikis, you can usually use code like this:: + + project, lang = "wikipedia", "es" + try: + site = bot.wiki.get_site(project=project, lang=lang) + except earwigbot.SiteNotFoundError: + # Load site info from http://es.wikipedia.org/w/api.php: + site = bot.wiki.add_site(project=project, lang=lang) + +This works because EarwigBot assumes that the URL for the site is +``"//{lang}.{project}.org"`` and the API is at ``/w/api.php``; this might +change if you're dealing with non-WMF wikis, where the code might look +something more like:: + + project, lang = "mywiki", "it" + try: + site = bot.wiki.get_site(project=project, lang=lang) + except earwigbot.SiteNotFoundError: + # Load site info from http://mysite.net/mywiki/it/s/api.php: + base_url = "http://mysite.net/" + project + "/" + lang + db_name = lang + project + "_p" + sql = {host: "sql.mysite.net", db: db_name} + site = bot.wiki.add_site(base_url=base_url, script_path="/s", sql=sql) + +``remove_site()`` does the opposite of ``add_site()``: give it a site's name +or a project/lang pair like ``get_site()`` takes, and it'll remove that site +from the sites database. + +Sites +~~~~~ + +``Site`` objects provide the following attributes: + +- ``name``: the site's name (or "wikiid"), like ``"enwiki"`` +- ``project``: the site's project name, like ``"wikipedia"`` +- ``lang``: the site's language code, like ``"en"`` +- ``domain``: the site's web domain, like ``"en.wikipedia.org"`` + +and the following methods: + +- ``api_query(**kwargs)``: does an API query with the given keyword arguments + as params +- ``sql_query(query, params=(), ...)``: does an SQL query and yields its + results (as a generator) +- ``get_replag()``: returns the estimated database replication lag (if we have + the site's SQL connection info) +- ``namespace_id_to_name(id, all=False)``: given a namespace ID, returns the + primary associated namespace name (or a list of all names when ``all`` is + ``True``) +- ``namespace_name_to_id(name)``: given a namespace name, returns the + associated namespace ID +- ``get_page(title, follow_redirects=False)``: returns a ``Page`` object for + the given title (or a ``Category`` object if the page's namespace is + "``Category:``") +- ``get_category(catname, follow_redirects=False)``: returns a ``Category`` + object for the given title (sans namespace) +- ``get_user(username)``: returns a ``User`` object for the given username + +Pages (and Categories) +~~~~~~~~~~~~~~~~~~~~~~ + +Create ``Page`` objects with ``site.get_page(title)``, +``page.toggle_talk()``, ``user.get_userpage()``, or ``user.get_talkpage()``. +They provide the following attributes: + +- ``title``: the page's title, or pagename +- ``exists``: whether the page exists +- ``pageid``: an integer ID representing the page +- ``url``: the page's URL +- ``namespace``: the page's namespace as an integer +- ``protection``: the page's current protection status +- ``is_talkpage``: ``True`` if the page is a talkpage, else ``False`` +- ``is_redirect``: ``True`` if the page is a redirect, else ``False`` + +and the following methods: + +- ``reload()``: forcibly reload the page's attributes (emphasis on *reload* - + this is only necessary if there is reason to believe they have changed) +- ``toggle_talk(...)``: returns a content page's talk page, or vice versa +- ``get()``: returns page content +- ``get_redirect_target()``: if the page is a redirect, returns its destination +- ``get_creator()``: returns a ``User`` object representing the first user to + edit the page +- ``edit(text, summary, minor=False, bot=True, force=False)``: replaces the + page's content with ``text`` or creates a new page +- ``add_section(text, title, minor=False, bot=True, force=False)``: adds a new + section named ``title`` at the bottom of the page +- ``copyvio_check(...)``: checks the page for copyright violations +- ``copyvio_compare(url, ...)``: checks the page like ``copyvio_check()``, but + against a specific URL + +Additionally, ``Category`` objects (created with ``site.get_category(name)`` or +``site.get_page(title)`` where ``title`` is in the ``Category:`` namespace) +provide the following additional method: + +- ``get_members(use_sql=False, limit=None)``: returns a list of page titles in + the category (limit is ``50`` by default if using the API) + +Users +~~~~~ + +Create ``User`` objects with ``site.get_user(name)`` or +``page.get_creator()``. They provide the following attributes: + +- ``name``: the user's username +- ``exists``: ``True`` if the user exists, or ``False`` if they do not +- ``userid``: an integer ID representing the user +- ``blockinfo``: information about any current blocks on the user (``False`` if + no block, or a dict of ``{"by": blocking_user, "reason": block_reason, + "expiry": block_expire_time}``) +- ``groups``: a list of the user's groups +- ``rights``: a list of the user's rights +- ``editcount``: the number of edits made by the user +- ``registration``: the time the user registered as a ``time.struct_time`` +- ``emailable``: ``True`` if you can email the user, ``False`` if you cannot +- ``gender``: the user's gender (``"male"``, ``"female"``, or ``"unknown"``) + +and the following methods: + +- ``reload()``: forcibly reload the user's attributes (emphasis on *reload* - + this is only necessary if there is reason to believe they have changed) +- ``get_userpage()``: returns a ``Page`` object representing the user's + userpage +- ``get_talkpage()``: returns a ``Page`` object representing the user's + talkpage + +Additional features +~~~~~~~~~~~~~~~~~~~ + +Not all aspects of the toolset are covered here. Explore `its code and +docstrings`_ to learn how to use it in a more hands-on fashion. For reference, +``bot.wiki`` is an instance of ``earwigbot.wiki.SitesDB`` tied to the +``sites.db`` file in the bot's working directory. + +Tips +---- + +- Logging_ is a fantastic way to monitor the bot's progress as it runs. It has + a slew of built-in loggers, and enabling log retention (so logs are saved to + ``logs/`` in the working directory) is highly recommended. In the normal + setup, there are three log files, each of which "rotate" at a specific time + (``filename.log`` becomes ``filename.log.2012-04-10``, for example). The + ``debug.log`` file rotates every hour, and maintains six hours of logs of + every level (``DEBUG`` and up). ``bot.log`` rotates every day at midnight, + and maintains seven days of non-debug logs (``INFO`` and up). Finally, + ``error.log`` rotates every Sunday night, and maintains four weeks of logs + indicating unexpected events (``WARNING`` and up). + + To use logging in your commands or tasks (recommended), ``BaseCommand`` and + ``BaseTask`` provide ``logger`` attributes configured for the specific + command or task. If you're working with other classes, ``bot.logger`` is the + root logger (``logging.getLogger("earwigbot")`` by default), so you can use + ``getChild`` to make your logger. For example, task loggers are essentially + ``bot.logger.getChild("tasks").getChild(task.name)``. + +- A very useful IRC command is "``!reload``", which reloads all commands and + tasks without restarting the bot. [3]_ Combined with using the `!git plugin`_ + for pulling repositories from IRC, this can provide a seamless command/task + development workflow if the bot runs on an external server and you set up + its working directory as a git repo. + +- You can run a task by itself instead of the entire bot with ``earwigbot + path/to/working/dir --task task_name``. + +- Questions, comments, or suggestions about the documentation? `Let me know`_ + so I can improve it for other people. + +Footnotes +--------- + +.. [1] ``python setup.py install``/``develop`` may require root, or use the + ``--user`` switch to install for the current user only. + +.. [2] ``Data`` objects are instances of ``earwigbot.irc.Data`` that contain + information about a single message sent on IRC. Their useful attributes + are ``chan`` (channel the message was sent from, equal to ``nick`` if + it's a private message), ``nick`` (nickname of the sender), ``ident`` + (ident_ of the sender), ``host`` (hostname of the sender), ``msg`` (text + of the sent message), ``is_command`` (boolean telling whether or not + this message is a bot command, i.e., whether it is prefixed by ``!``), + ``command`` (if the message is a command, this is the name of the + command used), and ``args`` (if the message is a command, this is a list + of the command arguments - for example, if issuing "``!part ##earwig + Goodbye guys``", ``args`` will equal ``["##earwig", "Goodbye", + "guys"]``). Note that not all ``Data`` objects will have all of these + attributes: ``Data`` objects generated by private messages will, but + ones generated by joins will only have ``chan``, ``nick``, ``ident``, + and ``host``. + +.. [3] In reality, all this does is call ``bot.commands.load()`` and + ``bot.tasks.load()``! + +.. _EarwigBot: http://en.wikipedia.org/wiki/User:EarwigBot +.. _Python: http://python.org/ +.. _Wikipedia: http://en.wikipedia.org/ +.. _IRC: http://en.wikipedia.org/wiki/Internet_Relay_Chat +.. _PyPI: http://packages.python.org/earwigbot +.. _Pywikipedia framework: http://pywikipediabot.sourceforge.net/ +.. _copyright violation detector: http://en.wikipedia.org/wiki/Wikipedia:Bots/Requests_for_approval/EarwigBot_1 +.. _several ongoing tasks: http://en.wikipedia.org/wiki/User:EarwigBot#Tasks +.. _my instance of EarwigBot: http://en.wikipedia.org/wiki/User:EarwigBot +.. _earwigbot-plugins: https://github.com/earwig/earwigbot-plugins +.. _Python Package Index: http://pypi.python.org +.. _get pip: http://pypi.python.org/pypi/pip +.. _git flow: http://nvie.com/posts/a-successful-git-branching-model/ +.. _explanation of YAML: http://en.wikipedia.org/wiki/YAML +.. _earwigbot.bot.Bot: https://github.com/earwig/earwigbot/blob/develop/earwigbot/bot.py +.. _earwigbot.config.BotConfig: https://github.com/earwig/earwigbot/blob/develop/earwigbot/config.py +.. _earwigbot.commands.BaseCommand: https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/__init__.py +.. _afc_status: https://github.com/earwig/earwigbot-plugins/blob/develop/commands/afc_status.py +.. _chanops: https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/chanops.py +.. _test: https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/test.py +.. _earwigbot.tasks.BaseTask: https://github.com/earwig/earwigbot/blob/develop/earwigbot/tasks/__init__.py +.. _wikiproject_tagger: https://github.com/earwig/earwigbot/blob/develop/earwigbot/tasks/wikiproject_tagger.py +.. _afc_statistics: https://github.com/earwig/earwigbot-plugins/blob/develop/tasks/afc_statistics.py +.. _its code and docstrings: https://github.com/earwig/earwigbot/tree/develop/earwigbot/wiki +.. _logging: http://docs.python.org/library/logging.html +.. _Let me know: ben.kurtovic@verizon.net +.. _!git plugin: https://github.com/earwig/earwigbot-plugins/blob/develop/commands/git.py +.. _ident: http://en.wikipedia.org/wiki/Ident diff --git a/docs/Makefile b/docs/Makefile new file mode 100644 index 0000000..80e5ec7 --- /dev/null +++ b/docs/Makefile @@ -0,0 +1,153 @@ +# Makefile for Sphinx documentation +# + +# You can set these variables from the command line. +SPHINXOPTS = +SPHINXBUILD = sphinx-build +PAPER = +BUILDDIR = _build + +# Internal variables. +PAPEROPT_a4 = -D latex_paper_size=a4 +PAPEROPT_letter = -D latex_paper_size=letter +ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . +# the i18n builder cannot share the environment and doctrees with the others +I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . + +.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext + +help: + @echo "Please use \`make ' where is one of" + @echo " html to make standalone HTML files" + @echo " dirhtml to make HTML files named index.html in directories" + @echo " singlehtml to make a single large HTML file" + @echo " pickle to make pickle files" + @echo " json to make JSON files" + @echo " htmlhelp to make HTML files and a HTML help project" + @echo " qthelp to make HTML files and a qthelp project" + @echo " devhelp to make HTML files and a Devhelp project" + @echo " epub to make an epub" + @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" + @echo " latexpdf to make LaTeX files and run them through pdflatex" + @echo " text to make text files" + @echo " man to make manual pages" + @echo " texinfo to make Texinfo files" + @echo " info to make Texinfo files and run them through makeinfo" + @echo " gettext to make PO message catalogs" + @echo " changes to make an overview of all changed/added/deprecated items" + @echo " linkcheck to check all external links for integrity" + @echo " doctest to run all doctests embedded in the documentation (if enabled)" + +clean: + -rm -rf $(BUILDDIR)/* + +html: + $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html + @echo + @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." + +dirhtml: + $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml + @echo + @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." + +singlehtml: + $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml + @echo + @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." + +pickle: + $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle + @echo + @echo "Build finished; now you can process the pickle files." + +json: + $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json + @echo + @echo "Build finished; now you can process the JSON files." + +htmlhelp: + $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp + @echo + @echo "Build finished; now you can run HTML Help Workshop with the" \ + ".hhp project file in $(BUILDDIR)/htmlhelp." + +qthelp: + $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp + @echo + @echo "Build finished; now you can run "qcollectiongenerator" with the" \ + ".qhcp project file in $(BUILDDIR)/qthelp, like this:" + @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/EarwigBot.qhcp" + @echo "To view the help file:" + @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/EarwigBot.qhc" + +devhelp: + $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp + @echo + @echo "Build finished." + @echo "To view the help file:" + @echo "# mkdir -p $$HOME/.local/share/devhelp/EarwigBot" + @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/EarwigBot" + @echo "# devhelp" + +epub: + $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub + @echo + @echo "Build finished. The epub file is in $(BUILDDIR)/epub." + +latex: + $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex + @echo + @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." + @echo "Run \`make' in that directory to run these through (pdf)latex" \ + "(use \`make latexpdf' here to do that automatically)." + +latexpdf: + $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex + @echo "Running LaTeX files through pdflatex..." + $(MAKE) -C $(BUILDDIR)/latex all-pdf + @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." + +text: + $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text + @echo + @echo "Build finished. The text files are in $(BUILDDIR)/text." + +man: + $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man + @echo + @echo "Build finished. The manual pages are in $(BUILDDIR)/man." + +texinfo: + $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo + @echo + @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." + @echo "Run \`make' in that directory to run these through makeinfo" \ + "(use \`make info' here to do that automatically)." + +info: + $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo + @echo "Running Texinfo files through makeinfo..." + make -C $(BUILDDIR)/texinfo info + @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." + +gettext: + $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale + @echo + @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." + +changes: + $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes + @echo + @echo "The overview file is in $(BUILDDIR)/changes." + +linkcheck: + $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck + @echo + @echo "Link check complete; look for any errors in the above output " \ + "or in $(BUILDDIR)/linkcheck/output.txt." + +doctest: + $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest + @echo "Testing of doctests in the sources finished, look at the " \ + "results in $(BUILDDIR)/doctest/output.txt." diff --git a/docs/api/earwigbot.commands.rst b/docs/api/earwigbot.commands.rst new file mode 100644 index 0000000..544a88d --- /dev/null +++ b/docs/api/earwigbot.commands.rst @@ -0,0 +1,163 @@ +commands Package +================ + +:mod:`commands` Package +----------------------- + +.. automodule:: earwigbot.commands + :members: + :undoc-members: + :show-inheritance: + +:mod:`_old` Module +------------------ + +.. automodule:: earwigbot.commands._old + :members: + :undoc-members: + :show-inheritance: + +:mod:`afc_report` Module +------------------------ + +.. automodule:: earwigbot.commands.afc_report + :members: + :undoc-members: + :show-inheritance: + +:mod:`afc_status` Module +------------------------ + +.. automodule:: earwigbot.commands.afc_status + :members: + :undoc-members: + :show-inheritance: + +:mod:`calc` Module +------------------ + +.. automodule:: earwigbot.commands.calc + :members: + :undoc-members: + :show-inheritance: + +:mod:`chanops` Module +--------------------- + +.. automodule:: earwigbot.commands.chanops + :members: + :undoc-members: + :show-inheritance: + +:mod:`crypt` Module +------------------- + +.. automodule:: earwigbot.commands.crypt + :members: + :undoc-members: + :show-inheritance: + +:mod:`ctcp` Module +------------------ + +.. automodule:: earwigbot.commands.ctcp + :members: + :undoc-members: + :show-inheritance: + +:mod:`editcount` Module +----------------------- + +.. automodule:: earwigbot.commands.editcount + :members: + :undoc-members: + :show-inheritance: + +:mod:`git` Module +----------------- + +.. automodule:: earwigbot.commands.git + :members: + :undoc-members: + :show-inheritance: + +:mod:`help` Module +------------------ + +.. automodule:: earwigbot.commands.help + :members: + :undoc-members: + :show-inheritance: + +:mod:`link` Module +------------------ + +.. automodule:: earwigbot.commands.link + :members: + :undoc-members: + :show-inheritance: + +:mod:`praise` Module +-------------------- + +.. automodule:: earwigbot.commands.praise + :members: + :undoc-members: + :show-inheritance: + +:mod:`quit` Module +------------------ + +.. automodule:: earwigbot.commands.quit + :members: + :undoc-members: + :show-inheritance: + +:mod:`registration` Module +-------------------------- + +.. automodule:: earwigbot.commands.registration + :members: + :undoc-members: + :show-inheritance: + +:mod:`remind` Module +-------------------- + +.. automodule:: earwigbot.commands.remind + :members: + :undoc-members: + :show-inheritance: + +:mod:`replag` Module +-------------------- + +.. automodule:: earwigbot.commands.replag + :members: + :undoc-members: + :show-inheritance: + +:mod:`rights` Module +-------------------- + +.. automodule:: earwigbot.commands.rights + :members: + :undoc-members: + :show-inheritance: + +:mod:`test` Module +------------------ + +.. automodule:: earwigbot.commands.test + :members: + :undoc-members: + :show-inheritance: + +:mod:`threads` Module +--------------------- + +.. automodule:: earwigbot.commands.threads + :members: + :undoc-members: + :show-inheritance: + diff --git a/docs/api/earwigbot.irc.rst b/docs/api/earwigbot.irc.rst new file mode 100644 index 0000000..93f327d --- /dev/null +++ b/docs/api/earwigbot.irc.rst @@ -0,0 +1,51 @@ +irc Package +=========== + +:mod:`irc` Package +------------------ + +.. automodule:: earwigbot.irc + :members: + :undoc-members: + :show-inheritance: + +:mod:`connection` Module +------------------------ + +.. automodule:: earwigbot.irc.connection + :members: + :undoc-members: + :show-inheritance: + +:mod:`data` Module +------------------ + +.. automodule:: earwigbot.irc.data + :members: + :undoc-members: + :show-inheritance: + +:mod:`frontend` Module +---------------------- + +.. automodule:: earwigbot.irc.frontend + :members: + :undoc-members: + :show-inheritance: + +:mod:`rc` Module +---------------- + +.. automodule:: earwigbot.irc.rc + :members: + :undoc-members: + :show-inheritance: + +:mod:`watcher` Module +--------------------- + +.. automodule:: earwigbot.irc.watcher + :members: + :undoc-members: + :show-inheritance: + diff --git a/docs/api/earwigbot.rst b/docs/api/earwigbot.rst new file mode 100644 index 0000000..f74e44f --- /dev/null +++ b/docs/api/earwigbot.rst @@ -0,0 +1,61 @@ +earwigbot Package +================= + +:mod:`earwigbot` Package +------------------------ + +.. automodule:: earwigbot.__init__ + :members: + :undoc-members: + :show-inheritance: + +:mod:`blowfish` Module +---------------------- + +.. automodule:: earwigbot.blowfish + :members: + :undoc-members: + :show-inheritance: + +:mod:`bot` Module +----------------- + +.. automodule:: earwigbot.bot + :members: + :undoc-members: + :show-inheritance: + +:mod:`config` Module +-------------------- + +.. automodule:: earwigbot.config + :members: + :undoc-members: + :show-inheritance: + +:mod:`managers` Module +---------------------- + +.. automodule:: earwigbot.managers + :members: + :undoc-members: + :show-inheritance: + +:mod:`util` Module +------------------ + +.. automodule:: earwigbot.util + :members: + :undoc-members: + :show-inheritance: + +Subpackages +----------- + +.. toctree:: + + earwigbot.commands + earwigbot.irc + earwigbot.tasks + earwigbot.wiki + diff --git a/docs/api/earwigbot.tasks.rst b/docs/api/earwigbot.tasks.rst new file mode 100644 index 0000000..47cbfe6 --- /dev/null +++ b/docs/api/earwigbot.tasks.rst @@ -0,0 +1,91 @@ +tasks Package +============= + +:mod:`tasks` Package +-------------------- + +.. automodule:: earwigbot.tasks + :members: + :undoc-members: + :show-inheritance: + +:mod:`afc_catdelink` Module +--------------------------- + +.. automodule:: earwigbot.tasks.afc_catdelink + :members: + :undoc-members: + :show-inheritance: + +:mod:`afc_copyvios` Module +-------------------------- + +.. automodule:: earwigbot.tasks.afc_copyvios + :members: + :undoc-members: + :show-inheritance: + +:mod:`afc_dailycats` Module +--------------------------- + +.. automodule:: earwigbot.tasks.afc_dailycats + :members: + :undoc-members: + :show-inheritance: + +:mod:`afc_history` Module +------------------------- + +.. automodule:: earwigbot.tasks.afc_history + :members: + :undoc-members: + :show-inheritance: + +:mod:`afc_statistics` Module +---------------------------- + +.. automodule:: earwigbot.tasks.afc_statistics + :members: + :undoc-members: + :show-inheritance: + +:mod:`afc_undated` Module +------------------------- + +.. automodule:: earwigbot.tasks.afc_undated + :members: + :undoc-members: + :show-inheritance: + +:mod:`blptag` Module +-------------------- + +.. automodule:: earwigbot.tasks.blptag + :members: + :undoc-members: + :show-inheritance: + +:mod:`feed_dailycats` Module +---------------------------- + +.. automodule:: earwigbot.tasks.feed_dailycats + :members: + :undoc-members: + :show-inheritance: + +:mod:`wikiproject_tagger` Module +-------------------------------- + +.. automodule:: earwigbot.tasks.wikiproject_tagger + :members: + :undoc-members: + :show-inheritance: + +:mod:`wrongmime` Module +----------------------- + +.. automodule:: earwigbot.tasks.wrongmime + :members: + :undoc-members: + :show-inheritance: + diff --git a/docs/api/earwigbot.wiki.rst b/docs/api/earwigbot.wiki.rst new file mode 100644 index 0000000..9938da8 --- /dev/null +++ b/docs/api/earwigbot.wiki.rst @@ -0,0 +1,75 @@ +wiki Package +============ + +:mod:`wiki` Package +------------------- + +.. automodule:: earwigbot.wiki + :members: + :undoc-members: + :show-inheritance: + +:mod:`category` Module +---------------------- + +.. automodule:: earwigbot.wiki.category + :members: + :undoc-members: + :show-inheritance: + +:mod:`constants` Module +----------------------- + +.. automodule:: earwigbot.wiki.constants + :members: + :undoc-members: + :show-inheritance: + +:mod:`copyright` Module +----------------------- + +.. automodule:: earwigbot.wiki.copyright + :members: + :undoc-members: + :show-inheritance: + +:mod:`exceptions` Module +------------------------ + +.. automodule:: earwigbot.wiki.exceptions + :members: + :undoc-members: + :show-inheritance: + +:mod:`page` Module +------------------ + +.. automodule:: earwigbot.wiki.page + :members: + :undoc-members: + :show-inheritance: + +:mod:`site` Module +------------------ + +.. automodule:: earwigbot.wiki.site + :members: + :undoc-members: + :show-inheritance: + +:mod:`sitesdb` Module +--------------------- + +.. automodule:: earwigbot.wiki.sitesdb + :members: + :undoc-members: + :show-inheritance: + +:mod:`user` Module +------------------ + +.. automodule:: earwigbot.wiki.user + :members: + :undoc-members: + :show-inheritance: + diff --git a/docs/api/modules.rst b/docs/api/modules.rst new file mode 100644 index 0000000..7c4c110 --- /dev/null +++ b/docs/api/modules.rst @@ -0,0 +1,7 @@ +earwigbot +========= + +.. toctree:: + :maxdepth: 4 + + earwigbot diff --git a/docs/conf.py b/docs/conf.py new file mode 100644 index 0000000..92b2d74 --- /dev/null +++ b/docs/conf.py @@ -0,0 +1,242 @@ +# -*- coding: utf-8 -*- +# +# EarwigBot documentation build configuration file, created by +# sphinx-quickstart on Sun Apr 29 01:42:25 2012. +# +# This file is execfile()d with the current directory set to its containing dir. +# +# Note that not all possible configuration values are present in this +# autogenerated file. +# +# All configuration values have a default; values that are commented out +# serve to show the default. + +import sys, os + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +sys.path.insert(0, os.path.abspath('..')) + +# -- General configuration ----------------------------------------------------- + +# If your documentation needs a minimal Sphinx version, state it here. +#needs_sphinx = '1.0' + +# Add any Sphinx extension module names here, as strings. They can be extensions +# coming with Sphinx (named 'sphinx.ext.*') or your custom ones. +extensions = ['sphinx.ext.autodoc', 'sphinx.ext.coverage', 'sphinx.ext.viewcode'] + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# The suffix of source filenames. +source_suffix = '.rst' + +# The encoding of source files. +#source_encoding = 'utf-8-sig' + +# The master toctree document. +master_doc = 'index' + +# General information about the project. +project = u'EarwigBot' +copyright = u'2009, 2010, 2011, 2012 by Ben Kurtovic' + +# The version info for the project you're documenting, acts as replacement for +# |version| and |release|, also used in various other places throughout the +# built documents. +# +# The short X.Y version. +version = '0.1' +# The full version, including alpha/beta/rc tags. +release = '0.1.dev' + +# The language for content autogenerated by Sphinx. Refer to documentation +# for a list of supported languages. +#language = None + +# There are two options for replacing |today|: either, you set today to some +# non-false value, then it is used: +#today = '' +# Else, today_fmt is used as the format for a strftime call. +#today_fmt = '%B %d, %Y' + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +exclude_patterns = ['_build'] + +# The reST default role (used for this markup: `text`) to use for all documents. +#default_role = None + +# If true, '()' will be appended to :func: etc. cross-reference text. +#add_function_parentheses = True + +# If true, the current module name will be prepended to all description +# unit titles (such as .. function::). +#add_module_names = True + +# If true, sectionauthor and moduleauthor directives will be shown in the +# output. They are ignored by default. +#show_authors = False + +# The name of the Pygments (syntax highlighting) style to use. +pygments_style = 'sphinx' + +# A list of ignored prefixes for module index sorting. +#modindex_common_prefix = [] + + +# -- Options for HTML output --------------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +html_theme = 'nature' + +# Theme options are theme-specific and customize the look and feel of a theme +# further. For a list of options available for each theme, see the +# documentation. +#html_theme_options = {} + +# Add any paths that contain custom themes here, relative to this directory. +#html_theme_path = [] + +# The name for this set of Sphinx documents. If None, it defaults to +# " v documentation". +#html_title = None + +# A shorter title for the navigation bar. Default is the same as html_title. +#html_short_title = None + +# The name of an image file (relative to this directory) to place at the top +# of the sidebar. +#html_logo = None + +# The name of an image file (within the static path) to use as favicon of the +# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 +# pixels large. +#html_favicon = None + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +html_static_path = ['_static'] + +# If not '', a 'Last updated on:' timestamp is inserted at every page bottom, +# using the given strftime format. +#html_last_updated_fmt = '%b %d, %Y' + +# If true, SmartyPants will be used to convert quotes and dashes to +# typographically correct entities. +#html_use_smartypants = True + +# Custom sidebar templates, maps document names to template names. +#html_sidebars = {} + +# Additional templates that should be rendered to pages, maps page names to +# template names. +#html_additional_pages = {} + +# If false, no module index is generated. +#html_domain_indices = True + +# If false, no index is generated. +#html_use_index = True + +# If true, the index is split into individual pages for each letter. +#html_split_index = False + +# If true, links to the reST sources are added to the pages. +#html_show_sourcelink = True + +# If true, "Created using Sphinx" is shown in the HTML footer. Default is True. +#html_show_sphinx = True + +# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. +#html_show_copyright = True + +# If true, an OpenSearch description file will be output, and all pages will +# contain a tag referring to it. The value of this option must be the +# base URL from which the finished HTML is served. +#html_use_opensearch = '' + +# This is the file name suffix for HTML files (e.g. ".xhtml"). +#html_file_suffix = None + +# Output file base name for HTML help builder. +htmlhelp_basename = 'EarwigBotdoc' + + +# -- Options for LaTeX output -------------------------------------------------- + +latex_elements = { +# The paper size ('letterpaper' or 'a4paper'). +#'papersize': 'letterpaper', + +# The font size ('10pt', '11pt' or '12pt'). +#'pointsize': '10pt', + +# Additional stuff for the LaTeX preamble. +#'preamble': '', +} + +# Grouping the document tree into LaTeX files. List of tuples +# (source start file, target name, title, author, documentclass [howto/manual]). +latex_documents = [ + ('index', 'EarwigBot.tex', u'EarwigBot Documentation', + u'Ben Kurtovic', 'manual'), +] + +# The name of an image file (relative to this directory) to place at the top of +# the title page. +#latex_logo = None + +# For "manual" documents, if this is true, then toplevel headings are parts, +# not chapters. +#latex_use_parts = False + +# If true, show page references after internal links. +#latex_show_pagerefs = False + +# If true, show URL addresses after external links. +#latex_show_urls = False + +# Documents to append as an appendix to all manuals. +#latex_appendices = [] + +# If false, no module index is generated. +#latex_domain_indices = True + + +# -- Options for manual page output -------------------------------------------- + +# One entry per manual page. List of tuples +# (source start file, name, description, authors, manual section). +man_pages = [ + ('index', 'earwigbot', u'EarwigBot Documentation', + [u'Ben Kurtovic'], 1) +] + +# If true, show URL addresses after external links. +#man_show_urls = False + + +# -- Options for Texinfo output ------------------------------------------------ + +# Grouping the document tree into Texinfo files. List of tuples +# (source start file, target name, title, author, +# dir menu entry, description, category) +texinfo_documents = [ + ('index', 'EarwigBot', u'EarwigBot Documentation', + u'Ben Kurtovic', 'EarwigBot', 'One line description of project.', + 'Miscellaneous'), +] + +# Documents to append as an appendix to all manuals. +#texinfo_appendices = [] + +# If false, no module index is generated. +#texinfo_domain_indices = True + +# How to display URL addresses: 'footnote', 'no', or 'inline'. +#texinfo_show_urls = 'footnote' diff --git a/docs/customizing.rst b/docs/customizing.rst new file mode 100644 index 0000000..dde9b67 --- /dev/null +++ b/docs/customizing.rst @@ -0,0 +1,238 @@ +Customizing +=========== + +The bot's working directory contains a :file:`commands` subdirectory and a +:file:`tasks` subdirectory. Custom IRC commands can be placed in the former, +whereas custom wiki bot tasks go into the latter. Developing custom modules is +explained in detail in this documentation. + +Note that custom commands will override built-in commands and tasks with the +same name. + +:py:class:`~earwigbot.bot.Bot` and :py:class:`~earwigbot.bot.BotConfig` +----------------------------------------------------------------------- + +:py:class:`earwigbot.bot.Bot` is EarwigBot's main class. You don't have to +instantiate this yourself, but it's good to be familiar with its attributes and +methods, because it is the main way to communicate with other parts of the bot. +A :py:class:`~earwigbot.bot.Bot` object is accessible as an attribute of +commands and tasks (i.e., :py:attr:`self.bot`). + +The most useful attributes are: + +- :py:attr:`~earwigbot.bot.Bot.config`: an instance of + :py:class:`~earwigbot.config.BotConfig`, for accessing the bot's + configuration data (see below). + +- :py:attr:`~earwigbot.bot.Bot.commands`: the bot's + :py:class:`~earwigbot.managers.CommandManager`, which is used internally to + run IRC commands (through + :py:meth:`commands.call() `, which + you shouldn't have to use); you can safely reload all commands with + :py:meth:`commands.load() `. + +- :py:attr:`~earwigbot.bot.Bot.tasks`: the bot's + :py:class:`~earwigbot.managers.TaskManager`, which can be used to start tasks + with :py:meth:`tasks.start(task_name, **kwargs) + `. :py:meth:`tasks.load() + ` can be used to safely reload all + tasks. + +- :py:attr:`~earwigbot.bot.Bot.frontend` / + :py:attr:`~earwigbot.bot.Bot.watcher`: instances of + :py:class:`earwigbot.irc.Frontend ` and + :py:class:`earwigbot.irc.Watcher `, + respectively, which represent the bot's connections to these two servers; you + can, for example, send a message to the frontend with + :py:meth:`frontend.say(chan, msg) + ` (more on communicating with IRC + below). + +- :py:attr:`~earwigbot.bot.Bot.wiki`: interface with the + :doc:`Wiki Toolset `. + +- Finally, :py:meth:`~earwigbot.bot.Bot.restart` (restarts IRC components and + reloads config, commands, and tasks) and :py:meth:`~earwigbot.bot.Bot.stop` + can be used almost anywhere. Both take an optional "reason" that will be + logged and used as the quit message when disconnecting from IRC. + +:py:class:`earwigbot.config.BotConfig` stores configuration information for the +bot. Its docstring explains what each attribute is used for, but essentially +each "node" (one of :py:attr:`config.components`, :py:attr:`wiki`, +:py:attr:`tasks`, :py:attr:`tasks`, or :py:attr:`metadata`) maps to a section +of the bot's :file:`config.yml` file. For example, if :file:`config.yml` +includes something like:: + + irc: + frontend: + nick: MyAwesomeBot + channels: + - "##earwigbot" + - "#channel" + - "#other-channel" + +...then :py:attr:`config.irc["frontend"]["nick"]` will be ``"MyAwesomeBot"`` +and :py:attr:`config.irc["frontend"]["channels"]` will be +``["##earwigbot", "#channel", "#other-channel"]``. + +Custom IRC commands +------------------- + +Custom commands are subclasses of :py:class:`earwigbot.commands.BaseCommand` +that override :py:class:`~earwigbot.commands.BaseCommand`'s +:py:meth:`~earwigbot.commands.BaseCommand.process` (and optionally +:py:meth:`~earwigbot.commands.BaseCommand.check`) methods. + +:py:class:`~earwigbot.commands.BaseCommand`'s docstrings should explain what +each attribute and method is for and what they should be overridden with, but +these are the basics: + +- Class attribute :py:attr:`~earwigbot.commands.BaseCommand.name` is the name + of the command. This must be specified. + +- Class attribute :py:attr:`~earwigbot.commands.BaseCommand.hooks` is a list of + the "IRC events" that this command might respond to. It defaults to + ``["msg"]``, but options include ``"msg_private"`` (for private messages + only), ``"msg_public"`` (for channel messages only), and ``"join"`` (for when + a user joins a channel). See the afc_status_ plugin for a command that + responds to other hook types. + +- Method :py:meth:`~earwigbot.commands.BaseCommand.check` is passed a + :py:class:`~earwigbot.irc.data.Data` [1]_ object, and should return ``True`` + if you want to respond to this message, or ``False`` otherwise. The default + behavior is to return ``True`` only if + :py:attr:`data.is_command` is ``True`` and :py:attr:`data.command` == + :py:attr:`~earwigbot.commands.BaseCommand.name`, which is suitable for most + cases. A common, straightforward reason for overriding is if a command has + aliases (see chanops_ for an example). Note that by returning ``True``, you + prevent any other commands from responding to this message. + +- Method :py:meth:`~earwigbot.commands.BaseCommand.process` is passed the same + :py:class:`~earwigbot.irc.data.Data` object as + :py:meth:`~earwigbot.commands.BaseCommand.check`, but only if + :py:meth:`~earwigbot.commands.BaseCommand.check` returned ``True``. This is + where the bulk of your command goes. To respond to IRC messages, there are a + number of methods of :py:class:`~earwigbot.commands.BaseCommand` at your + disposal. See the the test_ command for a simple example, or look in + :py:class:`~earwigbot.commands.BaseCommand`'s + :py:meth:`~earwigbot.commands.BaseCommand.__init__` method for the full list. + + The most common ones are :py:meth:`say(chan_or_user, msg) + `, :py:meth:`reply(data, msg) + ` (convenience function; sends + a reply to the issuer of the command in the channel it was received), + :py:meth:`action(chan_or_user, msg) + `, + :py:meth:`notice(chan_or_user, msg) + `, :py:meth:`join(chan) + `, and + :py:meth:`part(chan) `. + +It's important to name the command class :py:class:`Command` within the file, +or else the bot might not recognize it as a command. The name of the file +doesn't really matter and need not match the command's name, but this is +recommended for readability. + +The bot has a wide selection of built-in commands and plugins to act as sample +code and/or to give ideas. Start with test_, and then check out chanops_ and +afc_status_ for some more complicated scripts. + +Custom bot tasks +---------------- + +Custom tasks are subclasses of :py:class:`earwigbot.tasks.BaseTask` that +override :py:class:`~earwigbot.tasks.BaseTask`'s +:py:meth:`~earwigbot.tasks.BaseTask.run` (and optionally +:py:meth:`~earwigbot.tasks.BaseTask.setup`) methods. + +:py:class:`~earwigbot.tasks.BaseTask`'s docstrings should explain what each +attribute and method is for and what they should be overridden with, but these +are the basics: + +- Class attribute :py:attr:`~earwigbot.tasks.BaseTask.name` is the name of the + task. This must be specified. + +- Class attribute :py:attr:`~earwigbot.tasks.BaseTask.number` can be used to + store an optional "task number", possibly for use in edit summaries (to be + generated with :py:meth:`~earwigbot.tasks.BaseTask.make_summary`). For + example, EarwigBot's :py:attr:`config.wiki["summary"]` is + ``"([[WP:BOT|Bot]]; [[User:EarwigBot#Task $1|Task $1]]): $2"``, which the + task class's :py:meth:`make_summary(comment) + ` method will take and replace + ``$1`` with the task number and ``$2`` with the details of the edit. + + Additionally, :py:meth:`~earwigbot.tasks.BaseTask.shutoff_enabled` (which + checks whether the bot has been told to stop on-wiki by checking the content + of a particular page) can check a different page for each task using similar + variables. EarwigBot's :py:attr:`config.wiki["shutoff"]["page"]` is + ``"User:$1/Shutoff/Task $2"``; ``$1`` is substituted with the bot's username, + and ``$2`` is substituted with the task number, so, e.g., task #14 checks the + page ``[[User:EarwigBot/Shutoff/Task 14]].`` If the page's content does *not* + match :py:attr:`config.wiki["shutoff"]["disabled"]` (``"run"`` by default), + then shutoff is considered to be *enabled* and + :py:meth:`~earwigbot.tasks.BaseTask.shutoff_enabled` will return ``True``, + indicating the task should not run. If you don't intend to use either of + these methods, feel free to leave this attribute blank. + +- Method :py:meth:`~earwigbot.tasks.BaseTask.setup` is called *once* with no + arguments immediately after the task is first loaded. Does nothing by + default; treat it like an :py:meth:`__init__` if you want + (:py:meth:`~earwigbot.tasks.BaseTask.__init__` does things by default and a + dedicated setup method is often easier than overriding + :py:meth:`~earwigbot.tasks.BaseTask.__init__` and using :py:obj:`super`). + +- Method :py:meth:`~earwigbot.tasks.BaseTask.run` is called with any number of + keyword arguments every time the task is executed (by + :py:meth:`tasks.start(task_name, **kwargs) + `, usually). This is where the bulk of + the task's code goes. For interfacing with MediaWiki sites, read up on the + :doc:`Wiki Toolset `. + +Tasks have access to :py:attr:`config.tasks[task_name]` for config information, +which is a node in :file:`config.yml` like every other attribute of +:py:attr:`bot.config`. This can be used to store, for example, edit summaries, +or templates to append to user talk pages, so that these can be easily changed +without modifying the task itself. + +It's important to name the task class :py:class:`Task` within the file, or else +the bot might not recognize it as a task. The name of the file doesn't really +matter and need not match the task's name, but this is recommended for +readability. + +See the built-in wikiproject_tagger_ task for a relatively straightforward +task, or the afc_statistics_ plugin for a more complicated one. + +.. rubric:: Footnotes + +.. [1] :py:class:`~earwigbot.irc.data.Data` objects are instances of + :py:class:`earwigbot.irc.Data ` that contain + information about a single message sent on IRC. Their useful attributes + are :py:attr:`~earwigbot.irc.data.Data.chan` (channel the message was + sent from, equal to :py:attr:`~earwigbot.irc.data.Data.nick` if it's a + private message), :py:attr:`~earwigbot.irc.data.Data.nick` (nickname of + the sender), :py:attr:`~earwigbot.irc.data.Data.ident` (ident_ of the + sender), :py:attr:`~earwigbot.irc.data.Data.host` (hostname of the + sender), :py:attr:`~earwigbot.irc.data.Data.msg` (text of the sent + message), :py:attr:`~earwigbot.irc.data.Data.is_command` (boolean + telling whether or not this message is a bot command, e.g., whether it + is prefixed by ``!``), :py:attr:`~earwigbot.irc.data.Data.command` (if + the message is a command, this is the name of the command used), and + :py:attr:`~earwigbot.irc.data.Data.args` (if the message is a command, + this is a list of the command arguments - for example, if issuing + "``!part ##earwig Goodbye guys``", + :py:attr:`~earwigbot.irc.data.Data.args` will equal + ``["##earwig", "Goodbye", "guys"]``). Note that not all + :py:class:`~earwigbot.irc.data.Data` objects will have all of these + attributes: :py:class:`~earwigbot.irc.data.Data` objects generated by + private messages will, but ones generated by joins will only have + :py:attr:`~earwigbot.irc.data.Data.chan`, + :py:attr:`~earwigbot.irc.data.Data.nick`, + :py:attr:`~earwigbot.irc.data.Data.ident`, + and :py:attr:`~earwigbot.irc.data.Data.host`. + +.. _afc_status: https://github.com/earwig/earwigbot-plugins/blob/develop/commands/afc_status.py +.. _chanops: https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/chanops.py +.. _test: https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/test.py +.. _wikiproject_tagger: https://github.com/earwig/earwigbot/blob/develop/earwigbot/tasks/wikiproject_tagger.py +.. _afc_statistics: https://github.com/earwig/earwigbot-plugins/blob/develop/tasks/afc_statistics.py +.. _ident: http://en.wikipedia.org/wiki/Ident diff --git a/docs/index.rst b/docs/index.rst new file mode 100644 index 0000000..8d446dc --- /dev/null +++ b/docs/index.rst @@ -0,0 +1,48 @@ +EarwigBot v0.1 Documentation +============================ + +EarwigBot_ is a Python_ robot that edits Wikipedia_ and interacts with people +over IRC_. + +History +------- + +Development began, based on the `Pywikipedia framework`_, in early 2009. +Approval for its fist task, a `copyright violation detector`_, was carried out +in May, and the bot has been running consistently ever since (with the +exception of Jan/Feb 2011). It currently handles `several ongoing tasks`_ +ranging from statistics generation to category cleanup, and on-demand tasks +such as WikiProject template tagging. Since it started running, the bot has +made over 50,000 edits. + +A project to rewrite it from scratch began in early April 2011, thus moving +away from the Pywikipedia framework and allowing for less overall code, better +integration between bot parts, and easier maintenance. + +.. _EarwigBot: http://en.wikipedia.org/wiki/User:EarwigBot +.. _Python: http://python.org/ +.. _Wikipedia: http://en.wikipedia.org/ +.. _IRC: http://en.wikipedia.org/wiki/Internet_Relay_Chat +.. _Pywikipedia framework: http://pywikipediabot.sourceforge.net/ +.. _copyright violation detector: http://en.wikipedia.org/wiki/Wikipedia:Bots/Requests_for_approval/EarwigBot_1 +.. _several ongoing tasks: http://en.wikipedia.org/wiki/User:EarwigBot#Tasks + +Contents +-------- + +.. toctree:: + :maxdepth: 2 + + installation + setup + customizing + toolset + tips + API Reference + +Indices and tables +------------------ + +* :ref:`genindex` +* :ref:`modindex` +* :ref:`search` diff --git a/docs/installation.rst b/docs/installation.rst new file mode 100644 index 0000000..12fc907 --- /dev/null +++ b/docs/installation.rst @@ -0,0 +1,55 @@ +Installation +============ + +This package contains the core :py:mod:`earwigbot`, abstracted enough that it +should be usable and customizable by anyone running a bot on a MediaWiki site. +Since it is component-based, the IRC components can be disabled if desired. IRC +commands and bot tasks specific to `my instance of EarwigBot`_ that I don't +feel the average user will need are available from the repository +`earwigbot-plugins`_. + +It's recommended to run the bot's unit tests before installing. Run +:command:`python setup.py test` from the project's root directory. Note that +some tests require an internet connection, and others may take a while to run. +Coverage is currently rather incomplete. + +Latest release (v0.1) +--------------------- + +EarwigBot is available from the `Python Package Index`_, so you can install the +latest release with :command:`pip install earwigbot` (`get pip`_). + +You can also install it from source [1]_ directly:: + + curl -Lo earwigbot.tgz https://github.com/earwig/earwigbot/tarball/v0.1 + tar -xf earwigbot.tgz + cd earwig-earwigbot-* + python setup.py install + cd .. + rm -r earwigbot.tgz earwig-earwigbot-* + +Development version +------------------- + +You can install the development version of the bot from :command:`git` by using +setuptools/`distribute`_'s :command:`develop` command [1]_, probably on the +``develop`` branch which contains (usually) working code. ``master`` contains +the latest release. EarwigBot uses `git flow`_, so you're free to browse by +tags or by new features (``feature/*`` branches):: + + git clone git://github.com/earwig/earwigbot.git earwigbot + cd earwigbot + python setup.py develop + +.. rubric:: Footnotes + +.. [1] :command:`python setup.py install`/:command:`develop` may require root, + or use the :command:`--user` switch to install for the current user + only. + +.. _my instance of EarwigBot: http://en.wikipedia.org/wiki/User:EarwigBot +.. _earwigbot-plugins: https://github.com/earwig/earwigbot-plugins +.. _Python Package Index: http://pypi.python.org +.. _get pip: http://pypi.python.org/pypi/pip +.. _distribute: http://pypi.python.org/pypi/distribute +.. _git flow: http://nvie.com/posts/a-successful-git-branching-model/ diff --git a/docs/setup.rst b/docs/setup.rst new file mode 100644 index 0000000..81523df --- /dev/null +++ b/docs/setup.rst @@ -0,0 +1,28 @@ +Setup +===== + +The bot stores its data in a "working directory", including its config file and +databases. This is also the location where you will place custom IRC commands +and bot tasks, which will be explained later. It doesn't matter where this +directory is, as long as the bot can write to it. + +Start the bot with :command:`earwigbot path/to/working/dir`, or just +:command:`earwigbot` if the working directory is the current directory. It will +notice that no :file:`config.yml` file exists and take you through the setup +process. + +There is currently no way to edit the :file:`config.yml` file from within the +bot after it has been created, but YAML is a very straightforward format, so +you should be able to make any necessary changes yourself. Check out the +`explanation of YAML`_ on Wikipedia for help. + +After setup, the bot will start. This means it will connect to the IRC servers +it has been configured for, schedule bot tasks to run at specific times, and +then wait for instructions (as commands on IRC). For a list of commands, say +"``!help``" (commands are messages prefixed with an exclamation mark). + +You can stop the bot at any time with :kbd:`Control-c`, same as you stop a +normal Python program, and it will try to exit safely. You can also use the +"``!quit``" command on IRC. + +.. _explanation of YAML: http://en.wikipedia.org/wiki/YAML diff --git a/docs/tips.rst b/docs/tips.rst new file mode 100644 index 0000000..4f4052e --- /dev/null +++ b/docs/tips.rst @@ -0,0 +1,46 @@ +Tips +==== + +- Logging_ is a fantastic way to monitor the bot's progress as it runs. It has + a slew of built-in loggers, and enabling log retention (so logs are saved to + :file:`logs/` in the working directory) is highly recommended. In the normal + setup, there are three log files, each of which "rotate" at a specific time + (:file:`filename.log` becomes :file:`filename.log.2012-04-10`, for example). + The :file:`debug.log` file rotates every hour, and maintains six hours of + logs of every level (``DEBUG`` and up). :file:`bot.log` rotates every day at + midnight, and maintains seven days of non-debug logs (``INFO`` and up). + Finally, :file:`error.log` rotates every Sunday night, and maintains four + weeks of logs indicating unexpected events (``WARNING`` and up). + + To use logging in your commands or tasks (recommended), + :py:class:~earwigbot.commands.BaseCommand` and + :py:class:~earwigbot.tasks.BaseTask` provide :py:attr:`logger` attributes + configured for the specific command or task. If you're working with other + classes, :py:attr:`bot.logger` is the root logger + (:py:obj:`logging.getLogger("earwigbot")` by default), so you can use + :py:func:`~logging.Logger.getChild` to make your logger. For example, task + loggers are essentially + :py:attr:`bot.logger.getChild("tasks").getChild(task.name) `. + +- A very useful IRC command is "``!reload``", which reloads all commands and + tasks without restarting the bot. [1]_ Combined with using the `!git plugin`_ + for pulling repositories from IRC, this can provide a seamless command/task + development workflow if the bot runs on an external server and you set up + its working directory as a git repo. + +- You can run a task by itself instead of the entire bot with + :command:`earwigbot path/to/working/dir --task task_name`. + +- Questions, comments, or suggestions about the documentation? `Let me know`_, + or `create an issue`_ so I can improve it for other people. + +.. rubric:: Footnotes + +.. [1] In reality, all this does is call :py:meth:`bot.commands.load() + ` and + :py:meth:`bot.tasks.load() `! + +.. _logging: http://docs.python.org/library/logging.html +.. _!git plugin: https://github.com/earwig/earwigbot-plugins/blob/develop/commands/git.py +.. _Let me know: ben.kurtovic@verizon.net +.. _create an issue: https://github.com/earwig/earwigbot/issues diff --git a/docs/toolset.rst b/docs/toolset.rst new file mode 100644 index 0000000..cae1beb --- /dev/null +++ b/docs/toolset.rst @@ -0,0 +1,220 @@ +The Wiki Toolset +================ + +EarwigBot's answer to the `Pywikipedia framework`_ is the Wiki Toolset +(:py:mod:`earwigbot.wiki`), which you will mainly access through +:py:attr:`bot.wiki `. + +:py:attr:`bot.wiki ` provides three methods for the +management of Sites - :py:meth:`~earwigbot.wiki.sitesdb.SitesDB.get_site`, +:py:meth:`~earwigbot.wiki.sitesdb.SitesDB.add_site`, and +:py:meth:`~earwigbot.wiki.sitesdb.SitesDB.remove_site`. Sites are objects that +simply represent a MediaWiki site. A single instance of EarwigBot (i.e. a +single *working directory*) is expected to relate to a single site or group of +sites using the same login info (like all WMF wikis with `CentralAuth`_). + +Load your default site (the one that you picked during setup) with +``site = bot.wiki.get_site()``. + +Dealing with other sites +~~~~~~~~~~~~~~~~~~~~~~~~ + +*Skip this section if you're only working with one site.* + +If a site is *already known to the bot* (meaning that it is stored in the +:file:`sites.db` file, which includes just your default wiki at first), you can +load a site with ``site = bot.wiki.get_site(name)``, where ``name`` might be +``"enwiki"`` or ``"frwiktionary"`` (you can also do +``site = bot.wiki.get_site(project="wikipedia", lang="en")``). Recall that not +giving any arguments to ``get_site()`` will return the default site. + +:py:meth:`~earwigbot.wiki.sitesdb.SitesDB.add_site` is used to add new sites to +the sites database. It may be called with similar arguments as +:py:meth:`~earwigbot.wiki.sitesdb.SitesDB.get_site`, but the difference is +important. :py:meth:`~earwigbot.wiki.sitesdb.SitesDB.get_site` only needs +enough information to identify the site in its database, which is usually just +its name; the database stores all other necessary connection info. With +:py:meth:`~earwigbot.wiki.sitesdb.SitesDB.add_site`, you need to provide enough +connection info so the toolset can successfully access the site's API/SQL +databases and store that information for later. That might not be much; for WMF +wikis, you can usually use code like this:: + + project, lang = "wikipedia", "es" + try: + site = bot.wiki.get_site(project=project, lang=lang) + except earwigbot.SiteNotFoundError: + # Load site info from http://es.wikipedia.org/w/api.php: + site = bot.wiki.add_site(project=project, lang=lang) + +This works because EarwigBot assumes that the URL for the site is +``"//{lang}.{project}.org"`` and the API is at ``/w/api.php``; this might +change if you're dealing with non-WMF wikis, where the code might look +something more like:: + + project, lang = "mywiki", "it" + try: + site = bot.wiki.get_site(project=project, lang=lang) + except earwigbot.SiteNotFoundError: + # Load site info from http://mysite.net/mywiki/it/s/api.php: + base_url = "http://mysite.net/" + project + "/" + lang + db_name = lang + project + "_p" + sql = {host: "sql.mysite.net", db: db_name} + site = bot.wiki.add_site(base_url=base_url, script_path="/s", sql=sql) + +:py:meth:`~earwigbot.wiki.sitesdb.SitesDB.remove_site` does the opposite of +:py:meth:`~earwigbot.wiki.sitesdb.SitesDB.add_site`: give it a site's name or a +project/lang pair like :py:meth:`~earwigbot.wiki.sitesdb.SitesDB.get_site` +takes, and it'll remove that site from the sites database. + +Sites +~~~~~ + +:py:class:`earwigbot.wiki.Site ` objects provide the +following attributes: + +- :py:attr:`~earwigbot.wiki.site.Site.name`: the site's name (or "wikiid"), + like ``"enwiki"`` +- :py:attr:`~earwigbot.wiki.site.Site.project`: the site's project name, like + ``"wikipedia"`` +- :py:attr:`~earwigbot.wiki.site.Site.lang`: the site's language code, like + ``"en"`` +- :py:attr:`~earwigbot.wiki.site.Site.domain`: the site's web domain, like + ``"en.wikipedia.org"`` + +and the following methods: + +- :py:meth:`api_query(**kwargs) `: does an + API query with the given keyword arguments as params +- :py:meth:`sql_query(query, params=(), ...) + `: does an SQL query and yields its + results (as a generator) +- :py:meth:`~earwigbot.wiki.site.Site.get_replag`: returns the estimated + database replication lag (if we have the site's SQL connection info) +- :py:meth:`namespace_id_to_name(id, all=False) + `: given a namespace ID, + returns the primary associated namespace name (or a list of all names when + ``all`` is ``True``) +- :py:meth:`namespace_name_to_id(name) + `: given a namespace name, + returns the associated namespace ID +- :py:meth:`get_page(title, follow_redirects=False) + `: returns a ``Page`` object for the given + title (or a :py:class:`~earwigbot.wiki.category.Category` object if the + page's namespace is "``Category:``") +- :py:meth:`get_category(catname, follow_redirects=False) + `: returns a ``Category`` object for + the given title (sans namespace) +- :py:meth:`get_user(username) `: returns a + :py:class:`~earwigbot.wiki.user.User` object for the given username + +Pages and categories +~~~~~~~~~~~~~~~~~~~~ + +Create :py:class:`earwigbot.wiki.Page ` objects with +:py:meth:`site.get_page(title) `, +:py:meth:`page.toggle_talk() `, +:py:meth:`user.get_userpage() `, or +:py:meth:`user.get_talkpage() `. They +provide the following attributes: + +- :py:attr:`~earwigbot.wiki.page.Page.title`: the page's title, or pagename +- :py:attr:`~earwigbot.wiki.page.Page.exists`: whether the page exists +- :py:attr:`~earwigbot.wiki.page.Page.pageid`: an integer ID representing the + page +- :py:attr:`~earwigbot.wiki.page.Page.url`: the page's URL +- :py:attr:`~earwigbot.wiki.page.Page.namespace`: the page's namespace as an + integer +- :py:attr:`~earwigbot.wiki.page.Page.protection`: the page's current + protection status +- :py:attr:`~earwigbot.wiki.page.Page.is_talkpage`: ``True`` if the page is a + talkpage, else ``False`` +- :py:attr:`~earwigbot.wiki.page.Page.is_redirect`: ``True`` if the page is a + redirect, else ``False`` + +and the following methods: + +- :py:meth:`~earwigbot.wiki.page.Page.reload`: forcibly reload the page's + attributes (emphasis on *reload* - this is only necessary if there is reason + to believe they have changed) +- :py:meth:`toggle_talk(...) `: returns a + content page's talk page, or vice versa +- :py:meth:`~earwigbot.wiki.page.Page.get`: returns page content +- :py:meth:`~earwigbot.wiki.page.Page.get_redirect_target`: if the page is a + redirect, returns its destination +- :py:meth:`~earwigbot.wiki.page.Page.get_creator`: returns a + :py:class:`~earwigbot.wiki.user.User` object representing the first user to + edit the page +- :py:meth:`edit(text, summary, minor=False, bot=True, force=False) + `: replaces the page's content with ``text`` + or creates a new page +- :py:meth:`add_section(text, title, minor=False, bot=True, force=False) + `: adds a new section named ``title`` + at the bottom of the page +- :py:meth:`copyvio_check(...) + `: checks the page for + copyright violations +- :py:meth:`copyvio_compare(url, ...) + `: checks the page like + :py:meth:`~earwigbot.wiki.copyvios.CopyvioMixin.copyvio_check`, but + against a specific URL + +Additionally, :py:class:`~earwigbot.wiki.category.Category` objects (created +with :py:meth:`site.get_category(name) ` +or :py:meth:`site.get_page(title) ` where +``title`` is in the ``Category:`` namespace) provide the following additional +method: + +- :py:meth:`get_members(use_sql=False, limit=None) + `: returns a list of page + titles in the category (limit is ``50`` by default if using the API) + +Users +~~~~~ + +Create :py:class:`earwigbot.wiki.User ` objects with +:py:meth:`site.get_user(name) ` or +:py:meth:`page.get_creator() `. They +provide the following attributes: + +- :py:attr:`~earwigbot.wiki.user.User.name`: the user's username +- :py:attr:`~earwigbot.wiki.user.User.exists`: ``True`` if the user exists, or + ``False`` if they do not +- :py:attr:`~earwigbot.wiki.user.User.userid`: an integer ID representing the + user +- :py:attr:`~earwigbot.wiki.user.User.blockinfo`: information about any current + blocks on the user (``False`` if no block, or a dict of + ``{"by": blocking_user, "reason": block_reason, + "expiry": block_expire_time}``) +- :py:attr:`~earwigbot.wiki.user.User.groups`: a list of the user's groups +- :py:attr:`~earwigbot.wiki.user.User.rights`: a list of the user's rights +- :py:attr:`~earwigbot.wiki.user.User.editcount`: the number of edits made by + the user +- :py:attr:`~earwigbot.wiki.user.User.registration`: the time the user + registered as a :py:obj:`time.struct_time` +- :py:attr:`~earwigbot.wiki.user.User.emailable`: ``True`` if you can email the + user, ``False`` if you cannot +- :py:attr:`~earwigbot.wiki.user.User.gender`: the user's gender (``"male"``, + ``"female"``, or ``"unknown"``) + +and the following methods: + +- :py:meth:`~earwigbot.wiki.user.User.reload`: forcibly reload the user's + attributes (emphasis on *reload* - this is only necessary if there is reason + to believe they have changed) +- :py:meth:`~earwigbot.wiki.user.User.get_userpage`: returns a + :py:class:`~earwigbot.wiki.page.Page` object representing the user's userpage +- :py:meth:`~earwigbot.wiki.user.User.get_talkpage`: returns a + :py:class:`~earwigbot.wiki.page.Page` object representing the user's talkpage + +Additional features +~~~~~~~~~~~~~~~~~~~ + +Not all aspects of the toolset are covered here. Explore `its code and +docstrings`_ to learn how to use it in a more hands-on fashion. For reference, +:py:attr:`bot.wiki ` is an instance of +:py:class:`earwigbot.wiki.SitesDB ` tied to the +:file:`sites.db` file in the bot's working directory. + +.. _Pywikipedia framework: http://pywikipediabot.sourceforge.net/ +.. _CentralAuth: http://www.mediawiki.org/wiki/Extension:CentralAuth +.. _its code and docstrings: https://github.com/earwig/earwigbot/tree/develop/earwigbot/wiki diff --git a/earwigbot/__init__.py b/earwigbot/__init__.py index c3a0d64..fa766e0 100644 --- a/earwigbot/__init__.py +++ b/earwigbot/__init__.py @@ -22,9 +22,10 @@ """ EarwigBot is a Python robot that edits Wikipedia and interacts with people over -IRC. - http://earwig.github.com/earwig/earwigbot +IRC. - https://github.com/earwig/earwigbot -See README.md for a basic overview, or the docs/ directory for details. +See README.rst for an overview, or the docs/ directory for details. This +documentation is also available online at http://packages.python.org/earwigbot. """ __author__ = "Ben Kurtovic" @@ -35,17 +36,17 @@ __email__ = "ben.kurtovic@verizon.net" __release__ = False if not __release__: - def _add_git_commit_id_to_version(version): + def _add_git_commit_id_to_version_string(version): from git import Repo from os.path import split, dirname path = split(dirname(__file__))[0] commit_id = Repo(path).head.object.hexsha return version + ".git+" + commit_id[:8] try: - __version__ = _add_git_commit_id_to_version(__version__) + __version__ = _add_git_commit_id_to_version_string(__version__) except Exception: pass finally: - del _add_git_commit_id_to_version + del _add_git_commit_id_to_version_string from earwigbot import bot, commands, config, irc, managers, tasks, util, wiki diff --git a/earwigbot/bot.py b/earwigbot/bot.py index aa0697a..30bcfeb 100644 --- a/earwigbot/bot.py +++ b/earwigbot/bot.py @@ -39,6 +39,7 @@ class Bot(object): EarwigBot has three components that can run independently of each other: an IRC front-end, an IRC watcher, and a wiki scheduler. + * The IRC front-end runs on a normal IRC server and expects users to interact with it/give it commands. * The IRC watcher runs on a wiki recent-changes server and listens for diff --git a/earwigbot/commands/__init__.py b/earwigbot/commands/__init__.py index 8e08011..46c38be 100644 --- a/earwigbot/commands/__init__.py +++ b/earwigbot/commands/__init__.py @@ -20,20 +20,18 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. -""" -EarwigBot's IRC Commands - -This package provides the IRC "commands" used by the bot's front-end component. -This module contains the BaseCommand class (import with -`from earwigbot.commands import BaseCommand`), whereas the package contains -various built-in commands. Additional commands can be installed as plugins in -the bot's working directory. -""" - __all__ = ["BaseCommand"] class BaseCommand(object): - """A base class for commands on IRC. + """ + EarwigBot's Base IRC Command + + This package provides built-in IRC "commands" used by the bot's front-end + component. Additional commands can be installed as plugins in the bot's + working directory. + + This class (import with `from earwigbot.commands import BaseCommand`), + can be subclassed to create custom IRC commands. This docstring is reported to the user when they use !help . """ diff --git a/earwigbot/commands/chanops.py b/earwigbot/commands/chanops.py index 9783411..a343aee 100644 --- a/earwigbot/commands/chanops.py +++ b/earwigbot/commands/chanops.py @@ -29,9 +29,7 @@ class Command(BaseCommand): def check(self, data): cmnds = ["chanops", "voice", "devoice", "op", "deop", "join", "part"] - if data.is_command and data.command in cmnds: - return True - return False + return data.is_command and data.command in cmnds def process(self, data): if data.command == "chanops": @@ -77,11 +75,11 @@ class Command(BaseCommand): reason = None if data.args: if data.args[0].startswith("#"): - # !part #channel reason for parting + # "!part #channel reason for parting" channel = data.args[0] if data.args[1:]: reason = " ".join(data.args[1:]) - else: # !part reason for parting; assume current channel + else: # "!part reason for parting"; assume current channel reason = " ".join(data.args) msg = "Requested by {0}".format(data.nick) diff --git a/earwigbot/config.py b/earwigbot/config.py index 780b480..8b7b20a 100644 --- a/earwigbot/config.py +++ b/earwigbot/config.py @@ -40,6 +40,7 @@ class BotConfig(object): from scratch at the inital bot run. BotConfig has a few properties and functions, including the following: + * config.root_dir - bot's working directory; contains config.yml, logs/ * config.path - path to the bot's config file * config.components - enabled components @@ -50,6 +51,7 @@ class BotConfig(object): * config.schedule() - tasks scheduled to run at a given time BotConfig also has some functions used in config loading: + * config.load() - loads and parses our config file, returning True if passwords are stored encrypted or False otherwise; can also be used to easily reload config @@ -150,7 +152,10 @@ class BotConfig(object): #else: # is_encrypted = False raise NotImplementedError() - # yaml.dumps() + # yaml.dumps() config.yml file (self._config_path) + # Create root_dir/, root_dir/commands/, root_dir/tasks/ + # Give a reasonable message after config has been created regarding + # what to do next... @property def root_dir(self): diff --git a/earwigbot/irc/frontend.py b/earwigbot/irc/frontend.py index 35accd8..3133da7 100644 --- a/earwigbot/irc/frontend.py +++ b/earwigbot/irc/frontend.py @@ -57,7 +57,7 @@ class Frontend(IRCConnection): data.nick, data.ident, data.host = self.sender_regex.findall(line[0])[0] data.chan = line[2] data.parse_args() - self.bot.commands.check("join", data) + self.bot.commands.call("join", data) elif line[1] == "PRIVMSG": data.nick, data.ident, data.host = self.sender_regex.findall(line[0])[0] @@ -69,13 +69,13 @@ class Frontend(IRCConnection): # This is a privmsg to us, so set 'chan' as the nick of the # sender, then check for private-only command hooks: data.chan = data.nick - self.bot.commands.check("msg_private", data) + self.bot.commands.call("msg_private", data) else: # Check for public-only command hooks: - self.bot.commands.check("msg_public", data) + self.bot.commands.call("msg_public", data) # Check for command hooks that apply to all messages: - self.bot.commands.check("msg", data) + self.bot.commands.call("msg", data) elif line[0] == "PING": # If we are pinged, pong back self.pong(line[1]) diff --git a/earwigbot/managers.py b/earwigbot/managers.py index 74f4ea5..7610ca7 100644 --- a/earwigbot/managers.py +++ b/earwigbot/managers.py @@ -163,8 +163,8 @@ class CommandManager(_ResourceManager): e = "Error executing command '{0}':" self.logger.exception(e.format(data.command)) - def check(self, hook, data): - """Given an IRC event, check if there's anything we can respond to.""" + def call(self, hook, data): + """Given a hook type and a Data object, respond appropriately.""" self.lock.acquire() for command in self._resources.itervalues(): if hook in command.hooks and self._wrap_check(command, data): diff --git a/earwigbot/tasks/__init__.py b/earwigbot/tasks/__init__.py index bfa7ef9..d830f1e 100644 --- a/earwigbot/tasks/__init__.py +++ b/earwigbot/tasks/__init__.py @@ -20,24 +20,24 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. -""" -EarwigBot's Bot Tasks - -This package provides the wiki bot "tasks" EarwigBot runs. This module contains -the BaseTask class (import with `from earwigbot.tasks import BaseTask`), -whereas the package contains various built-in tasks. Additional tasks can be -installed as plugins in the bot's working directory. - -To run a task, use bot.tasks.start(name, **kwargs). **kwargs get passed to the -Task's run() function. -""" - from earwigbot import wiki __all__ = ["BaseTask"] class BaseTask(object): - """A base class for bot tasks that edit Wikipedia.""" + """ + EarwigBot's Base Bot Task + + This package provides built-in wiki bot "tasks" EarwigBot runs. Additional + tasks can be installed as plugins in the bot's working directory. + + This class (import with `from earwigbot.tasks import BaseTask`) can be + subclassed to create custom bot tasks. + + To run a task, use :py:meth:`bot.tasks.start(name, **kwargs) + `. ``**kwargs`` get passed to the + Task's run() function. + """ name = None number = 0 diff --git a/earwigbot/tasks/blptag.py b/earwigbot/tasks/blptag.py index 76f80be..695dce6 100644 --- a/earwigbot/tasks/blptag.py +++ b/earwigbot/tasks/blptag.py @@ -25,8 +25,8 @@ from earwigbot.tasks import BaseTask __all__ = ["Task"] class Task(BaseTask): - """A task to add |blp=yes to {{WPB}} or {{WPBS}} when it is used along with - {{WP Biography}}.""" + """A task to add |blp=yes to ``{{WPB}}`` or ``{{WPBS}}`` when it is used + along with ``{{WP Biography}}``.""" name = "blptag" def setup(self): diff --git a/earwigbot/wiki/page.py b/earwigbot/wiki/page.py index 701500e..e6e9a4f 100644 --- a/earwigbot/wiki/page.py +++ b/earwigbot/wiki/page.py @@ -38,19 +38,23 @@ class Page(CopyrightMixin): about the page, getting page content, and so on. Category is a subclass of Page with additional methods. + Attributes: + title -- the page's title, or pagename + exists -- whether the page exists + pageid -- an integer ID representing the page + url -- the page's URL + namespace -- the page's namespace as an integer + protection -- the page's current protection status + is_talkpage -- True if the page is a talkpage, else False + is_redirect -- True if the page is a redirect, else False + Public methods: - title -- returns the page's title, or pagename - exists -- returns whether the page exists - pageid -- returns an integer ID representing the page - url -- returns the page's URL - namespace -- returns the page's namespace as an integer - protection -- returns the page's current protection status - creator -- returns the page's creator (first user to edit) - is_talkpage -- returns True if the page is a talkpage, else False - is_redirect -- returns True if the page is a redirect, else False + reload -- forcibly reload the page's attributes toggle_talk -- returns a content page's talk page, or vice versa get -- returns page content get_redirect_target -- if the page is a redirect, returns its destination + get_creator -- returns a User object representing the first person + to edit the page edit -- replaces the page's content or creates a new page add_section -- adds a new section at the bottom of the page copyvio_check -- checks the page for copyright violations diff --git a/earwigbot/wiki/site.py b/earwigbot/wiki/site.py index bdea9e6..6505965 100644 --- a/earwigbot/wiki/site.py +++ b/earwigbot/wiki/site.py @@ -56,16 +56,18 @@ class Site(object): instances, tools.add_site() for adding new ones to config, and tools.del_site() for removing old ones from config, should suffice. + Attributes: + name -- the site's name (or "wikiid"), like "enwiki" + project -- the site's project name, like "wikipedia" + lang -- the site's language code, like "en" + domain -- the site's web domain, like "en.wikipedia.org" + Public methods: - name -- returns our name (or "wikiid"), like "enwiki" - project -- returns our project name, like "wikipedia" - lang -- returns our language code, like "en" - domain -- returns our web domain, like "en.wikipedia.org" api_query -- does an API query with the given kwargs as params sql_query -- does an SQL query and yields its results get_replag -- returns the estimated database replication lag namespace_id_to_name -- given a namespace ID, returns associated name(s) - namespace_name_to_id -- given a namespace name, returns associated id + namespace_name_to_id -- given a namespace name, returns the associated ID get_page -- returns a Page object for the given title get_category -- returns a Category object for the given title get_user -- returns a User object for the given username diff --git a/earwigbot/wiki/user.py b/earwigbot/wiki/user.py index 2747e2d..6c51051 100644 --- a/earwigbot/wiki/user.py +++ b/earwigbot/wiki/user.py @@ -36,17 +36,20 @@ class User(object): information about the user, such as editcount and user rights, methods for returning the user's userpage and talkpage, etc. + Attributes: + name -- the user's username + exists -- True if the user exists, or False if they do not + userid -- an integer ID representing the user + blockinfo -- information about any current blocks on the user + groups -- a list of the user's groups + rights -- a list of the user's rights + editcount -- the number of edits made by the user + registration -- the time the user registered as a time.struct_time + emailable -- True if you can email the user, False if you cannot + gender -- the user's gender ("male", "female", or "unknown") + Public methods: - name -- returns the user's username - exists -- returns True if the user exists, False if they do not - userid -- returns an integer ID representing the user - blockinfo -- returns information about a current block on the user - groups -- returns a list of the user's groups - rights -- returns a list of the user's rights - editcount -- returns the number of edits made by the user - registration -- returns the time the user registered as a time.struct_time - emailable -- returns True if you can email the user, False if you cannot - gender -- returns the user's gender ("male", "female", or "unknown") + reload -- forcibly reload the user's attributes get_userpage -- returns a Page object representing the user's userpage get_talkpage -- returns a Page object representing the user's talkpage """