Conflicts: earwigbot/wiki/constants.py earwigbot/wiki/copyvios/__init__.py earwigbot/wiki/page.pytags/v0.1^2
@@ -1,19 +1,6 @@ | |||||
# Ignore python bytecode: | |||||
*.pyc | *.pyc | ||||
# Ignore bot-specific config file: | |||||
config.json | |||||
# Ignore logs directory: | |||||
logs/ | |||||
# Ignore cookies file: | |||||
.cookies | |||||
# Ignore OS X's crud: | |||||
*.egg | |||||
*.egg-info | |||||
.DS_Store | .DS_Store | ||||
# Ignore pydev's nonsense: | |||||
.project | |||||
.pydevproject | |||||
.settings/ | |||||
build | |||||
docs/_build |
@@ -1,39 +0,0 @@ | |||||
[EarwigBot](http://en.wikipedia.org/wiki/User:EarwigBot) is a | |||||
[Python](http://python.org/) robot that edits | |||||
[Wikipedia](http://en.wikipedia.org/) and interacts with people over | |||||
[IRC](http://en.wikipedia.org/wiki/Internet_Relay_Chat). | |||||
# History | |||||
Development began, based on the | |||||
[Pywikipedia framework](http://pywikipediabot.sourceforge.net/), in early 2009. | |||||
Approval for its fist task, a | |||||
[copyright violation detector](http://en.wikipedia.org/wiki/Wikipedia:Bots/Requests_for_approval/EarwigBot_1), | |||||
was carried out in May, and the bot has been running consistently ever since | |||||
(with the exception of Jan/Feb 2011). It currently handles | |||||
[several ongoing tasks](http://en.wikipedia.org/wiki/User:EarwigBot#Tasks), | |||||
ranging from statistics generation to category cleanup, and on-demand tasks | |||||
such as WikiProject template tagging. Since it started running, the bot has | |||||
made over 45,000 edits. | |||||
A project to rewrite it from scratch began in early April 2011, thus moving | |||||
away from the Pywikipedia framework and allowing for less overall code, better | |||||
integration between bot parts, and easier maintenance. | |||||
# Installation | |||||
## Dependencies | |||||
EarwigBot uses the MySQL library | |||||
[oursql](http://packages.python.org/oursql/) (>= 0.9.2) for communicating with | |||||
MediaWiki databases, and some tasks use their own tables for storage. | |||||
Additionally, the afc_history task uses | |||||
[matplotlib](http://matplotlib.sourceforge.net/) and | |||||
[numpy](http://numpy.scipy.org/) for graphing AfC statistics. Neither of these | |||||
modules are required for the main bot itself. | |||||
`earwigbot.wiki.copyright` requires access to a search engine for detecting | |||||
copyright violations. Currently, | |||||
[Yahoo! BOSS](http://developer.yahoo.com/search/boss/) is the only engine | |||||
supported, and this requires | |||||
[oauth2](https://github.com/simplegeo/python-oauth2). |
@@ -0,0 +1,205 @@ | |||||
EarwigBot | |||||
========= | |||||
EarwigBot_ is a Python_ robot that edits Wikipedia_ and interacts with people | |||||
over IRC_. This file provides a basic overview of how to install and setup the | |||||
bot; more detailed information is located in the ``docs/`` directory (available | |||||
online at PyPI_). | |||||
History | |||||
------- | |||||
Development began, based on the `Pywikipedia framework`_, in early 2009. | |||||
Approval for its fist task, a `copyright violation detector`_, was carried out | |||||
in May, and the bot has been running consistently ever since (with the | |||||
exception of Jan/Feb 2011). It currently handles `several ongoing tasks`_ | |||||
ranging from statistics generation to category cleanup, and on-demand tasks | |||||
such as WikiProject template tagging. Since it started running, the bot has | |||||
made over 50,000 edits. | |||||
A project to rewrite it from scratch began in early April 2011, thus moving | |||||
away from the Pywikipedia framework and allowing for less overall code, better | |||||
integration between bot parts, and easier maintenance. | |||||
Installation | |||||
------------ | |||||
This package contains the core ``earwigbot``, abstracted enough that it should | |||||
be usable and customizable by anyone running a bot on a MediaWiki site. Since | |||||
it is component-based, the IRC components can be disabled if desired. IRC | |||||
commands and bot tasks specific to `my instance of EarwigBot`_ that I don't | |||||
feel the average user will need are available from the repository | |||||
`earwigbot-plugins`_. | |||||
It's recommended to run the bot's unit tests before installing. Run ``python | |||||
setup.py test`` from the project's root directory. Note that some | |||||
tests require an internet connection, and others may take a while to run. | |||||
Coverage is currently rather incomplete. | |||||
Latest release (v0.1) | |||||
~~~~~~~~~~~~~~~~~~~~~ | |||||
EarwigBot is available from the `Python Package Index`_, so you can install the | |||||
latest release with ``pip install earwigbot`` (`get pip`_). | |||||
You can also install it from source [1]_ directly:: | |||||
curl -Lo earwigbot.tgz https://github.com/earwig/earwigbot/tarball/v0.1 | |||||
tar -xf earwigbot.tgz | |||||
cd earwig-earwigbot-* | |||||
python setup.py install | |||||
cd .. | |||||
rm -r earwigbot.tgz earwig-earwigbot-* | |||||
Development version | |||||
~~~~~~~~~~~~~~~~~~~ | |||||
You can install the development version of the bot from ``git`` by using | |||||
setuptools/distribute's ``develop`` command [1]_, probably on the ``develop`` | |||||
branch which contains (usually) working code. ``master`` contains the latest | |||||
release. EarwigBot uses `git flow`_, so you're free to | |||||
browse by tags or by new features (``feature/*`` branches):: | |||||
git clone git://github.com/earwig/earwigbot.git earwigbot | |||||
cd earwigbot | |||||
python setup.py develop | |||||
Setup | |||||
----- | |||||
The bot stores its data in a "working directory", including its config file and | |||||
databases. This is also the location where you will place custom IRC commands | |||||
and bot tasks, which will be explained later. It doesn't matter where this | |||||
directory is, as long as the bot can write to it. | |||||
Start the bot with ``earwigbot path/to/working/dir``, or just ``earwigbot`` if | |||||
the working directory is the current directory. It will notice that no | |||||
``config.yml`` file exists and take you through the setup process. | |||||
There is currently no way to edit the ``config.yml`` file from within the bot | |||||
after it has been created, but YAML is a very straightforward format, so you | |||||
should be able to make any necessary changes yourself. Check out the | |||||
`explanation of YAML`_ on Wikipedia for help. | |||||
After setup, the bot will start. This means it will connect to the IRC servers | |||||
it has been configured for, schedule bot tasks to run at specific times, and | |||||
then wait for instructions (as commands on IRC). For a list of commands, say | |||||
"``!help``" (commands are messages prefixed with an exclamation mark). | |||||
You can stop the bot at any time with Control+C, same as you stop a normal | |||||
Python program, and it will try to exit safely. You can also use the | |||||
"``!quit``" command on IRC. | |||||
Customizing | |||||
----------- | |||||
The bot's working directory contains a ``commands`` subdirectory and a | |||||
``tasks`` subdirectory. Custom IRC commands can be placed in the former, | |||||
whereas custom wiki bot tasks go into the latter. Developing custom modules is | |||||
explained below, and in more detail through the bot's documentation on PyPI_ | |||||
(or in the ``docs/`` dir). | |||||
Note that custom commands will override built-in commands and tasks with the | |||||
same name. | |||||
``Bot`` and ``BotConfig`` | |||||
~~~~~~~~~~~~~~~~~~~~~~~~~ | |||||
`earwigbot.bot.Bot`_ is EarwigBot's main class. You don't have to instantiate | |||||
this yourself, but it's good to be familiar with its attributes and methods, | |||||
because it is the main way to communicate with other parts of the bot. A | |||||
``Bot`` object is accessible as an attribute of commands and tasks (i.e., | |||||
``self.bot``). | |||||
`earwigbot.config.BotConfig`_ stores configuration information for the bot. Its | |||||
docstring explains what each attribute is used for, but essentially each "node" | |||||
(one of ``config.components``, ``wiki``, ``irc``, ``commands``, ``tasks``, and | |||||
``metadata``) maps to a section of the bot's ``config.yml`` file. For example, | |||||
if ``config.yml`` includes something like:: | |||||
irc: | |||||
frontend: | |||||
nick: MyAwesomeBot | |||||
channels: | |||||
- "##earwigbot" | |||||
- "#channel" | |||||
- "#other-channel" | |||||
...then ``config.irc["frontend"]["nick"]`` will be ``"MyAwesomeBot"`` and | |||||
``config.irc["frontend"]["channels"]`` will be ``["##earwigbot", "#channel", | |||||
"#other-channel"]``. | |||||
Custom IRC commands | |||||
~~~~~~~~~~~~~~~~~~~ | |||||
Custom commands are subclasses of `earwigbot.commands.Command`_ that override | |||||
``Command``'s ``process()`` (and optionally ``check()`` or ``setup()``) | |||||
methods. | |||||
The bot has a wide selection of built-in commands and plugins to act as sample | |||||
code and/or to give ideas. Start with test_, and then check out chanops_ and | |||||
afc_status_ for some more complicated scripts. | |||||
Custom bot tasks | |||||
~~~~~~~~~~~~~~~~ | |||||
Custom tasks are subclasses of `earwigbot.tasks.Task`_ that override ``Task``'s | |||||
``run()`` (and optionally ``setup()``) methods. | |||||
See the built-in wikiproject_tagger_ task for a relatively straightforward | |||||
task, or the afc_statistics_ plugin for a more complicated one. | |||||
The Wiki Toolset | |||||
---------------- | |||||
EarwigBot's answer to the `Pywikipedia framework`_ is the Wiki Toolset | |||||
(``earwigbot.wiki``), which you will mainly access through ``bot.wiki``. | |||||
``bot.wiki`` provides three methods for the management of Sites - | |||||
``get_site()``, ``add_site()``, and ``remove_site()``. Sites are objects that | |||||
simply represent a MediaWiki site. A single instance of EarwigBot (i.e. a | |||||
single *working directory*) is expected to relate to a single site or group of | |||||
sites using the same login info (like all WMF wikis with CentralAuth). | |||||
Load your default site (the one that you picked during setup) with | |||||
``site = bot.wiki.get_site()``. | |||||
Not all aspects of the toolset are covered in the docs. Explore `its code and | |||||
docstrings`_ to learn how to use it in a more hands-on fashion. For reference, | |||||
``bot.wiki`` is an instance of ``earwigbot.wiki.SitesDB`` tied to the | |||||
``sites.db`` file in the bot's working directory. | |||||
Footnotes | |||||
--------- | |||||
- Questions, comments, or suggestions about the documentation? `Let me know`_ | |||||
so I can improve it for other people. | |||||
.. [1] ``python setup.py install``/``develop`` may require root, or use the | |||||
``--user`` switch to install for the current user only. | |||||
.. _EarwigBot: http://en.wikipedia.org/wiki/User:EarwigBot | |||||
.. _Python: http://python.org/ | |||||
.. _Wikipedia: http://en.wikipedia.org/ | |||||
.. _IRC: http://en.wikipedia.org/wiki/Internet_Relay_Chat | |||||
.. _PyPI: http://packages.python.org/earwigbot | |||||
.. _Pywikipedia framework: http://pywikipediabot.sourceforge.net/ | |||||
.. _copyright violation detector: http://en.wikipedia.org/wiki/Wikipedia:Bots/Requests_for_approval/EarwigBot_1 | |||||
.. _several ongoing tasks: http://en.wikipedia.org/wiki/User:EarwigBot#Tasks | |||||
.. _my instance of EarwigBot: http://en.wikipedia.org/wiki/User:EarwigBot | |||||
.. _earwigbot-plugins: https://github.com/earwig/earwigbot-plugins | |||||
.. _Python Package Index: http://pypi.python.org | |||||
.. _get pip: http://pypi.python.org/pypi/pip | |||||
.. _git flow: http://nvie.com/posts/a-successful-git-branching-model/ | |||||
.. _explanation of YAML: http://en.wikipedia.org/wiki/YAML | |||||
.. _earwigbot.bot.Bot: https://github.com/earwig/earwigbot/blob/develop/earwigbot/bot.py | |||||
.. _earwigbot.config.BotConfig: https://github.com/earwig/earwigbot/blob/develop/earwigbot/config.py | |||||
.. _earwigbot.commands.Command: https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/__init__.py | |||||
.. _test: https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/test.py | |||||
.. _chanops: https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/chanops.py | |||||
.. _afc_status: https://github.com/earwig/earwigbot-plugins/blob/develop/commands/afc_status.py | |||||
.. _earwigbot.tasks.Task: https://github.com/earwig/earwigbot/blob/develop/earwigbot/tasks/__init__.py | |||||
.. _wikiproject_tagger: https://github.com/earwig/earwigbot/blob/develop/earwigbot/tasks/wikiproject_tagger.py | |||||
.. _afc_statistics: https://github.com/earwig/earwigbot-plugins/blob/develop/tasks/afc_statistics.py | |||||
.. _its code and docstrings: https://github.com/earwig/earwigbot/tree/develop/earwigbot/wiki | |||||
.. _Let me know: ben.kurtovic@verizon.net |
@@ -1,70 +0,0 @@ | |||||
#! /usr/bin/env python | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
""" | |||||
EarwigBot | |||||
This is a thin wrapper for EarwigBot's main bot code, specified by bot_script. | |||||
The wrapper will automatically restart the bot when it shuts down (from | |||||
!restart, for example). It requests the bot's password at startup and reuses it | |||||
every time the bot restarts internally, so you do not need to re-enter the | |||||
password after using !restart. | |||||
For information about the bot as a whole, see the attached README.md file (in | |||||
markdown format!), the docs/ directory, and the LICENSE file for licensing | |||||
information. EarwigBot is released under the MIT license. | |||||
""" | |||||
from getpass import getpass | |||||
from subprocess import Popen, PIPE | |||||
from os import path | |||||
from sys import executable | |||||
from time import sleep | |||||
import earwigbot | |||||
bot_script = path.join(earwigbot.__path__[0], "runner.py") | |||||
def main(): | |||||
print "EarwigBot v{0}\n".format(earwigbot.__version__) | |||||
is_encrypted = earwigbot.config.config.load() | |||||
if is_encrypted: # Passwords in the config file are encrypted | |||||
key = getpass("Enter key to unencrypt bot passwords: ") | |||||
else: | |||||
key = None | |||||
while 1: | |||||
bot = Popen([executable, bot_script], stdin=PIPE) | |||||
print >> bot.stdin, path.dirname(path.abspath(__file__)) | |||||
if is_encrypted: | |||||
print >> bot.stdin, key | |||||
return_code = bot.wait() | |||||
if return_code == 1: | |||||
exit() # Let critical exceptions in the subprocess cause us to | |||||
# exit as well | |||||
else: | |||||
sleep(5) # Sleep between bot runs following a non-critical | |||||
# subprocess exit | |||||
if __name__ == "__main__": | |||||
main() |
@@ -0,0 +1,153 @@ | |||||
# Makefile for Sphinx documentation | |||||
# | |||||
# You can set these variables from the command line. | |||||
SPHINXOPTS = | |||||
SPHINXBUILD = sphinx-build | |||||
PAPER = | |||||
BUILDDIR = _build | |||||
# Internal variables. | |||||
PAPEROPT_a4 = -D latex_paper_size=a4 | |||||
PAPEROPT_letter = -D latex_paper_size=letter | |||||
ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . | |||||
# the i18n builder cannot share the environment and doctrees with the others | |||||
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . | |||||
.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext | |||||
help: | |||||
@echo "Please use \`make <target>' where <target> is one of" | |||||
@echo " html to make standalone HTML files" | |||||
@echo " dirhtml to make HTML files named index.html in directories" | |||||
@echo " singlehtml to make a single large HTML file" | |||||
@echo " pickle to make pickle files" | |||||
@echo " json to make JSON files" | |||||
@echo " htmlhelp to make HTML files and a HTML help project" | |||||
@echo " qthelp to make HTML files and a qthelp project" | |||||
@echo " devhelp to make HTML files and a Devhelp project" | |||||
@echo " epub to make an epub" | |||||
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" | |||||
@echo " latexpdf to make LaTeX files and run them through pdflatex" | |||||
@echo " text to make text files" | |||||
@echo " man to make manual pages" | |||||
@echo " texinfo to make Texinfo files" | |||||
@echo " info to make Texinfo files and run them through makeinfo" | |||||
@echo " gettext to make PO message catalogs" | |||||
@echo " changes to make an overview of all changed/added/deprecated items" | |||||
@echo " linkcheck to check all external links for integrity" | |||||
@echo " doctest to run all doctests embedded in the documentation (if enabled)" | |||||
clean: | |||||
-rm -rf $(BUILDDIR)/* | |||||
html: | |||||
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html | |||||
@echo | |||||
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html." | |||||
dirhtml: | |||||
$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml | |||||
@echo | |||||
@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." | |||||
singlehtml: | |||||
$(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml | |||||
@echo | |||||
@echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." | |||||
pickle: | |||||
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle | |||||
@echo | |||||
@echo "Build finished; now you can process the pickle files." | |||||
json: | |||||
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json | |||||
@echo | |||||
@echo "Build finished; now you can process the JSON files." | |||||
htmlhelp: | |||||
$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp | |||||
@echo | |||||
@echo "Build finished; now you can run HTML Help Workshop with the" \ | |||||
".hhp project file in $(BUILDDIR)/htmlhelp." | |||||
qthelp: | |||||
$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp | |||||
@echo | |||||
@echo "Build finished; now you can run "qcollectiongenerator" with the" \ | |||||
".qhcp project file in $(BUILDDIR)/qthelp, like this:" | |||||
@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/EarwigBot.qhcp" | |||||
@echo "To view the help file:" | |||||
@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/EarwigBot.qhc" | |||||
devhelp: | |||||
$(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp | |||||
@echo | |||||
@echo "Build finished." | |||||
@echo "To view the help file:" | |||||
@echo "# mkdir -p $$HOME/.local/share/devhelp/EarwigBot" | |||||
@echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/EarwigBot" | |||||
@echo "# devhelp" | |||||
epub: | |||||
$(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub | |||||
@echo | |||||
@echo "Build finished. The epub file is in $(BUILDDIR)/epub." | |||||
latex: | |||||
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex | |||||
@echo | |||||
@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." | |||||
@echo "Run \`make' in that directory to run these through (pdf)latex" \ | |||||
"(use \`make latexpdf' here to do that automatically)." | |||||
latexpdf: | |||||
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex | |||||
@echo "Running LaTeX files through pdflatex..." | |||||
$(MAKE) -C $(BUILDDIR)/latex all-pdf | |||||
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." | |||||
text: | |||||
$(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text | |||||
@echo | |||||
@echo "Build finished. The text files are in $(BUILDDIR)/text." | |||||
man: | |||||
$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man | |||||
@echo | |||||
@echo "Build finished. The manual pages are in $(BUILDDIR)/man." | |||||
texinfo: | |||||
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo | |||||
@echo | |||||
@echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." | |||||
@echo "Run \`make' in that directory to run these through makeinfo" \ | |||||
"(use \`make info' here to do that automatically)." | |||||
info: | |||||
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo | |||||
@echo "Running Texinfo files through makeinfo..." | |||||
make -C $(BUILDDIR)/texinfo info | |||||
@echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." | |||||
gettext: | |||||
$(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale | |||||
@echo | |||||
@echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." | |||||
changes: | |||||
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes | |||||
@echo | |||||
@echo "The overview file is in $(BUILDDIR)/changes." | |||||
linkcheck: | |||||
$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck | |||||
@echo | |||||
@echo "Link check complete; look for any errors in the above output " \ | |||||
"or in $(BUILDDIR)/linkcheck/output.txt." | |||||
doctest: | |||||
$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest | |||||
@echo "Testing of doctests in the sources finished, look at the " \ | |||||
"results in $(BUILDDIR)/doctest/output.txt." |
@@ -0,0 +1,9 @@ | |||||
commands Package | |||||
================ | |||||
:mod:`commands` Package | |||||
----------------------- | |||||
.. automodule:: earwigbot.commands | |||||
:members: | |||||
:undoc-members: |
@@ -0,0 +1,46 @@ | |||||
irc Package | |||||
=========== | |||||
:mod:`irc` Package | |||||
------------------ | |||||
.. automodule:: earwigbot.irc | |||||
:members: | |||||
:undoc-members: | |||||
:mod:`connection` Module | |||||
------------------------ | |||||
.. automodule:: earwigbot.irc.connection | |||||
:members: | |||||
:undoc-members: | |||||
:mod:`data` Module | |||||
------------------ | |||||
.. automodule:: earwigbot.irc.data | |||||
:members: | |||||
:undoc-members: | |||||
:mod:`frontend` Module | |||||
---------------------- | |||||
.. automodule:: earwigbot.irc.frontend | |||||
:members: | |||||
:undoc-members: | |||||
:show-inheritance: | |||||
:mod:`rc` Module | |||||
---------------- | |||||
.. automodule:: earwigbot.irc.rc | |||||
:members: | |||||
:undoc-members: | |||||
:mod:`watcher` Module | |||||
--------------------- | |||||
.. automodule:: earwigbot.irc.watcher | |||||
:members: | |||||
:undoc-members: | |||||
:show-inheritance: |
@@ -0,0 +1,56 @@ | |||||
earwigbot Package | |||||
================= | |||||
:mod:`earwigbot` Package | |||||
------------------------ | |||||
.. automodule:: earwigbot.__init__ | |||||
:members: | |||||
:undoc-members: | |||||
:mod:`bot` Module | |||||
----------------- | |||||
.. automodule:: earwigbot.bot | |||||
:members: | |||||
:undoc-members: | |||||
:mod:`config` Module | |||||
-------------------- | |||||
.. automodule:: earwigbot.config | |||||
:members: | |||||
:undoc-members: | |||||
:mod:`exceptions` Module | |||||
------------------------ | |||||
.. automodule:: earwigbot.exceptions | |||||
:members: | |||||
:undoc-members: | |||||
:show-inheritance: | |||||
:mod:`managers` Module | |||||
---------------------- | |||||
.. automodule:: earwigbot.managers | |||||
:members: _ResourceManager, CommandManager, TaskManager | |||||
:undoc-members: | |||||
:show-inheritance: | |||||
:mod:`util` Module | |||||
------------------ | |||||
.. automodule:: earwigbot.util | |||||
:members: | |||||
:undoc-members: | |||||
Subpackages | |||||
----------- | |||||
.. toctree:: | |||||
earwigbot.commands | |||||
earwigbot.irc | |||||
earwigbot.tasks | |||||
earwigbot.wiki |
@@ -0,0 +1,9 @@ | |||||
tasks Package | |||||
============= | |||||
:mod:`tasks` Package | |||||
-------------------- | |||||
.. automodule:: earwigbot.tasks | |||||
:members: | |||||
:undoc-members: |
@@ -0,0 +1,59 @@ | |||||
wiki Package | |||||
============ | |||||
:mod:`wiki` Package | |||||
------------------- | |||||
.. automodule:: earwigbot.wiki | |||||
:members: | |||||
:undoc-members: | |||||
:mod:`category` Module | |||||
---------------------- | |||||
.. automodule:: earwigbot.wiki.category | |||||
:members: | |||||
:undoc-members: | |||||
:mod:`constants` Module | |||||
----------------------- | |||||
.. automodule:: earwigbot.wiki.constants | |||||
:members: | |||||
:undoc-members: | |||||
:mod:`copyright` Module | |||||
----------------------- | |||||
.. automodule:: earwigbot.wiki.copyright | |||||
:members: | |||||
:undoc-members: | |||||
:mod:`page` Module | |||||
------------------ | |||||
.. automodule:: earwigbot.wiki.page | |||||
:members: | |||||
:undoc-members: | |||||
:show-inheritance: | |||||
:mod:`site` Module | |||||
------------------ | |||||
.. automodule:: earwigbot.wiki.site | |||||
:members: | |||||
:undoc-members: | |||||
:mod:`sitesdb` Module | |||||
--------------------- | |||||
.. automodule:: earwigbot.wiki.sitesdb | |||||
:members: | |||||
:undoc-members: | |||||
:mod:`user` Module | |||||
------------------ | |||||
.. automodule:: earwigbot.wiki.user | |||||
:members: | |||||
:undoc-members: |
@@ -0,0 +1,7 @@ | |||||
earwigbot | |||||
========= | |||||
.. toctree:: | |||||
:maxdepth: 4 | |||||
earwigbot |
@@ -0,0 +1,242 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# EarwigBot documentation build configuration file, created by | |||||
# sphinx-quickstart on Sun Apr 29 01:42:25 2012. | |||||
# | |||||
# This file is execfile()d with the current directory set to its containing dir. | |||||
# | |||||
# Note that not all possible configuration values are present in this | |||||
# autogenerated file. | |||||
# | |||||
# All configuration values have a default; values that are commented out | |||||
# serve to show the default. | |||||
import sys, os | |||||
# If extensions (or modules to document with autodoc) are in another directory, | |||||
# add these directories to sys.path here. If the directory is relative to the | |||||
# documentation root, use os.path.abspath to make it absolute, like shown here. | |||||
sys.path.insert(0, os.path.abspath('..')) | |||||
# -- General configuration ----------------------------------------------------- | |||||
# If your documentation needs a minimal Sphinx version, state it here. | |||||
#needs_sphinx = '1.0' | |||||
# Add any Sphinx extension module names here, as strings. They can be extensions | |||||
# coming with Sphinx (named 'sphinx.ext.*') or your custom ones. | |||||
extensions = ['sphinx.ext.autodoc', 'sphinx.ext.coverage', 'sphinx.ext.viewcode'] | |||||
# Add any paths that contain templates here, relative to this directory. | |||||
templates_path = ['_templates'] | |||||
# The suffix of source filenames. | |||||
source_suffix = '.rst' | |||||
# The encoding of source files. | |||||
#source_encoding = 'utf-8-sig' | |||||
# The master toctree document. | |||||
master_doc = 'index' | |||||
# General information about the project. | |||||
project = u'EarwigBot' | |||||
copyright = u'2009, 2010, 2011, 2012 by Ben Kurtovic' | |||||
# The version info for the project you're documenting, acts as replacement for | |||||
# |version| and |release|, also used in various other places throughout the | |||||
# built documents. | |||||
# | |||||
# The short X.Y version. | |||||
version = '0.1' | |||||
# The full version, including alpha/beta/rc tags. | |||||
release = '0.1.dev' | |||||
# The language for content autogenerated by Sphinx. Refer to documentation | |||||
# for a list of supported languages. | |||||
#language = None | |||||
# There are two options for replacing |today|: either, you set today to some | |||||
# non-false value, then it is used: | |||||
#today = '' | |||||
# Else, today_fmt is used as the format for a strftime call. | |||||
#today_fmt = '%B %d, %Y' | |||||
# List of patterns, relative to source directory, that match files and | |||||
# directories to ignore when looking for source files. | |||||
exclude_patterns = ['_build'] | |||||
# The reST default role (used for this markup: `text`) to use for all documents. | |||||
#default_role = None | |||||
# If true, '()' will be appended to :func: etc. cross-reference text. | |||||
#add_function_parentheses = True | |||||
# If true, the current module name will be prepended to all description | |||||
# unit titles (such as .. function::). | |||||
#add_module_names = True | |||||
# If true, sectionauthor and moduleauthor directives will be shown in the | |||||
# output. They are ignored by default. | |||||
#show_authors = False | |||||
# The name of the Pygments (syntax highlighting) style to use. | |||||
pygments_style = 'sphinx' | |||||
# A list of ignored prefixes for module index sorting. | |||||
#modindex_common_prefix = [] | |||||
# -- Options for HTML output --------------------------------------------------- | |||||
# The theme to use for HTML and HTML Help pages. See the documentation for | |||||
# a list of builtin themes. | |||||
html_theme = 'nature' | |||||
# Theme options are theme-specific and customize the look and feel of a theme | |||||
# further. For a list of options available for each theme, see the | |||||
# documentation. | |||||
#html_theme_options = {} | |||||
# Add any paths that contain custom themes here, relative to this directory. | |||||
#html_theme_path = [] | |||||
# The name for this set of Sphinx documents. If None, it defaults to | |||||
# "<project> v<release> documentation". | |||||
#html_title = None | |||||
# A shorter title for the navigation bar. Default is the same as html_title. | |||||
#html_short_title = None | |||||
# The name of an image file (relative to this directory) to place at the top | |||||
# of the sidebar. | |||||
#html_logo = None | |||||
# The name of an image file (within the static path) to use as favicon of the | |||||
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 | |||||
# pixels large. | |||||
#html_favicon = None | |||||
# Add any paths that contain custom static files (such as style sheets) here, | |||||
# relative to this directory. They are copied after the builtin static files, | |||||
# so a file named "default.css" will overwrite the builtin "default.css". | |||||
html_static_path = ['_static'] | |||||
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom, | |||||
# using the given strftime format. | |||||
#html_last_updated_fmt = '%b %d, %Y' | |||||
# If true, SmartyPants will be used to convert quotes and dashes to | |||||
# typographically correct entities. | |||||
#html_use_smartypants = True | |||||
# Custom sidebar templates, maps document names to template names. | |||||
#html_sidebars = {} | |||||
# Additional templates that should be rendered to pages, maps page names to | |||||
# template names. | |||||
#html_additional_pages = {} | |||||
# If false, no module index is generated. | |||||
#html_domain_indices = True | |||||
# If false, no index is generated. | |||||
#html_use_index = True | |||||
# If true, the index is split into individual pages for each letter. | |||||
#html_split_index = False | |||||
# If true, links to the reST sources are added to the pages. | |||||
#html_show_sourcelink = True | |||||
# If true, "Created using Sphinx" is shown in the HTML footer. Default is True. | |||||
#html_show_sphinx = True | |||||
# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. | |||||
#html_show_copyright = True | |||||
# If true, an OpenSearch description file will be output, and all pages will | |||||
# contain a <link> tag referring to it. The value of this option must be the | |||||
# base URL from which the finished HTML is served. | |||||
#html_use_opensearch = '' | |||||
# This is the file name suffix for HTML files (e.g. ".xhtml"). | |||||
#html_file_suffix = None | |||||
# Output file base name for HTML help builder. | |||||
htmlhelp_basename = 'EarwigBotdoc' | |||||
# -- Options for LaTeX output -------------------------------------------------- | |||||
latex_elements = { | |||||
# The paper size ('letterpaper' or 'a4paper'). | |||||
#'papersize': 'letterpaper', | |||||
# The font size ('10pt', '11pt' or '12pt'). | |||||
#'pointsize': '10pt', | |||||
# Additional stuff for the LaTeX preamble. | |||||
#'preamble': '', | |||||
} | |||||
# Grouping the document tree into LaTeX files. List of tuples | |||||
# (source start file, target name, title, author, documentclass [howto/manual]). | |||||
latex_documents = [ | |||||
('index', 'EarwigBot.tex', u'EarwigBot Documentation', | |||||
u'Ben Kurtovic', 'manual'), | |||||
] | |||||
# The name of an image file (relative to this directory) to place at the top of | |||||
# the title page. | |||||
#latex_logo = None | |||||
# For "manual" documents, if this is true, then toplevel headings are parts, | |||||
# not chapters. | |||||
#latex_use_parts = False | |||||
# If true, show page references after internal links. | |||||
#latex_show_pagerefs = False | |||||
# If true, show URL addresses after external links. | |||||
#latex_show_urls = False | |||||
# Documents to append as an appendix to all manuals. | |||||
#latex_appendices = [] | |||||
# If false, no module index is generated. | |||||
#latex_domain_indices = True | |||||
# -- Options for manual page output -------------------------------------------- | |||||
# One entry per manual page. List of tuples | |||||
# (source start file, name, description, authors, manual section). | |||||
man_pages = [ | |||||
('index', 'earwigbot', u'EarwigBot Documentation', | |||||
[u'Ben Kurtovic'], 1) | |||||
] | |||||
# If true, show URL addresses after external links. | |||||
#man_show_urls = False | |||||
# -- Options for Texinfo output ------------------------------------------------ | |||||
# Grouping the document tree into Texinfo files. List of tuples | |||||
# (source start file, target name, title, author, | |||||
# dir menu entry, description, category) | |||||
texinfo_documents = [ | |||||
('index', 'EarwigBot', u'EarwigBot Documentation', | |||||
u'Ben Kurtovic', 'EarwigBot', 'One line description of project.', | |||||
'Miscellaneous'), | |||||
] | |||||
# Documents to append as an appendix to all manuals. | |||||
#texinfo_appendices = [] | |||||
# If false, no module index is generated. | |||||
#texinfo_domain_indices = True | |||||
# How to display URL addresses: 'footnote', 'no', or 'inline'. | |||||
#texinfo_show_urls = 'footnote' |
@@ -0,0 +1,240 @@ | |||||
Customizing | |||||
=========== | |||||
The bot's working directory contains a :file:`commands` subdirectory and a | |||||
:file:`tasks` subdirectory. Custom IRC commands can be placed in the former, | |||||
whereas custom wiki bot tasks go into the latter. Developing custom modules is | |||||
explained in detail in this documentation. | |||||
Note that custom commands will override built-in commands and tasks with the | |||||
same name. | |||||
:py:class:`~earwigbot.bot.Bot` and :py:class:`~earwigbot.bot.BotConfig` | |||||
----------------------------------------------------------------------- | |||||
:py:class:`earwigbot.bot.Bot` is EarwigBot's main class. You don't have to | |||||
instantiate this yourself, but it's good to be familiar with its attributes and | |||||
methods, because it is the main way to communicate with other parts of the bot. | |||||
A :py:class:`~earwigbot.bot.Bot` object is accessible as an attribute of | |||||
commands and tasks (i.e., :py:attr:`self.bot`). | |||||
The most useful attributes are: | |||||
- :py:attr:`~earwigbot.bot.Bot.config`: an instance of | |||||
:py:class:`~earwigbot.config.BotConfig`, for accessing the bot's | |||||
configuration data (see below). | |||||
- :py:attr:`~earwigbot.bot.Bot.commands`: the bot's | |||||
:py:class:`~earwigbot.managers.CommandManager`, which is used internally to | |||||
run IRC commands (through | |||||
:py:meth:`commands.call() <earwigbot.managers.CommandManager.call>`, which | |||||
you shouldn't have to use); you can safely reload all commands with | |||||
:py:meth:`commands.load() <earwigbot.managers._ResourceManager.load>`. | |||||
- :py:attr:`~earwigbot.bot.Bot.tasks`: the bot's | |||||
:py:class:`~earwigbot.managers.TaskManager`, which can be used to start tasks | |||||
with :py:meth:`tasks.start(task_name, **kwargs) | |||||
<earwigbot.managers.TaskManager.start>`. :py:meth:`tasks.load() | |||||
<earwigbot.managers._ResourceManager.load>` can be used to safely reload all | |||||
tasks. | |||||
- :py:attr:`~earwigbot.bot.Bot.frontend` / | |||||
:py:attr:`~earwigbot.bot.Bot.watcher`: instances of | |||||
:py:class:`earwigbot.irc.Frontend <earwigbot.irc.frontend.Frontend>` and | |||||
:py:class:`earwigbot.irc.Watcher <earwigbot.irc.watcher.Watcher>`, | |||||
respectively, which represent the bot's connections to these two servers; you | |||||
can, for example, send a message to the frontend with | |||||
:py:meth:`frontend.say(chan, msg) | |||||
<earwigbot.irc.connection.IRCConnection.say>` (more on communicating with IRC | |||||
below). | |||||
- :py:attr:`~earwigbot.bot.Bot.wiki`: interface with the | |||||
:doc:`Wiki Toolset <toolset>`. | |||||
- Finally, :py:meth:`~earwigbot.bot.Bot.restart` (restarts IRC components and | |||||
reloads config, commands, and tasks) and :py:meth:`~earwigbot.bot.Bot.stop` | |||||
can be used almost anywhere. Both take an optional "reason" that will be | |||||
logged and used as the quit message when disconnecting from IRC. | |||||
:py:class:`earwigbot.config.BotConfig` stores configuration information for the | |||||
bot. Its docstrings explains what each attribute is used for, but essentially | |||||
each "node" (one of :py:attr:`config.components | |||||
<earwigbot.config.BotConfig.components>`, | |||||
:py:attr:`~earwigbot.config.BotConfig.wiki`, | |||||
:py:attr:`~earwigbot.config.BotConfig.irc`, | |||||
:py:attr:`~earwigbot.config.BotConfig.commands`, | |||||
:py:attr:`~earwigbot.config.BotConfig.tasks`, or | |||||
:py:attr:`~earwigbot.config.BotConfig.metadata`) maps to a section | |||||
of the bot's :file:`config.yml` file. For example, if :file:`config.yml` | |||||
includes something like:: | |||||
irc: | |||||
frontend: | |||||
nick: MyAwesomeBot | |||||
channels: | |||||
- "##earwigbot" | |||||
- "#channel" | |||||
- "#other-channel" | |||||
...then :py:attr:`config.irc["frontend"]["nick"]` will be ``"MyAwesomeBot"`` | |||||
and :py:attr:`config.irc["frontend"]["channels"]` will be | |||||
``["##earwigbot", "#channel", "#other-channel"]``. | |||||
Custom IRC commands | |||||
------------------- | |||||
Custom commands are subclasses of :py:class:`earwigbot.commands.Command` that | |||||
override :py:class:`~earwigbot.commands.Command`'s | |||||
:py:meth:`~earwigbot.commands.Command.process` (and optionally | |||||
:py:meth:`~earwigbot.commands.Command.check` or | |||||
:py:meth:`~earwigbot.commands.Command.setup`) methods. | |||||
:py:class:`~earwigbot.commands.Command`'s docstrings should explain what each | |||||
attribute and method is for and what they should be overridden with, but these | |||||
are the basics: | |||||
- Class attribute :py:attr:`~earwigbot.commands.Command.name` is the name of | |||||
the command. This must be specified. | |||||
- Class attribute :py:attr:`~earwigbot.commands.Command.commands` is a list of | |||||
names that will trigger this command. It defaults to the command's | |||||
:py:attr:`~earwigbot.commands.Command.name`, but you can override it with | |||||
multiple names to serve as aliases. This is handled by the default | |||||
:py:meth:`~earwigbot.commands.Command.check` implementation (see below), so | |||||
if :py:meth:`~earwigbot.commands.Command.check` is overridden, this is | |||||
ignored by everything except the help_ command (so ``!help alias`` will | |||||
trigger help for the actual command). | |||||
- Class attribute :py:attr:`~earwigbot.commands.Command.hooks` is a list of the | |||||
"IRC events" that this command might respond to. It defaults to ``["msg"]``, | |||||
but options include ``"msg_private"`` (for private messages only), | |||||
``"msg_public"`` (for channel messages only), and ``"join"`` (for when a user | |||||
joins a channel). See the afc_status_ plugin for a command that responds to | |||||
other hook types. | |||||
- Method :py:meth:`~earwigbot.commands.Command.setup` is called *once* with no | |||||
arguments immediately after the command is first loaded. Does nothing by | |||||
default; treat it like an :py:meth:`__init__` if you want | |||||
(:py:meth:`~earwigbot.tasks.Command.__init__` does things by default and a | |||||
dedicated setup method is often easier than overriding | |||||
:py:meth:`~earwigbot.tasks.Command.__init__` and using :py:obj:`super`). | |||||
- Method :py:meth:`~earwigbot.commands.Command.check` is passed a | |||||
:py:class:`~earwigbot.irc.data.Data` object, and should return ``True`` if | |||||
you want to respond to this message, or ``False`` otherwise. The default | |||||
behavior is to return ``True`` only if :py:attr:`data.is_command` is ``True`` | |||||
and :py:attr:`data.command` ``==`` | |||||
:py:attr:`~earwigbot.commands.Command.name` (or :py:attr:`data.command | |||||
<earwigbot.irc.data.Data.command>` is in | |||||
:py:attr:`~earwigbot.commands.Command.commands` if that list is overriden; | |||||
see above), which is suitable for most cases. A possible reason for | |||||
overriding is if you want to do something in response to events from a | |||||
specific channel only. Note that by returning ``True``, you prevent any other | |||||
commands from responding to this message. | |||||
- Method :py:meth:`~earwigbot.commands.Command.process` is passed the same | |||||
:py:class:`~earwigbot.irc.data.Data` object as | |||||
:py:meth:`~earwigbot.commands.Command.check`, but only if | |||||
:py:meth:`~earwigbot.commands.Command.check` returned ``True``. This is where | |||||
the bulk of your command goes. To respond to IRC messages, there are a number | |||||
of methods of :py:class:`~earwigbot.commands.Command` at your disposal. See | |||||
the test_ command for a simple example, or look in | |||||
:py:class:`~earwigbot.commands.Command`'s | |||||
:py:meth:`~earwigbot.commands.Command.__init__` method for the full list. | |||||
The most common ones are :py:meth:`say(chan_or_user, msg) | |||||
<earwigbot.irc.connection.IRCConnection.say>`, :py:meth:`reply(data, msg) | |||||
<earwigbot.irc.connection.IRCConnection.reply>` (convenience function; sends | |||||
a reply to the issuer of the command in the channel it was received), | |||||
:py:meth:`action(chan_or_user, msg) | |||||
<earwigbot.irc.connection.IRCConnection.action>`, | |||||
:py:meth:`notice(chan_or_user, msg) | |||||
<earwigbot.irc.connection.IRCConnection.notice>`, :py:meth:`join(chan) | |||||
<earwigbot.irc.connection.IRCConnection.join>`, and | |||||
:py:meth:`part(chan) <earwigbot.irc.connection.IRCConnection.part>`. | |||||
Commands have access to :py:attr:`config.commands[command_name]` for config | |||||
information, which is a node in :file:`config.yml` like every other attribute | |||||
of :py:attr:`bot.config`. This can be used to store, for example, API keys or | |||||
SQL connection info, so that these can be easily changed without modifying the | |||||
command itself. | |||||
The command *class* doesn't need a specific name, but it should logically | |||||
follow the command's name. The filename doesn't matter, but it is recommended | |||||
to match the command name for readability. Multiple command classes are allowed | |||||
in one file. | |||||
The bot has a wide selection of built-in commands and plugins to act as sample | |||||
code and/or to give ideas. Start with test_, and then check out chanops_ and | |||||
afc_status_ for some more complicated scripts. | |||||
Custom bot tasks | |||||
---------------- | |||||
Custom tasks are subclasses of :py:class:`earwigbot.tasks.Task` that | |||||
override :py:class:`~earwigbot.tasks.Task`'s | |||||
:py:meth:`~earwigbot.tasks.Task.run` (and optionally | |||||
:py:meth:`~earwigbot.tasks.Task.setup`) methods. | |||||
:py:class:`~earwigbot.tasks.Task`'s docstrings should explain what each | |||||
attribute and method is for and what they should be overridden with, but these | |||||
are the basics: | |||||
- Class attribute :py:attr:`~earwigbot.tasks.Task.name` is the name of the | |||||
task. This must be specified. | |||||
- Class attribute :py:attr:`~earwigbot.tasks.Task.number` can be used to store | |||||
an optional "task number", possibly for use in edit summaries (to be | |||||
generated with :py:meth:`~earwigbot.tasks.Task.make_summary`). For | |||||
example, EarwigBot's :py:attr:`config.wiki["summary"]` is | |||||
``"([[WP:BOT|Bot]]; [[User:EarwigBot#Task $1|Task $1]]): $2"``, which the | |||||
task class's :py:meth:`make_summary(comment) | |||||
<earwigbot.tasks.Task.make_summary>` method will take and replace | |||||
``$1`` with the task number and ``$2`` with the details of the edit. | |||||
Additionally, :py:meth:`~earwigbot.tasks.Task.shutoff_enabled` (which checks | |||||
whether the bot has been told to stop on-wiki by checking the content of a | |||||
particular page) can check a different page for each task using similar | |||||
variables. EarwigBot's :py:attr:`config.wiki["shutoff"]["page"]` is | |||||
``"User:$1/Shutoff/Task $2"``; ``$1`` is substituted with the bot's username, | |||||
and ``$2`` is substituted with the task number, so, e.g., task #14 checks the | |||||
page ``[[User:EarwigBot/Shutoff/Task 14]].`` If the page's content does *not* | |||||
match :py:attr:`config.wiki["shutoff"]["disabled"]` (``"run"`` by default), | |||||
then shutoff is considered to be *enabled* and | |||||
:py:meth:`~earwigbot.tasks.Task.shutoff_enabled` will return ``True``, | |||||
indicating the task should not run. If you don't intend to use either of | |||||
these methods, feel free to leave this attribute blank. | |||||
- Method :py:meth:`~earwigbot.tasks.Task.setup` is called *once* with no | |||||
arguments immediately after the task is first loaded. Does nothing by | |||||
default; treat it like an :py:meth:`__init__` if you want | |||||
(:py:meth:`~earwigbot.tasks.Task.__init__` does things by default and a | |||||
dedicated setup method is often easier than overriding | |||||
:py:meth:`~earwigbot.tasks.Task.__init__` and using :py:obj:`super`). | |||||
- Method :py:meth:`~earwigbot.tasks.Task.run` is called with any number of | |||||
keyword arguments every time the task is executed (by | |||||
:py:meth:`tasks.start(task_name, **kwargs) | |||||
<earwigbot.managers.TaskManager.start>`, usually). This is where the bulk of | |||||
the task's code goes. For interfacing with MediaWiki sites, read up on the | |||||
:doc:`Wiki Toolset <toolset>`. | |||||
Tasks have access to :py:attr:`config.tasks[task_name]` for config information, | |||||
which is a node in :file:`config.yml` like every other attribute of | |||||
:py:attr:`bot.config`. This can be used to store, for example, edit summaries | |||||
or templates to append to user talk pages, so that these can be easily changed | |||||
without modifying the task itself. | |||||
The task *class* doesn't need a specific name, but it should logically follow | |||||
the task's name. The filename doesn't matter, but it is recommended to match | |||||
the task name for readability. Multiple tasks classes are allowed in one file. | |||||
See the built-in wikiproject_tagger_ task for a relatively straightforward | |||||
task, or the afc_statistics_ plugin for a more complicated one. | |||||
.. _help: https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/help.py | |||||
.. _afc_status: https://github.com/earwig/earwigbot-plugins/blob/develop/commands/afc_status.py | |||||
.. _test: https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/test.py | |||||
.. _chanops: https://github.com/earwig/earwigbot/blob/develop/earwigbot/commands/chanops.py | |||||
.. _wikiproject_tagger: https://github.com/earwig/earwigbot/blob/develop/earwigbot/tasks/wikiproject_tagger.py | |||||
.. _afc_statistics: https://github.com/earwig/earwigbot-plugins/blob/develop/tasks/afc_statistics.py |
@@ -0,0 +1,48 @@ | |||||
EarwigBot v0.1 Documentation | |||||
============================ | |||||
EarwigBot_ is a Python_ robot that edits Wikipedia_ and interacts with people | |||||
over IRC_. | |||||
History | |||||
------- | |||||
Development began, based on the `Pywikipedia framework`_, in early 2009. | |||||
Approval for its fist task, a `copyright violation detector`_, was carried out | |||||
in May, and the bot has been running consistently ever since (with the | |||||
exception of Jan/Feb 2011). It currently handles `several ongoing tasks`_ | |||||
ranging from statistics generation to category cleanup, and on-demand tasks | |||||
such as WikiProject template tagging. Since it started running, the bot has | |||||
made over 50,000 edits. | |||||
A project to rewrite it from scratch began in early April 2011, thus moving | |||||
away from the Pywikipedia framework and allowing for less overall code, better | |||||
integration between bot parts, and easier maintenance. | |||||
.. _EarwigBot: http://en.wikipedia.org/wiki/User:EarwigBot | |||||
.. _Python: http://python.org/ | |||||
.. _Wikipedia: http://en.wikipedia.org/ | |||||
.. _IRC: http://en.wikipedia.org/wiki/Internet_Relay_Chat | |||||
.. _Pywikipedia framework: http://pywikipediabot.sourceforge.net/ | |||||
.. _copyright violation detector: http://en.wikipedia.org/wiki/Wikipedia:Bots/Requests_for_approval/EarwigBot_1 | |||||
.. _several ongoing tasks: http://en.wikipedia.org/wiki/User:EarwigBot#Tasks | |||||
Contents | |||||
-------- | |||||
.. toctree:: | |||||
:maxdepth: 2 | |||||
installation | |||||
setup | |||||
customizing | |||||
toolset | |||||
tips | |||||
API Reference <api/modules> | |||||
Indices and tables | |||||
------------------ | |||||
* :ref:`genindex` | |||||
* :ref:`modindex` | |||||
* :ref:`search` |
@@ -0,0 +1,55 @@ | |||||
Installation | |||||
============ | |||||
This package contains the core :py:mod:`earwigbot`, abstracted enough that it | |||||
should be usable and customizable by anyone running a bot on a MediaWiki site. | |||||
Since it is component-based, the IRC components can be disabled if desired. IRC | |||||
commands and bot tasks specific to `my instance of EarwigBot`_ that I don't | |||||
feel the average user will need are available from the repository | |||||
`earwigbot-plugins`_. | |||||
It's recommended to run the bot's unit tests before installing. Run | |||||
:command:`python setup.py test` from the project's root directory. Note that | |||||
some tests require an internet connection, and others may take a while to run. | |||||
Coverage is currently rather incomplete. | |||||
Latest release (v0.1) | |||||
--------------------- | |||||
EarwigBot is available from the `Python Package Index`_, so you can install the | |||||
latest release with :command:`pip install earwigbot` (`get pip`_). | |||||
You can also install it from source [1]_ directly:: | |||||
curl -Lo earwigbot.tgz https://github.com/earwig/earwigbot/tarball/v0.1 | |||||
tar -xf earwigbot.tgz | |||||
cd earwig-earwigbot-* | |||||
python setup.py install | |||||
cd .. | |||||
rm -r earwigbot.tgz earwig-earwigbot-* | |||||
Development version | |||||
------------------- | |||||
You can install the development version of the bot from :command:`git` by using | |||||
setuptools/`distribute`_'s :command:`develop` command [1]_, probably on the | |||||
``develop`` branch which contains (usually) working code. ``master`` contains | |||||
the latest release. EarwigBot uses `git flow`_, so you're free to browse by | |||||
tags or by new features (``feature/*`` branches):: | |||||
git clone git://github.com/earwig/earwigbot.git earwigbot | |||||
cd earwigbot | |||||
python setup.py develop | |||||
.. rubric:: Footnotes | |||||
.. [1] :command:`python setup.py install`/:command:`develop` may require root, | |||||
or use the :command:`--user` switch to install for the current user | |||||
only. | |||||
.. _my instance of EarwigBot: http://en.wikipedia.org/wiki/User:EarwigBot | |||||
.. _earwigbot-plugins: https://github.com/earwig/earwigbot-plugins | |||||
.. _Python Package Index: http://pypi.python.org | |||||
.. _get pip: http://pypi.python.org/pypi/pip | |||||
.. _distribute: http://pypi.python.org/pypi/distribute | |||||
.. _git flow: http://nvie.com/posts/a-successful-git-branching-model/ |
@@ -0,0 +1,28 @@ | |||||
Setup | |||||
===== | |||||
The bot stores its data in a "working directory", including its config file and | |||||
databases. This is also the location where you will place custom IRC commands | |||||
and bot tasks, which will be explained later. It doesn't matter where this | |||||
directory is, as long as the bot can write to it. | |||||
Start the bot with :command:`earwigbot path/to/working/dir`, or just | |||||
:command:`earwigbot` if the working directory is the current directory. It will | |||||
notice that no :file:`config.yml` file exists and take you through the setup | |||||
process. | |||||
There is currently no way to edit the :file:`config.yml` file from within the | |||||
bot after it has been created, but YAML is a very straightforward format, so | |||||
you should be able to make any necessary changes yourself. Check out the | |||||
`explanation of YAML`_ on Wikipedia for help. | |||||
After setup, the bot will start. This means it will connect to the IRC servers | |||||
it has been configured for, schedule bot tasks to run at specific times, and | |||||
then wait for instructions (as commands on IRC). For a list of commands, say | |||||
"``!help``" (commands are messages prefixed with an exclamation mark). | |||||
You can stop the bot at any time with :kbd:`Control-c`, same as you stop a | |||||
normal Python program, and it will try to exit safely. You can also use the | |||||
"``!quit``" command on IRC. | |||||
.. _explanation of YAML: http://en.wikipedia.org/wiki/YAML |
@@ -0,0 +1,46 @@ | |||||
Tips | |||||
==== | |||||
- Logging_ is a fantastic way to monitor the bot's progress as it runs. It has | |||||
a slew of built-in loggers, and enabling log retention (so logs are saved to | |||||
:file:`logs/` in the working directory) is highly recommended. In the normal | |||||
setup, there are three log files, each of which "rotate" at a specific time | |||||
(:file:`filename.log` becomes :file:`filename.log.2012-04-10`, for example). | |||||
The :file:`debug.log` file rotates every hour, and maintains six hours of | |||||
logs of every level (``DEBUG`` and up). :file:`bot.log` rotates every day at | |||||
midnight, and maintains seven days of non-debug logs (``INFO`` and up). | |||||
Finally, :file:`error.log` rotates every Sunday night, and maintains four | |||||
weeks of logs indicating unexpected events (``WARNING`` and up). | |||||
To use logging in your commands or tasks (recommended), | |||||
:py:class:~earwigbot.commands.BaseCommand` and | |||||
:py:class:~earwigbot.tasks.BaseTask` provide :py:attr:`logger` attributes | |||||
configured for the specific command or task. If you're working with other | |||||
classes, :py:attr:`bot.logger` is the root logger | |||||
(:py:obj:`logging.getLogger("earwigbot")` by default), so you can use | |||||
:py:func:`~logging.Logger.getChild` to make your logger. For example, task | |||||
loggers are essentially | |||||
:py:attr:`bot.logger.getChild("tasks").getChild(task.name) <bot.logger>`. | |||||
- A very useful IRC command is "``!reload``", which reloads all commands and | |||||
tasks without restarting the bot. [1]_ Combined with using the `!git plugin`_ | |||||
for pulling repositories from IRC, this can provide a seamless command/task | |||||
development workflow if the bot runs on an external server and you set up | |||||
its working directory as a git repo. | |||||
- You can run a task by itself instead of the entire bot with | |||||
:command:`earwigbot path/to/working/dir --task task_name`. | |||||
- Questions, comments, or suggestions about the documentation? `Let me know`_, | |||||
or `create an issue`_ so I can improve it for other people. | |||||
.. rubric:: Footnotes | |||||
.. [1] In reality, all this does is call :py:meth:`bot.commands.load() | |||||
<earwigbot.managers._ResourceManager.load>` and | |||||
:py:meth:`bot.tasks.load() <earwigbot.managers._ResourceManager.load>`! | |||||
.. _logging: http://docs.python.org/library/logging.html | |||||
.. _!git plugin: https://github.com/earwig/earwigbot-plugins/blob/develop/commands/git.py | |||||
.. _Let me know: ben.kurtovic@verizon.net | |||||
.. _create an issue: https://github.com/earwig/earwigbot/issues |
@@ -0,0 +1,244 @@ | |||||
The Wiki Toolset | |||||
================ | |||||
EarwigBot's answer to the `Pywikipedia framework`_ is the Wiki Toolset | |||||
(:py:mod:`earwigbot.wiki`), which you will mainly access through | |||||
:py:attr:`bot.wiki <earwigbot.bot.Bot.wiki>`. | |||||
:py:attr:`bot.wiki <earwigbot.bot.Bot.wiki>` provides three methods for the | |||||
management of Sites - :py:meth:`~earwigbot.wiki.sitesdb.SitesDB.get_site`, | |||||
:py:meth:`~earwigbot.wiki.sitesdb.SitesDB.add_site`, and | |||||
:py:meth:`~earwigbot.wiki.sitesdb.SitesDB.remove_site`. Sites are objects that | |||||
simply represent a MediaWiki site. A single instance of EarwigBot (i.e. a | |||||
single *working directory*) is expected to relate to a single site or group of | |||||
sites using the same login info (like all WMF wikis with `CentralAuth`_). | |||||
Load your default site (the one that you picked during setup) with | |||||
``site = bot.wiki.get_site()``. | |||||
Dealing with other sites | |||||
~~~~~~~~~~~~~~~~~~~~~~~~ | |||||
*Skip this section if you're only working with one site.* | |||||
If a site is *already known to the bot* (meaning that it is stored in the | |||||
:file:`sites.db` file, which includes just your default wiki at first), you can | |||||
load a site with ``site = bot.wiki.get_site(name)``, where ``name`` might be | |||||
``"enwiki"`` or ``"frwiktionary"`` (you can also do | |||||
``site = bot.wiki.get_site(project="wikipedia", lang="en")``). Recall that not | |||||
giving any arguments to ``get_site()`` will return the default site. | |||||
:py:meth:`~earwigbot.wiki.sitesdb.SitesDB.add_site` is used to add new sites to | |||||
the sites database. It may be called with similar arguments as | |||||
:py:meth:`~earwigbot.wiki.sitesdb.SitesDB.get_site`, but the difference is | |||||
important. :py:meth:`~earwigbot.wiki.sitesdb.SitesDB.get_site` only needs | |||||
enough information to identify the site in its database, which is usually just | |||||
its name; the database stores all other necessary connection info. With | |||||
:py:meth:`~earwigbot.wiki.sitesdb.SitesDB.add_site`, you need to provide enough | |||||
connection info so the toolset can successfully access the site's API/SQL | |||||
databases and store that information for later. That might not be much; for WMF | |||||
wikis, you can usually use code like this:: | |||||
project, lang = "wikipedia", "es" | |||||
try: | |||||
site = bot.wiki.get_site(project=project, lang=lang) | |||||
except earwigbot.SiteNotFoundError: | |||||
# Load site info from http://es.wikipedia.org/w/api.php: | |||||
site = bot.wiki.add_site(project=project, lang=lang) | |||||
This works because EarwigBot assumes that the URL for the site is | |||||
``"//{lang}.{project}.org"`` and the API is at ``/w/api.php``; this might | |||||
change if you're dealing with non-WMF wikis, where the code might look | |||||
something more like:: | |||||
project, lang = "mywiki", "it" | |||||
try: | |||||
site = bot.wiki.get_site(project=project, lang=lang) | |||||
except earwigbot.SiteNotFoundError: | |||||
# Load site info from http://mysite.net/mywiki/it/s/api.php: | |||||
base_url = "http://mysite.net/" + project + "/" + lang | |||||
db_name = lang + project + "_p" | |||||
sql = {host: "sql.mysite.net", db: db_name} | |||||
site = bot.wiki.add_site(base_url=base_url, script_path="/s", sql=sql) | |||||
:py:meth:`~earwigbot.wiki.sitesdb.SitesDB.remove_site` does the opposite of | |||||
:py:meth:`~earwigbot.wiki.sitesdb.SitesDB.add_site`: give it a site's name or a | |||||
project/lang pair like :py:meth:`~earwigbot.wiki.sitesdb.SitesDB.get_site` | |||||
takes, and it'll remove that site from the sites database. | |||||
Sites | |||||
~~~~~ | |||||
:py:class:`earwigbot.wiki.Site <earwigbot.wiki.site.Site>` objects provide the | |||||
following attributes: | |||||
- :py:attr:`~earwigbot.wiki.site.Site.name`: the site's name (or "wikiid"), | |||||
like ``"enwiki"`` | |||||
- :py:attr:`~earwigbot.wiki.site.Site.project`: the site's project name, like | |||||
``"wikipedia"`` | |||||
- :py:attr:`~earwigbot.wiki.site.Site.lang`: the site's language code, like | |||||
``"en"`` | |||||
- :py:attr:`~earwigbot.wiki.site.Site.domain`: the site's web domain, like | |||||
``"en.wikipedia.org"`` | |||||
- :py:attr:`~earwigbot.wiki.site.Site.url`: the site's full base URL, like | |||||
``"https://en.wikipedia.org"`` | |||||
and the following methods: | |||||
- :py:meth:`api_query(**kwargs) <earwigbot.wiki.site.Site.api_query>`: does an | |||||
API query with the given keyword arguments as params | |||||
- :py:meth:`sql_query(query, params=(), ...) | |||||
<earwigbot.wiki.site.Site.sql_query>`: does an SQL query and yields its | |||||
results (as a generator) | |||||
- :py:meth:`~earwigbot.wiki.site.Site.get_replag`: returns the estimated | |||||
database replication lag (if we have the site's SQL connection info) | |||||
- :py:meth:`namespace_id_to_name(id, all=False) | |||||
<earwigbot.wiki.site.Site.namespace_id_to_name>`: given a namespace ID, | |||||
returns the primary associated namespace name (or a list of all names when | |||||
``all`` is ``True``) | |||||
- :py:meth:`namespace_name_to_id(name) | |||||
<earwigbot.wiki.site.Site.namespace_name_to_id>`: given a namespace name, | |||||
returns the associated namespace ID | |||||
- :py:meth:`get_page(title, follow_redirects=False, ...) | |||||
<earwigbot.wiki.site.Site.get_page>`: returns a ``Page`` object for the given | |||||
title (or a :py:class:`~earwigbot.wiki.category.Category` object if the | |||||
page's namespace is "``Category:``") | |||||
- :py:meth:`get_category(catname, follow_redirects=False, ...) | |||||
<earwigbot.wiki.site.Site.get_category>`: returns a ``Category`` object for | |||||
the given title (sans namespace) | |||||
- :py:meth:`get_user(username) <earwigbot.wiki.site.Site.get_user>`: returns a | |||||
:py:class:`~earwigbot.wiki.user.User` object for the given username | |||||
- :py:meth:`delegate(services, ...) <earwigbot.wiki.site.Site.delegate>`: | |||||
delegates a task to either the API or SQL depending on various conditions, | |||||
such as server lag | |||||
Pages and categories | |||||
~~~~~~~~~~~~~~~~~~~~ | |||||
Create :py:class:`earwigbot.wiki.Page <earwigbot.wiki.page.Page>` objects with | |||||
:py:meth:`site.get_page(title) <earwigbot.wiki.site.Site.get_page>`, | |||||
:py:meth:`page.toggle_talk() <earwigbot.wiki.page.Page.toggle_talk>`, | |||||
:py:meth:`user.get_userpage() <earwigbot.wiki.user.User.get_userpage>`, or | |||||
:py:meth:`user.get_talkpage() <earwigbot.wiki.user.User.get_talkpage>`. They | |||||
provide the following attributes: | |||||
- :py:attr:`~earwigbot.wiki.page.Page.site`: the page's corresponding | |||||
:py:class:`~earwigbot.wiki.site.Site` object | |||||
- :py:attr:`~earwigbot.wiki.page.Page.title`: the page's title, or pagename | |||||
- :py:attr:`~earwigbot.wiki.page.Page.exists`: whether or not the page exists | |||||
- :py:attr:`~earwigbot.wiki.page.Page.pageid`: an integer ID representing the | |||||
page | |||||
- :py:attr:`~earwigbot.wiki.page.Page.url`: the page's URL | |||||
- :py:attr:`~earwigbot.wiki.page.Page.namespace`: the page's namespace as an | |||||
integer | |||||
- :py:attr:`~earwigbot.wiki.page.Page.protection`: the page's current | |||||
protection status | |||||
- :py:attr:`~earwigbot.wiki.page.Page.is_talkpage`: ``True`` if the page is a | |||||
talkpage, else ``False`` | |||||
- :py:attr:`~earwigbot.wiki.page.Page.is_redirect`: ``True`` if the page is a | |||||
redirect, else ``False`` | |||||
and the following methods: | |||||
- :py:meth:`~earwigbot.wiki.page.Page.reload`: forcibly reloads the page's | |||||
attributes (emphasis on *reload* - this is only necessary if there is reason | |||||
to believe they have changed) | |||||
- :py:meth:`toggle_talk(...) <earwigbot.wiki.page.Page.toggle_talk>`: returns a | |||||
content page's talk page, or vice versa | |||||
- :py:meth:`~earwigbot.wiki.page.Page.get`: returns page content | |||||
- :py:meth:`~earwigbot.wiki.page.Page.get_redirect_target`: if the page is a | |||||
redirect, returns its destination | |||||
- :py:meth:`~earwigbot.wiki.page.Page.get_creator`: returns a | |||||
:py:class:`~earwigbot.wiki.user.User` object representing the first user to | |||||
edit the page | |||||
- :py:meth:`edit(text, summary, minor=False, bot=True, force=False) | |||||
<earwigbot.wiki.page.Page.edit>`: replaces the page's content with ``text`` | |||||
or creates a new page | |||||
- :py:meth:`add_section(text, title, minor=False, bot=True, force=False) | |||||
<earwigbot.wiki.page.Page.add_section>`: adds a new section named ``title`` | |||||
at the bottom of the page | |||||
- :py:meth:`copyvio_check(...) | |||||
<earwigbot.wiki.copyvios.CopyvioMixIn.copyvio_check>`: checks the page for | |||||
copyright violations | |||||
- :py:meth:`copyvio_compare(url, ...) | |||||
<earwigbot.wiki.copyvios.CopyvioMixIn.copyvio_compare>`: checks the page like | |||||
:py:meth:`~earwigbot.wiki.copyvios.CopyvioMixIn.copyvio_check`, but | |||||
against a specific URL | |||||
- :py:meth:`check_exclusion(username=None, optouts=None) | |||||
<earwigbot.wiki.page.Page.check_exclusion>`: checks whether or not we are | |||||
allowed to edit the page per ``{{bots}}``/``{{nobots}}`` | |||||
Additionally, :py:class:`~earwigbot.wiki.category.Category` objects (created | |||||
with :py:meth:`site.get_category(name) <earwigbot.wiki.site.Site.get_category>` | |||||
or :py:meth:`site.get_page(title) <earwigbot.wiki.site.Site.get_page>` where | |||||
``title`` is in the ``Category:`` namespace) provide the following additional | |||||
attributes: | |||||
- :py:attr:`~earwigbot.wiki.category.Category.size`: the total number of | |||||
members in the category | |||||
- :py:attr:`~earwigbot.wiki.category.Category.pages`: the number of pages in | |||||
the category | |||||
- :py:attr:`~earwigbot.wiki.category.Category.files`: the number of files in | |||||
the category | |||||
- :py:attr:`~earwigbot.wiki.category.Category.subcats`: the number of | |||||
subcategories in the category | |||||
And the following additional method: | |||||
- :py:meth:`get_members(limit=None, ...) | |||||
<earwigbot.wiki.category.Category.get_members>`: iterates over | |||||
:py:class:`~earwigbot.wiki.page.Page`\ s in the category, until either the | |||||
category is exhausted or (if given) ``limit`` is reached | |||||
Users | |||||
~~~~~ | |||||
Create :py:class:`earwigbot.wiki.User <earwigbot.wiki.user.User>` objects with | |||||
:py:meth:`site.get_user(name) <earwigbot.wiki.site.Site.get_user>` or | |||||
:py:meth:`page.get_creator() <earwigbot.wiki.page.Page.get_creator>`. They | |||||
provide the following attributes: | |||||
- :py:attr:`~earwigbot.wiki.user.User.site`: the user's corresponding | |||||
:py:class:`~earwigbot.wiki.site.Site` object | |||||
- :py:attr:`~earwigbot.wiki.user.User.name`: the user's username | |||||
- :py:attr:`~earwigbot.wiki.user.User.exists`: ``True`` if the user exists, or | |||||
``False`` if they do not | |||||
- :py:attr:`~earwigbot.wiki.user.User.userid`: an integer ID representing the | |||||
user | |||||
- :py:attr:`~earwigbot.wiki.user.User.blockinfo`: information about any current | |||||
blocks on the user (``False`` if no block, or a dict of | |||||
``{"by": blocking_user, "reason": block_reason, | |||||
"expiry": block_expire_time}``) | |||||
- :py:attr:`~earwigbot.wiki.user.User.groups`: a list of the user's groups | |||||
- :py:attr:`~earwigbot.wiki.user.User.rights`: a list of the user's rights | |||||
- :py:attr:`~earwigbot.wiki.user.User.editcount`: the number of edits made by | |||||
the user | |||||
- :py:attr:`~earwigbot.wiki.user.User.registration`: the time the user | |||||
registered as a :py:obj:`time.struct_time` | |||||
- :py:attr:`~earwigbot.wiki.user.User.emailable`: ``True`` if you can email the | |||||
user, ``False`` if you cannot | |||||
- :py:attr:`~earwigbot.wiki.user.User.gender`: the user's gender (``"male"``, | |||||
``"female"``, or ``"unknown"``) | |||||
and the following methods: | |||||
- :py:meth:`~earwigbot.wiki.user.User.reload`: forcibly reloads the user's | |||||
attributes (emphasis on *reload* - this is only necessary if there is reason | |||||
to believe they have changed) | |||||
- :py:meth:`~earwigbot.wiki.user.User.get_userpage`: returns a | |||||
:py:class:`~earwigbot.wiki.page.Page` object representing the user's userpage | |||||
- :py:meth:`~earwigbot.wiki.user.User.get_talkpage`: returns a | |||||
:py:class:`~earwigbot.wiki.page.Page` object representing the user's talkpage | |||||
Additional features | |||||
~~~~~~~~~~~~~~~~~~~ | |||||
Not all aspects of the toolset are covered here. Explore `its code and | |||||
docstrings`_ to learn how to use it in a more hands-on fashion. For reference, | |||||
:py:attr:`bot.wiki <earwigbot.bot.Bot.wiki>` is an instance of | |||||
:py:class:`earwigbot.wiki.SitesDB <earwigbot.wiki.sitesdb.SitesDB>` tied to the | |||||
:file:`sites.db` file in the bot's working directory. | |||||
.. _Pywikipedia framework: http://pywikipediabot.sourceforge.net/ | |||||
.. _CentralAuth: http://www.mediawiki.org/wiki/Extension:CentralAuth | |||||
.. _its code and docstrings: https://github.com/earwig/earwigbot/tree/develop/earwigbot/wiki |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -21,17 +21,42 @@ | |||||
# SOFTWARE. | # SOFTWARE. | ||||
""" | """ | ||||
EarwigBot - http://earwig.github.com/earwig/earwigbot | |||||
See README.md for a basic overview, or the docs/ directory for details. | |||||
`EarwigBot <https://github.com/earwig/earwigbot>`_ is a Python robot that edits | |||||
Wikipedia and interacts with people over IRC. | |||||
See :file:`README.rst` for an overview, or the :file:`docs/` directory for | |||||
details. This documentation is also available `online | |||||
<http://packages.python.org/earwigbot>`_. | |||||
""" | """ | ||||
__author__ = "Ben Kurtovic" | __author__ = "Ben Kurtovic" | ||||
__copyright__ = "Copyright (C) 2009, 2010, 2011 by Ben Kurtovic" | |||||
__copyright__ = "Copyright (C) 2009, 2010, 2011, 2012 by Ben Kurtovic" | |||||
__license__ = "MIT License" | __license__ = "MIT License" | ||||
__version__ = "0.1.dev" | __version__ = "0.1.dev" | ||||
__email__ = "ben.kurtovic@verizon.net" | __email__ = "ben.kurtovic@verizon.net" | ||||
__release__ = False | |||||
if not __release__: | |||||
def _get_git_commit_id(): | |||||
"""Return the ID of the git HEAD commit.""" | |||||
from git import Repo | |||||
from os.path import split, dirname | |||||
path = split(dirname(__file__))[0] | |||||
commit_id = Repo(path).head.object.hexsha | |||||
return commit_id[:8] | |||||
try: | |||||
__version__ += ".git+" + _get_git_commit_id() | |||||
except Exception: | |||||
pass | |||||
finally: | |||||
del _get_git_commit_id | |||||
from earwigbot import ( | |||||
blowfish, config, classes, commands, config, frontend, main, rules, tasks, | |||||
tests, watcher, wiki | |||||
) | |||||
from earwigbot import bot | |||||
from earwigbot import commands | |||||
from earwigbot import config | |||||
from earwigbot import exceptions | |||||
from earwigbot import irc | |||||
from earwigbot import managers | |||||
from earwigbot import tasks | |||||
from earwigbot import util | |||||
from earwigbot import wiki |
@@ -1,556 +0,0 @@ | |||||
#! /usr/bin/env python | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# blowfish.py | |||||
# Copyright (C) 2002 Michael Gilfix <mgilfix@eecs.tufts.edu> | |||||
# Copyright (C) 2011, 2012 Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# This module is open source; you can redistribute it and/or | |||||
# modify it under the terms of the GPL or Artistic License. | |||||
# These licenses are available at http://www.opensource.org | |||||
# | |||||
# This software must be used and distributed in accordance | |||||
# with the law. The author claims no liability for its | |||||
# misuse. | |||||
# | |||||
# This program is distributed in the hope that it will be useful, | |||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of | |||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. | |||||
# | |||||
""" | |||||
Blowfish Encryption | |||||
This module is a pure python implementation of Bruce Schneier's encryption | |||||
scheme 'Blowfish'. Blowish is a 16-round Feistel Network cipher and offers | |||||
substantial speed gains over DES. | |||||
The key is a string of length anywhere between 64 and 448 bits, or equivalently | |||||
8 and 56 bytes. The encryption and decryption functions operate on 64-bit | |||||
blocks, or 8 byte strings. | |||||
The entire Blowfish() class (excluding verify_key()) is by Michael Gilfix | |||||
<mgilfix@eecs.tufts.edu>. | |||||
Blowfish.verify_key(), exception classes, encrypt() and decrypt() wrappers, and | |||||
interactive mode are by Ben Kurtovic <ben.kurtovic@verizon.net>. | |||||
""" | |||||
class BlowfishError(Exception): | |||||
"""Base exception class for errors involving blowfish | |||||
encryption/decryption.""" | |||||
class BlockSizeError(BlowfishError): | |||||
"""Attempted to handle a block not 8 bytes in length.""" | |||||
class KeyLengthError(BlowfishError): | |||||
"""Attempted to use a key that is either less than 8 bytes or more than 56 | |||||
bytes in length.""" | |||||
class DecryptionError(BlowfishError): | |||||
"""Attempted to decrypt malformed cyphertext (e.g., not evenly divisible | |||||
into 8-byte blocks) or attempted to decrypt using a bad key.""" | |||||
class Blowfish(object): | |||||
"""Blowfish encryption Scheme | |||||
This class implements the encryption and decryption | |||||
functionality of the Blowfish cipher. | |||||
Public functions: | |||||
def __init__ (self, key) | |||||
Creates an instance of blowfish using 'key' | |||||
as the encryption key. Key is a string of | |||||
length ranging from 8 to 56 bytes (64 to 448 | |||||
bits). Once the instance of the object is | |||||
created, the key is no longer necessary. | |||||
def encrypt (self, data): | |||||
Encrypt an 8 byte (64-bit) block of text | |||||
where 'data' is an 8 byte string. Returns an | |||||
8-byte encrypted string. | |||||
def decrypt (self, data): | |||||
Decrypt an 8 byte (64-bit) encrypted block | |||||
of text, where 'data' is the 8 byte encrypted | |||||
string. Returns an 8-byte string of plaintext. | |||||
def cipher (self, xl, xr, direction): | |||||
Encrypts a 64-bit block of data where xl is | |||||
the upper 32-bits and xr is the lower 32-bits. | |||||
'direction' is the direction to apply the | |||||
cipher, either ENCRYPT or DECRYPT constants. | |||||
returns a tuple of either encrypted or decrypted | |||||
data of the left half and right half of the | |||||
64-bit block. | |||||
Private members: | |||||
def __round_func (self, xl) | |||||
Performs an obscuring function on the 32-bit | |||||
block of data 'xl', which is the left half of | |||||
the 64-bit block of data. Returns the 32-bit | |||||
result as a long integer. | |||||
""" | |||||
# Cipher directions | |||||
ENCRYPT = 0 | |||||
DECRYPT = 1 | |||||
# For the __round_func | |||||
modulus = long (2) ** 32 | |||||
def __init__ (self, key): | |||||
self.verify_key(key) | |||||
self.p_boxes = [ | |||||
0x243F6A88, 0x85A308D3, 0x13198A2E, 0x03707344, | |||||
0xA4093822, 0x299F31D0, 0x082EFA98, 0xEC4E6C89, | |||||
0x452821E6, 0x38D01377, 0xBE5466CF, 0x34E90C6C, | |||||
0xC0AC29B7, 0xC97C50DD, 0x3F84D5B5, 0xB5470917, | |||||
0x9216D5D9, 0x8979FB1B | |||||
] | |||||
self.s_boxes = [ | |||||
[ | |||||
0xD1310BA6, 0x98DFB5AC, 0x2FFD72DB, 0xD01ADFB7, | |||||
0xB8E1AFED, 0x6A267E96, 0xBA7C9045, 0xF12C7F99, | |||||
0x24A19947, 0xB3916CF7, 0x0801F2E2, 0x858EFC16, | |||||
0x636920D8, 0x71574E69, 0xA458FEA3, 0xF4933D7E, | |||||
0x0D95748F, 0x728EB658, 0x718BCD58, 0x82154AEE, | |||||
0x7B54A41D, 0xC25A59B5, 0x9C30D539, 0x2AF26013, | |||||
0xC5D1B023, 0x286085F0, 0xCA417918, 0xB8DB38EF, | |||||
0x8E79DCB0, 0x603A180E, 0x6C9E0E8B, 0xB01E8A3E, | |||||
0xD71577C1, 0xBD314B27, 0x78AF2FDA, 0x55605C60, | |||||
0xE65525F3, 0xAA55AB94, 0x57489862, 0x63E81440, | |||||
0x55CA396A, 0x2AAB10B6, 0xB4CC5C34, 0x1141E8CE, | |||||
0xA15486AF, 0x7C72E993, 0xB3EE1411, 0x636FBC2A, | |||||
0x2BA9C55D, 0x741831F6, 0xCE5C3E16, 0x9B87931E, | |||||
0xAFD6BA33, 0x6C24CF5C, 0x7A325381, 0x28958677, | |||||
0x3B8F4898, 0x6B4BB9AF, 0xC4BFE81B, 0x66282193, | |||||
0x61D809CC, 0xFB21A991, 0x487CAC60, 0x5DEC8032, | |||||
0xEF845D5D, 0xE98575B1, 0xDC262302, 0xEB651B88, | |||||
0x23893E81, 0xD396ACC5, 0x0F6D6FF3, 0x83F44239, | |||||
0x2E0B4482, 0xA4842004, 0x69C8F04A, 0x9E1F9B5E, | |||||
0x21C66842, 0xF6E96C9A, 0x670C9C61, 0xABD388F0, | |||||
0x6A51A0D2, 0xD8542F68, 0x960FA728, 0xAB5133A3, | |||||
0x6EEF0B6C, 0x137A3BE4, 0xBA3BF050, 0x7EFB2A98, | |||||
0xA1F1651D, 0x39AF0176, 0x66CA593E, 0x82430E88, | |||||
0x8CEE8619, 0x456F9FB4, 0x7D84A5C3, 0x3B8B5EBE, | |||||
0xE06F75D8, 0x85C12073, 0x401A449F, 0x56C16AA6, | |||||
0x4ED3AA62, 0x363F7706, 0x1BFEDF72, 0x429B023D, | |||||
0x37D0D724, 0xD00A1248, 0xDB0FEAD3, 0x49F1C09B, | |||||
0x075372C9, 0x80991B7B, 0x25D479D8, 0xF6E8DEF7, | |||||
0xE3FE501A, 0xB6794C3B, 0x976CE0BD, 0x04C006BA, | |||||
0xC1A94FB6, 0x409F60C4, 0x5E5C9EC2, 0x196A2463, | |||||
0x68FB6FAF, 0x3E6C53B5, 0x1339B2EB, 0x3B52EC6F, | |||||
0x6DFC511F, 0x9B30952C, 0xCC814544, 0xAF5EBD09, | |||||
0xBEE3D004, 0xDE334AFD, 0x660F2807, 0x192E4BB3, | |||||
0xC0CBA857, 0x45C8740F, 0xD20B5F39, 0xB9D3FBDB, | |||||
0x5579C0BD, 0x1A60320A, 0xD6A100C6, 0x402C7279, | |||||
0x679F25FE, 0xFB1FA3CC, 0x8EA5E9F8, 0xDB3222F8, | |||||
0x3C7516DF, 0xFD616B15, 0x2F501EC8, 0xAD0552AB, | |||||
0x323DB5FA, 0xFD238760, 0x53317B48, 0x3E00DF82, | |||||
0x9E5C57BB, 0xCA6F8CA0, 0x1A87562E, 0xDF1769DB, | |||||
0xD542A8F6, 0x287EFFC3, 0xAC6732C6, 0x8C4F5573, | |||||
0x695B27B0, 0xBBCA58C8, 0xE1FFA35D, 0xB8F011A0, | |||||
0x10FA3D98, 0xFD2183B8, 0x4AFCB56C, 0x2DD1D35B, | |||||
0x9A53E479, 0xB6F84565, 0xD28E49BC, 0x4BFB9790, | |||||
0xE1DDF2DA, 0xA4CB7E33, 0x62FB1341, 0xCEE4C6E8, | |||||
0xEF20CADA, 0x36774C01, 0xD07E9EFE, 0x2BF11FB4, | |||||
0x95DBDA4D, 0xAE909198, 0xEAAD8E71, 0x6B93D5A0, | |||||
0xD08ED1D0, 0xAFC725E0, 0x8E3C5B2F, 0x8E7594B7, | |||||
0x8FF6E2FB, 0xF2122B64, 0x8888B812, 0x900DF01C, | |||||
0x4FAD5EA0, 0x688FC31C, 0xD1CFF191, 0xB3A8C1AD, | |||||
0x2F2F2218, 0xBE0E1777, 0xEA752DFE, 0x8B021FA1, | |||||
0xE5A0CC0F, 0xB56F74E8, 0x18ACF3D6, 0xCE89E299, | |||||
0xB4A84FE0, 0xFD13E0B7, 0x7CC43B81, 0xD2ADA8D9, | |||||
0x165FA266, 0x80957705, 0x93CC7314, 0x211A1477, | |||||
0xE6AD2065, 0x77B5FA86, 0xC75442F5, 0xFB9D35CF, | |||||
0xEBCDAF0C, 0x7B3E89A0, 0xD6411BD3, 0xAE1E7E49, | |||||
0x00250E2D, 0x2071B35E, 0x226800BB, 0x57B8E0AF, | |||||
0x2464369B, 0xF009B91E, 0x5563911D, 0x59DFA6AA, | |||||
0x78C14389, 0xD95A537F, 0x207D5BA2, 0x02E5B9C5, | |||||
0x83260376, 0x6295CFA9, 0x11C81968, 0x4E734A41, | |||||
0xB3472DCA, 0x7B14A94A, 0x1B510052, 0x9A532915, | |||||
0xD60F573F, 0xBC9BC6E4, 0x2B60A476, 0x81E67400, | |||||
0x08BA6FB5, 0x571BE91F, 0xF296EC6B, 0x2A0DD915, | |||||
0xB6636521, 0xE7B9F9B6, 0xFF34052E, 0xC5855664, | |||||
0x53B02D5D, 0xA99F8FA1, 0x08BA4799, 0x6E85076A | |||||
], | |||||
[ | |||||
0x4B7A70E9, 0xB5B32944, 0xDB75092E, 0xC4192623, | |||||
0xAD6EA6B0, 0x49A7DF7D, 0x9CEE60B8, 0x8FEDB266, | |||||
0xECAA8C71, 0x699A17FF, 0x5664526C, 0xC2B19EE1, | |||||
0x193602A5, 0x75094C29, 0xA0591340, 0xE4183A3E, | |||||
0x3F54989A, 0x5B429D65, 0x6B8FE4D6, 0x99F73FD6, | |||||
0xA1D29C07, 0xEFE830F5, 0x4D2D38E6, 0xF0255DC1, | |||||
0x4CDD2086, 0x8470EB26, 0x6382E9C6, 0x021ECC5E, | |||||
0x09686B3F, 0x3EBAEFC9, 0x3C971814, 0x6B6A70A1, | |||||
0x687F3584, 0x52A0E286, 0xB79C5305, 0xAA500737, | |||||
0x3E07841C, 0x7FDEAE5C, 0x8E7D44EC, 0x5716F2B8, | |||||
0xB03ADA37, 0xF0500C0D, 0xF01C1F04, 0x0200B3FF, | |||||
0xAE0CF51A, 0x3CB574B2, 0x25837A58, 0xDC0921BD, | |||||
0xD19113F9, 0x7CA92FF6, 0x94324773, 0x22F54701, | |||||
0x3AE5E581, 0x37C2DADC, 0xC8B57634, 0x9AF3DDA7, | |||||
0xA9446146, 0x0FD0030E, 0xECC8C73E, 0xA4751E41, | |||||
0xE238CD99, 0x3BEA0E2F, 0x3280BBA1, 0x183EB331, | |||||
0x4E548B38, 0x4F6DB908, 0x6F420D03, 0xF60A04BF, | |||||
0x2CB81290, 0x24977C79, 0x5679B072, 0xBCAF89AF, | |||||
0xDE9A771F, 0xD9930810, 0xB38BAE12, 0xDCCF3F2E, | |||||
0x5512721F, 0x2E6B7124, 0x501ADDE6, 0x9F84CD87, | |||||
0x7A584718, 0x7408DA17, 0xBC9F9ABC, 0xE94B7D8C, | |||||
0xEC7AEC3A, 0xDB851DFA, 0x63094366, 0xC464C3D2, | |||||
0xEF1C1847, 0x3215D908, 0xDD433B37, 0x24C2BA16, | |||||
0x12A14D43, 0x2A65C451, 0x50940002, 0x133AE4DD, | |||||
0x71DFF89E, 0x10314E55, 0x81AC77D6, 0x5F11199B, | |||||
0x043556F1, 0xD7A3C76B, 0x3C11183B, 0x5924A509, | |||||
0xF28FE6ED, 0x97F1FBFA, 0x9EBABF2C, 0x1E153C6E, | |||||
0x86E34570, 0xEAE96FB1, 0x860E5E0A, 0x5A3E2AB3, | |||||
0x771FE71C, 0x4E3D06FA, 0x2965DCB9, 0x99E71D0F, | |||||
0x803E89D6, 0x5266C825, 0x2E4CC978, 0x9C10B36A, | |||||
0xC6150EBA, 0x94E2EA78, 0xA5FC3C53, 0x1E0A2DF4, | |||||
0xF2F74EA7, 0x361D2B3D, 0x1939260F, 0x19C27960, | |||||
0x5223A708, 0xF71312B6, 0xEBADFE6E, 0xEAC31F66, | |||||
0xE3BC4595, 0xA67BC883, 0xB17F37D1, 0x018CFF28, | |||||
0xC332DDEF, 0xBE6C5AA5, 0x65582185, 0x68AB9802, | |||||
0xEECEA50F, 0xDB2F953B, 0x2AEF7DAD, 0x5B6E2F84, | |||||
0x1521B628, 0x29076170, 0xECDD4775, 0x619F1510, | |||||
0x13CCA830, 0xEB61BD96, 0x0334FE1E, 0xAA0363CF, | |||||
0xB5735C90, 0x4C70A239, 0xD59E9E0B, 0xCBAADE14, | |||||
0xEECC86BC, 0x60622CA7, 0x9CAB5CAB, 0xB2F3846E, | |||||
0x648B1EAF, 0x19BDF0CA, 0xA02369B9, 0x655ABB50, | |||||
0x40685A32, 0x3C2AB4B3, 0x319EE9D5, 0xC021B8F7, | |||||
0x9B540B19, 0x875FA099, 0x95F7997E, 0x623D7DA8, | |||||
0xF837889A, 0x97E32D77, 0x11ED935F, 0x16681281, | |||||
0x0E358829, 0xC7E61FD6, 0x96DEDFA1, 0x7858BA99, | |||||
0x57F584A5, 0x1B227263, 0x9B83C3FF, 0x1AC24696, | |||||
0xCDB30AEB, 0x532E3054, 0x8FD948E4, 0x6DBC3128, | |||||
0x58EBF2EF, 0x34C6FFEA, 0xFE28ED61, 0xEE7C3C73, | |||||
0x5D4A14D9, 0xE864B7E3, 0x42105D14, 0x203E13E0, | |||||
0x45EEE2B6, 0xA3AAABEA, 0xDB6C4F15, 0xFACB4FD0, | |||||
0xC742F442, 0xEF6ABBB5, 0x654F3B1D, 0x41CD2105, | |||||
0xD81E799E, 0x86854DC7, 0xE44B476A, 0x3D816250, | |||||
0xCF62A1F2, 0x5B8D2646, 0xFC8883A0, 0xC1C7B6A3, | |||||
0x7F1524C3, 0x69CB7492, 0x47848A0B, 0x5692B285, | |||||
0x095BBF00, 0xAD19489D, 0x1462B174, 0x23820E00, | |||||
0x58428D2A, 0x0C55F5EA, 0x1DADF43E, 0x233F7061, | |||||
0x3372F092, 0x8D937E41, 0xD65FECF1, 0x6C223BDB, | |||||
0x7CDE3759, 0xCBEE7460, 0x4085F2A7, 0xCE77326E, | |||||
0xA6078084, 0x19F8509E, 0xE8EFD855, 0x61D99735, | |||||
0xA969A7AA, 0xC50C06C2, 0x5A04ABFC, 0x800BCADC, | |||||
0x9E447A2E, 0xC3453484, 0xFDD56705, 0x0E1E9EC9, | |||||
0xDB73DBD3, 0x105588CD, 0x675FDA79, 0xE3674340, | |||||
0xC5C43465, 0x713E38D8, 0x3D28F89E, 0xF16DFF20, | |||||
0x153E21E7, 0x8FB03D4A, 0xE6E39F2B, 0xDB83ADF7 | |||||
], | |||||
[ | |||||
0xE93D5A68, 0x948140F7, 0xF64C261C, 0x94692934, | |||||
0x411520F7, 0x7602D4F7, 0xBCF46B2E, 0xD4A20068, | |||||
0xD4082471, 0x3320F46A, 0x43B7D4B7, 0x500061AF, | |||||
0x1E39F62E, 0x97244546, 0x14214F74, 0xBF8B8840, | |||||
0x4D95FC1D, 0x96B591AF, 0x70F4DDD3, 0x66A02F45, | |||||
0xBFBC09EC, 0x03BD9785, 0x7FAC6DD0, 0x31CB8504, | |||||
0x96EB27B3, 0x55FD3941, 0xDA2547E6, 0xABCA0A9A, | |||||
0x28507825, 0x530429F4, 0x0A2C86DA, 0xE9B66DFB, | |||||
0x68DC1462, 0xD7486900, 0x680EC0A4, 0x27A18DEE, | |||||
0x4F3FFEA2, 0xE887AD8C, 0xB58CE006, 0x7AF4D6B6, | |||||
0xAACE1E7C, 0xD3375FEC, 0xCE78A399, 0x406B2A42, | |||||
0x20FE9E35, 0xD9F385B9, 0xEE39D7AB, 0x3B124E8B, | |||||
0x1DC9FAF7, 0x4B6D1856, 0x26A36631, 0xEAE397B2, | |||||
0x3A6EFA74, 0xDD5B4332, 0x6841E7F7, 0xCA7820FB, | |||||
0xFB0AF54E, 0xD8FEB397, 0x454056AC, 0xBA489527, | |||||
0x55533A3A, 0x20838D87, 0xFE6BA9B7, 0xD096954B, | |||||
0x55A867BC, 0xA1159A58, 0xCCA92963, 0x99E1DB33, | |||||
0xA62A4A56, 0x3F3125F9, 0x5EF47E1C, 0x9029317C, | |||||
0xFDF8E802, 0x04272F70, 0x80BB155C, 0x05282CE3, | |||||
0x95C11548, 0xE4C66D22, 0x48C1133F, 0xC70F86DC, | |||||
0x07F9C9EE, 0x41041F0F, 0x404779A4, 0x5D886E17, | |||||
0x325F51EB, 0xD59BC0D1, 0xF2BCC18F, 0x41113564, | |||||
0x257B7834, 0x602A9C60, 0xDFF8E8A3, 0x1F636C1B, | |||||
0x0E12B4C2, 0x02E1329E, 0xAF664FD1, 0xCAD18115, | |||||
0x6B2395E0, 0x333E92E1, 0x3B240B62, 0xEEBEB922, | |||||
0x85B2A20E, 0xE6BA0D99, 0xDE720C8C, 0x2DA2F728, | |||||
0xD0127845, 0x95B794FD, 0x647D0862, 0xE7CCF5F0, | |||||
0x5449A36F, 0x877D48FA, 0xC39DFD27, 0xF33E8D1E, | |||||
0x0A476341, 0x992EFF74, 0x3A6F6EAB, 0xF4F8FD37, | |||||
0xA812DC60, 0xA1EBDDF8, 0x991BE14C, 0xDB6E6B0D, | |||||
0xC67B5510, 0x6D672C37, 0x2765D43B, 0xDCD0E804, | |||||
0xF1290DC7, 0xCC00FFA3, 0xB5390F92, 0x690FED0B, | |||||
0x667B9FFB, 0xCEDB7D9C, 0xA091CF0B, 0xD9155EA3, | |||||
0xBB132F88, 0x515BAD24, 0x7B9479BF, 0x763BD6EB, | |||||
0x37392EB3, 0xCC115979, 0x8026E297, 0xF42E312D, | |||||
0x6842ADA7, 0xC66A2B3B, 0x12754CCC, 0x782EF11C, | |||||
0x6A124237, 0xB79251E7, 0x06A1BBE6, 0x4BFB6350, | |||||
0x1A6B1018, 0x11CAEDFA, 0x3D25BDD8, 0xE2E1C3C9, | |||||
0x44421659, 0x0A121386, 0xD90CEC6E, 0xD5ABEA2A, | |||||
0x64AF674E, 0xDA86A85F, 0xBEBFE988, 0x64E4C3FE, | |||||
0x9DBC8057, 0xF0F7C086, 0x60787BF8, 0x6003604D, | |||||
0xD1FD8346, 0xF6381FB0, 0x7745AE04, 0xD736FCCC, | |||||
0x83426B33, 0xF01EAB71, 0xB0804187, 0x3C005E5F, | |||||
0x77A057BE, 0xBDE8AE24, 0x55464299, 0xBF582E61, | |||||
0x4E58F48F, 0xF2DDFDA2, 0xF474EF38, 0x8789BDC2, | |||||
0x5366F9C3, 0xC8B38E74, 0xB475F255, 0x46FCD9B9, | |||||
0x7AEB2661, 0x8B1DDF84, 0x846A0E79, 0x915F95E2, | |||||
0x466E598E, 0x20B45770, 0x8CD55591, 0xC902DE4C, | |||||
0xB90BACE1, 0xBB8205D0, 0x11A86248, 0x7574A99E, | |||||
0xB77F19B6, 0xE0A9DC09, 0x662D09A1, 0xC4324633, | |||||
0xE85A1F02, 0x09F0BE8C, 0x4A99A025, 0x1D6EFE10, | |||||
0x1AB93D1D, 0x0BA5A4DF, 0xA186F20F, 0x2868F169, | |||||
0xDCB7DA83, 0x573906FE, 0xA1E2CE9B, 0x4FCD7F52, | |||||
0x50115E01, 0xA70683FA, 0xA002B5C4, 0x0DE6D027, | |||||
0x9AF88C27, 0x773F8641, 0xC3604C06, 0x61A806B5, | |||||
0xF0177A28, 0xC0F586E0, 0x006058AA, 0x30DC7D62, | |||||
0x11E69ED7, 0x2338EA63, 0x53C2DD94, 0xC2C21634, | |||||
0xBBCBEE56, 0x90BCB6DE, 0xEBFC7DA1, 0xCE591D76, | |||||
0x6F05E409, 0x4B7C0188, 0x39720A3D, 0x7C927C24, | |||||
0x86E3725F, 0x724D9DB9, 0x1AC15BB4, 0xD39EB8FC, | |||||
0xED545578, 0x08FCA5B5, 0xD83D7CD3, 0x4DAD0FC4, | |||||
0x1E50EF5E, 0xB161E6F8, 0xA28514D9, 0x6C51133C, | |||||
0x6FD5C7E7, 0x56E14EC4, 0x362ABFCE, 0xDDC6C837, | |||||
0xD79A3234, 0x92638212, 0x670EFA8E, 0x406000E0 | |||||
], | |||||
[ | |||||
0x3A39CE37, 0xD3FAF5CF, 0xABC27737, 0x5AC52D1B, | |||||
0x5CB0679E, 0x4FA33742, 0xD3822740, 0x99BC9BBE, | |||||
0xD5118E9D, 0xBF0F7315, 0xD62D1C7E, 0xC700C47B, | |||||
0xB78C1B6B, 0x21A19045, 0xB26EB1BE, 0x6A366EB4, | |||||
0x5748AB2F, 0xBC946E79, 0xC6A376D2, 0x6549C2C8, | |||||
0x530FF8EE, 0x468DDE7D, 0xD5730A1D, 0x4CD04DC6, | |||||
0x2939BBDB, 0xA9BA4650, 0xAC9526E8, 0xBE5EE304, | |||||
0xA1FAD5F0, 0x6A2D519A, 0x63EF8CE2, 0x9A86EE22, | |||||
0xC089C2B8, 0x43242EF6, 0xA51E03AA, 0x9CF2D0A4, | |||||
0x83C061BA, 0x9BE96A4D, 0x8FE51550, 0xBA645BD6, | |||||
0x2826A2F9, 0xA73A3AE1, 0x4BA99586, 0xEF5562E9, | |||||
0xC72FEFD3, 0xF752F7DA, 0x3F046F69, 0x77FA0A59, | |||||
0x80E4A915, 0x87B08601, 0x9B09E6AD, 0x3B3EE593, | |||||
0xE990FD5A, 0x9E34D797, 0x2CF0B7D9, 0x022B8B51, | |||||
0x96D5AC3A, 0x017DA67D, 0xD1CF3ED6, 0x7C7D2D28, | |||||
0x1F9F25CF, 0xADF2B89B, 0x5AD6B472, 0x5A88F54C, | |||||
0xE029AC71, 0xE019A5E6, 0x47B0ACFD, 0xED93FA9B, | |||||
0xE8D3C48D, 0x283B57CC, 0xF8D56629, 0x79132E28, | |||||
0x785F0191, 0xED756055, 0xF7960E44, 0xE3D35E8C, | |||||
0x15056DD4, 0x88F46DBA, 0x03A16125, 0x0564F0BD, | |||||
0xC3EB9E15, 0x3C9057A2, 0x97271AEC, 0xA93A072A, | |||||
0x1B3F6D9B, 0x1E6321F5, 0xF59C66FB, 0x26DCF319, | |||||
0x7533D928, 0xB155FDF5, 0x03563482, 0x8ABA3CBB, | |||||
0x28517711, 0xC20AD9F8, 0xABCC5167, 0xCCAD925F, | |||||
0x4DE81751, 0x3830DC8E, 0x379D5862, 0x9320F991, | |||||
0xEA7A90C2, 0xFB3E7BCE, 0x5121CE64, 0x774FBE32, | |||||
0xA8B6E37E, 0xC3293D46, 0x48DE5369, 0x6413E680, | |||||
0xA2AE0810, 0xDD6DB224, 0x69852DFD, 0x09072166, | |||||
0xB39A460A, 0x6445C0DD, 0x586CDECF, 0x1C20C8AE, | |||||
0x5BBEF7DD, 0x1B588D40, 0xCCD2017F, 0x6BB4E3BB, | |||||
0xDDA26A7E, 0x3A59FF45, 0x3E350A44, 0xBCB4CDD5, | |||||
0x72EACEA8, 0xFA6484BB, 0x8D6612AE, 0xBF3C6F47, | |||||
0xD29BE463, 0x542F5D9E, 0xAEC2771B, 0xF64E6370, | |||||
0x740E0D8D, 0xE75B1357, 0xF8721671, 0xAF537D5D, | |||||
0x4040CB08, 0x4EB4E2CC, 0x34D2466A, 0x0115AF84, | |||||
0xE1B00428, 0x95983A1D, 0x06B89FB4, 0xCE6EA048, | |||||
0x6F3F3B82, 0x3520AB82, 0x011A1D4B, 0x277227F8, | |||||
0x611560B1, 0xE7933FDC, 0xBB3A792B, 0x344525BD, | |||||
0xA08839E1, 0x51CE794B, 0x2F32C9B7, 0xA01FBAC9, | |||||
0xE01CC87E, 0xBCC7D1F6, 0xCF0111C3, 0xA1E8AAC7, | |||||
0x1A908749, 0xD44FBD9A, 0xD0DADECB, 0xD50ADA38, | |||||
0x0339C32A, 0xC6913667, 0x8DF9317C, 0xE0B12B4F, | |||||
0xF79E59B7, 0x43F5BB3A, 0xF2D519FF, 0x27D9459C, | |||||
0xBF97222C, 0x15E6FC2A, 0x0F91FC71, 0x9B941525, | |||||
0xFAE59361, 0xCEB69CEB, 0xC2A86459, 0x12BAA8D1, | |||||
0xB6C1075E, 0xE3056A0C, 0x10D25065, 0xCB03A442, | |||||
0xE0EC6E0E, 0x1698DB3B, 0x4C98A0BE, 0x3278E964, | |||||
0x9F1F9532, 0xE0D392DF, 0xD3A0342B, 0x8971F21E, | |||||
0x1B0A7441, 0x4BA3348C, 0xC5BE7120, 0xC37632D8, | |||||
0xDF359F8D, 0x9B992F2E, 0xE60B6F47, 0x0FE3F11D, | |||||
0xE54CDA54, 0x1EDAD891, 0xCE6279CF, 0xCD3E7E6F, | |||||
0x1618B166, 0xFD2C1D05, 0x848FD2C5, 0xF6FB2299, | |||||
0xF523F357, 0xA6327623, 0x93A83531, 0x56CCCD02, | |||||
0xACF08162, 0x5A75EBB5, 0x6E163697, 0x88D273CC, | |||||
0xDE966292, 0x81B949D0, 0x4C50901B, 0x71C65614, | |||||
0xE6C6C7BD, 0x327A140A, 0x45E1D006, 0xC3F27B9A, | |||||
0xC9AA53FD, 0x62A80F00, 0xBB25BFE2, 0x35BDD2F6, | |||||
0x71126905, 0xB2040222, 0xB6CBCF7C, 0xCD769C2B, | |||||
0x53113EC0, 0x1640E3D3, 0x38ABBD60, 0x2547ADF0, | |||||
0xBA38209C, 0xF746CE76, 0x77AFA1C5, 0x20756060, | |||||
0x85CBFE4E, 0x8AE88DD8, 0x7AAAF9B0, 0x4CF9AA7E, | |||||
0x1948C25C, 0x02FB8A8C, 0x01C36AE4, 0xD6EBE1F9, | |||||
0x90D4F869, 0xA65CDEA0, 0x3F09252D, 0xC208E69F, | |||||
0xB74E6132, 0xCE77E25B, 0x578FDFE3, 0x3AC372E6 | |||||
] | |||||
] | |||||
# Cycle through the p-boxes and round-robin XOR the | |||||
# key with the p-boxes | |||||
key_len = len (key) | |||||
index = 0 | |||||
for i in range (len (self.p_boxes)): | |||||
val = (ord (key[index % key_len]) << 24) + \ | |||||
(ord (key[(index + 1) % key_len]) << 16) + \ | |||||
(ord (key[(index + 2) % key_len]) << 8) + \ | |||||
ord (key[(index + 3) % key_len]) | |||||
self.p_boxes[i] = self.p_boxes[i] ^ val | |||||
index = index + 4 | |||||
# For the chaining process | |||||
l, r = 0, 0 | |||||
# Begin chain replacing the p-boxes | |||||
for i in range (0, len (self.p_boxes), 2): | |||||
l, r = self.cipher (l, r, self.ENCRYPT) | |||||
self.p_boxes[i] = l | |||||
self.p_boxes[i + 1] = r | |||||
# Chain replace the s-boxes | |||||
for i in range (len (self.s_boxes)): | |||||
for j in range (0, len (self.s_boxes[i]), 2): | |||||
l, r = self.cipher (l, r, self.ENCRYPT) | |||||
self.s_boxes[i][j] = l | |||||
self.s_boxes[i][j + 1] = r | |||||
def cipher(self, xl, xr, direction): | |||||
if direction == self.ENCRYPT: | |||||
for i in range (16): | |||||
xl = xl ^ self.p_boxes[i] | |||||
xr = self.__round_func (xl) ^ xr | |||||
xl, xr = xr, xl | |||||
xl, xr = xr, xl | |||||
xr = xr ^ self.p_boxes[16] | |||||
xl = xl ^ self.p_boxes[17] | |||||
else: | |||||
for i in range (17, 1, -1): | |||||
xl = xl ^ self.p_boxes[i] | |||||
xr = self.__round_func (xl) ^ xr | |||||
xl, xr = xr, xl | |||||
xl, xr = xr, xl | |||||
xr = xr ^ self.p_boxes[1] | |||||
xl = xl ^ self.p_boxes[0] | |||||
return xl, xr | |||||
def __round_func(self, xl): | |||||
a = (xl & 0xFF000000) >> 24 | |||||
b = (xl & 0x00FF0000) >> 16 | |||||
c = (xl & 0x0000FF00) >> 8 | |||||
d = xl & 0x000000FF | |||||
# Perform all ops as longs then and out the last 32-bits to | |||||
# obtain the integer | |||||
f = (long (self.s_boxes[0][a]) + long (self.s_boxes[1][b])) % self.modulus | |||||
f = f ^ long (self.s_boxes[2][c]) | |||||
f = f + long (self.s_boxes[3][d]) | |||||
f = (f % self.modulus) & 0xFFFFFFFF | |||||
return f | |||||
def encrypt(self, data): | |||||
if not len(data) == 8: | |||||
e = "blocks must be 8 bytes long, but tried to encrypt one {0} bytes long" | |||||
raise BlockSizeError(e.format(len(data))) | |||||
# Use big endianess since that's what everyone else uses | |||||
xl = ord (data[3]) | (ord (data[2]) << 8) | (ord (data[1]) << 16) | (ord (data[0]) << 24) | |||||
xr = ord (data[7]) | (ord (data[6]) << 8) | (ord (data[5]) << 16) | (ord (data[4]) << 24) | |||||
cl, cr = self.cipher (xl, xr, self.ENCRYPT) | |||||
chars = ''.join ([ | |||||
chr ((cl >> 24) & 0xFF), chr ((cl >> 16) & 0xFF), chr ((cl >> 8) & 0xFF), chr (cl & 0xFF), | |||||
chr ((cr >> 24) & 0xFF), chr ((cr >> 16) & 0xFF), chr ((cr >> 8) & 0xFF), chr (cr & 0xFF) | |||||
]) | |||||
return chars | |||||
def decrypt(self, data): | |||||
if not len(data) == 8: | |||||
e = "blocks must be 8 bytes long, but tried to decrypt one {0} bytes long" | |||||
raise BlockSizeError(e.format(len(data))) | |||||
# Use big endianess since that's what everyone else uses | |||||
cl = ord (data[3]) | (ord (data[2]) << 8) | (ord (data[1]) << 16) | (ord (data[0]) << 24) | |||||
cr = ord (data[7]) | (ord (data[6]) << 8) | (ord (data[5]) << 16) | (ord (data[4]) << 24) | |||||
xl, xr = self.cipher (cl, cr, self.DECRYPT) | |||||
chars = ''.join ([ | |||||
chr ((xl >> 24) & 0xFF), chr ((xl >> 16) & 0xFF), chr ((xl >> 8) & 0xFF), chr (xl & 0xFF), | |||||
chr ((xr >> 24) & 0xFF), chr ((xr >> 16) & 0xFF), chr ((xr >> 8) & 0xFF), chr (xr & 0xFF) | |||||
]) | |||||
return chars | |||||
def blocksize(self): | |||||
return 8 | |||||
def key_length(self): | |||||
return 56 | |||||
def key_bits(self): | |||||
return 56 * 8 | |||||
def verify_key(self, key): | |||||
"""Make sure our key is not too short or too long. | |||||
If there's a problem, raise KeyTooShortError() or KeyTooLongError(). | |||||
""" | |||||
if not key: | |||||
raise KeyLengthError("no key given") | |||||
if len(key) < 8: | |||||
e = "key is {0} bytes long, but it must be at least 8" | |||||
raise KeyLengthError(e.format(len(key))) | |||||
if len(key) > 56: | |||||
e = "key is {0} bytes long, but it must be less than 56" | |||||
raise KeyLengthError(e.format(len(key))) | |||||
def encrypt(key, plaintext): | |||||
"""Encrypt any length of plaintext using a given key that must be between | |||||
8 and 56 bytes in length. This is a convienence function that can handle | |||||
plaintext that is not a single block in length. It will auto-pad blocks | |||||
that are less than 8 bytes with spaces that are automatically removed by | |||||
decrypt(). Actual spaces in plaintext are preserved.""" | |||||
cypher = Blowfish(key) | |||||
msg = "TRUE{0}|{1}".format(len(plaintext), plaintext) | |||||
while len(msg) % 8 > 0: | |||||
msg += " " # pad message to form complete 8-byte blocks | |||||
blocks = [msg[f:f+8] for f in range(0, len(msg), 8)] | |||||
cyphertext = ''.join(map(cypher.encrypt, blocks)) | |||||
return cyphertext.encode('hex') | |||||
def decrypt(key, cyphertext): | |||||
"""Decrypt the result of encrypt() using the original key, or raise | |||||
DecryptionError().""" | |||||
cypher = Blowfish(key) | |||||
try: | |||||
cyphertext = cyphertext.decode("hex") | |||||
except (TypeError, AttributeError) as error: | |||||
e = error.message | |||||
raise DecryptionError("cyphertext could not be decoded: " + e.lower()) | |||||
if len(cyphertext) % 8 > 0: | |||||
e = "cyphertext cannot be broken into 8-byte blocks evenly" | |||||
raise DecryptionError(e) | |||||
blocks = [cyphertext[f:f+8] for f in range(0, len(cyphertext), 8)] | |||||
msg = ''.join(map(cypher.decrypt, blocks)) | |||||
# Sanity check to ensure valid decryption: | |||||
if not msg.startswith("TRUE"): | |||||
e = "the given key is incorrect, or part of the cyphertext is malformed" | |||||
raise DecryptionError(e) | |||||
size, msg = msg[4:].split("|", 1) | |||||
while len(msg) > int(size): | |||||
msg = msg[:-1] # Remove the padding that we applied earlier | |||||
return msg | |||||
if __name__ == '__main__': | |||||
action = raw_input("Would you like to [e]ncrypt or [d]ecrypt? ") | |||||
if action.lower().startswith("e"): | |||||
key = raw_input("Enter a key: ") | |||||
plaintext = raw_input("Enter a message to encrypt: ") | |||||
print "\n" + encrypt(key, plaintext) | |||||
elif action.lower().startswith("d"): | |||||
key = raw_input("Enter a key: ") | |||||
cyphertext = raw_input("Enter a message to decrypt: ") | |||||
print "\n" + decrypt(key, cyphertext) | |||||
else: | |||||
print "Unknown action: '{0}'".format(action) |
@@ -0,0 +1,212 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
import logging | |||||
from threading import Lock, Thread, enumerate as enumerate_threads | |||||
from time import sleep, time | |||||
from earwigbot import __version__ | |||||
from earwigbot.config import BotConfig | |||||
from earwigbot.irc import Frontend, Watcher | |||||
from earwigbot.managers import CommandManager, TaskManager | |||||
from earwigbot.wiki import SitesDB | |||||
__all__ = ["Bot"] | |||||
class Bot(object): | |||||
""" | |||||
**EarwigBot: Main Bot Class** | |||||
The :py:class:`Bot` class is the core of EarwigBot, essentially responsible | |||||
for starting the various bot components and making sure they are all happy. | |||||
EarwigBot has three components that can run independently of each other: an | |||||
IRC front-end, an IRC watcher, and a wiki scheduler. | |||||
- The IRC front-end runs on a normal IRC server and expects users to | |||||
interact with it/give it commands. | |||||
- The IRC watcher runs on a wiki recent-changes server and listens for | |||||
edits. Users cannot interact with this part of the bot. | |||||
- The wiki scheduler runs wiki-editing bot tasks in separate threads at | |||||
user-defined times through a cron-like interface. | |||||
The :py:class:`Bot` object is accessible from within commands and tasks as | |||||
:py:attr:`self.bot`. This is the primary way to access data from other | |||||
components of the bot. For example, our | |||||
:py:class:`~earwigbot.config.BotConfig` object is accessable from | |||||
:py:attr:`bot.config`, tasks can be started with | |||||
:py:meth:`bot.tasks.start() <earwigbot.managers.TaskManager.start>`, and | |||||
sites can be loaded from the wiki toolset with | |||||
:py:meth:`bot.wiki.get_site() <earwigbot.wiki.sitesdb.SitesDB.get_site>`. | |||||
""" | |||||
def __init__(self, root_dir, level=logging.INFO): | |||||
self.config = BotConfig(root_dir, level) | |||||
self.logger = logging.getLogger("earwigbot") | |||||
self.commands = CommandManager(self) | |||||
self.tasks = TaskManager(self) | |||||
self.wiki = SitesDB(self) | |||||
self.frontend = None | |||||
self.watcher = None | |||||
self.component_lock = Lock() | |||||
self._keep_looping = True | |||||
self.config.load() | |||||
self.commands.load() | |||||
self.tasks.load() | |||||
def __repr__(self): | |||||
"""Return the canonical string representation of the Bot.""" | |||||
return "Bot(config={0!r})".format(self.config) | |||||
def __str__(self): | |||||
"""Return a nice string representation of the Bot.""" | |||||
return "<Bot at {0}>".format(self.config.root_dir) | |||||
def _dispatch_irc_component(self, name, klass): | |||||
"""Create a new IRC component, record it internally, and start it.""" | |||||
component = klass(self) | |||||
setattr(self, name, component) | |||||
Thread(name="irc_" + name, target=component.loop).start() | |||||
def _start_irc_components(self): | |||||
"""Start the IRC frontend/watcher in separate threads if enabled.""" | |||||
if self.config.components.get("irc_frontend"): | |||||
self.logger.info("Starting IRC frontend") | |||||
self._dispatch_irc_component("frontend", Frontend) | |||||
if self.config.components.get("irc_watcher"): | |||||
self.logger.info("Starting IRC watcher") | |||||
self._dispatch_irc_component("watcher", Watcher) | |||||
def _start_wiki_scheduler(self): | |||||
"""Start the wiki scheduler in a separate thread if enabled.""" | |||||
def wiki_scheduler(): | |||||
while self._keep_looping: | |||||
time_start = time() | |||||
self.tasks.schedule() | |||||
time_end = time() | |||||
time_diff = time_start - time_end | |||||
if time_diff < 60: # Sleep until the next minute | |||||
sleep(60 - time_diff) | |||||
if self.config.components.get("wiki_scheduler"): | |||||
self.logger.info("Starting wiki scheduler") | |||||
thread = Thread(name="wiki_scheduler", target=wiki_scheduler) | |||||
thread.daemon = True # Stop if other threads stop | |||||
thread.start() | |||||
def _keep_irc_component_alive(self, name, klass): | |||||
"""Ensure that IRC components stay connected, else restart them.""" | |||||
component = getattr(self, name) | |||||
if component: | |||||
component.keep_alive() | |||||
if component.is_stopped(): | |||||
log = "IRC {0} has stopped; restarting".format(name) | |||||
self.logger.warn(log) | |||||
self._dispatch_irc_component(name, klass) | |||||
def _stop_irc_components(self, msg): | |||||
"""Request the IRC frontend and watcher to stop if enabled.""" | |||||
if self.frontend: | |||||
self.frontend.stop(msg) | |||||
if self.watcher: | |||||
self.watcher.stop(msg) | |||||
def _stop_daemon_threads(self): | |||||
"""Notify the user of which threads are going to be killed. | |||||
Unfortunately, there is no method right now of stopping command and | |||||
task threads safely. This is because there is no way to tell them to | |||||
stop like the IRC components can be told; furthermore, they are run as | |||||
daemons, and daemon threads automatically stop without calling any | |||||
__exit__ or try/finally code when all non-daemon threads stop. They | |||||
were originally implemented as regular non-daemon threads, but this | |||||
meant there was no way to completely stop the bot if tasks were | |||||
running, because all other threads would exit and threading would | |||||
absorb KeyboardInterrupts. | |||||
The advantage of this is that stopping the bot is truly guarenteed to | |||||
*stop* the bot, while the disadvantage is that the threads are given no | |||||
advance warning of their forced shutdown. | |||||
""" | |||||
tasks = [] | |||||
non_tasks = self.config.components.keys() + ["MainThread", "reminder"] | |||||
for thread in enumerate_threads(): | |||||
if thread.name not in non_tasks and thread.is_alive(): | |||||
tasks.append(thread.name) | |||||
if tasks: | |||||
log = "The following commands or tasks will be killed: {0}" | |||||
self.logger.warn(log.format(" ".join(tasks))) | |||||
def run(self): | |||||
"""Main entry point into running the bot. | |||||
Starts all config-enabled components and then enters an idle loop, | |||||
ensuring that all components remain online and restarting components | |||||
that get disconnected from their servers. | |||||
""" | |||||
self.logger.info("Starting bot (EarwigBot {0})".format(__version__)) | |||||
self._start_irc_components() | |||||
self._start_wiki_scheduler() | |||||
while self._keep_looping: | |||||
with self.component_lock: | |||||
self._keep_irc_component_alive("frontend", Frontend) | |||||
self._keep_irc_component_alive("watcher", Watcher) | |||||
sleep(2) | |||||
def restart(self, msg=None): | |||||
"""Reload config, commands, tasks, and safely restart IRC components. | |||||
This is thread-safe, and it will gracefully stop IRC components before | |||||
reloading anything. Note that you can safely reload commands or tasks | |||||
without restarting the bot with :py:meth:`bot.commands.load() | |||||
<earwigbot.managers._ResourceManager.load>` or | |||||
:py:meth:`bot.tasks.load() <earwigbot.managers._ResourceManager.load>`. | |||||
These should not interfere with running components or tasks. | |||||
If given, *msg* will be used as our quit message. | |||||
""" | |||||
if msg: | |||||
self.logger.info('Restarting bot ("{0}")'.format(msg)) | |||||
else: | |||||
self.logger.info("Restarting bot") | |||||
with self.component_lock: | |||||
self._stop_irc_components(msg) | |||||
self.config.load() | |||||
self.commands.load() | |||||
self.tasks.load() | |||||
self._start_irc_components() | |||||
def stop(self, msg=None): | |||||
"""Gracefully stop all bot components. | |||||
If given, *msg* will be used as our quit message. | |||||
""" | |||||
if msg: | |||||
self.logger.info('Stopping bot ("{0}")'.format(msg)) | |||||
else: | |||||
self.logger.info("Stopping bot") | |||||
with self.component_lock: | |||||
self._stop_irc_components(msg) | |||||
self._keep_looping = False | |||||
self._stop_daemon_threads() |
@@ -1,75 +0,0 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
import logging | |||||
__all__ = ["BaseCommand"] | |||||
class BaseCommand(object): | |||||
"""A base class for commands on IRC. | |||||
This docstring is reported to the user when they use !help <command>. | |||||
""" | |||||
# This is the command's name, as reported to the user when they use !help: | |||||
name = None | |||||
# Hooks are "msg", "msg_private", "msg_public", and "join". "msg" is the | |||||
# default behavior; if you wish to override that, change the value in your | |||||
# command subclass: | |||||
hooks = ["msg"] | |||||
def __init__(self, connection): | |||||
"""Constructor for new commands. | |||||
This is called once when the command is loaded (from | |||||
commands._load_command()). `connection` is a Connection object, | |||||
allowing us to do self.connection.say(), self.connection.send(), etc, | |||||
from within a method. | |||||
""" | |||||
self.connection = connection | |||||
logger_name = ".".join(("earwigbot", "commands", self.name)) | |||||
self.logger = logging.getLogger(logger_name) | |||||
self.logger.setLevel(logging.DEBUG) | |||||
def check(self, data): | |||||
"""Returns whether this command should be called in response to 'data'. | |||||
Given a Data() instance, return True if we should respond to this | |||||
activity, or False if we should ignore it or it doesn't apply to us. | |||||
Most commands return True if data.command == self.name, otherwise they | |||||
return False. This is the default behavior of check(); you need only | |||||
override it if you wish to change that. | |||||
""" | |||||
if data.is_command and data.command == self.name: | |||||
return True | |||||
return False | |||||
def process(self, data): | |||||
"""Main entry point for doing a command. | |||||
Handle an activity (usually a message) on IRC. At this point, thanks | |||||
to self.check() which is called automatically by the command handler, | |||||
we know this is something we should respond to, so (usually) something | |||||
like 'if data.command != "command_name": return' is unnecessary. | |||||
""" | |||||
pass |
@@ -1,117 +0,0 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
import logging | |||||
from earwigbot.config import config | |||||
from earwigbot import wiki | |||||
__all__ = ["BaseTask"] | |||||
class BaseTask(object): | |||||
"""A base class for bot tasks that edit Wikipedia.""" | |||||
name = None | |||||
number = 0 | |||||
def __init__(self): | |||||
"""Constructor for new tasks. | |||||
This is called once immediately after the task class is loaded by | |||||
the task manager (in tasks._load_task()). | |||||
""" | |||||
pass | |||||
def _setup_logger(self): | |||||
"""Set up a basic module-level logger.""" | |||||
logger_name = ".".join(("earwigbot", "tasks", self.name)) | |||||
self.logger = logging.getLogger(logger_name) | |||||
self.logger.setLevel(logging.DEBUG) | |||||
def run(self, **kwargs): | |||||
"""Main entry point to run a given task. | |||||
This is called directly by tasks.start() and is the main way to make a | |||||
task do stuff. kwargs will be any keyword arguments passed to start() | |||||
which are entirely optional. | |||||
The same task instance is preserved between runs, so you can | |||||
theoretically store data in self (e.g. | |||||
start('mytask', action='store', data='foo')) and then use it later | |||||
(e.g. start('mytask', action='save')). | |||||
""" | |||||
pass | |||||
def make_summary(self, comment): | |||||
"""Makes an edit summary by filling in variables in a config value. | |||||
config.wiki["summary"] is used, where $2 is replaced by the main | |||||
summary body, given as a method arg, and $1 is replaced by the task | |||||
number. | |||||
If the config value is not found, we just return the arg as-is. | |||||
""" | |||||
try: | |||||
summary = config.wiki["summary"] | |||||
except KeyError: | |||||
return comment | |||||
return summary.replace("$1", str(self.number)).replace("$2", comment) | |||||
def shutoff_enabled(self, site=None): | |||||
"""Returns whether on-wiki shutoff is enabled for this task. | |||||
We check a certain page for certain content. This is determined by | |||||
our config file: config.wiki["shutoff"]["page"] is used as the title, | |||||
with $1 replaced by our username and $2 replaced by the task number, | |||||
and config.wiki["shutoff"]["disabled"] is used as the content. | |||||
If the page has that content or the page does not exist, then shutoff | |||||
is "disabled", meaning the bot is supposed to run normally, and we | |||||
return False. If the page's content is something other than what we | |||||
expect, shutoff is enabled, and we return True. | |||||
If a site is not provided, we'll try to use self.site if it's set. | |||||
Otherwise, we'll use our default site. | |||||
""" | |||||
if not site: | |||||
try: | |||||
site = self.site | |||||
except AttributeError: | |||||
site = wiki.get_site() | |||||
try: | |||||
cfg = config.wiki["shutoff"] | |||||
except KeyError: | |||||
return False | |||||
title = cfg.get("page", "User:$1/Shutoff/Task $2") | |||||
username = site.get_user().name() | |||||
title = title.replace("$1", username).replace("$2", str(self.number)) | |||||
page = site.get_page(title) | |||||
try: | |||||
content = page.get() | |||||
except wiki.PageNotFoundError: | |||||
return False | |||||
if content == cfg.get("disabled", "run"): | |||||
return False | |||||
self.logger.warn("Emergency task shutoff has been enabled!") | |||||
return True |
@@ -1,115 +0,0 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
import socket | |||||
import threading | |||||
__all__ = ["BrokenSocketException", "Connection"] | |||||
class BrokenSocketException(Exception): | |||||
"""A socket has broken, because it is not sending data. Raised by | |||||
Connection.get().""" | |||||
pass | |||||
class Connection(object): | |||||
"""A class to interface with IRC.""" | |||||
def __init__(self, host=None, port=None, nick=None, ident=None, | |||||
realname=None, logger=None): | |||||
self.host = host | |||||
self.port = port | |||||
self.nick = nick | |||||
self.ident = ident | |||||
self.realname = realname | |||||
self.logger = logger | |||||
# A lock to prevent us from sending two messages at once: | |||||
self.lock = threading.Lock() | |||||
def connect(self): | |||||
"""Connect to our IRC server.""" | |||||
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) | |||||
try: | |||||
self.sock.connect((self.host, self.port)) | |||||
except socket.error: | |||||
self.logger.critical("Couldn't connect to IRC server", exc_info=1) | |||||
exit(1) | |||||
self.send("NICK %s" % self.nick) | |||||
self.send("USER %s %s * :%s" % (self.ident, self.host, self.realname)) | |||||
def close(self): | |||||
"""Close our connection with the IRC server.""" | |||||
try: | |||||
self.sock.shutdown(socket.SHUT_RDWR) # shut down connection first | |||||
except socket.error: | |||||
pass # ignore if the socket is already down | |||||
self.sock.close() | |||||
def get(self, size=4096): | |||||
"""Receive (i.e. get) data from the server.""" | |||||
data = self.sock.recv(4096) | |||||
if not data: | |||||
# Socket isn't giving us any data, so it is dead or broken: | |||||
raise BrokenSocketException() | |||||
return data | |||||
def send(self, msg): | |||||
"""Send data to the server.""" | |||||
# Ensure that we only send one message at a time with a blocking lock: | |||||
with self.lock: | |||||
self.sock.sendall(msg + "\r\n") | |||||
self.logger.debug(msg) | |||||
def say(self, target, msg): | |||||
"""Send a private message to a target on the server.""" | |||||
message = "".join(("PRIVMSG ", target, " :", msg)) | |||||
self.send(message) | |||||
def reply(self, data, msg): | |||||
"""Send a private message as a reply to a user on the server.""" | |||||
message = "".join((chr(2), data.nick, chr(0x0f), ": ", msg)) | |||||
self.say(data.chan, message) | |||||
def action(self, target, msg): | |||||
"""Send a private message to a target on the server as an action.""" | |||||
message = "".join((chr(1), "ACTION ", msg, chr(1))) | |||||
self.say(target, message) | |||||
def notice(self, target, msg): | |||||
"""Send a notice to a target on the server.""" | |||||
message = "".join(("NOTICE ", target, " :", msg)) | |||||
self.send(message) | |||||
def join(self, chan): | |||||
"""Join a channel on the server.""" | |||||
message = " ".join(("JOIN", chan)) | |||||
self.send(message) | |||||
def part(self, chan): | |||||
"""Part from a channel on the server.""" | |||||
message = " ".join(("PART", chan)) | |||||
self.send(message) | |||||
def mode(self, chan, level, msg): | |||||
"""Send a mode message to the server.""" | |||||
message = " ".join(("MODE", chan, level, msg)) | |||||
self.send(message) |
@@ -1,79 +0,0 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
import re | |||||
__all__ = ["KwargParseException", "Data"] | |||||
class KwargParseException(Exception): | |||||
"""Couldn't parse a certain keyword argument in self.args, probably because | |||||
it was given incorrectly: e.g., no value (abc), just a value (=xyz), just | |||||
an equal sign (=), instead of the correct (abc=xyz).""" | |||||
pass | |||||
class Data(object): | |||||
"""Store data from an individual line received on IRC.""" | |||||
def __init__(self, line): | |||||
self.line = line | |||||
self.chan = self.nick = self.ident = self.host = self.msg = "" | |||||
def parse_args(self): | |||||
"""Parse command args from self.msg into self.command and self.args.""" | |||||
args = self.msg.strip().split() | |||||
while "" in args: | |||||
args.remove("") | |||||
# Isolate command arguments: | |||||
self.args = args[1:] | |||||
self.is_command = False # is this message a command? | |||||
try: | |||||
self.command = args[0] | |||||
except IndexError: | |||||
self.command = None | |||||
try: | |||||
if self.command.startswith('!') or self.command.startswith('.'): | |||||
self.is_command = True | |||||
self.command = self.command[1:] # Strip the '!' or '.' | |||||
self.command = self.command.lower() | |||||
except AttributeError: | |||||
pass | |||||
def parse_kwargs(self): | |||||
"""Parse keyword arguments embedded in self.args. | |||||
Parse a command given as "!command key1=value1 key2=value2..." into a | |||||
dict, self.kwargs, like {'key1': 'value2', 'key2': 'value2'...}. | |||||
""" | |||||
self.kwargs = {} | |||||
for arg in self.args[2:]: | |||||
try: | |||||
key, value = re.findall("^(.*?)\=(.*?)$", arg)[0] | |||||
except IndexError: | |||||
raise KwargParseException(arg) | |||||
if key and value: | |||||
self.kwargs[key] = value | |||||
else: | |||||
raise KwargParseException(arg) |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -20,92 +20,103 @@ | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||||
# SOFTWARE. | # SOFTWARE. | ||||
""" | |||||
EarwigBot's IRC Command Manager | |||||
This package provides the IRC "commands" used by the bot's front-end component. | |||||
In __init__, you can find some functions used to load and run these commands. | |||||
""" | |||||
import logging | |||||
import os | |||||
import sys | |||||
from earwigbot.classes import BaseCommand | |||||
from earwigbot.config import config | |||||
__all__ = ["load", "get_all", "check"] | |||||
__all__ = ["Command"] | |||||
# Base directory when searching for commands: | |||||
base_dir = os.path.dirname(os.path.abspath(__file__)) | |||||
# Store commands in a dict, where the key is the command's name and the value | |||||
# is an instance of the command's class: | |||||
_commands = {} | |||||
# Logger for this module: | |||||
logger = logging.getLogger("earwigbot.tasks") | |||||
def _load_command(connection, filename): | |||||
"""Try to load a specific command from a module, identified by file name. | |||||
Given a Connection object and a filename, we'll first try to import it, | |||||
and if that works, make an instance of the 'Command' class inside (assuming | |||||
it is an instance of BaseCommand), add it to _commands, and report the | |||||
addition to the user. Any problems along the way will either be ignored or | |||||
reported. | |||||
class Command(object): | |||||
""" | """ | ||||
global _commands | |||||
# Strip .py from the end of the filename and join with our package name: | |||||
name = ".".join(("commands", filename[:-3])) | |||||
try: | |||||
__import__(name) | |||||
except: | |||||
logger.exception("Couldn't load file {0}".format(filename)) | |||||
return | |||||
command = sys.modules[name].Command(connection) | |||||
if not isinstance(command, BaseCommand): | |||||
return | |||||
**EarwigBot: Base IRC Command** | |||||
_commands[command.name] = command | |||||
logger.debug("Added command {0}".format(command.name)) | |||||
This package provides built-in IRC "commands" used by the bot's front-end | |||||
component. Additional commands can be installed as plugins in the bot's | |||||
working directory. | |||||
def load(connection): | |||||
"""Load all valid commands into the _commands global variable. | |||||
This class (import with ``from earwigbot.commands import Command``), can be | |||||
subclassed to create custom IRC commands. | |||||
`connection` is a Connection object that is given to each command's | |||||
constructor. | |||||
This docstring is reported to the user when they type ``"!help | |||||
<command>"``. | |||||
""" | """ | ||||
files = os.listdir(base_dir) | |||||
files.sort() | |||||
for filename in files: | |||||
if filename.startswith("_") or not filename.endswith(".py"): | |||||
continue | |||||
try: | |||||
_load_command(connection, filename) | |||||
except AttributeError: | |||||
pass # The file is doesn't contain a command, so just move on | |||||
msg = "Found {0} commands: {1}" | |||||
logger.info(msg.format(len(_commands), ", ".join(_commands.keys()))) | |||||
def get_all(): | |||||
"""Return our dict of all loaded commands.""" | |||||
return _commands | |||||
def check(hook, data): | |||||
"""Given an event on IRC, check if there's anything we can respond to.""" | |||||
# Parse command arguments into data.command and data.args: | |||||
data.parse_args() | |||||
for command in _commands.values(): | |||||
if hook in command.hooks: | |||||
if command.check(data): | |||||
try: | |||||
command.process(data) | |||||
except: | |||||
logger.exception("Error executing command '{0}'".format(data.command)) | |||||
break | |||||
# The command's name, as reported to the user when they use !help: | |||||
name = None | |||||
# A list of names that will trigger this command. If left empty, it will | |||||
# be triggered by the command's name and its name only: | |||||
commands = [] | |||||
# Hooks are "msg", "msg_private", "msg_public", and "join". "msg" is the | |||||
# default behavior; if you wish to override that, change the value in your | |||||
# command subclass: | |||||
hooks = ["msg"] | |||||
def __init__(self, bot): | |||||
"""Constructor for new commands. | |||||
This is called once when the command is loaded (from | |||||
:py:meth:`commands.load() <earwigbot.managers._ResourceManager.load>`). | |||||
*bot* is out base :py:class:`~earwigbot.bot.Bot` object. Don't override | |||||
this directly; if you do, remember to place | |||||
``super(Command, self).__init()`` first. Use :py:meth:`setup` for | |||||
typical command-init/setup needs. | |||||
""" | |||||
self.bot = bot | |||||
self.config = bot.config | |||||
self.logger = bot.commands.logger.getChild(self.name) | |||||
# Convenience functions: | |||||
self.say = lambda target, msg, hidelog=False: self.bot.frontend.say(target, msg, hidelog) | |||||
self.reply = lambda data, msg, hidelog=False: self.bot.frontend.reply(data, msg, hidelog) | |||||
self.action = lambda target, msg, hidelog=False: self.bot.frontend.action(target, msg, hidelog) | |||||
self.notice = lambda target, msg, hidelog=False: self.bot.frontend.notice(target, msg, hidelog) | |||||
self.join = lambda chan, hidelog=False: self.bot.frontend.join(chan, hidelog) | |||||
self.part = lambda chan, msg=None, hidelog=False: self.bot.frontend.part(chan, msg, hidelog) | |||||
self.mode = lambda t, level, msg, hidelog=False: self.bot.frontend.mode(t, level, msg, hidelog) | |||||
self.ping = lambda target, hidelog=False: self.bot.frontend.ping(target, hidelog) | |||||
self.pong = lambda target, hidelog=False: self.bot.frontend.pong(target, hidelog) | |||||
self.setup() | |||||
def __repr__(self): | |||||
"""Return the canonical string representation of the Command.""" | |||||
res = "Command(name={0!r}, commands={1!r}, hooks={2!r}, bot={3!r})" | |||||
return res.format(self.name, self.commands, self.hooks, self.bot) | |||||
def __str__(self): | |||||
"""Return a nice string representation of the Command.""" | |||||
return "<Command {0} of {1}>".format(self.name, self.bot) | |||||
def setup(self): | |||||
"""Hook called immediately after the command is loaded. | |||||
Does nothing by default; feel free to override. | |||||
""" | |||||
pass | |||||
def check(self, data): | |||||
"""Return whether this command should be called in response to *data*. | |||||
Given a :py:class:`~earwigbot.irc.data.Data` instance, return ``True`` | |||||
if we should respond to this activity, or ``False`` if we should ignore | |||||
it and move on. Be aware that since this is called for each message | |||||
sent on IRC, it should be cheap to execute and unlikely to throw | |||||
exceptions. | |||||
Most commands return ``True`` only if :py:attr:`data.command | |||||
<earwigbot.irc.data.Data.command>` ``==`` :py:attr:`self.name <name>`, | |||||
or :py:attr:`data.command <earwigbot.irc.data.Data.command>` is in | |||||
:py:attr:`self.commands <commands>` if that list is overriden. This is | |||||
the default behavior; you should only override it if you wish to change | |||||
that. | |||||
""" | |||||
if self.commands: | |||||
return data.is_command and data.command in self.commands | |||||
return data.is_command and data.command == self.name | |||||
def process(self, data): | |||||
"""Main entry point for doing a command. | |||||
Handle an activity (usually a message) on IRC. At this point, thanks | |||||
to :py:meth:`check` which is called automatically by the command | |||||
handler, we know this is something we should respond to. Place your | |||||
command's body here. | |||||
""" | |||||
pass |
@@ -1,979 +0,0 @@ | |||||
# -*- coding: utf-8 -*- | |||||
###### | |||||
###### NOTE: | |||||
###### This is an old commands file from the previous version of EarwigBot. | |||||
###### It is not used by the new EarwigBot and is simply here for reference | |||||
###### when developing new commands. | |||||
###### | |||||
### EarwigBot | |||||
## Import basics. | |||||
import sys, socket, string, time, codecs, os, traceback, thread, re, urllib, web, math, unicodedata | |||||
## Import our functions. | |||||
import config | |||||
## Set up constants. | |||||
HOST, PORT, NICK, IDENT, REALNAME, CHANS, REPORT_CHAN, WELCOME_CHAN, HOST2, CHAN2, OWNER, ADMINS, ADMINS_R, PASS = config.host, config.port, config.nick, config.ident, config.realname, config.chans, config.report_chan, config.welcome_chan, config.host2, config.chan2, config.owner, config.admins, config.admin_readable, config.password | |||||
def get_commandList(): | |||||
return {'quiet': 'quiet', | |||||
'welcome': 'welcome', | |||||
'greet': 'welcome', | |||||
'linker': 'linker', | |||||
'auth': 'auth', | |||||
'access': 'access', | |||||
'join': 'join', | |||||
'part': 'part', | |||||
'restart': 'restart', | |||||
'quit': 'quit', | |||||
'die': 'quit', | |||||
'msg': 'msg', | |||||
'me': 'me', | |||||
'calc': 'calc', | |||||
'dice': 'dice', | |||||
'tock': 'tock', | |||||
'beats': 'beats', | |||||
'copyvio': 'copyvio', | |||||
'copy': 'copyvio', | |||||
'copyright': 'copyvio', | |||||
'dict': 'dictionary', | |||||
'dictionary': 'dictionary', | |||||
'ety': 'etymology', | |||||
'etymology': 'etymology', | |||||
'lang': 'langcode', | |||||
'langcode': 'langcode', | |||||
'num': 'number', | |||||
'number': 'number', | |||||
'count': 'number', | |||||
'c': 'number', | |||||
'nick': 'nick', | |||||
'op': 'op', | |||||
'deop': 'deop', | |||||
'voice': 'voice', | |||||
'devoice': 'devoice', | |||||
'pend': 'pending', | |||||
'pending': 'pending', | |||||
'sub': 'submissions', | |||||
'submissions': 'submissions', | |||||
'praise': 'praise', | |||||
'leonard': 'leonard', | |||||
'groovedog': 'groovedog', | |||||
'earwig': 'earwig', | |||||
'macmed': 'macmed', | |||||
'cubs197': 'cubs197', | |||||
'sparksboy': 'sparksboy', | |||||
'tim_song': 'tim_song', | |||||
'tim': 'tim_song', | |||||
'blurpeace': 'blurpeace', | |||||
'sausage': 'sausage', | |||||
'mindstormskid': 'mindstormskid', | |||||
'mcjohn': 'mcjohn', | |||||
'fetchcomms': 'fetchcomms', | |||||
'trout': 'trout', | |||||
'kill': 'kill', | |||||
'destroy': 'kill', | |||||
'murder': 'kill', | |||||
'fish': 'fish', | |||||
'report': 'report', | |||||
'commands': 'commands', | |||||
'help': 'help', | |||||
'doc': 'help', | |||||
'documentation': 'help', | |||||
'mysql': 'mysql', | |||||
'remind': 'reminder', | |||||
'reminder': 'reminder', | |||||
'notes': 'notes', | |||||
'note': 'notes', | |||||
'about': 'notes', | |||||
'data': 'notes', | |||||
'database': 'notes', | |||||
'hash': 'hash', | |||||
'lookup': 'lookup', | |||||
'ip': 'lookup' | |||||
} | |||||
def main(command, line, line2, nick, chan, host, auth, notice, say, reply, s): | |||||
try: | |||||
parse(command, line, line2, nick, chan, host, auth, notice, say, reply, s) | |||||
except Exception: | |||||
trace = traceback.format_exc() # Traceback. | |||||
print trace # Print. | |||||
lines = list(reversed(trace.splitlines())) # Convert lines to process traceback.... | |||||
report2 = [lines[0].strip()] | |||||
for line in lines: | |||||
line = line.strip() | |||||
if line.startswith('File "/'): | |||||
report2.append(line[0].lower() + line[1:]) | |||||
break | |||||
else: report2.append('source unknown') | |||||
say(report2[0] + ' (' + report2[1] + ')', chan) | |||||
def parse(command, line, line2, nick, chan, host, auth, notice, say, reply, s): | |||||
authy = auth(host) | |||||
if command == "access": | |||||
a = 'The bot\'s owner is "%s".' % OWNER | |||||
b = 'The bot\'s admins are "%s".' % ', '.join(ADMINS_R) | |||||
reply(a, chan, nick) | |||||
reply(b, chan, nick) | |||||
return | |||||
if command == "join": | |||||
if authy == "owner" or authy == "admin": | |||||
try: | |||||
channel = line2[4] | |||||
except Exception: | |||||
channel = chan | |||||
s.send("JOIN %s\r\n" % channel) | |||||
else: | |||||
reply("You aren't authorized to use that command.", chan, nick) | |||||
return | |||||
if command == "part": | |||||
if authy == "owner" or authy == "admin": | |||||
try: | |||||
channel = line2[4] | |||||
except Exception: | |||||
channel = chan | |||||
s.send("PART %s\r\n" % channel) | |||||
else: | |||||
reply("You aren't authorized to use that command.", chan, nick) | |||||
return | |||||
if command == "restart": | |||||
import thread | |||||
if authy == "owner": | |||||
s.send("QUIT\r\n") | |||||
time.sleep(5) | |||||
os.system("nice -15 python main.py") | |||||
exit() | |||||
else: | |||||
reply("Only the owner, %s, can stop the bot. This incident will be reported." % OWNER, chan, nick) | |||||
return | |||||
if command == "quit" or command == "die": | |||||
if authy != "owner": | |||||
if command != "suicide": | |||||
reply("Only the owner, %s, can stop the bot. This incident will be reported." % OWNER, chan, nick) | |||||
else: | |||||
say("\x01ACTION hands %s a gun... have fun :D\x01" % nick, nick) | |||||
else: | |||||
if command == "suicide": | |||||
say("\x01ACTION stabs himself with a knife.\x01", chan) | |||||
time.sleep(0.2) | |||||
try: | |||||
s.send("QUIT :%s\r\n" % ' '.join(line2[4:])) | |||||
except Exception: | |||||
s.send("QUIT\r\n") | |||||
__import__('os')._exit(0) | |||||
return | |||||
if command == "msg": | |||||
if authy == "owner" or authy == "admin": | |||||
say(' '.join(line2[5:]), line2[4]) | |||||
else: | |||||
reply("You aren't authorized to use that command.", chan, nick) | |||||
return | |||||
if command == "me": | |||||
if authy == "owner" or authy == "admin": | |||||
say("\x01ACTION %s\x01" % ' '.join(line2[5:]), line2[4]) | |||||
else: | |||||
reply("You aren't authorized to use that command.", chan, nick) | |||||
return | |||||
if command == "calc": | |||||
r_result = re.compile(r'(?i)<A NAME=results>(.*?)</A>') | |||||
r_tag = re.compile(r'<\S+.*?>') | |||||
subs = [ | |||||
(' in ', ' -> '), | |||||
(' over ', ' / '), | |||||
(u'£', 'GBP '), | |||||
(u'€', 'EUR '), | |||||
('\$', 'USD '), | |||||
(r'\bKB\b', 'kilobytes'), | |||||
(r'\bMB\b', 'megabytes'), | |||||
(r'\bGB\b', 'kilobytes'), | |||||
('kbps', '(kilobits / second)'), | |||||
('mbps', '(megabits / second)') | |||||
] | |||||
try: | |||||
q = ' '.join(line2[4:]) | |||||
except Exception: | |||||
say("0?", chan) | |||||
return | |||||
query = q[:] | |||||
for a, b in subs: | |||||
query = re.sub(a, b, query) | |||||
query = query.rstrip(' \t') | |||||
precision = 5 | |||||
if query[-3:] in ('GBP', 'USD', 'EUR', 'NOK'): | |||||
precision = 2 | |||||
query = web.urllib.quote(query.encode('utf-8')) | |||||
uri = 'http://futureboy.us/fsp/frink.fsp?fromVal=' | |||||
bytes = web.get(uri + query) | |||||
m = r_result.search(bytes) | |||||
if m: | |||||
result = m.group(1) | |||||
result = r_tag.sub('', result) # strip span.warning tags | |||||
result = result.replace('>', '>') | |||||
result = result.replace('(undefined symbol)', '(?) ') | |||||
if '.' in result: | |||||
try: result = str(round(float(result), precision)) | |||||
except ValueError: pass | |||||
if not result.strip(): | |||||
result = '?' | |||||
elif ' in ' in q: | |||||
result += ' ' + q.split(' in ', 1)[1] | |||||
say(q + ' = ' + result[:350], chan) | |||||
else: reply("Sorry, can't calculate that.", chan, nick) | |||||
return | |||||
if command == "dice": | |||||
import random | |||||
try: | |||||
set = range(int(line2[4]), int(line2[5]) + 1) | |||||
except Exception: | |||||
set = range(1, 7) | |||||
num = random.choice(set) | |||||
reply("You rolled a %s." % num, chan, nick) | |||||
if len(set) < 30: | |||||
say("Set consisted of %s." % set, nick) | |||||
else: | |||||
say("Set consisted of %s... and %s others." % (set[:30], len(set) - 30), nick) | |||||
return | |||||
if command == "tock": | |||||
u = urllib.urlopen('http://tycho.usno.navy.mil/cgi-bin/timer.pl') | |||||
info = u.info() | |||||
u.close() | |||||
say('"' + info['Date'] + '" - tycho.usno.navy.mil', chan) | |||||
return | |||||
if command == "beats": | |||||
beats = ((time.time() + 3600) % 86400) / 86.4 | |||||
beats = int(math.floor(beats)) | |||||
say('@%03i' % beats, chan) | |||||
return | |||||
if command == "copyvio" or command == "copy" or command == "copyright": | |||||
url = "http://en.wikipedia.org/wiki/User:EarwigBot/AfC copyvios" | |||||
query = urllib.urlopen(url) | |||||
data = query.read() | |||||
url = "http://toolserver.org/~earwig/earwigbot/pywikipedia/error.txt" | |||||
query = urllib.urlopen(url) | |||||
data2 = query.read() | |||||
if "critical" in data2: | |||||
text = "AfC copyvio situation is CRITICAL: Major disaster." | |||||
elif "exceed" in data2: | |||||
text = "AfC copyvio situation is CRITICAL: Queries exceeded error." | |||||
elif "spam" in data2: | |||||
text = "AfC copyvio situation is CRITICAL: Spamfilter error." | |||||
elif "<h3>" in data: | |||||
text = "AfC copyvio situation is BAD: Unsolved copyvios at [[User:EarwigBot/AfC copyvios]]" | |||||
else: | |||||
text = "AfC copyvio situation is OK: OK." | |||||
reply(text, chan, nick) | |||||
return | |||||
if command == "dict" or command == "dictionary": | |||||
def trim(thing): | |||||
if thing.endswith(' '): | |||||
thing = thing[:-6] | |||||
return thing.strip(' :.') | |||||
r_li = re.compile(r'(?ims)<li>.*?</li>') | |||||
r_tag = re.compile(r'<[^>]+>') | |||||
r_parens = re.compile(r'(?<=\()(?:[^()]+|\([^)]+\))*(?=\))') | |||||
r_word = re.compile(r'^[A-Za-z0-9\' -]+$') | |||||
uri = 'http://encarta.msn.com/dictionary_/%s.html' | |||||
r_info = re.compile(r'(?:ResultBody"><br /><br />(.*?) )|(?:<b>(.*?)</b>)') | |||||
try: | |||||
word = line2[4] | |||||
except Exception: | |||||
reply("Please enter a word.", chan, nick) | |||||
return | |||||
word = urllib.quote(word.encode('utf-8')) | |||||
bytes = web.get(uri % word) | |||||
results = {} | |||||
wordkind = None | |||||
for kind, sense in r_info.findall(bytes): | |||||
kind, sense = trim(kind), trim(sense) | |||||
if kind: wordkind = kind | |||||
elif sense: | |||||
results.setdefault(wordkind, []).append(sense) | |||||
result = word.encode('utf-8') + ' - ' | |||||
for key in sorted(results.keys()): | |||||
if results[key]: | |||||
result += (key or '') + ' 1. ' + results[key][0] | |||||
if len(results[key]) > 1: | |||||
result += ', 2. ' + results[key][1] | |||||
result += '; ' | |||||
result = result.rstrip('; ') | |||||
if result.endswith('-') and (len(result) < 30): | |||||
reply('Sorry, no definition found.', chan, nick) | |||||
else: say(result, chan) | |||||
return | |||||
if command == "ety" or command == "etymology": | |||||
etyuri = 'http://etymonline.com/?term=%s' | |||||
etysearch = 'http://etymonline.com/?search=%s' | |||||
r_definition = re.compile(r'(?ims)<dd[^>]*>.*?</dd>') | |||||
r_tag = re.compile(r'<(?!!)[^>]+>') | |||||
r_whitespace = re.compile(r'[\t\r\n ]+') | |||||
abbrs = [ | |||||
'cf', 'lit', 'etc', 'Ger', 'Du', 'Skt', 'Rus', 'Eng', 'Amer.Eng', 'Sp', | |||||
'Fr', 'N', 'E', 'S', 'W', 'L', 'Gen', 'J.C', 'dial', 'Gk', | |||||
'19c', '18c', '17c', '16c', 'St', 'Capt', 'obs', 'Jan', 'Feb', 'Mar', | |||||
'Apr', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec', 'c', 'tr', 'e', 'g' | |||||
] | |||||
t_sentence = r'^.*?(?<!%s)(?:\.(?= [A-Z0-9]|\Z)|\Z)' | |||||
r_sentence = re.compile(t_sentence % ')(?<!'.join(abbrs)) | |||||
def unescape(s): | |||||
s = s.replace('>', '>') | |||||
s = s.replace('<', '<') | |||||
s = s.replace('&', '&') | |||||
return s | |||||
def text(html): | |||||
html = r_tag.sub('', html) | |||||
html = r_whitespace.sub(' ', html) | |||||
return unescape(html).strip() | |||||
try: | |||||
word = line2[4] | |||||
except Exception: | |||||
reply("Please enter a word.", chan, nick) | |||||
return | |||||
def ety(word): | |||||
if len(word) > 25: | |||||
raise ValueError("Word too long: %s[...]" % word[:10]) | |||||
word = {'axe': 'ax/axe'}.get(word, word) | |||||
bytes = web.get(etyuri % word) | |||||
definitions = r_definition.findall(bytes) | |||||
if not definitions: | |||||
return None | |||||
defn = text(definitions[0]) | |||||
m = r_sentence.match(defn) | |||||
if not m: | |||||
return None | |||||
sentence = m.group(0) | |||||
try: | |||||
sentence = unicode(sentence, 'iso-8859-1') | |||||
sentence = sentence.encode('utf-8') | |||||
except: pass | |||||
maxlength = 275 | |||||
if len(sentence) > maxlength: | |||||
sentence = sentence[:maxlength] | |||||
words = sentence[:-5].split(' ') | |||||
words.pop() | |||||
sentence = ' '.join(words) + ' [...]' | |||||
sentence = '"' + sentence.replace('"', "'") + '"' | |||||
return sentence + ' - ' + (etyuri % word) | |||||
try: | |||||
result = ety(word.encode('utf-8')) | |||||
except IOError: | |||||
msg = "Can't connect to etymonline.com (%s)" % (etyuri % word) | |||||
reply(msg, chan, nick) | |||||
return | |||||
except AttributeError: | |||||
result = None | |||||
if result is not None: | |||||
reply(result, chan, nick) | |||||
else: | |||||
uri = etysearch % word | |||||
msg = 'Can\'t find the etymology for "%s". Try %s' % (word, uri) | |||||
reply(msg, chan, nick) | |||||
return | |||||
if command == "num" or command == "number" or command == "count" or command == "c": | |||||
try: | |||||
params = string.lower(line2[4]) | |||||
except Exception: | |||||
params = False | |||||
if params == "old" or params == "afc" or params == "a": | |||||
number = unicode(int(len(re.findall("title=", urllib.urlopen("http://en.wikipedia.org/w/api.php?action=query&list=categorymembers&cmtitle=Category:Pending_AfC_submissions&cmlimit=500").read()))) - 2) | |||||
reply("There are currently %s pending AfC submissions." % number, chan, nick) | |||||
elif params == "redirect" or params == "redir" or params == "redirs" or params == "redirects" or params == "r": | |||||
redir_data = urllib.urlopen("http://en.wikipedia.org/w/index.php?title=Wikipedia:Articles_for_creation/Redirects").read() | |||||
redirs = (string.count(redir_data, "<h2>") - 1) - (string.count(redir_data, '<table class="navbox collapsible collapsed" style="text-align: left; border: 0px; margin-top: 0.2em;">')) | |||||
reply("There are currently %s open redirect requests." % redirs, chan, nick) | |||||
elif params == "files" or params == "ffu" or params == "file" or params == "image" or params == "images" or params == "ifu" or params == "f": | |||||
file_data = re.sub("<h2>Contents</h2>", "", urllib.urlopen("http://en.wikipedia.org/w/index.php?title=Wikipedia:Files_for_upload").read()) | |||||
files = (string.count(file_data, "<h2>") - 1) - (string.count(file_data, '<table class="navbox collapsible collapsed" style="text-align: left; border: 0px; margin-top: 0.2em;">')) | |||||
reply("There are currently %s open file upload requests." % files, chan, nick) | |||||
elif params == "aggregate" or params == "agg": | |||||
subs = unicode(int(len(re.findall("title=", urllib.urlopen("http://en.wikipedia.org/w/api.php?action=query&list=categorymembers&cmtitle=Category:Pending_AfC_submissions&cmlimit=500").read()))) - 2) | |||||
redir_data = urllib.urlopen("http://en.wikipedia.org/w/index.php?title=Wikipedia:Articles_for_creation/Redirects").read() | |||||
file_data = re.sub("<h2>Contents</h2>", "", urllib.urlopen("http://en.wikipedia.org/w/index.php?title=Wikipedia:Files_for_upload").read()) | |||||
redirs = (string.count(redir_data, "<h2><span class=\"editsection\">")) - (string.count(redir_data, '<table class="navbox collapsible collapsed" style="text-align: left; border: 0px; margin-top: 0.2em;">')) | |||||
files = (string.count(file_data, "<h2>") - 1) - (string.count(file_data, '<table class="navbox collapsible collapsed" style="text-align: left; border: 0px; margin-top: 0.2em;">')) | |||||
aggregate = (int(subs) * 5) + (int(redirs) * 2) + (int(files) * 2) | |||||
if aggregate == 0: | |||||
stat = "clear" | |||||
elif aggregate < 60: | |||||
stat = "almost clear" | |||||
elif aggregate < 125: | |||||
stat = "small backlog" | |||||
elif aggregate < 175: | |||||
stat = "average backlog" | |||||
elif aggregate < 250: | |||||
stat = "backlogged" | |||||
elif aggregate < 300: | |||||
stat = "heavily backlogged" | |||||
else: | |||||
stat = "severely backlogged" | |||||
reply("Aggregate is currently %s (%s)." % (aggregate, stat), chan, nick) | |||||
else: | |||||
subs = unicode(int(len(re.findall("title=", urllib.urlopen("http://en.wikipedia.org/w/api.php?action=query&list=categorymembers&cmtitle=Category:Pending_AfC_submissions&cmlimit=500").read()))) - 2) | |||||
redir_data = urllib.urlopen("http://en.wikipedia.org/w/index.php?title=Wikipedia:Articles_for_creation/Redirects").read() | |||||
file_data = re.sub("<h2>Contents</h2>", "", urllib.urlopen("http://en.wikipedia.org/w/index.php?title=Wikipedia:Files_for_upload").read()) | |||||
redirs = (string.count(redir_data, "<h2><span class=\"editsection\">")) - (string.count(redir_data, '<table class="navbox collapsible collapsed" style="text-align: left; border: 0px; margin-top: 0.2em;">')) | |||||
files = (string.count(file_data, "<h2>") - 1) - (string.count(file_data, '<table class="navbox collapsible collapsed" style="text-align: left; border: 0px; margin-top: 0.2em;">')) | |||||
reply("There are currently %s pending submissions, %s open redirect requests, and %s open file upload requests." % (subs, redirs, files), chan, nick) | |||||
return | |||||
if command == "nick": | |||||
if authy == "owner": | |||||
try: | |||||
new_nick = line2[4] | |||||
except Exception: | |||||
reply("Please specify a nick to change to.", chan, nick) | |||||
return | |||||
s.send("NICK %s\r\n" % new_nick) | |||||
else: | |||||
reply("You aren't authorized to use that command.", chan, nick) | |||||
return | |||||
if command == "op" or command == "deop" or command == "voice" or command == "devoice": | |||||
if authy == "owner" or authy == "admin": | |||||
try: | |||||
user = line2[4] | |||||
except Exception: | |||||
user = nick | |||||
say("%s %s %s" % (command, chan, user), "ChanServ") | |||||
else: | |||||
reply("You aren't authorized to use that command.", chan, nick) | |||||
return | |||||
if command == "pend" or command == "pending": | |||||
say("Pending submissions status page: <http://en.wikipedia.org/wiki/WP:AFC/S>.", chan) | |||||
say("Pending submissions category: <http://en.wikipedia.org/wiki/Category:Pending_AfC_submissions>.", chan) | |||||
return | |||||
if command == "sub" or command == "submissions": | |||||
try: | |||||
number = int(line2[4]) | |||||
except Exception: | |||||
reply("Please enter a number.", chan, nick) | |||||
return | |||||
do_url = False | |||||
try: | |||||
if "url" in line2[5:]: do_url = True | |||||
except Exception: | |||||
pass | |||||
url = "http://en.wikipedia.org/w/api.php?action=query&list=categorymembers&cmtitle=Category:Pending_AfC_submissions&cmlimit=500&cmsort=timestamp" | |||||
query = urllib.urlopen(url) | |||||
data = query.read() | |||||
pages = re.findall("title="(.*?)"", data) | |||||
try: | |||||
pages.remove("Wikipedia:Articles for creation/Redirects") | |||||
except Exception: | |||||
pass | |||||
try: | |||||
pages.remove("Wikipedia:Files for upload") | |||||
except Exception: | |||||
pass | |||||
pages.reverse() | |||||
pages = pages[:number] | |||||
if not do_url: | |||||
s = string.join(pages, "]], [[") | |||||
s = "[[%s]]" % s | |||||
else: | |||||
s = string.join(pages, ">, <http://en.wikipedia.org/wiki/") | |||||
s = "<http://en.wikipedia.org/wiki/%s>" % s | |||||
s = re.sub(" ", "_", s) | |||||
s = re.sub(">,_<", ">, <", s) | |||||
report = "\x02First %s pending AfC submissions:\x0F %s" % (number, s) | |||||
say(report, chan) | |||||
return | |||||
if command == "praise" or command == "leonard" or command == "groovedog" or command == "earwig" or command == "macmed" or command == "cubs197" or command == "sparksboy" or command == "tim_song" or command == "tim" or command == "sausage" or command == "mindstormskid" or command == "mcjohn" or command == "fetchcomms" or command == "blurpeace": | |||||
bad = False | |||||
if command == "leonard": | |||||
special = "AfC redirect reviewer" | |||||
user = "Leonard^Bloom" | |||||
elif command == "groovedog": | |||||
special = "heh" | |||||
user = "GrooveDog" | |||||
elif command == "earwig": | |||||
special = "Python programmer" | |||||
user = "Earwig" | |||||
elif command == "macmed": | |||||
special = "CSD tagger" | |||||
user = "MacMed" | |||||
elif command == "mindstormskid": | |||||
special = "Lego fanatic" | |||||
user = "MindstormsKid" | |||||
elif command == "cubs197": | |||||
special = "IRC dude" | |||||
user = "Cubs197" | |||||
elif command == "sparksboy": | |||||
special = "pet owner" | |||||
user = "SparksBoy" | |||||
elif command == "tim_song" or command == "tim": | |||||
special = "JavaScript programmer" | |||||
user = "Tim_Song" | |||||
elif command == "sausage": | |||||
special = "helper" | |||||
user = "chzz" | |||||
elif command == "mcjohn": | |||||
special = "edit summary writer" | |||||
user = "McJohn" | |||||
elif command == "fetchcomms": | |||||
special = "n00b" | |||||
user = "Fetchcomms" | |||||
elif command == "blurpeace": | |||||
special = "Commons admin" | |||||
user = "Blurpeace" | |||||
else: | |||||
say("Only a true fool would use that command, %s." % nick, chan) | |||||
# say("The users who you can praise are: Leonard^Bloom, GrooveDog, Earwig, MacMed, Cubs197, SparksBoy, MindstormsKid, Chzz, McJohn, Tim_Song, Fetchcomms, and Blurpeace.", chan) | |||||
return | |||||
if not bad: | |||||
say("\x02%s\x0F is the bestest %s evah!" % (user, special), chan) | |||||
if bad: | |||||
say("\x02%s\x0F is worstest %s evah!" % (user, special), chan) | |||||
return | |||||
if command == "trout": | |||||
try: | |||||
user = line2[4] | |||||
user = ' '.join(line2[4:]) | |||||
except Exception: | |||||
reply("Hahahahahahahaha...", chan, nick) | |||||
return | |||||
normal = unicodedata.normalize('NFKD', unicode(string.lower(user))) | |||||
if "itself" in normal: | |||||
reply("I'm not that stupid ;)", chan, nick) | |||||
return | |||||
elif "earwigbot" in normal: | |||||
reply("I'm not that stupid ;)", chan, nick) | |||||
elif "earwig" not in normal and "ear wig" not in normal: | |||||
text = 'slaps %s around a bit with a large trout.' % user | |||||
msg = '\x01ACTION %s\x01' % text | |||||
say(msg, chan) | |||||
else: | |||||
reply("I refuse to hurt anything with \"Earwig\" in its name :P", chan, nick) | |||||
return | |||||
if command == "kill" or command == "destroy" or command == "murder": | |||||
reply("Who do you think I am? The Mafia?", chan, nick) | |||||
return | |||||
if command == "fish": | |||||
try: | |||||
user = line2[4] | |||||
fish = ' '.join(line2[5:]) | |||||
except Exception: | |||||
reply("Hahahahahahahaha...", chan, nick) | |||||
return | |||||
normal = unicodedata.normalize('NFKD', unicode(string.lower(user))) | |||||
if "itself" in normal: | |||||
reply("I'm not that stupid ;)", chan, nick) | |||||
return | |||||
elif "earwigbot" in normal: | |||||
reply("I'm not that stupid ;)", chan, nick) | |||||
elif "earwig" not in normal and "ear wig" not in normal: | |||||
text = 'slaps %s around a bit with a %s.' % (user, fish) | |||||
msg = '\x01ACTION %s\x01' % text | |||||
say(msg, chan) | |||||
else: | |||||
reply("I refuse to hurt anything with \"Earwig\" in its name :P", chan, nick) | |||||
return | |||||
if command == "report": | |||||
def find_status(name="", talk=False): | |||||
enname = re.sub(" ", "_", name) | |||||
if talk == True: | |||||
enname = "Wikipedia_talk:Articles_for_creation/%s" % enname | |||||
if talk == False: | |||||
enname = "Wikipedia:Articles_for_creation/%s" % enname | |||||
url = "http://en.wikipedia.org/w/api.php?action=query&titles=%s&prop=revisions&rvprop=content" % enname | |||||
query = urllib.urlopen(url) | |||||
data = query.read() | |||||
status = "" | |||||
if "{{AFC submission|D" in data or "{{AFC submission|d" in data: | |||||
reason = re.findall("(D|d)\|(.*?)\|", data) | |||||
if reason[0][1] != "reason": | |||||
status = "Declined, reason is '%s'" % reason[0][1] | |||||
if reason[0][1] == "reason": | |||||
status = "Declined, reason is a custom reason" | |||||
if "{{AFC submission|H" in data or "{{AFC submission|h" in data: | |||||
reason = re.findall("(H|h)\|(.*?)\|", data) | |||||
if reason[0][1] != "reason": | |||||
status = "Held, reason is '%s'" % reason[0][1] | |||||
if reason[0][1] == "reason": | |||||
status = "Held, reason is a custom reason" | |||||
if "{{AFC submission||" in data: | |||||
status = "Pending" | |||||
if "{{AFC submission|R" in data or "{{AFC submission|r" in data: | |||||
status = "Reviewing" | |||||
if not status: | |||||
exist = exists(name=enname) | |||||
if exist == True: | |||||
status = "Accepted" | |||||
if exist == False: | |||||
status = "Not found" | |||||
return status | |||||
def exists(name=""): | |||||
url = "http://en.wikipedia.org/wiki/%s" % name | |||||
query = urllib.urlopen(url) | |||||
data = query.read() | |||||
if "Wikipedia does not have a" in data: | |||||
return False | |||||
return True | |||||
def get_submitter(name="", talk=False): | |||||
enname = re.sub(" ", "_", name) | |||||
if talk == True: | |||||
enname = "Wikipedia_talk:Articles_for_creation/%s" % enname | |||||
if talk == False: | |||||
enname = "Wikipedia:Articles_for_creation/%s" % enname | |||||
url = "http://en.wikipedia.org/w/api.php?action=query&titles=%s&prop=revisions&rvprop=user&rvdir=newer&rvlimit=1" % enname | |||||
query = urllib.urlopen(url) | |||||
data = query.read() | |||||
extract = re.findall("user="(.*?)"", data) | |||||
if "anon=" in data: | |||||
anon = True | |||||
else: | |||||
anon = False | |||||
try: | |||||
return extract[0], anon | |||||
except BaseException: | |||||
print extract | |||||
return "", anon | |||||
try: | |||||
rawSub = line2[4] | |||||
rawSub = ' '.join(line2[4:]) | |||||
except Exception: | |||||
reply("You need to specify a submission name in order to use %s!" % command, chan, nick) | |||||
return | |||||
talk = False | |||||
if "[[" in rawSub and "]]" in rawSub: | |||||
name = re.sub("\[\[(.*)\]\]", "\\1", rawSub) | |||||
name = re.sub(" ", "_", name) | |||||
name = urllib.quote(name, ":/") | |||||
name = "http://en.wikipedia.org/wiki/%s" % name | |||||
if "talk:" in name: | |||||
talk = True | |||||
elif "http://" in rawSub: | |||||
name = rawSub | |||||
if "talk:" in name: | |||||
talk = True | |||||
elif "en.wikipedia.org" in rawSub: | |||||
name = "http://%s" % rawSub | |||||
if "talk:" in name: | |||||
talk = True | |||||
elif "Wikipedia:" in rawSub or "Wikipedia_talk:" in rawSub or "Wikipedia talk:" in rawSub: | |||||
name = re.sub(" ", "_", rawSub) | |||||
name = urllib.quote(name, ":/") | |||||
name = "http://en.wikipedia.org/wiki/%s" % name | |||||
if "talk:" in name: | |||||
talk = True | |||||
else: | |||||
url = "http://en.wikipedia.org/wiki/" | |||||
pagename = re.sub(" ", "_", rawSub) | |||||
pagename = urllib.quote(pagename, ":/") | |||||
pagename = "Wikipedia:Articles_for_creation/%s" % pagename | |||||
page = urllib.urlopen("%s%s" % (url, pagename)) | |||||
text = page.read() | |||||
name = "http://en.wikipedia.org/wiki/%s" % pagename | |||||
if "Wikipedia does not have a" in text: | |||||
pagename = re.sub(" ", "_", rawSub) | |||||
pagename = urllib.quote(pagename, ":/") | |||||
pagename = "Wikipedia_talk:Articles_for_creation/%s" % pagename | |||||
page = urllib.urlopen("%s%s" % (url, pagename)) | |||||
name = "http://en.wikipedia.org/wiki/%s" % pagename | |||||
talk = True | |||||
unname = re.sub("http://en.wikipedia.org/wiki/Wikipedia:Articles_for_creation/", "", name) | |||||
unname = re.sub("http://en.wikipedia.org/wiki/Wikipedia_talk:Articles_for_creation/", "", unname) | |||||
unname = re.sub("_", " ", unname) | |||||
if "talk" in unname: | |||||
talk = True | |||||
submitter, anon = get_submitter(name=unname, talk=talk) | |||||
status = find_status(name=unname, talk=talk) | |||||
if submitter != "": | |||||
if anon == True: | |||||
submitter_page = "Special:Contributions/%s" % submitter | |||||
if anon == False: | |||||
unsubmit = re.sub(" ", "_", submitter) | |||||
unsubmit = urllib.quote(unsubmit, ":/") | |||||
submitter_page = "User:%s" % unsubmit | |||||
if status == "Accepted": | |||||
submitterm = "Reviewer" | |||||
else: | |||||
submitterm = "Submitter" | |||||
line1 = "\x02AfC submission report for %s:" % unname | |||||
line2 = "\x02URL: \x0301\x0F%s" % name | |||||
if submitter != "": | |||||
line3 = "\x02%s: \x0F\x0302%s (\x0301\x0Fhttp://en.wikipedia.org/wiki/%s)." % (submitterm, submitter, submitter_page) | |||||
line4 = "\x02Status: \x0F\x0302%s." % status | |||||
say(line1, chan) | |||||
time.sleep(0.1) | |||||
say(line2, chan) | |||||
time.sleep(0.1) | |||||
if submitter != "": | |||||
say(line3, chan) | |||||
time.sleep(0.1) | |||||
say(line4, chan) | |||||
return | |||||
if command == "commands": | |||||
if chan.startswith("#"): | |||||
reply("Please use that command in a private message.", chan, nick) | |||||
return | |||||
others2 = get_commandList().values() | |||||
others = [] | |||||
for com in others2: | |||||
if com == "copyvio" or com == "number" or com == "pending" or com == "report" or com == "submissions" or com == "access" or com == "help" or com == "join" or com == "linker" or com == "nick" or com == "op" or com == "part" or com == "quiet" or com == "quit" or com == "restart" or com == "voice" or com == "welcome" or com == "fish" or com == "praise" or com == "trout" or com == "notes": | |||||
continue | |||||
if com in others: continue | |||||
others.append(com) | |||||
others.sort() | |||||
say("\x02AFC commands:\x0F copyvio, number, pending, report, submissions.", chan) | |||||
time.sleep(0.1) | |||||
say("\x02Bot operation and channel maintaince commands:\x0F access, help, join, linker, nick, op, part, quiet, quit, restart, voice, welcome.", chan) | |||||
time.sleep(0.1) | |||||
say("\x02Fun commands:\x0F fish, praise, trout, and numerous easter eggs", chan) | |||||
time.sleep(0.1) | |||||
say("\x02Other commands:\x0F %s" % ', '.join(others), chan) | |||||
time.sleep(0.1) | |||||
say("The bot maintains a mini-wiki. Type \"!notes help\" for more information.", chan) | |||||
time.sleep(0.1) | |||||
say("See http://enwp.org/User:The_Earwig/Bots/IRC for details. For help on a specific command, type '!help command'.", chan) | |||||
return | |||||
if command == "help" or command == "doc" or command == "documentation": | |||||
try: | |||||
com = line2[4] | |||||
except Exception: | |||||
reply("Hi, I'm a bot that does work for Articles for Creation. You can find information about me at http://enwp.org/User:The_Earwig/Bots/IRC. Say \"!commands\" to me in a private message for some of my abilities. Earwig is my owner and creator, and you can contact him at http://enwp.org/User_talk:The_Earwig.", chan, nick) | |||||
return | |||||
say("Sorry, command documentation has not been implemented yet.", chan) | |||||
return | |||||
if command == "mysql": | |||||
if authy != "owner": | |||||
reply("You aren't authorized to use this command.", chan, nick) | |||||
return | |||||
import MySQLdb | |||||
try: | |||||
strings = line2[4] | |||||
strings = ' '.join(line2[4:]) | |||||
if "db:" in strings: | |||||
database = re.findall("db\:(.*?)\s", strings)[0] | |||||
else: | |||||
database = "enwiki_p" | |||||
if "time:" in strings: | |||||
times = int(re.findall("time\:(.*?)\s", strings)[0]) | |||||
else: | |||||
times = 60 | |||||
file = re.findall("file\:(.*?)\s", strings)[0] | |||||
sqlquery = re.findall("query\:(.*?)\Z", strings)[0] | |||||
except Exception: | |||||
reply("You did not specify enough data for the bot to continue.", chan, nick) | |||||
return | |||||
database2 = database[:-2] + "-p" | |||||
db = MySQLdb.connect(db=database, host="%s.rrdb.toolserver.org" % database2, read_default_file="/home/earwig/.my.cnf") | |||||
db.query(sqlquery) | |||||
r = db.use_result() | |||||
data = r.fetch_row(0) | |||||
try: | |||||
f = codecs.open("/home/earwig/public_html/reports/%s/%s" % (database[:-2], file), 'r') | |||||
reply("A file already exists with that name.", chan, nick) | |||||
return | |||||
except Exception: | |||||
pass | |||||
f = codecs.open("/home/earwig/public_html/reports/%s/%s" % (database[:-2], file), 'a', 'utf-8') | |||||
for line in data: | |||||
new_line = [] | |||||
for l in line: | |||||
new_line.append(str(l)) | |||||
f.write(' '.join(new_line) + "\n") | |||||
f.close() | |||||
reply("Query completed successfully. See http://toolserver.org/~earwig/reports/%s/%s. I will delete the report in %s seconds." % (database[:-2], file, times), chan, nick) | |||||
time.sleep(times) | |||||
os.remove("/home/earwig/public_html/reports/%s/%s" % (database[:-2], file)) | |||||
return | |||||
if command == "remind" or command == "reminder": | |||||
try: | |||||
times = int(line2[4]) | |||||
content = ' '.join(line2[5:]) | |||||
except Exception: | |||||
reply("Please specify a time and a note in the following format: !remind <time> <note>.", chan, nick) | |||||
return | |||||
reply("Set reminder for \"%s\" in %s seconds." % (content, times), chan, nick) | |||||
time.sleep(times) | |||||
reply(content, chan, nick) | |||||
return | |||||
if command == "notes" or command == "note" or command == "about" or command == "data" or command == "database": | |||||
try: | |||||
action = line2[4] | |||||
except BaseException: | |||||
reply("What do you want me to do? Type \"!notes help\" for more information.", chan, nick) | |||||
return | |||||
import MySQLdb | |||||
db = MySQLdb.connect(db="u_earwig_ircbot", host="sql", read_default_file="/home/earwig/.my.cnf") | |||||
specify = ' '.join(line2[5:]) | |||||
if action == "help" or action == "manual": | |||||
shortCommandList = "read, write, change, undo, delete, move, author, category, list, report, developer" | |||||
if specify == "read": | |||||
say("To read an entry, type \"!notes read <entry>\".", chan) | |||||
elif specify == "write": | |||||
say("To write a new entry, type \"!notes write <entry> <content>\". This will create a new entry only if one does not exist, see the below command...", chan) | |||||
elif specify == "change": | |||||
say("To change an entry, type \"!notes change <entry> <new content>\". The old entry will be stored in the database, so it can be undone later.", chan) | |||||
elif specify == "undo": | |||||
say("To undo a change, type \"!notes undo <entry>\".", chan) | |||||
elif specify == "delete": | |||||
say("To delete an entry, type \"!notes delete <entry>\". For security reasons, only bot admins can do this.", chan) | |||||
elif specify == "move": | |||||
say("To move an entry, type \"!notes move <old_title> <new_title>\".", chan) | |||||
elif specify == "author": | |||||
say("To return the author of an entry, type \"!notes author <entry>\".", chan) | |||||
elif specify == "category" or specify == "cat": | |||||
say("To change an entry's category, type \"!notes category <entry> <category>\".", chan) | |||||
elif specify == "list": | |||||
say("To list all categories in the database, type \"!notes list\". Type \"!notes list <category>\" to get all entries in a certain category.", chan) | |||||
elif specify == "report": | |||||
say("To give some statistics about the mini-wiki, including some debugging information, type \"!notes report\" in a PM.", chan) | |||||
elif specify == "developer": | |||||
say("To do developer work, such as writing to the database directly, type \"!notes developer <command>\". This can only be done by the bot owner.", chan) | |||||
else: | |||||
db.query("SELECT * FROM version;") | |||||
r = db.use_result() | |||||
data = r.fetch_row(0) | |||||
version = data[0] | |||||
reply("The Earwig Mini-Wiki: running v%s." % version, chan, nick) | |||||
reply("The full list of commands, for reference, are: %s." % shortCommandList, chan, nick) | |||||
reply("For an explaination of a certain command, type \"!notes help <command>\".", chan, nick) | |||||
reply("You can also access the database from the Toolserver: http://toolserver.org/~earwig/cgi-bin/irc_database.py", chan, nick) | |||||
time.sleep(0.4) | |||||
return | |||||
elif action == "read": | |||||
specify = string.lower(specify) | |||||
if " " in specify: specify = string.split(specify, " ")[0] | |||||
if not specify or "\"" in specify: | |||||
reply("Please include the name of the entry you would like to read after the command, e.g. !notes read earwig", chan, nick) | |||||
return | |||||
try: | |||||
db.query("SELECT entry_content FROM entries WHERE entry_title = \"%s\";" % specify) | |||||
r = db.use_result() | |||||
data = r.fetch_row(0) | |||||
entry = data[0][0] | |||||
say("Entry \"\x02%s\x0F\": %s" % (specify, entry), chan) | |||||
except Exception: | |||||
reply("There is no entry titled \"\x02%s\x0F\"." % specify, chan, nick) | |||||
return | |||||
elif action == "delete" or action == "remove": | |||||
specify = string.lower(specify) | |||||
if " " in specify: specify = string.split(specify, " ")[0] | |||||
if not specify or "\"" in specify: | |||||
reply("Please include the name of the entry you would like to delete after the command, e.g. !notes delete earwig", chan, nick) | |||||
return | |||||
if authy == "owner" or authy == "admin": | |||||
try: | |||||
db.query("DELETE from entries where entry_title = \"%s\";" % specify) | |||||
r = db.use_result() | |||||
db.commit() | |||||
reply("The entry on \"\x02%s\x0F\" has been removed." % specify, chan, nick) | |||||
except Exception: | |||||
phenny.reply("Unable to remove the entry on \"\x02%s\x0F\", because it doesn't exist." % specify, chan, nick) | |||||
else: | |||||
reply("Only bot admins can remove entries.", chan, nick) | |||||
return | |||||
elif action == "developer": | |||||
if authy == "owner": | |||||
db.query(specify) | |||||
r = db.use_result() | |||||
try: | |||||
print r.fetch_row(0) | |||||
except Exception: | |||||
pass | |||||
db.commit() | |||||
reply("Done.", chan, nick) | |||||
else: | |||||
reply("Only the bot owner can modify the raw database.", chan, nick) | |||||
return | |||||
elif action == "write": | |||||
try: | |||||
write = line2[5] | |||||
content = ' '.join(line2[6:]) | |||||
except Exception: | |||||
reply("Please include some content in your entry.", chan, nick) | |||||
return | |||||
db.query("SELECT * from entries WHERE entry_title = \"%s\";" % write) | |||||
r = db.use_result() | |||||
data = r.fetch_row(0) | |||||
if data: | |||||
reply("An entry on %s already exists; please use \"!notes change %s %s\"." % (write, write, content), chan, nick) | |||||
return | |||||
content2 = content.replace('"', '\\' + '"') | |||||
db.query("INSERT INTO entries (entry_title, entry_author, entry_category, entry_content, entry_content_old) VALUES (\"%s\", \"%s\", \"uncategorized\", \"%s\", NULL);" % (write, nick, content2)) | |||||
db.commit() | |||||
reply("You have written an entry titled \"\x02%s\x0F\", with the following content: \"%s\"" % (write, content), chan, nick) | |||||
return | |||||
elif action == "change": | |||||
reply("NotImplementedError", chan, nick) | |||||
elif action == "undo": | |||||
reply("NotImplementedError", chan, nick) | |||||
elif action == "move": | |||||
reply("NotImplementedError", chan, nick) | |||||
elif action == "author": | |||||
try: | |||||
entry = line2[5] | |||||
except Exception: | |||||
reply("Please include the name of the entry you would like to get information for after the command, e.g. !notes author earwig", chan, nick) | |||||
return | |||||
db.query("SELECT entry_author from entries WHERE entry_title = \"%s\";" % entry) | |||||
r = db.use_result() | |||||
data = r.fetch_row(0) | |||||
if data: | |||||
say("The author of \"\x02%s\x0F\" is \x02%s\x0F." % (entry, data[0][0]), chan) | |||||
return | |||||
reply("There is no entry titled \"\x02%s\x0F\"." % entry, chan, nick) | |||||
return | |||||
elif action == "cat" or action == "category": | |||||
reply("NotImplementedError", chan, nick) | |||||
elif action == "list": | |||||
reply("NotImplementedError", chan, nick) | |||||
elif action == "report": | |||||
reply("NotImplementedError", chan, nick) | |||||
if command == "hash": | |||||
import hashlib | |||||
try: | |||||
hashVia = line2[4] | |||||
hashText = line2[5] | |||||
hashText = ' '.join(line2[5:]) | |||||
except Exception: | |||||
reply("Please provide a string and method to hash by.", chan, nick) | |||||
return | |||||
try: | |||||
hashed = eval("hashlib.%s(\"%s\").hexdigest()" % (hashVia, hashText)) | |||||
reply(hashed, chan, nick) | |||||
except Exception: | |||||
try: | |||||
hashing = hashlib.new(hashVia) | |||||
hashing.update(hashText) | |||||
hashed = hashing.hexdigest() | |||||
reply(hashed, chan, nick) | |||||
except Exception: | |||||
reply("Error.", chan, nick) | |||||
if command == "langcode" or command == "lang" or command == "language": | |||||
try: | |||||
lang = line2[4] | |||||
except Exception: | |||||
reply("Please specify an ISO code.", chan, nick) | |||||
return | |||||
data = urllib.urlopen("http://toolserver.org/~earwig/cgi-bin/swmt.py?action=iso").read() | |||||
data = string.split(data, "\n") | |||||
result = False | |||||
for datum in data: | |||||
if datum.startswith(lang): | |||||
result = re.findall(".*? (.*)", datum)[0] | |||||
break | |||||
if result: | |||||
reply(result, chan, nick) | |||||
return | |||||
reply("Not found.", chan, nick) | |||||
return | |||||
if command == "lookup" or command == "ip": | |||||
try: | |||||
hexIP = line2[4] | |||||
except Exception: | |||||
reply("Please specify a hex IP address.", chan, nick) | |||||
return | |||||
hexes = [hexIP[:2], hexIP[2:4], hexIP[4:6], hexIP[6:8]] | |||||
hashes = [] | |||||
for hexHash in hexes: | |||||
newHex = int(hexHash, 16) | |||||
hashes.append(newHex) | |||||
normalizedIP = "%s.%s.%s.%s" % (hashes[0], hashes[1], hashes[2], hashes[3]) | |||||
reply(normalizedIP, chan, nick) | |||||
return |
@@ -0,0 +1,34 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
from earwigbot.commands import Command | |||||
class AFCPending(Command): | |||||
"""Link the user to the pending AFC submissions page and category.""" | |||||
name = "pending" | |||||
commands = ["pending", "pend"] | |||||
def process(self, data): | |||||
msg1 = "pending submissions status page: http://enwp.org/WP:AFC/ST" | |||||
msg2 = "pending submissions category: http://enwp.org/CAT:PEND" | |||||
self.reply(data, msg1) | |||||
self.reply(data, msg2) |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -20,31 +20,29 @@ | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||||
# SOFTWARE. | # SOFTWARE. | ||||
import re | |||||
from earwigbot.classes import BaseCommand | |||||
from earwigbot import tasks | |||||
from earwigbot import wiki | from earwigbot import wiki | ||||
from earwigbot.commands import Command | |||||
class Command(BaseCommand): | |||||
class AFCReport(Command): | |||||
"""Get information about an AFC submission by name.""" | """Get information about an AFC submission by name.""" | ||||
name = "report" | name = "report" | ||||
def process(self, data): | def process(self, data): | ||||
self.site = wiki.get_site() | |||||
self.site._maxlag = None | |||||
self.site = self.bot.wiki.get_site() | |||||
self.data = data | self.data = data | ||||
try: | try: | ||||
self.statistics = tasks.get("afc_statistics") | |||||
self.statistics = self.bot.tasks.get("afc_statistics") | |||||
except KeyError: | except KeyError: | ||||
e = "Cannot run command: requires afc_statistics task." | |||||
e = "Cannot run command: requires afc_statistics task (from earwigbot_plugins)" | |||||
self.logger.error(e) | self.logger.error(e) | ||||
msg = "command requires afc_statistics task (from earwigbot_plugins)" | |||||
self.reply(data, msg) | |||||
return | return | ||||
if not data.args: | if not data.args: | ||||
msg = "what submission do you want me to give information about?" | msg = "what submission do you want me to give information about?" | ||||
self.connection.reply(data, msg) | |||||
self.reply(data, msg) | |||||
return | return | ||||
title = " ".join(data.args) | title = " ".join(data.args) | ||||
@@ -68,21 +66,20 @@ class Command(BaseCommand): | |||||
if page: | if page: | ||||
return self.report(page) | return self.report(page) | ||||
msg = "submission \x0302{0}\x0301 not found.".format(title) | |||||
self.connection.reply(data, msg) | |||||
self.reply(data, "submission \x0302{0}\x0301 not found.".format(title)) | |||||
def get_page(self, title): | def get_page(self, title): | ||||
page = self.site.get_page(title, follow_redirects=False) | page = self.site.get_page(title, follow_redirects=False) | ||||
if page.exists()[0]: | |||||
if page.exists == page.PAGE_EXISTS: | |||||
return page | return page | ||||
def report(self, page): | def report(self, page): | ||||
url = page.url().replace("en.wikipedia.org/wiki", "enwp.org") | |||||
short = self.statistics.get_short_title(page.title()) | |||||
url = page.url.replace("en.wikipedia.org/wiki", "enwp.org") | |||||
short = self.statistics.get_short_title(page.title) | |||||
status = self.get_status(page) | status = self.get_status(page) | ||||
user = self.site.get_user(page.creator()) | |||||
user_name = user.name() | |||||
user_url = user.get_talkpage().url() | |||||
user = page.get_creator() | |||||
user_name = user.name | |||||
user_url = user.get_talkpage().url | |||||
msg1 = "AfC submission report for \x0302{0}\x0301 ({1}):" | msg1 = "AfC submission report for \x0302{0}\x0301 ({1}):" | ||||
msg2 = "Status: \x0303{0}\x0301" | msg2 = "Status: \x0303{0}\x0301" | ||||
@@ -90,14 +87,14 @@ class Command(BaseCommand): | |||||
if status == "accepted": | if status == "accepted": | ||||
msg3 = "Reviewed by \x0302{0}\x0301 ({1})" | msg3 = "Reviewed by \x0302{0}\x0301 ({1})" | ||||
self.connection.reply(self.data, msg1.format(short, url)) | |||||
self.connection.say(self.data.chan, msg2.format(status)) | |||||
self.connection.say(self.data.chan, msg3.format(user_name, user_url)) | |||||
self.reply(self.data, msg1.format(short, url)) | |||||
self.say(self.data.chan, msg2.format(status)) | |||||
self.say(self.data.chan, msg3.format(user_name, user_url)) | |||||
def get_status(self, page): | def get_status(self, page): | ||||
if page.is_redirect(): | |||||
if page.is_redirect: | |||||
target = page.get_redirect_target() | target = page.get_redirect_target() | ||||
if self.site.get_page(target).namespace() == wiki.NS_MAIN: | |||||
if self.site.get_page(target).namespace == wiki.NS_MAIN: | |||||
return "accepted" | return "accepted" | ||||
return "redirect" | return "redirect" | ||||
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -22,36 +22,32 @@ | |||||
import re | import re | ||||
from earwigbot import wiki | |||||
from earwigbot.classes import BaseCommand | |||||
from earwigbot.config import config | |||||
from earwigbot.commands import Command | |||||
class Command(BaseCommand): | |||||
class AFCStatus(Command): | |||||
"""Get the number of pending AfC submissions, open redirect requests, and | """Get the number of pending AfC submissions, open redirect requests, and | ||||
open file upload requests.""" | open file upload requests.""" | ||||
name = "status" | name = "status" | ||||
commands = ["status", "count", "num", "number"] | |||||
hooks = ["join", "msg"] | hooks = ["join", "msg"] | ||||
def check(self, data): | def check(self, data): | ||||
commands = ["status", "count", "num", "number"] | |||||
if data.is_command and data.command in commands: | |||||
if data.is_command and data.command in self.commands: | |||||
return True | return True | ||||
try: | try: | ||||
if data.line[1] == "JOIN" and data.chan == "#wikipedia-en-afc": | if data.line[1] == "JOIN" and data.chan == "#wikipedia-en-afc": | ||||
if data.nick != config.irc["frontend"]["nick"]: | |||||
if data.nick != self.config.irc["frontend"]["nick"]: | |||||
return True | return True | ||||
except IndexError: | except IndexError: | ||||
pass | pass | ||||
return False | return False | ||||
def process(self, data): | def process(self, data): | ||||
self.site = wiki.get_site() | |||||
self.site._maxlag = None | |||||
self.site = self.bot.wiki.get_site() | |||||
if data.line[1] == "JOIN": | if data.line[1] == "JOIN": | ||||
status = " ".join(("\x02Current status:\x0F", self.get_status())) | status = " ".join(("\x02Current status:\x0F", self.get_status())) | ||||
self.connection.notice(data.nick, status) | |||||
self.notice(data.nick, status) | |||||
return | return | ||||
if data.args: | if data.args: | ||||
@@ -59,17 +55,17 @@ class Command(BaseCommand): | |||||
if action.startswith("sub") or action == "s": | if action.startswith("sub") or action == "s": | ||||
subs = self.count_submissions() | subs = self.count_submissions() | ||||
msg = "there are \x0305{0}\x0301 pending AfC submissions (\x0302WP:AFC\x0301)." | msg = "there are \x0305{0}\x0301 pending AfC submissions (\x0302WP:AFC\x0301)." | ||||
self.connection.reply(data, msg.format(subs)) | |||||
self.reply(data, msg.format(subs)) | |||||
elif action.startswith("redir") or action == "r": | elif action.startswith("redir") or action == "r": | ||||
redirs = self.count_redirects() | redirs = self.count_redirects() | ||||
msg = "there are \x0305{0}\x0301 open redirect requests (\x0302WP:AFC/R\x0301)." | msg = "there are \x0305{0}\x0301 open redirect requests (\x0302WP:AFC/R\x0301)." | ||||
self.connection.reply(data, msg.format(redirs)) | |||||
self.reply(data, msg.format(redirs)) | |||||
elif action.startswith("file") or action == "f": | elif action.startswith("file") or action == "f": | ||||
files = self.count_redirects() | files = self.count_redirects() | ||||
msg = "there are \x0305{0}\x0301 open file upload requests (\x0302WP:FFU\x0301)." | msg = "there are \x0305{0}\x0301 open file upload requests (\x0302WP:FFU\x0301)." | ||||
self.connection.reply(data, msg.format(files)) | |||||
self.reply(data, msg.format(files)) | |||||
elif action.startswith("agg") or action == "a": | elif action.startswith("agg") or action == "a": | ||||
try: | try: | ||||
@@ -80,21 +76,21 @@ class Command(BaseCommand): | |||||
agg_num = self.get_aggregate_number(agg_data) | agg_num = self.get_aggregate_number(agg_data) | ||||
except ValueError: | except ValueError: | ||||
msg = "\x0303{0}\x0301 isn't a number!" | msg = "\x0303{0}\x0301 isn't a number!" | ||||
self.connection.reply(data, msg.format(data.args[1])) | |||||
self.reply(data, msg.format(data.args[1])) | |||||
return | return | ||||
aggregate = self.get_aggregate(agg_num) | aggregate = self.get_aggregate(agg_num) | ||||
msg = "aggregate is \x0305{0}\x0301 (AfC {1})." | msg = "aggregate is \x0305{0}\x0301 (AfC {1})." | ||||
self.connection.reply(data, msg.format(agg_num, aggregate)) | |||||
self.reply(data, msg.format(agg_num, aggregate)) | |||||
elif action.startswith("nocolor") or action == "n": | elif action.startswith("nocolor") or action == "n": | ||||
self.connection.reply(data, self.get_status(color=False)) | |||||
self.reply(data, self.get_status(color=False)) | |||||
else: | else: | ||||
msg = "unknown argument: \x0303{0}\x0301. Valid args are 'subs', 'redirs', 'files', 'agg', 'nocolor'." | msg = "unknown argument: \x0303{0}\x0301. Valid args are 'subs', 'redirs', 'files', 'agg', 'nocolor'." | ||||
self.connection.reply(data, msg.format(data.args[0])) | |||||
self.reply(data, msg.format(data.args[0])) | |||||
else: | else: | ||||
self.connection.reply(data, self.get_status()) | |||||
self.reply(data, self.get_status()) | |||||
def get_status(self, color=True): | def get_status(self, color=True): | ||||
subs = self.count_submissions() | subs = self.count_submissions() | ||||
@@ -111,12 +107,9 @@ class Command(BaseCommand): | |||||
def count_submissions(self): | def count_submissions(self): | ||||
"""Returns the number of open AFC submissions (count of CAT:PEND).""" | """Returns the number of open AFC submissions (count of CAT:PEND).""" | ||||
cat = self.site.get_category("Pending AfC submissions") | |||||
subs = len(cat.members(limit=2500, use_sql=True)) | |||||
# Remove [[Wikipedia:Articles for creation/Redirects]] and | |||||
# Subtract two for [[Wikipedia:Articles for creation/Redirects]] and | |||||
# [[Wikipedia:Files for upload]], which aren't real submissions: | # [[Wikipedia:Files for upload]], which aren't real submissions: | ||||
return subs - 2 | |||||
return self.site.get_category("Pending AfC submissions").pages - 2 | |||||
def count_redirects(self): | def count_redirects(self): | ||||
"""Returns the number of open redirect submissions. Calculated as the | """Returns the number of open redirect submissions. Calculated as the | ||||
@@ -140,30 +133,30 @@ class Command(BaseCommand): | |||||
def get_aggregate(self, num): | def get_aggregate(self, num): | ||||
"""Returns a human-readable AFC status based on the number of pending | """Returns a human-readable AFC status based on the number of pending | ||||
AFC submissions, open redirect requests, and open FFU requests. This | AFC submissions, open redirect requests, and open FFU requests. This | ||||
does not match {{AFC status}} directly because my algorithm factors in | |||||
does not match {{AFC status}} directly because the algorithm factors in | |||||
WP:AFC/R and WP:FFU while the template only looks at the main | WP:AFC/R and WP:FFU while the template only looks at the main | ||||
submissions. My reasoning is that AFC/R and FFU are still part of | |||||
submissions. The reasoning is that AFC/R and FFU are still part of | |||||
the project, so even if there are no pending submissions, a backlog at | the project, so even if there are no pending submissions, a backlog at | ||||
FFU (for example) indicates that our work is *not* done and the | FFU (for example) indicates that our work is *not* done and the | ||||
project-wide backlog is most certainly *not* clear.""" | project-wide backlog is most certainly *not* clear.""" | ||||
if num == 0: | if num == 0: | ||||
return "is \x02\x0303clear\x0301\x0F" | return "is \x02\x0303clear\x0301\x0F" | ||||
elif num < 125: # < 25 subs | |||||
elif num <= 200: | |||||
return "is \x0303almost clear\x0301" | return "is \x0303almost clear\x0301" | ||||
elif num < 200: # < 40 subs | |||||
elif num <= 400: | |||||
return "is \x0312normal\x0301" | return "is \x0312normal\x0301" | ||||
elif num < 275: # < 55 subs | |||||
elif num <= 600: | |||||
return "is \x0307lightly backlogged\x0301" | return "is \x0307lightly backlogged\x0301" | ||||
elif num < 350: # < 70 subs | |||||
elif num <= 900: | |||||
return "is \x0304backlogged\x0301" | return "is \x0304backlogged\x0301" | ||||
elif num < 500: # < 100 subs | |||||
elif num <= 1200: | |||||
return "is \x02\x0304heavily backlogged\x0301\x0F" | return "is \x02\x0304heavily backlogged\x0301\x0F" | ||||
else: # >= 100 subs | |||||
else: | |||||
return "is \x02\x1F\x0304severely backlogged\x0301\x0F" | return "is \x02\x1F\x0304severely backlogged\x0301\x0F" | ||||
def get_aggregate_number(self, (subs, redirs, files)): | def get_aggregate_number(self, (subs, redirs, files)): | ||||
"""Returns an 'aggregate number' based on the real number of pending | """Returns an 'aggregate number' based on the real number of pending | ||||
submissions in CAT:PEND (subs), open redirect submissions in WP:AFC/R | submissions in CAT:PEND (subs), open redirect submissions in WP:AFC/R | ||||
(redirs), and open files-for-upload requests in WP:FFU (files).""" | (redirs), and open files-for-upload requests in WP:FFU (files).""" | ||||
num = (subs * 5) + (redirs * 2) + (files * 2) | |||||
num = subs + (redirs / 2) + (files / 2) | |||||
return num | return num |
@@ -0,0 +1,59 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
from earwigbot.commands import Command | |||||
class AFCSubmissions(Command): | |||||
"""Link the user directly to some pending AFC submissions.""" | |||||
name = "submissions" | |||||
commands = ["submissions", "subs"] | |||||
def setup(self): | |||||
try: | |||||
self.ignore_list = self.config.commands[self.name]["ignoreList"] | |||||
except KeyError: | |||||
try: | |||||
ignores = self.config.tasks["afc_statistics"]["ignoreList"] | |||||
self.ignore_list = ignores | |||||
except KeyError: | |||||
self.ignore_list = [] | |||||
def process(self, data): | |||||
if data.args: | |||||
try: | |||||
number = int(data.args[0]) | |||||
except ValueError: | |||||
self.reply(data, "argument must be a number.") | |||||
return | |||||
if number > 5: | |||||
msg = "cannot get more than five submissions at a time." | |||||
self.reply(data, msg) | |||||
return | |||||
else: | |||||
number = 3 | |||||
site = self.bot.wiki.get_site() | |||||
category = site.get_category("Pending AfC submissions") | |||||
members = category.get_members(limit=number + len(self.ignore_list)) | |||||
urls = [member.url for member in members if member.title not in self.ignore_list] | |||||
pages = ", ".join(urls[:number]) | |||||
self.reply(data, "{0} pending AfC subs: {1}".format(number, pages)) |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -23,16 +23,16 @@ | |||||
import re | import re | ||||
import urllib | import urllib | ||||
from earwigbot.classes import BaseCommand | |||||
from earwigbot.commands import Command | |||||
class Command(BaseCommand): | |||||
class Calc(Command): | |||||
"""A somewhat advanced calculator: see http://futureboy.us/fsp/frink.fsp | """A somewhat advanced calculator: see http://futureboy.us/fsp/frink.fsp | ||||
for details.""" | for details.""" | ||||
name = "calc" | name = "calc" | ||||
def process(self, data): | def process(self, data): | ||||
if not data.args: | if not data.args: | ||||
self.connection.reply(data, "what do you want me to calculate?") | |||||
self.reply(data, "what do you want me to calculate?") | |||||
return | return | ||||
query = ' '.join(data.args) | query = ' '.join(data.args) | ||||
@@ -47,7 +47,7 @@ class Command(BaseCommand): | |||||
match = r_result.search(result) | match = r_result.search(result) | ||||
if not match: | if not match: | ||||
self.connection.reply(data, "Calculation error.") | |||||
self.reply(data, "Calculation error.") | |||||
return | return | ||||
result = match.group(1) | result = match.group(1) | ||||
@@ -58,26 +58,26 @@ class Command(BaseCommand): | |||||
if not result: | if not result: | ||||
result = '?' | result = '?' | ||||
elif " in " in query: | |||||
elif " in " in query: | |||||
result += " " + query.split(" in ", 1)[1] | result += " " + query.split(" in ", 1)[1] | ||||
res = "%s = %s" % (query, result) | res = "%s = %s" % (query, result) | ||||
self.connection.reply(data, res) | |||||
self.reply(data, res) | |||||
def cleanup(self, query): | def cleanup(self, query): | ||||
fixes = [ | fixes = [ | ||||
(' in ', ' -> '), | |||||
(' over ', ' / '), | |||||
(u'£', 'GBP '), | |||||
(u'€', 'EUR '), | |||||
('\$', 'USD '), | |||||
(r'\bKB\b', 'kilobytes'), | |||||
(r'\bMB\b', 'megabytes'), | |||||
(r'\bGB\b', 'kilobytes'), | |||||
('kbps', '(kilobits / second)'), | |||||
(' in ', ' -> '), | |||||
(' over ', ' / '), | |||||
(u'£', 'GBP '), | |||||
(u'€', 'EUR '), | |||||
('\$', 'USD '), | |||||
(r'\bKB\b', 'kilobytes'), | |||||
(r'\bMB\b', 'megabytes'), | |||||
(r'\bGB\b', 'kilobytes'), | |||||
('kbps', '(kilobits / second)'), | |||||
('mbps', '(megabits / second)') | ('mbps', '(megabits / second)') | ||||
] | ] | ||||
for original, fix in fixes: | |||||
for original, fix in fixes: | |||||
query = re.sub(original, fix, query) | query = re.sub(original, fix, query) | ||||
return query.strip() | return query.strip() |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -20,36 +20,69 @@ | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||||
# SOFTWARE. | # SOFTWARE. | ||||
from earwigbot.classes import BaseCommand | |||||
from earwigbot.config import config | |||||
from earwigbot.commands import Command | |||||
class Command(BaseCommand): | |||||
"""Voice, devoice, op, or deop users in the channel.""" | |||||
class ChanOps(Command): | |||||
"""Voice, devoice, op, or deop users in the channel, or join or part from | |||||
other channels.""" | |||||
name = "chanops" | name = "chanops" | ||||
def check(self, data): | |||||
commands = ["chanops", "voice", "devoice", "op", "deop"] | |||||
if data.is_command and data.command in commands: | |||||
return True | |||||
return False | |||||
commands = ["chanops", "voice", "devoice", "op", "deop", "join", "part"] | |||||
def process(self, data): | def process(self, data): | ||||
if data.command == "chanops": | if data.command == "chanops": | ||||
msg = "available commands are !voice, !devoice, !op, and !deop." | |||||
self.connection.reply(data, msg) | |||||
msg = "available commands are !voice, !devoice, !op, !deop, !join, and !part." | |||||
self.reply(data, msg) | |||||
return | return | ||||
if data.host not in config.irc["permissions"]["admins"]: | |||||
msg = "you must be a bot admin to use this command." | |||||
self.connection.reply(data, msg) | |||||
if data.host not in self.config.irc["permissions"]["admins"]: | |||||
self.reply(data, "you must be a bot admin to use this command.") | |||||
return | return | ||||
# If it is just !op/!devoice/whatever without arguments, assume they | |||||
# want to do this to themselves: | |||||
if not data.args: | |||||
target = data.nick | |||||
if data.command == "join": | |||||
self.do_join(data) | |||||
elif data.command == "part": | |||||
self.do_part(data) | |||||
else: | else: | ||||
target = data.args[0] | |||||
# If it is just !op/!devoice/whatever without arguments, assume | |||||
# they want to do this to themselves: | |||||
if not data.args: | |||||
target = data.nick | |||||
else: | |||||
target = data.args[0] | |||||
command = data.command.upper() | |||||
self.say("ChanServ", " ".join((command, data.chan, target))) | |||||
log = "{0} requested {1} on {2} in {3}" | |||||
self.logger.info(log.format(data.nick, command, target, data.chan)) | |||||
def do_join(self, data): | |||||
if data.args: | |||||
channel = data.args[0] | |||||
if not channel.startswith("#"): | |||||
channel = "#" + channel | |||||
else: | |||||
msg = "you must specify a channel to join or part from." | |||||
self.reply(data, msg) | |||||
return | |||||
self.join(channel) | |||||
log = "{0} requested JOIN to {1}".format(data.nick, channel) | |||||
self.logger.info(log) | |||||
def do_part(self, data): | |||||
channel = data.chan | |||||
reason = None | |||||
if data.args: | |||||
if data.args[0].startswith("#"): | |||||
# "!part #channel reason for parting" | |||||
channel = data.args[0] | |||||
if data.args[1:]: | |||||
reason = " ".join(data.args[1:]) | |||||
else: # "!part reason for parting"; assume current channel | |||||
reason = " ".join(data.args) | |||||
msg = " ".join((data.command, data.chan, target)) | |||||
self.connection.say("ChanServ", msg) | |||||
msg = "Requested by {0}".format(data.nick) | |||||
log = "{0} requested PART from {1}".format(data.nick, channel) | |||||
if reason: | |||||
msg += ": {0}".format(reason) | |||||
log += ' ("{0}")'.format(reason) | |||||
self.part(channel, msg) | |||||
self.logger.info(log) |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -22,29 +22,25 @@ | |||||
import hashlib | import hashlib | ||||
from earwigbot.classes import BaseCommand | |||||
from earwigbot import blowfish | |||||
from Crypto.Cipher import Blowfish | |||||
class Command(BaseCommand): | |||||
from earwigbot.commands import Command | |||||
class Crypt(Command): | |||||
"""Provides hash functions with !hash (!hash list for supported algorithms) | """Provides hash functions with !hash (!hash list for supported algorithms) | ||||
and blowfish encryption with !encrypt and !decrypt.""" | |||||
and Blowfish encryption with !encrypt and !decrypt.""" | |||||
name = "crypt" | name = "crypt" | ||||
def check(self, data): | |||||
commands = ["crypt", "hash", "encrypt", "decrypt"] | |||||
if data.is_command and data.command in commands: | |||||
return True | |||||
return False | |||||
commands = ["crypt", "hash", "encrypt", "decrypt"] | |||||
def process(self, data): | def process(self, data): | ||||
if data.command == "crypt": | if data.command == "crypt": | ||||
msg = "available commands are !hash, !encrypt, and !decrypt." | msg = "available commands are !hash, !encrypt, and !decrypt." | ||||
self.connection.reply(data, msg) | |||||
self.reply(data, msg) | |||||
return | return | ||||
if not data.args: | if not data.args: | ||||
msg = "what do you want me to {0}?".format(data.command) | msg = "what do you want me to {0}?".format(data.command) | ||||
self.connection.reply(data, msg) | |||||
self.reply(data, msg) | |||||
return | return | ||||
if data.command == "hash": | if data.command == "hash": | ||||
@@ -52,29 +48,29 @@ class Command(BaseCommand): | |||||
if algo == "list": | if algo == "list": | ||||
algos = ', '.join(hashlib.algorithms) | algos = ', '.join(hashlib.algorithms) | ||||
msg = algos.join(("supported algorithms: ", ".")) | msg = algos.join(("supported algorithms: ", ".")) | ||||
self.connection.reply(data, msg) | |||||
self.reply(data, msg) | |||||
elif algo in hashlib.algorithms: | elif algo in hashlib.algorithms: | ||||
string = ' '.join(data.args[1:]) | string = ' '.join(data.args[1:]) | ||||
result = getattr(hashlib, algo)(string).hexdigest() | result = getattr(hashlib, algo)(string).hexdigest() | ||||
self.connection.reply(data, result) | |||||
self.reply(data, result) | |||||
else: | else: | ||||
msg = "unknown algorithm: '{0}'.".format(algo) | msg = "unknown algorithm: '{0}'.".format(algo) | ||||
self.connection.reply(data, msg) | |||||
self.reply(data, msg) | |||||
else: | else: | ||||
key = data.args[0] | key = data.args[0] | ||||
text = ' '.join(data.args[1:]) | |||||
text = " ".join(data.args[1:]) | |||||
if not text: | if not text: | ||||
msg = "a key was provided, but text to {0} was not." | msg = "a key was provided, but text to {0} was not." | ||||
self.connection.reply(data, msg.format(data.command)) | |||||
self.reply(data, msg.format(data.command)) | |||||
return | return | ||||
cipher = Blowfish.new(hashlib.sha256(key).digest()) | |||||
try: | try: | ||||
if data.command == "encrypt": | if data.command == "encrypt": | ||||
self.connection.reply(data, blowfish.encrypt(key, text)) | |||||
self.reply(data, cipher.encrypt(text).encode("hex")) | |||||
else: | else: | ||||
self.connection.reply(data, blowfish.decrypt(key, text)) | |||||
except blowfish.BlowfishError as error: | |||||
msg = "{0}: {1}.".format(error.__class__.__name__, error) | |||||
self.connection.reply(data, msg) | |||||
self.reply(data, cipher.decrypt(text.decode("hex"))) | |||||
except ValueError as error: | |||||
self.reply(data, error.message) |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -23,13 +23,12 @@ | |||||
import platform | import platform | ||||
import time | import time | ||||
import earwigbot | |||||
from earwigbot.classes import BaseCommand | |||||
from earwigbot.config import config | |||||
from earwigbot import __version__ | |||||
from earwigbot.commands import Command | |||||
class Command(BaseCommand): | |||||
"""Not an actual command, this module is used to respond to the CTCP | |||||
commands PING, TIME, and VERSION.""" | |||||
class CTCP(Command): | |||||
"""Not an actual command; this module implements responses to the CTCP | |||||
requests PING, TIME, and VERSION.""" | |||||
name = "ctcp" | name = "ctcp" | ||||
hooks = ["msg_private"] | hooks = ["msg_private"] | ||||
@@ -53,17 +52,17 @@ class Command(BaseCommand): | |||||
if command == "PING": | if command == "PING": | ||||
msg = " ".join(data.line[4:]) | msg = " ".join(data.line[4:]) | ||||
if msg: | if msg: | ||||
self.connection.notice(target, "\x01PING {0}\x01".format(msg)) | |||||
self.notice(target, "\x01PING {0}\x01".format(msg)) | |||||
else: | else: | ||||
self.connection.notice(target, "\x01PING\x01") | |||||
self.notice(target, "\x01PING\x01") | |||||
elif command == "TIME": | elif command == "TIME": | ||||
ts = time.strftime("%a, %d %b %Y %H:%M:%S %Z", time.localtime()) | ts = time.strftime("%a, %d %b %Y %H:%M:%S %Z", time.localtime()) | ||||
self.connection.notice(target, "\x01TIME {0}\x01".format(ts)) | |||||
self.notice(target, "\x01TIME {0}\x01".format(ts)) | |||||
elif command == "VERSION": | elif command == "VERSION": | ||||
default = "EarwigBot - $1 - Python/$2 https://github.com/earwig/earwigbot" | default = "EarwigBot - $1 - Python/$2 https://github.com/earwig/earwigbot" | ||||
vers = config.irc.get("version", default) | |||||
vers = vers.replace("$1", earwigbot.__version__) | |||||
vers = self.config.irc.get("version", default) | |||||
vers = vers.replace("$1", __version__) | |||||
vers = vers.replace("$2", platform.python_version()) | vers = vers.replace("$2", platform.python_version()) | ||||
self.connection.notice(target, "\x01VERSION {0}\x01".format(vers)) | |||||
self.notice(target, "\x01VERSION {0}\x01".format(vers)) |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -22,18 +22,13 @@ | |||||
from urllib import quote_plus | from urllib import quote_plus | ||||
from earwigbot.classes import BaseCommand | |||||
from earwigbot import wiki | |||||
from earwigbot import exceptions | |||||
from earwigbot.commands import Command | |||||
class Command(BaseCommand): | |||||
class Editcount(Command): | |||||
"""Return a user's edit count.""" | """Return a user's edit count.""" | ||||
name = "editcount" | name = "editcount" | ||||
def check(self, data): | |||||
commands = ["ec", "editcount"] | |||||
if data.is_command and data.command in commands: | |||||
return True | |||||
return False | |||||
commands = ["ec", "editcount"] | |||||
def process(self, data): | def process(self, data): | ||||
if not data.args: | if not data.args: | ||||
@@ -41,18 +36,18 @@ class Command(BaseCommand): | |||||
else: | else: | ||||
name = ' '.join(data.args) | name = ' '.join(data.args) | ||||
site = wiki.get_site() | |||||
site._maxlag = None | |||||
site = self.bot.wiki.get_site() | |||||
user = site.get_user(name) | user = site.get_user(name) | ||||
try: | try: | ||||
count = user.editcount() | |||||
except wiki.UserNotFoundError: | |||||
count = user.editcount | |||||
except exceptions.UserNotFoundError: | |||||
msg = "the user \x0302{0}\x0301 does not exist." | msg = "the user \x0302{0}\x0301 does not exist." | ||||
self.connection.reply(data, msg.format(name)) | |||||
self.reply(data, msg.format(name)) | |||||
return | return | ||||
safe = quote_plus(user.name()) | |||||
url = "http://toolserver.org/~tparis/pcount/index.php?name={0}&lang=en&wiki=wikipedia" | |||||
safe = quote_plus(user.name.encode("utf8")) | |||||
url = "http://toolserver.org/~tparis/pcount/index.php?name={0}&lang={1}&wiki={2}" | |||||
fullurl = url.format(safe, site.lang, site.project) | |||||
msg = "\x0302{0}\x0301 has {1} edits ({2})." | msg = "\x0302{0}\x0301 has {1} edits ({2})." | ||||
self.connection.reply(data, msg.format(name, count, url.format(safe))) | |||||
self.reply(data, msg.format(name, count, fullurl)) |
@@ -0,0 +1,72 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
import json | |||||
import urllib2 | |||||
from earwigbot.commands import Command | |||||
class Geolocate(Command): | |||||
"""Geolocate an IP address (via http://ipinfodb.com/).""" | |||||
name = "geolocate" | |||||
commands = ["geolocate", "locate", "geo", "ip"] | |||||
def setup(self): | |||||
self.config.decrypt(self.config.commands, self.name, "apiKey") | |||||
try: | |||||
self.key = self.config.commands[self.name]["apiKey"] | |||||
except KeyError: | |||||
self.key = None | |||||
log = 'Cannot use without an API key for http://ipinfodb.com/ stored as config.commands["{0}"]["apiKey"]' | |||||
self.logger.warn(log.format(self.name)) | |||||
def process(self, data): | |||||
if not data.args: | |||||
self.reply(data, "please specify an IP to lookup.") | |||||
return | |||||
if not self.key: | |||||
msg = 'I need an API key for http://ipinfodb.com/ stored as \x0303config.commands["{0}"]["apiKey"]\x0301.' | |||||
log = 'Need an API key for http://ipinfodb.com/ stored as config.commands["{0}"]["apiKey"]' | |||||
self.reply(data, msg.format(self.name) + ".") | |||||
self.logger.error(log.format(self.name)) | |||||
return | |||||
address = data.args[0] | |||||
url = "http://api.ipinfodb.com/v3/ip-city/?key={0}&ip={1}&format=json" | |||||
query = urllib2.urlopen(url.format(self.key, address)).read() | |||||
res = json.loads(query) | |||||
try: | |||||
country = res["countryName"].title() | |||||
region = res["regionName"].title() | |||||
city = res["cityName"].title() | |||||
latitude = res["latitude"] | |||||
longitude = res["longitude"] | |||||
utcoffset = res["timeZone"] | |||||
except KeyError: | |||||
self.reply(data, "IP \x0302{0}\x0301 not found.".format(address)) | |||||
return | |||||
msg = "{0}, {1}, {2} ({3}, {4}), UTC {5}" | |||||
geo = msg.format(country, region, city, latitude, longitude, utcoffset) | |||||
self.reply(data, geo) |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -20,61 +20,92 @@ | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||||
# SOFTWARE. | # SOFTWARE. | ||||
import shlex | |||||
import subprocess | |||||
import re | |||||
import time | |||||
import git | |||||
from earwigbot.classes import BaseCommand | |||||
from earwigbot.config import config | |||||
from earwigbot.commands import Command | |||||
class Command(BaseCommand): | |||||
class Git(Command): | |||||
"""Commands to interface with the bot's git repository; use '!git' for a | """Commands to interface with the bot's git repository; use '!git' for a | ||||
sub-command list.""" | sub-command list.""" | ||||
name = "git" | name = "git" | ||||
def setup(self): | |||||
try: | |||||
self.repos = self.config.commands[self.name]["repos"] | |||||
except KeyError: | |||||
self.repos = None | |||||
def process(self, data): | def process(self, data): | ||||
self.data = data | self.data = data | ||||
if data.host not in config.irc["permissions"]["owners"]: | |||||
if data.host not in self.config.irc["permissions"]["owners"]: | |||||
msg = "you must be a bot owner to use this command." | msg = "you must be a bot owner to use this command." | ||||
self.connection.reply(data, msg) | |||||
self.reply(data, msg) | |||||
return | return | ||||
if not data.args: | |||||
if not data.args or data.args[0] == "help": | |||||
self.do_help() | self.do_help() | ||||
return | return | ||||
if not self.repos: | |||||
self.reply(data, "no repos are specified in the config file.") | |||||
return | |||||
if data.args[0] == "help": | |||||
self.do_help() | |||||
command = data.args[0] | |||||
try: | |||||
repo_name = data.args[1] | |||||
except IndexError: | |||||
repos = self.get_repos() | |||||
msg = "which repo do you want to work with (options are {0})?" | |||||
self.reply(data, msg.format(repos)) | |||||
return | |||||
if repo_name not in self.repos: | |||||
repos = self.get_repos() | |||||
msg = "repository must be one of the following: {0}." | |||||
self.reply(data, msg.format(repos)) | |||||
return | |||||
self.repo = git.Repo(self.repos[repo_name]) | |||||
elif data.args[0] == "branch": | |||||
if command == "branch": | |||||
self.do_branch() | self.do_branch() | ||||
elif data.args[0] == "branches": | |||||
elif command == "branches": | |||||
self.do_branches() | self.do_branches() | ||||
elif data.args[0] == "checkout": | |||||
elif command == "checkout": | |||||
self.do_checkout() | self.do_checkout() | ||||
elif data.args[0] == "delete": | |||||
elif command == "delete": | |||||
self.do_delete() | self.do_delete() | ||||
elif data.args[0] == "pull": | |||||
elif command == "pull": | |||||
self.do_pull() | self.do_pull() | ||||
elif data.args[0] == "status": | |||||
elif command == "status": | |||||
self.do_status() | self.do_status() | ||||
else: # They asked us to do something we don't know | else: # They asked us to do something we don't know | ||||
msg = "unknown argument: \x0303{0}\x0301.".format(data.args[0]) | msg = "unknown argument: \x0303{0}\x0301.".format(data.args[0]) | ||||
self.connection.reply(data, msg) | |||||
self.reply(data, msg) | |||||
def exec_shell(self, command): | |||||
"""Execute a shell command and get the output.""" | |||||
command = shlex.split(command) | |||||
result = subprocess.check_output(command, stderr=subprocess.STDOUT) | |||||
if result: | |||||
result = result[:-1] # Strip newline | |||||
return result | |||||
def get_repos(self): | |||||
data = self.repos.iteritems() | |||||
repos = ["\x0302{0}\x0301 ({1})".format(k, v) for k, v in data] | |||||
return ", ".join(repos) | |||||
def get_remote(self): | |||||
try: | |||||
remote_name = self.data.args[2] | |||||
except IndexError: | |||||
remote_name = "origin" | |||||
try: | |||||
return getattr(self.repo.remotes, remote_name) | |||||
except AttributeError: | |||||
msg = "unknown remote: \x0302{0}\x0301.".format(remote_name) | |||||
self.reply(self.data, msg) | |||||
def get_time_since(self, date): | |||||
diff = time.mktime(time.gmtime()) - date | |||||
if diff < 60: | |||||
return "{0} seconds".format(int(diff)) | |||||
if diff < 60 * 60: | |||||
return "{0} minutes".format(int(diff / 60)) | |||||
if diff < 60 * 60 * 24: | |||||
return "{0} hours".format(int(diff / 60 / 60)) | |||||
return "{0} days".format(int(diff / 60 / 60 / 24)) | |||||
def do_help(self): | def do_help(self): | ||||
"""Display all commands.""" | """Display all commands.""" | ||||
@@ -86,110 +117,124 @@ class Command(BaseCommand): | |||||
"pull": "update everything from the remote server", | "pull": "update everything from the remote server", | ||||
"status": "check if we are up-to-date", | "status": "check if we are up-to-date", | ||||
} | } | ||||
msg = "" | |||||
subcommands = "" | |||||
for key in sorted(help.keys()): | for key in sorted(help.keys()): | ||||
msg += "\x0303{0}\x0301 ({1}), ".format(key, help[key]) | |||||
msg = msg[:-2] # Trim last comma and space | |||||
self.connection.reply(self.data, "sub-commands are: {0}.".format(msg)) | |||||
subcommands += "\x0303{0}\x0301 ({1}), ".format(key, help[key]) | |||||
subcommands = subcommands[:-2] # Trim last comma and space | |||||
msg = "sub-commands are: {0}; repos are: {1}. Syntax: !git \x0303subcommand\x0301 \x0302repo\x0301." | |||||
self.reply(self.data, msg.format(subcommands, self.get_repos())) | |||||
def do_branch(self): | def do_branch(self): | ||||
"""Get our current branch.""" | """Get our current branch.""" | ||||
branch = self.exec_shell("git name-rev --name-only HEAD") | |||||
branch = self.repo.active_branch.name | |||||
msg = "currently on branch \x0302{0}\x0301.".format(branch) | msg = "currently on branch \x0302{0}\x0301.".format(branch) | ||||
self.connection.reply(self.data, msg) | |||||
self.reply(self.data, msg) | |||||
def do_branches(self): | def do_branches(self): | ||||
"""Get a list of branches.""" | """Get a list of branches.""" | ||||
branches = self.exec_shell("git branch") | |||||
# Remove extraneous characters: | |||||
branches = branches.replace('\n* ', ', ') | |||||
branches = branches.replace('* ', ' ') | |||||
branches = branches.replace('\n ', ', ') | |||||
branches = branches.strip() | |||||
msg = "branches: \x0302{0}\x0301.".format(branches) | |||||
self.connection.reply(self.data, msg) | |||||
branches = [branch.name for branch in self.repo.branches] | |||||
msg = "branches: \x0302{0}\x0301.".format(", ".join(branches)) | |||||
self.reply(self.data, msg) | |||||
def do_checkout(self): | def do_checkout(self): | ||||
"""Switch branches.""" | """Switch branches.""" | ||||
try: | try: | ||||
branch = self.data.args[1] | |||||
except IndexError: # no branch name provided | |||||
self.connection.reply(self.data, "switch to which branch?") | |||||
target = self.data.args[2] | |||||
except IndexError: # No branch name provided | |||||
self.reply(self.data, "switch to which branch?") | |||||
return | return | ||||
current_branch = self.exec_shell("git name-rev --name-only HEAD") | |||||
current_branch = self.repo.active_branch.name | |||||
if target == current_branch: | |||||
msg = "already on \x0302{0}\x0301!".format(target) | |||||
self.reply(self.data, msg) | |||||
return | |||||
try: | try: | ||||
result = self.exec_shell("git checkout %s" % branch) | |||||
if "Already on" in result: | |||||
msg = "already on \x0302{0}\x0301!".format(branch) | |||||
self.connection.reply(self.data, msg) | |||||
else: | |||||
ms = "switched from branch \x0302{1}\x0301 to \x0302{1}\x0301." | |||||
msg = ms.format(current_branch, branch) | |||||
self.connection.reply(self.data, msg) | |||||
except subprocess.CalledProcessError: | |||||
# Git couldn't switch branches; assume the branch doesn't exist: | |||||
msg = "branch \x0302{0}\x0301 doesn't exist!".format(branch) | |||||
self.connection.reply(self.data, msg) | |||||
ref = getattr(self.repo.branches, target) | |||||
except AttributeError: | |||||
msg = "branch \x0302{0}\x0301 doesn't exist!".format(target) | |||||
self.reply(self.data, msg) | |||||
else: | |||||
ref.checkout() | |||||
ms = "switched from branch \x0302{0}\x0301 to \x0302{1}\x0301." | |||||
msg = ms.format(current_branch, target) | |||||
self.reply(self.data, msg) | |||||
log = "{0} checked out branch {1} of {2}" | |||||
logmsg = log.format(self.data.nick, target, self.repo.working_dir) | |||||
self.logger.info(logmsg) | |||||
def do_delete(self): | def do_delete(self): | ||||
"""Delete a branch, while making sure that we are not already on it.""" | """Delete a branch, while making sure that we are not already on it.""" | ||||
try: | try: | ||||
delete_branch = self.data.args[1] | |||||
except IndexError: # no branch name provided | |||||
self.connection.reply(self.data, "delete which branch?") | |||||
target = self.data.args[2] | |||||
except IndexError: # No branch name provided | |||||
self.reply(self.data, "delete which branch?") | |||||
return | return | ||||
current_branch = self.exec_shell("git name-rev --name-only HEAD") | |||||
if current_branch == delete_branch: | |||||
current_branch = self.repo.active_branch.name | |||||
if current_branch == target: | |||||
msg = "you're currently on this branch; please checkout to a different branch before deleting." | msg = "you're currently on this branch; please checkout to a different branch before deleting." | ||||
self.connection.reply(self.data, msg) | |||||
self.reply(self.data, msg) | |||||
return | return | ||||
try: | try: | ||||
self.exec_shell("git branch -d %s" % delete_branch) | |||||
ref = getattr(self.repo.branches, target) | |||||
except AttributeError: | |||||
msg = "branch \x0302{0}\x0301 doesn't exist!".format(target) | |||||
self.reply(self.data, msg) | |||||
else: | |||||
self.repo.git.branch("-d", ref) | |||||
msg = "branch \x0302{0}\x0301 has been deleted locally." | msg = "branch \x0302{0}\x0301 has been deleted locally." | ||||
self.connection.reply(self.data, msg.format(delete_branch)) | |||||
except subprocess.CalledProcessError: | |||||
# Git couldn't switch branches; assume the branch doesn't exist: | |||||
msg = "branch \x0302{0}\x0301 doesn't exist!".format(delete_branch) | |||||
self.connection.reply(self.data, msg) | |||||
self.reply(self.data, msg.format(target)) | |||||
log = "{0} deleted branch {1} of {2}" | |||||
logmsg = log.format(self.data.nick, target, self.repo.working_dir) | |||||
self.logger.info(logmsg) | |||||
def do_pull(self): | def do_pull(self): | ||||
"""Pull from our remote repository.""" | """Pull from our remote repository.""" | ||||
branch = self.exec_shell("git name-rev --name-only HEAD") | |||||
branch = self.repo.active_branch.name | |||||
msg = "pulling from remote (currently on \x0302{0}\x0301)..." | msg = "pulling from remote (currently on \x0302{0}\x0301)..." | ||||
self.connection.reply(self.data, msg.format(branch)) | |||||
self.reply(self.data, msg.format(branch)) | |||||
result = self.exec_shell("git pull") | |||||
if "Already up-to-date." in result: | |||||
self.connection.reply(self.data, "done; no new changes.") | |||||
remote = self.get_remote() | |||||
if not remote: | |||||
return | |||||
result = remote.pull() | |||||
updated = [info for info in result if info.flags != info.HEAD_UPTODATE] | |||||
if updated: | |||||
branches = ", ".join([info.ref.remote_head for info in updated]) | |||||
msg = "done; updates to \x0302{0}\x0301 (from {1})." | |||||
self.reply(self.data, msg.format(branches, remote.url)) | |||||
log = "{0} pulled {1} of {2} (updates to {3})" | |||||
self.logger.info(log.format(self.data.nick, remote.name, | |||||
self.repo.working_dir, branches)) | |||||
else: | else: | ||||
regex = "\s*((.*?)\sfile(.*?)tions?\(-\))" | |||||
changes = re.findall(regex, result)[0][0] | |||||
try: | |||||
cmnd_remt = "git config --get branch.{0}.remote".format(branch) | |||||
remote = self.exec_shell(cmnd_remt) | |||||
cmnd_url = "git config --get remote.{0}.url".format(remote) | |||||
url = self.exec_shell(cmnd_url) | |||||
msg = "done; {0} [from {1}].".format(changes, url) | |||||
self.connection.reply(self.data, msg) | |||||
except subprocess.CalledProcessError: | |||||
# Something in .git/config is not specified correctly, so we | |||||
# cannot get the remote's URL. However, pull was a success: | |||||
self.connection.reply(self.data, "done; %s." % changes) | |||||
self.reply(self.data, "done; no new changes.") | |||||
log = "{0} pulled {1} of {2} (no updates)" | |||||
self.logger.info(log.format(self.data.nick, remote.name, | |||||
self.repo.working_dir)) | |||||
def do_status(self): | def do_status(self): | ||||
"""Check whether we have anything to pull.""" | |||||
last = self.exec_shell('git log -n 1 --pretty="%ar"') | |||||
result = self.exec_shell("git fetch --dry-run") | |||||
if not result: # Nothing was fetched, so remote and local are equal | |||||
msg = "last commit was {0}. Local copy is \x02up-to-date\x0F with remote." | |||||
self.connection.reply(self.data, msg.format(last)) | |||||
"""Check if we have anything to pull.""" | |||||
remote = self.get_remote() | |||||
if not remote: | |||||
return | |||||
since = self.get_time_since(self.repo.head.object.committed_date) | |||||
result = remote.fetch(dry_run=True) | |||||
updated = [info for info in result if info.flags != info.HEAD_UPTODATE] | |||||
if updated: | |||||
branches = ", ".join([info.ref.remote_head for info in updated]) | |||||
msg = "last local commit was \x02{0}\x0F ago; updates to \x0302{1}\x0301." | |||||
self.reply(self.data, msg.format(since, branches)) | |||||
log = "{0} got status of {1} of {2} (updates to {3})" | |||||
self.logger.info(log.format(self.data.nick, remote.name, | |||||
self.repo.working_dir, branches)) | |||||
else: | else: | ||||
msg = "last local commit was {0}. Remote is \x02ahead\x0F of local copy." | |||||
self.connection.reply(self.data, msg.format(last)) | |||||
msg = "last commit was \x02{0}\x0F ago. Local copy is up-to-date with remote." | |||||
self.reply(self.data, msg.format(since)) | |||||
log = "{0} pulled {1} of {2} (no updates)" | |||||
self.logger.info(log.format(self.data.nick, remote.name, | |||||
self.repo.working_dir)) |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -22,47 +22,50 @@ | |||||
import re | import re | ||||
from earwigbot.classes import BaseCommand, Data | |||||
from earwigbot import commands | |||||
from earwigbot.commands import Command | |||||
class Command(BaseCommand): | |||||
class Help(Command): | |||||
"""Displays help information.""" | """Displays help information.""" | ||||
name = "help" | name = "help" | ||||
def check(self, data): | |||||
if data.is_command: | |||||
if data.command == "help": | |||||
return True | |||||
if not data.command and data.trigger == data.my_nick: | |||||
return True | |||||
return False | |||||
def process(self, data): | def process(self, data): | ||||
self.cmnds = commands.get_all() | |||||
if not data.args: | |||||
self.do_main_help(data) | |||||
else: | |||||
if not data.command: | |||||
self.do_hello(data) | |||||
elif data.args: | |||||
self.do_command_help(data) | self.do_command_help(data) | ||||
else: | |||||
self.do_main_help(data) | |||||
def do_main_help(self, data): | def do_main_help(self, data): | ||||
"""Give the user a general help message with a list of all commands.""" | """Give the user a general help message with a list of all commands.""" | ||||
msg = "Hi, I'm a bot! I have {0} commands loaded: {1}. You can get help for any command with '!help <command>'." | msg = "Hi, I'm a bot! I have {0} commands loaded: {1}. You can get help for any command with '!help <command>'." | ||||
cmnds = sorted(self.cmnds.keys()) | |||||
cmnds = sorted([cmnd.name for cmnd in self.bot.commands]) | |||||
msg = msg.format(len(cmnds), ', '.join(cmnds)) | msg = msg.format(len(cmnds), ', '.join(cmnds)) | ||||
self.connection.reply(data, msg) | |||||
self.reply(data, msg) | |||||
def do_command_help(self, data): | def do_command_help(self, data): | ||||
"""Give the user help for a specific command.""" | """Give the user help for a specific command.""" | ||||
command = data.args[0] | |||||
target = data.args[0] | |||||
# Create a dummy message to test which commands pick up the user's | |||||
# input: | |||||
dummy = Data(":foo!bar@example.com PRIVMSG #channel :msg!".split()) | |||||
dummy.command = command.lower() | |||||
dummy.is_command = True | |||||
for command in self.bot.commands: | |||||
if command.name == target or target in command.commands: | |||||
if command.__doc__: | |||||
doc = command.__doc__.replace("\n", "") | |||||
doc = re.sub("\s\s+", " ", doc) | |||||
msg = 'help for command \x0303{0}\x0301: "{1}"' | |||||
self.reply(data, msg.format(target, doc)) | |||||
return | |||||
for cmnd in self.cmnds.values(): | |||||
if not cmnd.check(dummy): | |||||
continue | |||||
if cmnd.__doc__: | |||||
doc = cmnd.__doc__.replace("\n", "") | |||||
doc = re.sub("\s\s+", " ", doc) | |||||
msg = "info for command \x0303{0}\x0301: \"{1}\"" | |||||
self.connection.reply(data, msg.format(command, doc)) | |||||
return | |||||
break | |||||
msg = "sorry, no help for \x0303{0}\x0301.".format(target) | |||||
self.reply(data, msg) | |||||
msg = "sorry, no help for \x0303{0}\x0301.".format(command) | |||||
self.connection.reply(data, msg) | |||||
def do_hello(self, data): | |||||
self.say(data.chan, "Yes, {0}?".format(data.nick)) |
@@ -0,0 +1,53 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
from earwigbot.commands import Command | |||||
class Langcode(Command): | |||||
"""Convert a language code into its name and a list of WMF sites in that | |||||
language.""" | |||||
name = "langcode" | |||||
commands = ["langcode", "lang", "language"] | |||||
def process(self, data): | |||||
if not data.args: | |||||
self.reply(data, "please specify a language code.") | |||||
return | |||||
code = data.args[0] | |||||
site = self.bot.wiki.get_site() | |||||
matrix = site.api_query(action="sitematrix")["sitematrix"] | |||||
del matrix["count"] | |||||
del matrix["specials"] | |||||
for site in matrix.itervalues(): | |||||
if site["code"] == code: | |||||
name = site["name"].encode("utf8") | |||||
localname = site["localname"].encode("utf8") | |||||
if name != localname: | |||||
name += " ({0})".format(localname) | |||||
sites = ", ".join([s["url"] for s in site["site"]]) | |||||
msg = "\x0302{0}\x0301 is {1} ({2})".format(code, name, sites) | |||||
self.reply(data, msg) | |||||
return | |||||
self.reply(data, "language \x0302{0}\x0301 not found.".format(code)) |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -23,35 +23,27 @@ | |||||
import re | import re | ||||
from urllib import quote | from urllib import quote | ||||
from earwigbot.classes import BaseCommand | |||||
from earwigbot.commands import Command | |||||
class Command(BaseCommand): | |||||
class Link(Command): | |||||
"""Convert a Wikipedia page name into a URL.""" | """Convert a Wikipedia page name into a URL.""" | ||||
name = "link" | name = "link" | ||||
def check(self, data): | |||||
# if ((data.is_command and data.command == "link") or | |||||
# (("[[" in data.msg and "]]" in data.msg) or | |||||
# ("{{" in data.msg and "}}" in data.msg))): | |||||
if data.is_command and data.command == "link": | |||||
return True | |||||
return False | |||||
def process(self, data): | def process(self, data): | ||||
msg = data.msg | msg = data.msg | ||||
if re.search("(\[\[(.*?)\]\])|(\{\{(.*?)\}\})", msg): | if re.search("(\[\[(.*?)\]\])|(\{\{(.*?)\}\})", msg): | ||||
links = self.parse_line(msg) | links = self.parse_line(msg) | ||||
links = " , ".join(links) | links = " , ".join(links) | ||||
self.connection.reply(data, links) | |||||
self.reply(data, links) | |||||
elif data.command == "link": | elif data.command == "link": | ||||
if not data.args: | if not data.args: | ||||
self.connection.reply(data, "what do you want me to link to?") | |||||
self.reply(data, "what do you want me to link to?") | |||||
return | return | ||||
pagename = ' '.join(data.args) | pagename = ' '.join(data.args) | ||||
link = self.parse_link(pagename) | link = self.parse_link(pagename) | ||||
self.connection.reply(data, link) | |||||
self.reply(data, link) | |||||
def parse_line(self, line): | def parse_line(self, line): | ||||
results = [] | results = [] | ||||
@@ -0,0 +1,167 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
from earwigbot.commands import Command | |||||
class Notes(Command): | |||||
"""A mini IRC-based wiki for storing notes, tips, and reminders.""" | |||||
name = "notes" | |||||
def process(self, data): | |||||
pass | |||||
class OldCommand(object): | |||||
def parse(self): | |||||
if command == "notes" or command == "note" or command == "about" or command == "data" or command == "database": | |||||
try: | |||||
action = line2[4] | |||||
except BaseException: | |||||
reply("What do you want me to do? Type \"!notes help\" for more information.", chan, nick) | |||||
return | |||||
import MySQLdb | |||||
db = MySQLdb.connect(db="u_earwig_ircbot", host="sql", read_default_file="/home/earwig/.my.cnf") | |||||
specify = ' '.join(line2[5:]) | |||||
if action == "help" or action == "manual": | |||||
shortCommandList = "read, write, change, undo, delete, move, author, category, list, report, developer" | |||||
if specify == "read": | |||||
say("To read an entry, type \"!notes read <entry>\".", chan) | |||||
elif specify == "write": | |||||
say("To write a new entry, type \"!notes write <entry> <content>\". This will create a new entry only if one does not exist, see the below command...", chan) | |||||
elif specify == "change": | |||||
say("To change an entry, type \"!notes change <entry> <new content>\". The old entry will be stored in the database, so it can be undone later.", chan) | |||||
elif specify == "undo": | |||||
say("To undo a change, type \"!notes undo <entry>\".", chan) | |||||
elif specify == "delete": | |||||
say("To delete an entry, type \"!notes delete <entry>\". For security reasons, only bot admins can do this.", chan) | |||||
elif specify == "move": | |||||
say("To move an entry, type \"!notes move <old_title> <new_title>\".", chan) | |||||
elif specify == "author": | |||||
say("To return the author of an entry, type \"!notes author <entry>\".", chan) | |||||
elif specify == "category" or specify == "cat": | |||||
say("To change an entry's category, type \"!notes category <entry> <category>\".", chan) | |||||
elif specify == "list": | |||||
say("To list all categories in the database, type \"!notes list\". Type \"!notes list <category>\" to get all entries in a certain category.", chan) | |||||
elif specify == "report": | |||||
say("To give some statistics about the mini-wiki, including some debugging information, type \"!notes report\" in a PM.", chan) | |||||
elif specify == "developer": | |||||
say("To do developer work, such as writing to the database directly, type \"!notes developer <command>\". This can only be done by the bot owner.", chan) | |||||
else: | |||||
db.query("SELECT * FROM version;") | |||||
r = db.use_result() | |||||
data = r.fetch_row(0) | |||||
version = data[0] | |||||
reply("The Earwig Mini-Wiki: running v%s." % version, chan, nick) | |||||
reply("The full list of commands, for reference, are: %s." % shortCommandList, chan, nick) | |||||
reply("For an explaination of a certain command, type \"!notes help <command>\".", chan, nick) | |||||
reply("You can also access the database from the Toolserver: http://toolserver.org/~earwig/cgi-bin/irc_database.py", chan, nick) | |||||
time.sleep(0.4) | |||||
return | |||||
elif action == "read": | |||||
specify = string.lower(specify) | |||||
if " " in specify: specify = string.split(specify, " ")[0] | |||||
if not specify or "\"" in specify: | |||||
reply("Please include the name of the entry you would like to read after the command, e.g. !notes read earwig", chan, nick) | |||||
return | |||||
try: | |||||
db.query("SELECT entry_content FROM entries WHERE entry_title = \"%s\";" % specify) | |||||
r = db.use_result() | |||||
data = r.fetch_row(0) | |||||
entry = data[0][0] | |||||
say("Entry \"\x02%s\x0F\": %s" % (specify, entry), chan) | |||||
except Exception: | |||||
reply("There is no entry titled \"\x02%s\x0F\"." % specify, chan, nick) | |||||
return | |||||
elif action == "delete" or action == "remove": | |||||
specify = string.lower(specify) | |||||
if " " in specify: specify = string.split(specify, " ")[0] | |||||
if not specify or "\"" in specify: | |||||
reply("Please include the name of the entry you would like to delete after the command, e.g. !notes delete earwig", chan, nick) | |||||
return | |||||
if authy == "owner" or authy == "admin": | |||||
try: | |||||
db.query("DELETE from entries where entry_title = \"%s\";" % specify) | |||||
r = db.use_result() | |||||
db.commit() | |||||
reply("The entry on \"\x02%s\x0F\" has been removed." % specify, chan, nick) | |||||
except Exception: | |||||
phenny.reply("Unable to remove the entry on \"\x02%s\x0F\", because it doesn't exist." % specify, chan, nick) | |||||
else: | |||||
reply("Only bot admins can remove entries.", chan, nick) | |||||
return | |||||
elif action == "developer": | |||||
if authy == "owner": | |||||
db.query(specify) | |||||
r = db.use_result() | |||||
try: | |||||
print r.fetch_row(0) | |||||
except Exception: | |||||
pass | |||||
db.commit() | |||||
reply("Done.", chan, nick) | |||||
else: | |||||
reply("Only the bot owner can modify the raw database.", chan, nick) | |||||
return | |||||
elif action == "write": | |||||
try: | |||||
write = line2[5] | |||||
content = ' '.join(line2[6:]) | |||||
except Exception: | |||||
reply("Please include some content in your entry.", chan, nick) | |||||
return | |||||
db.query("SELECT * from entries WHERE entry_title = \"%s\";" % write) | |||||
r = db.use_result() | |||||
data = r.fetch_row(0) | |||||
if data: | |||||
reply("An entry on %s already exists; please use \"!notes change %s %s\"." % (write, write, content), chan, nick) | |||||
return | |||||
content2 = content.replace('"', '\\' + '"') | |||||
db.query("INSERT INTO entries (entry_title, entry_author, entry_category, entry_content, entry_content_old) VALUES (\"%s\", \"%s\", \"uncategorized\", \"%s\", NULL);" % (write, nick, content2)) | |||||
db.commit() | |||||
reply("You have written an entry titled \"\x02%s\x0F\", with the following content: \"%s\"" % (write, content), chan, nick) | |||||
return | |||||
elif action == "change": | |||||
reply("NotImplementedError", chan, nick) | |||||
elif action == "undo": | |||||
reply("NotImplementedError", chan, nick) | |||||
elif action == "move": | |||||
reply("NotImplementedError", chan, nick) | |||||
elif action == "author": | |||||
try: | |||||
entry = line2[5] | |||||
except Exception: | |||||
reply("Please include the name of the entry you would like to get information for after the command, e.g. !notes author earwig", chan, nick) | |||||
return | |||||
db.query("SELECT entry_author from entries WHERE entry_title = \"%s\";" % entry) | |||||
r = db.use_result() | |||||
data = r.fetch_row(0) | |||||
if data: | |||||
say("The author of \"\x02%s\x0F\" is \x02%s\x0F." % (entry, data[0][0]), chan) | |||||
return | |||||
reply("There is no entry titled \"\x02%s\x0F\"." % entry, chan, nick) | |||||
return | |||||
elif action == "cat" or action == "category": | |||||
reply("NotImplementedError", chan, nick) | |||||
elif action == "list": | |||||
reply("NotImplementedError", chan, nick) | |||||
elif action == "report": | |||||
reply("NotImplementedError", chan, nick) |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -20,32 +20,29 @@ | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||||
# SOFTWARE. | # SOFTWARE. | ||||
import random | |||||
from earwigbot.classes import BaseCommand | |||||
from earwigbot.commands import Command | |||||
class Command(BaseCommand): | |||||
class Praise(Command): | |||||
"""Praise people!""" | """Praise people!""" | ||||
name = "praise" | name = "praise" | ||||
def setup(self): | |||||
try: | |||||
self.praises = self.config.commands[self.name]["praises"] | |||||
except KeyError: | |||||
self.praises = [] | |||||
def check(self, data): | def check(self, data): | ||||
commands = ["praise", "earwig", "leonard", "leonard^bloom", "groove", | |||||
"groovedog"] | |||||
return data.is_command and data.command in commands | |||||
check = data.command == "praise" or data.command in self.praises | |||||
return data.is_command and check | |||||
def process(self, data): | def process(self, data): | ||||
if data.command == "earwig": | |||||
msg = "\x02Earwig\x0F is the bestest Python programmer ever!" | |||||
elif data.command in ["leonard", "leonard^bloom"]: | |||||
msg = "\x02Leonard^Bloom\x0F is the biggest slacker ever!" | |||||
elif data.command in ["groove", "groovedog"]: | |||||
msg = "\x02GrooveDog\x0F is the bestest heh evar!" | |||||
else: | |||||
if not data.args: | |||||
msg = "You use this command to praise certain people. Who they are is a secret." | |||||
else: | |||||
msg = "You're doing it wrong." | |||||
self.connection.reply(data, msg) | |||||
if data.command in self.praises: | |||||
msg = self.praises[data.command] | |||||
self.say(data.chan, msg) | |||||
return | return | ||||
self.connection.say(data.chan, msg) | |||||
if not data.args: | |||||
msg = "You use this command to praise certain people. Who they are is a secret." | |||||
else: | |||||
msg = "you're doing it wrong." | |||||
self.reply(data, msg) |
@@ -0,0 +1,68 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
from earwigbot.commands import Command | |||||
class Quit(Command): | |||||
"""Quit, restart, or reload components from the bot. Only the owners can | |||||
run this command.""" | |||||
name = "quit" | |||||
commands = ["quit", "restart", "reload"] | |||||
def process(self, data): | |||||
if data.host not in self.config.irc["permissions"]["owners"]: | |||||
self.reply(data, "you must be a bot owner to use this command.") | |||||
return | |||||
if data.command == "quit": | |||||
self.do_quit(data) | |||||
elif data.command == "restart": | |||||
self.do_restart(data) | |||||
else: | |||||
self.do_reload(data) | |||||
def do_quit(self, data): | |||||
args = data.args | |||||
if data.trigger == data.my_nick: | |||||
reason = " ".join(args) | |||||
else: | |||||
if not args or args[0].lower() != data.my_nick: | |||||
self.reply(data, "to confirm this action, the first argument must be my name.") | |||||
return | |||||
reason = " ".join(args[1:]) | |||||
if reason: | |||||
self.bot.stop("Stopped by {0}: {1}".format(data.nick, reason)) | |||||
else: | |||||
self.bot.stop("Stopped by {0}".format(data.nick)) | |||||
def do_restart(self, data): | |||||
if data.args: | |||||
msg = " ".join(data.args) | |||||
self.bot.restart("Restarted by {0}: {1}".format(data.nick, msg)) | |||||
else: | |||||
self.bot.restart("Restarted by {0}".format(data.nick)) | |||||
def do_reload(self, data): | |||||
self.logger.info("{0} requested command/task reload".format(data.nick)) | |||||
self.bot.commands.load() | |||||
self.bot.tasks.load() | |||||
self.reply(data, "IRC commands and bot tasks reloaded.") |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -22,18 +22,13 @@ | |||||
import time | import time | ||||
from earwigbot.classes import BaseCommand | |||||
from earwigbot import wiki | |||||
from earwigbot import exceptions | |||||
from earwigbot.commands import Command | |||||
class Command(BaseCommand): | |||||
class Registration(Command): | |||||
"""Return when a user registered.""" | """Return when a user registered.""" | ||||
name = "registration" | name = "registration" | ||||
def check(self, data): | |||||
commands = ["registration", "age"] | |||||
if data.is_command and data.command in commands: | |||||
return True | |||||
return False | |||||
commands = ["registration", "reg", "age"] | |||||
def process(self, data): | def process(self, data): | ||||
if not data.args: | if not data.args: | ||||
@@ -41,30 +36,28 @@ class Command(BaseCommand): | |||||
else: | else: | ||||
name = ' '.join(data.args) | name = ' '.join(data.args) | ||||
site = wiki.get_site() | |||||
site._maxlag = None | |||||
site = self.bot.wiki.get_site() | |||||
user = site.get_user(name) | user = site.get_user(name) | ||||
try: | try: | ||||
reg = user.registration() | |||||
except wiki.UserNotFoundError: | |||||
reg = user.registration | |||||
except exceptions.UserNotFoundError: | |||||
msg = "the user \x0302{0}\x0301 does not exist." | msg = "the user \x0302{0}\x0301 does not exist." | ||||
self.connection.reply(data, msg.format(name)) | |||||
self.reply(data, msg.format(name)) | |||||
return | return | ||||
date = time.strftime("%b %d, %Y at %H:%M:%S UTC", reg) | date = time.strftime("%b %d, %Y at %H:%M:%S UTC", reg) | ||||
age = self.get_diff(time.mktime(reg), time.mktime(time.gmtime())) | age = self.get_diff(time.mktime(reg), time.mktime(time.gmtime())) | ||||
g = user.gender() | |||||
if g == "male": | |||||
if user.gender == "male": | |||||
gender = "He's" | gender = "He's" | ||||
elif g == "female": | |||||
elif user.gender == "female": | |||||
gender = "She's" | gender = "She's" | ||||
else: | else: | ||||
gender = "They're" | |||||
gender = "They're" # Singluar they? | |||||
msg = "\x0302{0}\x0301 registered on {1}. {2} {3} old." | msg = "\x0302{0}\x0301 registered on {1}. {2} {3} old." | ||||
self.connection.reply(data, msg.format(name, date, gender, age)) | |||||
self.reply(data, msg.format(name, date, gender, age)) | |||||
def get_diff(self, t1, t2): | def get_diff(self, t1, t2): | ||||
parts = {"years": 31536000, "days": 86400, "hours": 3600, | parts = {"years": 31536000, "days": 86400, "hours": 3600, | ||||
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -20,36 +20,32 @@ | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||||
# SOFTWARE. | # SOFTWARE. | ||||
import threading | |||||
from threading import Timer | |||||
import time | import time | ||||
from earwigbot.classes import BaseCommand | |||||
from earwigbot.commands import Command | |||||
class Command(BaseCommand): | |||||
class Remind(Command): | |||||
"""Set a message to be repeated to you in a certain amount of time.""" | """Set a message to be repeated to you in a certain amount of time.""" | ||||
name = "remind" | name = "remind" | ||||
def check(self, data): | |||||
if data.is_command and data.command in ["remind", "reminder"]: | |||||
return True | |||||
return False | |||||
commands = ["remind", "reminder"] | |||||
def process(self, data): | def process(self, data): | ||||
if not data.args: | if not data.args: | ||||
msg = "please specify a time (in seconds) and a message in the following format: !remind <time> <msg>." | msg = "please specify a time (in seconds) and a message in the following format: !remind <time> <msg>." | ||||
self.connection.reply(data, msg) | |||||
self.reply(data, msg) | |||||
return | return | ||||
try: | try: | ||||
wait = int(data.args[0]) | wait = int(data.args[0]) | ||||
except ValueError: | except ValueError: | ||||
msg = "the time must be given as an integer, in seconds." | msg = "the time must be given as an integer, in seconds." | ||||
self.connection.reply(data, msg) | |||||
self.reply(data, msg) | |||||
return | return | ||||
message = ' '.join(data.args[1:]) | message = ' '.join(data.args[1:]) | ||||
if not message: | if not message: | ||||
msg = "what message do you want me to give you when time is up?" | msg = "what message do you want me to give you when time is up?" | ||||
self.connection.reply(data, msg) | |||||
self.reply(data, msg) | |||||
return | return | ||||
end = time.localtime(time.time() + wait) | end = time.localtime(time.time() + wait) | ||||
@@ -58,14 +54,9 @@ class Command(BaseCommand): | |||||
msg = 'Set reminder for "{0}" in {1} seconds (ends {2}).' | msg = 'Set reminder for "{0}" in {1} seconds (ends {2}).' | ||||
msg = msg.format(message, wait, end_time_with_timezone) | msg = msg.format(message, wait, end_time_with_timezone) | ||||
self.connection.reply(data, msg) | |||||
self.reply(data, msg) | |||||
t_reminder = threading.Thread(target=self.reminder, | |||||
args=(data, message, wait)) | |||||
t_reminder = Timer(wait, self.reply, args=(data, message)) | |||||
t_reminder.name = "reminder " + end_time | t_reminder.name = "reminder " + end_time | ||||
t_reminder.daemon = True | t_reminder.daemon = True | ||||
t_reminder.start() | t_reminder.start() | ||||
def reminder(self, data, message, wait): | |||||
time.sleep(wait) | |||||
self.connection.reply(data, message) |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -24,16 +24,22 @@ from os.path import expanduser | |||||
import oursql | import oursql | ||||
from earwigbot.classes import BaseCommand | |||||
from earwigbot.commands import Command | |||||
class Command(BaseCommand): | |||||
class Replag(Command): | |||||
"""Return the replag for a specific database on the Toolserver.""" | """Return the replag for a specific database on the Toolserver.""" | ||||
name = "replag" | name = "replag" | ||||
def setup(self): | |||||
try: | |||||
self.default = self.config.commands[self.name]["default"] | |||||
except KeyError: | |||||
self.default = None | |||||
def process(self, data): | def process(self, data): | ||||
args = {} | args = {} | ||||
if not data.args: | if not data.args: | ||||
args["db"] = "enwiki_p" | |||||
args["db"] = self.default or self.bot.wiki.get_site().name + "_p" | |||||
else: | else: | ||||
args["db"] = data.args[0] | args["db"] = data.args[0] | ||||
args["host"] = args["db"].replace("_", "-") + ".rrdb.toolserver.org" | args["host"] = args["db"].replace("_", "-") + ".rrdb.toolserver.org" | ||||
@@ -41,10 +47,11 @@ class Command(BaseCommand): | |||||
conn = oursql.connect(**args) | conn = oursql.connect(**args) | ||||
with conn.cursor() as cursor: | with conn.cursor() as cursor: | ||||
query = "SELECT UNIX_TIMESTAMP() - UNIX_TIMESTAMP(rc_timestamp) FROM recentchanges ORDER BY rc_timestamp DESC LIMIT 1" | |||||
query = """SELECT UNIX_TIMESTAMP() - UNIX_TIMESTAMP(rc_timestamp) | |||||
FROM recentchanges ORDER BY rc_timestamp DESC LIMIT 1""" | |||||
cursor.execute(query) | cursor.execute(query) | ||||
replag = int(cursor.fetchall()[0][0]) | replag = int(cursor.fetchall()[0][0]) | ||||
conn.close() | conn.close() | ||||
msg = "Replag on \x0302{0}\x0301 is \x02{1}\x0F seconds." | |||||
self.connection.reply(data, msg.format(args["db"], replag)) | |||||
msg = "replag on \x0302{0}\x0301 is \x02{1}\x0F seconds." | |||||
self.reply(data, msg.format(args["db"], replag)) |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -20,18 +20,13 @@ | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||||
# SOFTWARE. | # SOFTWARE. | ||||
from earwigbot.classes import BaseCommand | |||||
from earwigbot import wiki | |||||
from earwigbot import exceptions | |||||
from earwigbot.commands import Command | |||||
class Command(BaseCommand): | |||||
class Rights(Command): | |||||
"""Retrieve a list of rights for a given username.""" | """Retrieve a list of rights for a given username.""" | ||||
name = "rights" | name = "rights" | ||||
def check(self, data): | |||||
commands = ["rights", "groups", "permissions", "privileges"] | |||||
if data.is_command and data.command in commands: | |||||
return True | |||||
return False | |||||
commands = ["rights", "groups", "permissions", "privileges"] | |||||
def process(self, data): | def process(self, data): | ||||
if not data.args: | if not data.args: | ||||
@@ -39,15 +34,14 @@ class Command(BaseCommand): | |||||
else: | else: | ||||
name = ' '.join(data.args) | name = ' '.join(data.args) | ||||
site = wiki.get_site() | |||||
site._maxlag = None | |||||
site = self.bot.wiki.get_site() | |||||
user = site.get_user(name) | user = site.get_user(name) | ||||
try: | try: | ||||
rights = user.groups() | |||||
except wiki.UserNotFoundError: | |||||
rights = user.groups | |||||
except exceptions.UserNotFoundError: | |||||
msg = "the user \x0302{0}\x0301 does not exist." | msg = "the user \x0302{0}\x0301 does not exist." | ||||
self.connection.reply(data, msg.format(name)) | |||||
self.reply(data, msg.format(name)) | |||||
return | return | ||||
try: | try: | ||||
@@ -55,4 +49,4 @@ class Command(BaseCommand): | |||||
except ValueError: | except ValueError: | ||||
pass | pass | ||||
msg = "the rights for \x0302{0}\x0301 are {1}." | msg = "the rights for \x0302{0}\x0301 are {1}." | ||||
self.connection.reply(data, msg.format(name, ', '.join(rights))) | |||||
self.reply(data, msg.format(name, ', '.join(rights))) |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -22,15 +22,16 @@ | |||||
import random | import random | ||||
from earwigbot.classes import BaseCommand | |||||
from earwigbot.commands import Command | |||||
class Command(BaseCommand): | |||||
class Test(Command): | |||||
"""Test the bot!""" | """Test the bot!""" | ||||
name = "test" | name = "test" | ||||
def process(self, data): | def process(self, data): | ||||
user = "\x02" + data.nick + "\x0F" # Wrap nick in bold | |||||
hey = random.randint(0, 1) | hey = random.randint(0, 1) | ||||
if hey: | if hey: | ||||
self.connection.say(data.chan, "Hey \x02%s\x0F!" % data.nick) | |||||
self.say(data.chan, "Hey {0}!".format(user)) | |||||
else: | else: | ||||
self.connection.say(data.chan, "'sup \x02%s\x0F?" % data.nick) | |||||
self.say(data.chan, "'sup {0}?".format(user)) |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -23,25 +23,18 @@ | |||||
import threading | import threading | ||||
import re | import re | ||||
from earwigbot import tasks | |||||
from earwigbot.classes import BaseCommand, Data, KwargParseException | |||||
from earwigbot.config import config | |||||
from earwigbot.commands import Command | |||||
class Command(BaseCommand): | |||||
class Threads(Command): | |||||
"""Manage wiki tasks from IRC, and check on thread status.""" | """Manage wiki tasks from IRC, and check on thread status.""" | ||||
name = "threads" | name = "threads" | ||||
def check(self, data): | |||||
commands = ["tasks", "task", "threads", "tasklist"] | |||||
if data.is_command and data.command in commands: | |||||
return True | |||||
return False | |||||
commands = ["tasks", "task", "threads", "tasklist"] | |||||
def process(self, data): | def process(self, data): | ||||
self.data = data | self.data = data | ||||
if data.host not in config.irc["permissions"]["owners"]: | |||||
if data.host not in self.config.irc["permissions"]["owners"]: | |||||
msg = "you must be a bot owner to use this command." | msg = "you must be a bot owner to use this command." | ||||
self.connection.reply(data, msg) | |||||
self.reply(data, msg) | |||||
return | return | ||||
if not data.args: | if not data.args: | ||||
@@ -49,7 +42,7 @@ class Command(BaseCommand): | |||||
self.do_list() | self.do_list() | ||||
else: | else: | ||||
msg = "no arguments provided. Maybe you wanted '!{0} list', '!{0} start', or '!{0} listall'?" | msg = "no arguments provided. Maybe you wanted '!{0} list', '!{0} start', or '!{0} listall'?" | ||||
self.connection.reply(data, msg.format(data.command)) | |||||
self.reply(data, msg.format(data.command)) | |||||
return | return | ||||
if data.args[0] == "list": | if data.args[0] == "list": | ||||
@@ -63,7 +56,7 @@ class Command(BaseCommand): | |||||
else: # They asked us to do something we don't know | else: # They asked us to do something we don't know | ||||
msg = "unknown argument: \x0303{0}\x0301.".format(data.args[0]) | msg = "unknown argument: \x0303{0}\x0301.".format(data.args[0]) | ||||
self.connection.reply(data, msg) | |||||
self.reply(data, msg) | |||||
def do_list(self): | def do_list(self): | ||||
"""With !tasks list (or abbreviation !tasklist), list all running | """With !tasks list (or abbreviation !tasklist), list all running | ||||
@@ -72,15 +65,14 @@ class Command(BaseCommand): | |||||
threads = threading.enumerate() | threads = threading.enumerate() | ||||
normal_threads = [] | normal_threads = [] | ||||
task_threads = [] | |||||
daemon_threads = [] | |||||
for thread in threads: | for thread in threads: | ||||
tname = thread.name | tname = thread.name | ||||
if tname == "MainThread": | if tname == "MainThread": | ||||
tname = self.get_main_thread_name() | |||||
t = "\x0302{0}\x0301 (as main thread, id {1})" | |||||
normal_threads.append(t.format(tname, thread.ident)) | |||||
elif tname in ["irc-frontend", "irc-watcher", "wiki-scheduler"]: | |||||
t = "\x0302MainThread\x0301 (id {0})" | |||||
normal_threads.append(t.format(thread.ident)) | |||||
elif tname in self.config.components: | |||||
t = "\x0302{0}\x0301 (id {1})" | t = "\x0302{0}\x0301 (id {1})" | ||||
normal_threads.append(t.format(tname, thread.ident)) | normal_threads.append(t.format(tname, thread.ident)) | ||||
elif tname.startswith("reminder"): | elif tname.startswith("reminder"): | ||||
@@ -90,28 +82,28 @@ class Command(BaseCommand): | |||||
else: | else: | ||||
tname, start_time = re.findall("^(.*?) \((.*?)\)$", tname)[0] | tname, start_time = re.findall("^(.*?) \((.*?)\)$", tname)[0] | ||||
t = "\x0302{0}\x0301 (id {1}, since {2})" | t = "\x0302{0}\x0301 (id {1}, since {2})" | ||||
task_threads.append(t.format(tname, thread.ident, start_time)) | |||||
daemon_threads.append(t.format(tname, thread.ident, | |||||
start_time)) | |||||
if task_threads: | |||||
msg = "\x02{0}\x0F threads active: {1}, and \x02{2}\x0F task threads: {3}." | |||||
if daemon_threads: | |||||
if len(daemon_threads) > 1: | |||||
msg = "\x02{0}\x0F threads active: {1}, and \x02{2}\x0F command/task threads: {3}." | |||||
else: | |||||
msg = "\x02{0}\x0F threads active: {1}, and \x02{2}\x0F command/task thread: {3}." | |||||
msg = msg.format(len(threads), ', '.join(normal_threads), | msg = msg.format(len(threads), ', '.join(normal_threads), | ||||
len(task_threads), ', '.join(task_threads)) | |||||
len(daemon_threads), ', '.join(daemon_threads)) | |||||
else: | else: | ||||
msg = "\x02{0}\x0F threads active: {1}, and \x020\x0F task threads." | |||||
msg = "\x02{0}\x0F threads active: {1}, and \x020\x0F command/task threads." | |||||
msg = msg.format(len(threads), ', '.join(normal_threads)) | msg = msg.format(len(threads), ', '.join(normal_threads)) | ||||
self.connection.reply(self.data, msg) | |||||
self.reply(self.data, msg) | |||||
def do_listall(self): | def do_listall(self): | ||||
"""With !tasks listall or !tasks all, list all loaded tasks, and report | """With !tasks listall or !tasks all, list all loaded tasks, and report | ||||
whether they are currently running or idle.""" | whether they are currently running or idle.""" | ||||
all_tasks = tasks.get_all().keys() | |||||
threads = threading.enumerate() | threads = threading.enumerate() | ||||
tasklist = [] | tasklist = [] | ||||
all_tasks.sort() | |||||
for task in all_tasks: | |||||
for task in sorted([task.name for task in self.bot.tasks]): | |||||
threadlist = [t for t in threads if t.name.startswith(task)] | threadlist = [t for t in threads if t.name.startswith(task)] | ||||
ids = [str(t.ident) for t in threadlist] | ids = [str(t.ident) for t in threadlist] | ||||
if not ids: | if not ids: | ||||
@@ -123,10 +115,10 @@ class Command(BaseCommand): | |||||
t = "\x0302{0}\x0301 (\x02active\x0F as ids {1})" | t = "\x0302{0}\x0301 (\x02active\x0F as ids {1})" | ||||
tasklist.append(t.format(task, ', '.join(ids))) | tasklist.append(t.format(task, ', '.join(ids))) | ||||
tasklist = ", ".join(tasklist) | |||||
tasks = ", ".join(tasklist) | |||||
msg = "{0} tasks loaded: {1}.".format(len(all_tasks), tasklist) | |||||
self.connection.reply(self.data, msg) | |||||
msg = "\x02{0}\x0F tasks loaded: {1}.".format(len(tasklist), tasks) | |||||
self.reply(self.data, msg) | |||||
def do_start(self): | def do_start(self): | ||||
"""With !tasks start, start any loaded task by name with or without | """With !tasks start, start any loaded task by name with or without | ||||
@@ -136,32 +128,16 @@ class Command(BaseCommand): | |||||
try: | try: | ||||
task_name = data.args[1] | task_name = data.args[1] | ||||
except IndexError: # No task name given | except IndexError: # No task name given | ||||
self.connection.reply(data, "what task do you want me to start?") | |||||
return | |||||
try: | |||||
data.parse_kwargs() | |||||
except KwargParseException, arg: | |||||
msg = "error parsing argument: \x0303{0}\x0301.".format(arg) | |||||
self.connection.reply(data, msg) | |||||
self.reply(data, "what task do you want me to start?") | |||||
return | return | ||||
if task_name not in tasks.get_all().keys(): | |||||
if task_name not in [task.name for task in self.bot.tasks]: | |||||
# This task does not exist or hasn't been loaded: | # This task does not exist or hasn't been loaded: | ||||
msg = "task could not be found; either bot/tasks/{0}.py doesn't exist, or it wasn't loaded correctly." | |||||
self.connection.reply(data, msg.format(task_name)) | |||||
msg = "task could not be found; either it doesn't exist, or it wasn't loaded correctly." | |||||
self.reply(data, msg.format(task_name)) | |||||
return | return | ||||
data.kwargs["fromIRC"] = True | data.kwargs["fromIRC"] = True | ||||
tasks.start(task_name, **data.kwargs) | |||||
self.bot.tasks.start(task_name, **data.kwargs) | |||||
msg = "task \x0302{0}\x0301 started.".format(task_name) | msg = "task \x0302{0}\x0301 started.".format(task_name) | ||||
self.connection.reply(data, msg) | |||||
def get_main_thread_name(self): | |||||
"""Return the "proper" name of the MainThread.""" | |||||
if "irc_frontend" in config.components: | |||||
return "irc-frontend" | |||||
elif "wiki_schedule" in config.components: | |||||
return "wiki-scheduler" | |||||
else: | |||||
return "irc-watcher" | |||||
self.reply(data, msg) |
@@ -0,0 +1,68 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
from datetime import datetime | |||||
from math import floor | |||||
from time import time | |||||
try: | |||||
import pytz | |||||
except ImportError: | |||||
pytz = None | |||||
from earwigbot.commands import Command | |||||
class Time(Command): | |||||
"""Report the current time in any timezone (UTC default), or in beats.""" | |||||
name = "time" | |||||
commands = ["time", "beats", "swatch"] | |||||
def process(self, data): | |||||
if data.command in ["beats", "swatch"]: | |||||
self.do_beats(data) | |||||
return | |||||
if data.args: | |||||
timezone = data.args[0] | |||||
else: | |||||
timezone = "UTC" | |||||
if timezone in ["beats", "swatch"]: | |||||
self.do_beats(data) | |||||
else: | |||||
self.do_time(data, timezone) | |||||
def do_beats(self, data): | |||||
beats = ((time() + 3600) % 86400) / 86.4 | |||||
beats = int(floor(beats)) | |||||
self.reply(data, "@{0:0>3}".format(beats)) | |||||
def do_time(self, data, timezone): | |||||
if not pytz: | |||||
msg = "this command requires the 'pytz' module: http://pytz.sourceforge.net/" | |||||
self.reply(data, msg) | |||||
return | |||||
try: | |||||
tzinfo = pytz.timezone(timezone) | |||||
except pytz.exceptions.UnknownTimeZoneError: | |||||
self.reply(data, "unknown timezone: {0}.".format(timezone)) | |||||
return | |||||
now = pytz.utc.localize(datetime.utcnow()).astimezone(tzinfo) | |||||
self.reply(data, now.strftime("%Y-%m-%d %H:%M:%S %Z")) |
@@ -0,0 +1,46 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
from unicodedata import normalize | |||||
from earwigbot.commands import Command | |||||
class Trout(Command): | |||||
"""Slap someone with a trout, or related fish.""" | |||||
name = "trout" | |||||
commands = ["trout", "whale"] | |||||
def setup(self): | |||||
try: | |||||
self.exceptions = self.config.commands[self.name]["exceptions"] | |||||
except KeyError: | |||||
self.exceptions = {} | |||||
def process(self, data): | |||||
animal = data.command | |||||
target = " ".join(data.args) or data.nick | |||||
normal = normalize("NFKD", target.decode("utf8")).lower() | |||||
if normal in self.exceptions: | |||||
self.reply(data, self.exceptions[normal]) | |||||
else: | |||||
msg = "slaps \x02{0}\x0F around a bit with a large {1}." | |||||
self.action(data.chan, msg.format(target, animal)) |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -20,105 +20,97 @@ | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||||
# SOFTWARE. | # SOFTWARE. | ||||
""" | |||||
EarwigBot's JSON Config File Parser | |||||
This handles all tasks involving reading and writing to our config file, | |||||
including encrypting and decrypting passwords and making a new config file from | |||||
scratch at the inital bot run. | |||||
Usually you'll just want to do "from earwigbot.config import config", which | |||||
returns a singleton _BotConfig object, with data accessible from various | |||||
attributes and functions: | |||||
* config.components - enabled components | |||||
* config.wiki - information about wiki-editing | |||||
* config.tasks - information for bot tasks | |||||
* config.irc - information about IRC | |||||
* config.metadata - miscellaneous information | |||||
* config.schedule() - tasks scheduled to run at a given time | |||||
Additionally, _BotConfig has some functions used in config loading: | |||||
* config.load() - loads and parses our config file, returning True if | |||||
passwords are stored encrypted or False otherwise | |||||
* config.decrypt() - given a key, decrypts passwords inside our config | |||||
variables; won't work if passwords aren't encrypted | |||||
""" | |||||
import json | |||||
from getpass import getpass | |||||
from hashlib import sha256 | |||||
import logging | import logging | ||||
import logging.handlers | import logging.handlers | ||||
from os import mkdir, path | from os import mkdir, path | ||||
from earwigbot import blowfish | |||||
from Crypto.Cipher import Blowfish | |||||
import bcrypt | |||||
import yaml | |||||
__all__ = ["config"] | |||||
from earwigbot.exceptions import NoConfigError | |||||
class _ConfigNode(object): | |||||
def __iter__(self): | |||||
for key in self.__dict__.iterkeys(): | |||||
yield key | |||||
__all__ = ["BotConfig"] | |||||
def __getitem__(self, item): | |||||
return self.__dict__.__getitem__(item) | |||||
class BotConfig(object): | |||||
""" | |||||
**EarwigBot: YAML Config File Manager** | |||||
def _dump(self): | |||||
data = self.__dict__.copy() | |||||
for key, val in data.iteritems(): | |||||
if isinstance(val, _ConfigNode): | |||||
data[key] = val.dump() | |||||
return data | |||||
This handles all tasks involving reading and writing to our config file, | |||||
including encrypting and decrypting passwords and making a new config file | |||||
from scratch at the inital bot run. | |||||
def _load(self, data): | |||||
self.__dict__ = data.copy() | |||||
BotConfig has a few attributes and methods, including the following: | |||||
def _decrypt(self, key, intermediates, item): | |||||
base = self.__dict__ | |||||
try: | |||||
for inter in intermediates: | |||||
base = base[inter] | |||||
except KeyError: | |||||
return | |||||
if item in base: | |||||
base[item] = blowfish.decrypt(key, base[item]) | |||||
- :py:attr:`root_dir`: bot's working directory; contains | |||||
:file:`config.yml`, :file:`logs/` | |||||
- :py:attr:`path`: path to the bot's config file | |||||
- :py:attr:`components`: enabled components | |||||
- :py:attr:`wiki`: information about wiki-editing | |||||
- :py:attr:`irc`: information about IRC | |||||
- :py:attr:`commands`: information about IRC commands | |||||
- :py:attr:`tasks`: information for bot tasks | |||||
- :py:attr:`metadata`: miscellaneous information | |||||
- :py:meth:`schedule`: tasks scheduled to run at a given time | |||||
def get(self, *args, **kwargs): | |||||
return self.__dict__.get(*args, **kwargs) | |||||
BotConfig also has some methods used in config loading: | |||||
- :py:meth:`load`: loads (or reloads) and parses our config file | |||||
- :py:meth:`decrypt`: decrypts an object in the config tree | |||||
""" | |||||
class _BotConfig(object): | |||||
def __init__(self): | |||||
self._script_dir = path.dirname(path.abspath(__file__)) | |||||
self._root_dir = path.split(self._script_dir)[0] | |||||
self._config_path = path.join(self._root_dir, "config.json") | |||||
def __init__(self, root_dir, level): | |||||
self._root_dir = root_dir | |||||
self._logging_level = level | |||||
self._config_path = path.join(self._root_dir, "config.yml") | |||||
self._log_dir = path.join(self._root_dir, "logs") | self._log_dir = path.join(self._root_dir, "logs") | ||||
self._decryption_key = None | |||||
self._decryption_cipher = None | |||||
self._data = None | self._data = None | ||||
self._components = _ConfigNode() | self._components = _ConfigNode() | ||||
self._wiki = _ConfigNode() | self._wiki = _ConfigNode() | ||||
self._tasks = _ConfigNode() | |||||
self._irc = _ConfigNode() | self._irc = _ConfigNode() | ||||
self._commands = _ConfigNode() | |||||
self._tasks = _ConfigNode() | |||||
self._metadata = _ConfigNode() | self._metadata = _ConfigNode() | ||||
self._nodes = [self._components, self._wiki, self._tasks, self._irc, | |||||
self._metadata] | |||||
self._nodes = [self._components, self._wiki, self._irc, self._commands, | |||||
self._tasks, self._metadata] | |||||
self._decryptable_nodes = [ # Default nodes to decrypt | |||||
(self._wiki, ("password",)), | |||||
(self._wiki, ("search", "credentials", "key")), | |||||
(self._wiki, ("search", "credentials", "secret")), | |||||
(self._irc, ("frontend", "nickservPassword")), | |||||
(self._irc, ("watcher", "nickservPassword")), | |||||
] | |||||
def __repr__(self): | |||||
"""Return the canonical string representation of the BotConfig.""" | |||||
res = "BotConfig(root_dir={0!r}, level={1!r})" | |||||
return res.format(self.root_dir, self.logging_level) | |||||
def __str__(self): | |||||
"""Return a nice string representation of the BotConfig.""" | |||||
return "<BotConfig at {0}>".format(self.root_dir) | |||||
def _load(self): | def _load(self): | ||||
"""Load data from our JSON config file (config.json) into _config.""" | |||||
"""Load data from our JSON config file (config.yml) into self._data.""" | |||||
filename = self._config_path | filename = self._config_path | ||||
with open(filename, 'r') as fp: | with open(filename, 'r') as fp: | ||||
try: | try: | ||||
self._data = json.load(fp) | |||||
except ValueError as error: | |||||
self._data = yaml.load(fp) | |||||
except yaml.YAMLError: | |||||
print "Error parsing config file {0}:".format(filename) | print "Error parsing config file {0}:".format(filename) | ||||
print error | |||||
exit(1) | |||||
raise | |||||
def _setup_logging(self): | def _setup_logging(self): | ||||
"""Configures the logging module so it works the way we want it to.""" | """Configures the logging module so it works the way we want it to.""" | ||||
log_dir = self._log_dir | log_dir = self._log_dir | ||||
logger = logging.getLogger("earwigbot") | logger = logging.getLogger("earwigbot") | ||||
logger.handlers = [] # Remove any handlers already attached to us | |||||
logger.setLevel(logging.DEBUG) | logger.setLevel(logging.DEBUG) | ||||
if self.metadata.get("enableLogging"): | if self.metadata.get("enableLogging"): | ||||
@@ -134,7 +126,7 @@ class _BotConfig(object): | |||||
else: | else: | ||||
msg = "log_dir ({0}) exists but is not a directory!" | msg = "log_dir ({0}) exists but is not a directory!" | ||||
print msg.format(log_dir) | print msg.format(log_dir) | ||||
exit(1) | |||||
return | |||||
main_handler = hand(logfile("bot.log"), "midnight", 1, 7) | main_handler = hand(logfile("bot.log"), "midnight", 1, 7) | ||||
error_handler = hand(logfile("error.log"), "W6", 1, 4) | error_handler = hand(logfile("error.log"), "W6", 1, 4) | ||||
@@ -148,41 +140,64 @@ class _BotConfig(object): | |||||
h.setFormatter(formatter) | h.setFormatter(formatter) | ||||
logger.addHandler(h) | logger.addHandler(h) | ||||
stream_handler = logging.StreamHandler() | |||||
stream_handler.setLevel(logging.DEBUG) | |||||
stream_handler.setFormatter(color_formatter) | |||||
logger.addHandler(stream_handler) | |||||
self._stream_handler = stream = logging.StreamHandler() | |||||
stream.setLevel(self._logging_level) | |||||
stream.setFormatter(color_formatter) | |||||
logger.addHandler(stream) | |||||
else: | |||||
logger.addHandler(logging.NullHandler()) | |||||
def _decrypt(self, node, nodes): | |||||
"""Try to decrypt the contents of a config node. Use self.decrypt().""" | |||||
try: | |||||
node._decrypt(self._decryption_cipher, nodes[:-1], nodes[-1]) | |||||
except ValueError: | |||||
print "Error decrypting passwords:" | |||||
raise | |||||
def _make_new(self): | def _make_new(self): | ||||
"""Make a new config file based on the user's input.""" | """Make a new config file based on the user's input.""" | ||||
encrypt = raw_input("Would you like to encrypt passwords stored in config.json? [y/n] ") | |||||
if encrypt.lower().startswith("y"): | |||||
is_encrypted = True | |||||
else: | |||||
is_encrypted = False | |||||
return is_encrypted | |||||
@property | |||||
def script_dir(self): | |||||
return self._script_dir | |||||
#m = "Would you like to encrypt passwords stored in config.yml? [y/n] " | |||||
#encrypt = raw_input(m) | |||||
#if encrypt.lower().startswith("y"): | |||||
# is_encrypted = True | |||||
#else: | |||||
# is_encrypted = False | |||||
raise NotImplementedError() | |||||
# yaml.dumps() config.yml file (self._config_path) | |||||
# Create root_dir/, root_dir/commands/, root_dir/tasks/ | |||||
# Give a reasonable message after config has been created regarding | |||||
# what to do next... | |||||
@property | @property | ||||
def root_dir(self): | def root_dir(self): | ||||
"""The bot's root directory containing its config file and more.""" | |||||
return self._root_dir | return self._root_dir | ||||
@property | @property | ||||
def config_path(self): | |||||
def logging_level(self): | |||||
"""The minimum logging level for messages logged via stdout.""" | |||||
return self._logging_level | |||||
@logging_level.setter | |||||
def logging_level(self, level): | |||||
self._logging_level = level | |||||
self._stream_handler.setLevel(level) | |||||
@property | |||||
def path(self): | |||||
"""The path to the bot's config file.""" | |||||
return self._config_path | return self._config_path | ||||
@property | @property | ||||
def log_dir(self): | def log_dir(self): | ||||
"""The directory containing the bot's logs.""" | |||||
return self._log_dir | return self._log_dir | ||||
@property | @property | ||||
def data(self): | |||||
"""The entire config file as a decoded JSON object.""" | |||||
return self._data | |||||
@property | |||||
def components(self): | def components(self): | ||||
"""A dict of enabled components.""" | """A dict of enabled components.""" | ||||
return self._components | return self._components | ||||
@@ -193,90 +208,103 @@ class _BotConfig(object): | |||||
return self._wiki | return self._wiki | ||||
@property | @property | ||||
def tasks(self): | |||||
"""A dict of information for bot tasks.""" | |||||
return self._tasks | |||||
@property | |||||
def irc(self): | def irc(self): | ||||
"""A dict of information about IRC.""" | """A dict of information about IRC.""" | ||||
return self._irc | return self._irc | ||||
@property | @property | ||||
def commands(self): | |||||
"""A dict of information for IRC commands.""" | |||||
return self._commands | |||||
@property | |||||
def tasks(self): | |||||
"""A dict of information for bot tasks.""" | |||||
return self._tasks | |||||
@property | |||||
def metadata(self): | def metadata(self): | ||||
"""A dict of miscellaneous information.""" | """A dict of miscellaneous information.""" | ||||
return self._metadata | return self._metadata | ||||
def is_loaded(self): | def is_loaded(self): | ||||
"""Return True if our config file has been loaded, otherwise False.""" | |||||
"""Return ``True`` if our config file has been loaded, or ``False``.""" | |||||
return self._data is not None | return self._data is not None | ||||
def is_encrypted(self): | def is_encrypted(self): | ||||
"""Return True if passwords are encrypted, otherwise False.""" | |||||
"""Return ``True`` if passwords are encrypted, otherwise ``False``.""" | |||||
return self.metadata.get("encryptPasswords", False) | return self.metadata.get("encryptPasswords", False) | ||||
def load(self, config_path=None, log_dir=None): | |||||
def load(self): | |||||
"""Load, or reload, our config file. | """Load, or reload, our config file. | ||||
First, check if we have a valid config file, and if not, notify the | First, check if we have a valid config file, and if not, notify the | ||||
user. If there is no config file at all, offer to make one, otherwise | user. If there is no config file at all, offer to make one, otherwise | ||||
exit. | exit. | ||||
Store data from our config file in five _ConfigNodes (components, | |||||
wiki, tasks, irc, metadata) for easy access (as well as the internal | |||||
_data variable). | |||||
If everything goes well, return True if stored passwords are | |||||
encrypted in the file, or False if they are not. | |||||
Data from the config file is stored in six | |||||
:py:class:`~earwigbot.config._ConfigNode`\ s (:py:attr:`components`, | |||||
:py:attr:`wiki`, :py:attr:`irc`, :py:attr:`commands`, :py:attr:`tasks`, | |||||
:py:attr:`metadata`) for easy access (as well as the lower-level | |||||
:py:attr:`data` attribute). If passwords are encrypted, we'll use | |||||
:py:func:`~getpass.getpass` for the key and then decrypt them. If the | |||||
config is being reloaded, encrypted items will be automatically | |||||
decrypted if they were decrypted earlier. | |||||
""" | """ | ||||
if config_path: | |||||
self._config_path = config_path | |||||
if log_dir: | |||||
self._log_dir = log_dir | |||||
if not path.exists(self._config_path): | if not path.exists(self._config_path): | ||||
print "You haven't configured the bot yet!" | |||||
choice = raw_input("Would you like to do this now? [y/n] ") | |||||
print "Config file not found:", self._config_path | |||||
choice = raw_input("Would you like to create a config file now? [y/n] ") | |||||
if choice.lower().startswith("y"): | if choice.lower().startswith("y"): | ||||
return self._make_new() | |||||
self._make_new() | |||||
else: | else: | ||||
exit(1) | |||||
raise NoConfigError() | |||||
self._load() | self._load() | ||||
data = self._data | data = self._data | ||||
self.components._load(data.get("components", {})) | self.components._load(data.get("components", {})) | ||||
self.wiki._load(data.get("wiki", {})) | self.wiki._load(data.get("wiki", {})) | ||||
self.tasks._load(data.get("tasks", {})) | |||||
self.irc._load(data.get("irc", {})) | self.irc._load(data.get("irc", {})) | ||||
self.commands._load(data.get("commands", {})) | |||||
self.tasks._load(data.get("tasks", {})) | |||||
self.metadata._load(data.get("metadata", {})) | self.metadata._load(data.get("metadata", {})) | ||||
self._setup_logging() | self._setup_logging() | ||||
return self.is_encrypted() | |||||
if self.is_encrypted(): | |||||
if not self._decryption_cipher: | |||||
key = getpass("Enter key to decrypt bot passwords: ") | |||||
self._decryption_cipher = Blowfish.new(sha256(key).digest()) | |||||
signature = self.metadata["signature"] | |||||
assert bcrypt.hashpw(key, signature) == signature | |||||
for node, nodes in self._decryptable_nodes: | |||||
self._decrypt(node, nodes) | |||||
def decrypt(self, node, *nodes): | def decrypt(self, node, *nodes): | ||||
"""Use self._decryption_key to decrypt an object in our config tree. | |||||
"""Decrypt an object in our config tree. | |||||
:py:attr:`_decryption_cipher` is used as our key, retrieved using | |||||
:py:func:`~getpass.getpass` in :py:meth:`load` if it wasn't already | |||||
specified. If this is called when passwords are not encrypted (check | |||||
with :py:meth:`is_encrypted`), nothing will happen. We'll also keep | |||||
track of this node if :py:meth:`load` is called again (i.e. to reload) | |||||
and automatically decrypt it. | |||||
If this is called when passwords are not encrypted (check with | |||||
config.is_encrypted()), nothing will happen. | |||||
Example usage:: | |||||
An example usage would be: | |||||
config.decrypt(config.irc, "frontend", "nickservPassword") | |||||
>>> config.decrypt(config.irc, "frontend", "nickservPassword") | |||||
# decrypts config.irc["frontend"]["nickservPassword"] | |||||
""" | """ | ||||
if not self.is_encrypted(): | |||||
return | |||||
try: | |||||
node._decrypt(self._decryption_key, nodes[:-1], nodes[-1]) | |||||
except blowfish.BlowfishError as error: | |||||
print "\nError decrypting passwords:" | |||||
print "{0}: {1}.".format(error.__class__.__name__, error) | |||||
exit(1) | |||||
signature = (node, nodes) | |||||
if signature in self._decryptable_nodes: | |||||
return # Already decrypted | |||||
self._decryptable_nodes.append(signature) | |||||
if self.is_encrypted(): | |||||
self._decrypt(node, nodes) | |||||
def schedule(self, minute, hour, month_day, month, week_day): | def schedule(self, minute, hour, month_day, month, week_day): | ||||
"""Return a list of tasks scheduled to run at the specified time. | """Return a list of tasks scheduled to run at the specified time. | ||||
The schedule data comes from our config file's 'schedule' field, which | |||||
is stored as self._data["schedule"]. Call this function as | |||||
config.schedule(args). | |||||
The schedule data comes from our config file's ``schedule`` field, | |||||
which is stored as :py:attr:`self.data["schedule"] <data>`. | |||||
""" | """ | ||||
# Tasks to run this turn, each as a list of either [task_name, kwargs], | # Tasks to run this turn, each as a list of either [task_name, kwargs], | ||||
# or just the task_name: | # or just the task_name: | ||||
@@ -305,6 +333,57 @@ class _BotConfig(object): | |||||
return tasks | return tasks | ||||
class _ConfigNode(object): | |||||
def __iter__(self): | |||||
for key in self.__dict__: | |||||
yield key | |||||
def __getitem__(self, item): | |||||
return self.__dict__.__getitem__(item) | |||||
def _dump(self): | |||||
data = self.__dict__.copy() | |||||
for key, val in data.iteritems(): | |||||
if isinstance(val, _ConfigNode): | |||||
data[key] = val._dump() | |||||
return data | |||||
def _load(self, data): | |||||
self.__dict__ = data.copy() | |||||
def _decrypt(self, cipher, intermediates, item): | |||||
base = self.__dict__ | |||||
for inter in intermediates: | |||||
try: | |||||
base = base[inter] | |||||
except KeyError: | |||||
return | |||||
if item in base: | |||||
ciphertext = base[item].decode("hex") | |||||
base[item] = cipher.decrypt(ciphertext).rstrip("\x00") | |||||
def get(self, *args, **kwargs): | |||||
return self.__dict__.get(*args, **kwargs) | |||||
def keys(self): | |||||
return self.__dict__.keys() | |||||
def values(self): | |||||
return self.__dict__.values() | |||||
def items(self): | |||||
return self.__dict__.items() | |||||
def iterkeys(self): | |||||
return self.__dict__.iterkeys() | |||||
def itervalues(self): | |||||
return self.__dict__.itervalues() | |||||
def iteritems(self): | |||||
return self.__dict__.iteritems() | |||||
class _BotFormatter(logging.Formatter): | class _BotFormatter(logging.Formatter): | ||||
def __init__(self, color=False): | def __init__(self, color=False): | ||||
self._format = super(_BotFormatter, self).format | self._format = super(_BotFormatter, self).format | ||||
@@ -330,6 +409,3 @@ class _BotFormatter(logging.Formatter): | |||||
if record.levelno == logging.CRITICAL: | if record.levelno == logging.CRITICAL: | ||||
record.lvl = l.join(("\x1b[1m\x1b[31m", "\x1b[0m")) # Bold red | record.lvl = l.join(("\x1b[1m\x1b[31m", "\x1b[0m")) # Bold red | ||||
return record | return record | ||||
config = _BotConfig() |
@@ -0,0 +1,256 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
""" | |||||
**EarwigBot: Exceptions** | |||||
This module contains all exceptions used by EarwigBot:: | |||||
EarwigBotError | |||||
+-- NoConfigError | |||||
+-- IRCError | |||||
| +-- BrokenSocketError | |||||
+-- WikiToolsetError | |||||
+-- SiteNotFoundError | |||||
+-- ServiceError | |||||
| +-- APIError | |||||
| +-- SQLError | |||||
+-- NoServiceError | |||||
+-- LoginError | |||||
+-- NamespaceNotFoundError | |||||
+-- PageNotFoundError | |||||
+-- InvalidPageError | |||||
+-- RedirectError | |||||
+-- UserNotFoundError | |||||
+-- EditError | |||||
| +-- PermissionsError | |||||
| +-- EditConflictError | |||||
| +-- NoContentError | |||||
| +-- ContentTooBigError | |||||
| +-- SpamDetectedError | |||||
| +-- FilteredError | |||||
+-- CopyvioCheckError | |||||
+-- UnknownSearchEngineError | |||||
+-- UnsupportedSearchEngineError | |||||
+-- SearchQueryError | |||||
""" | |||||
class EarwigBotError(Exception): | |||||
"""Base exception class for errors in EarwigBot.""" | |||||
class NoConfigError(EarwigBotError): | |||||
"""The bot cannot be run without a config file. | |||||
This occurs if no config file exists, and the user said they did not want | |||||
one to be created. | |||||
""" | |||||
class IRCError(EarwigBotError): | |||||
"""Base exception class for errors in IRC-relation sections of the bot.""" | |||||
class BrokenSocketError(IRCError): | |||||
"""A socket has broken, because it is not sending data. | |||||
Raised by :py:meth:`IRCConnection._get | |||||
<earwigbot.irc.connection.IRCConnection._get>`. | |||||
""" | |||||
class WikiToolsetError(EarwigBotError): | |||||
"""Base exception class for errors in the Wiki Toolset.""" | |||||
class SiteNotFoundError(WikiToolsetError): | |||||
"""A particular site could not be found in the sites database. | |||||
Raised by :py:class:`~earwigbot.wiki.sitesdb.SitesDB`. | |||||
""" | |||||
class ServiceError(WikiToolsetError): | |||||
"""Base exception class for an error within a service (the API or SQL). | |||||
This is caught by :py:meth:`Site.delegate | |||||
<earwigbot.wiki.site.Site.delegate>` to indicate a service is | |||||
non-functional so another, less-preferred one can be tried. | |||||
""" | |||||
class APIError(ServiceError): | |||||
"""Couldn't connect to a site's API. | |||||
Perhaps the server doesn't exist, our URL is wrong or incomplete, or | |||||
there are temporary problems on their end. | |||||
Raised by :py:meth:`Site.api_query <earwigbot.wiki.site.Site.api_query>`. | |||||
""" | |||||
class SQLError(ServiceError): | |||||
"""Some error involving SQL querying occurred. | |||||
Raised by :py:meth:`Site.sql_query <earwigbot.wiki.site.Site.sql_query>`. | |||||
""" | |||||
class NoServiceError(WikiToolsetError): | |||||
"""No service is functioning to handle a specific task. | |||||
Raised by :py:meth:`Site.delegate <earwigbot.wiki.site.Site.delegate>`. | |||||
""" | |||||
class LoginError(WikiToolsetError): | |||||
"""An error occured while trying to login. | |||||
Perhaps the username/password is incorrect. | |||||
Raised by :py:meth:`Site._login <earwigbot.wiki.site.Site._login>`. | |||||
""" | |||||
class NamespaceNotFoundError(WikiToolsetError): | |||||
"""A requested namespace name or namespace ID does not exist. | |||||
Raised by :py:meth:`Site.namespace_id_to_name | |||||
<earwigbot.wiki.site.Site.namespace_id_to_name>` and | |||||
:py:meth:`Site.namespace_name_to_id | |||||
<earwigbot.wiki.site.Site.namespace_name_to_id>`. | |||||
""" | |||||
class PageNotFoundError(WikiToolsetError): | |||||
"""Attempted to get information about a page that does not exist. | |||||
Raised by :py:class:`~earwigbot.wiki.page.Page`. | |||||
""" | |||||
class InvalidPageError(WikiToolsetError): | |||||
"""Attempted to get information about a page whose title is invalid. | |||||
Raised by :py:class:`~earwigbot.wiki.page.Page`. | |||||
""" | |||||
class RedirectError(WikiToolsetError): | |||||
"""A redirect-only method was called on a malformed or non-redirect page. | |||||
Raised by :py:meth:`Page.get_redirect_target | |||||
<earwigbot.wiki.page.Page.get_redirect_target>`. | |||||
""" | |||||
class UserNotFoundError(WikiToolsetError): | |||||
"""Attempted to get certain information about a user that does not exist. | |||||
Raised by :py:class:`~earwigbot.wiki.user.User`. | |||||
""" | |||||
class EditError(WikiToolsetError): | |||||
"""An error occured while editing. | |||||
This is used as a base class for all editing errors; this one specifically | |||||
is used only when a generic error occurs that we don't know about. | |||||
Raised by :py:meth:`Page.edit <earwigbot.wiki.page.Page.edit>` and | |||||
:py:meth:`Page.add_section <earwigbot.wiki.page.Page.add_section>`. | |||||
""" | |||||
class PermissionsError(EditError): | |||||
"""A permissions error ocurred while editing. | |||||
We tried to do something we don't have permission to, like trying to delete | |||||
a page as a non-admin, or trying to edit a page without login information | |||||
and AssertEdit enabled. | |||||
Raised by :py:meth:`Page.edit <earwigbot.wiki.page.Page.edit>` and | |||||
:py:meth:`Page.add_section <earwigbot.wiki.page.Page.add_section>`. | |||||
""" | |||||
class EditConflictError(EditError): | |||||
"""We gotten an edit conflict or a (rarer) delete/recreate conflict. | |||||
Raised by :py:meth:`Page.edit <earwigbot.wiki.page.Page.edit>` and | |||||
:py:meth:`Page.add_section <earwigbot.wiki.page.Page.add_section>`. | |||||
""" | |||||
class NoContentError(EditError): | |||||
"""We tried to create a page or new section with no content. | |||||
Raised by :py:meth:`Page.edit <earwigbot.wiki.page.Page.edit>` and | |||||
:py:meth:`Page.add_section <earwigbot.wiki.page.Page.add_section>`. | |||||
""" | |||||
class ContentTooBigError(EditError): | |||||
"""The edit we tried to push exceeded the article size limit. | |||||
Raised by :py:meth:`Page.edit <earwigbot.wiki.page.Page.edit>` and | |||||
:py:meth:`Page.add_section <earwigbot.wiki.page.Page.add_section>`. | |||||
""" | |||||
class SpamDetectedError(EditError): | |||||
"""The spam filter refused our edit. | |||||
Raised by :py:meth:`Page.edit <earwigbot.wiki.page.Page.edit>` and | |||||
:py:meth:`Page.add_section <earwigbot.wiki.page.Page.add_section>`. | |||||
""" | |||||
class FilteredError(EditError): | |||||
"""The edit filter refused our edit. | |||||
Raised by :py:meth:`Page.edit <earwigbot.wiki.page.Page.edit>` and | |||||
:py:meth:`Page.add_section <earwigbot.wiki.page.Page.add_section>`. | |||||
""" | |||||
class CopyvioCheckError(WikiToolsetError): | |||||
"""An error occured when checking a page for copyright violations. | |||||
This is a base class for multiple exceptions; usually one of those will be | |||||
raised instead of this. | |||||
Raised by :py:meth:`Page.copyvio_check | |||||
<earwigbot.wiki.copyvios.CopyvioMixIn.copyvio_check>` and | |||||
:py:meth:`Page.copyvio_compare | |||||
<earwigbot.wiki.copyvios.CopyvioMixIn.copyvio_compare>`. | |||||
""" | |||||
class UnknownSearchEngineError(CopyvioCheckError): | |||||
"""Attempted to do a copyvio check with an unknown search engine. | |||||
Search engines are specified in :file:`config.yml` as | |||||
:py:attr:`config.wiki["search"]["engine"]`. | |||||
Raised by :py:meth:`Page.copyvio_check | |||||
<earwigbot.wiki.copyvios.CopyvioMixIn.copyvio_check>` and | |||||
:py:meth:`Page.copyvio_compare | |||||
<earwigbot.wiki.copyvios.CopyvioMixIn.copyvio_compare>`. | |||||
""" | |||||
class UnsupportedSearchEngineError(CopyvioCheckError): | |||||
"""Attmpted to do a copyvio check using an unavailable engine. | |||||
This might occur if, for example, an engine requires oauth2 but the package | |||||
couldn't be imported. | |||||
Raised by :py:meth:`Page.copyvio_check | |||||
<earwigbot.wiki.copyvios.CopyvioMixIn.copyvio_check>` and | |||||
:py:meth:`Page.copyvio_compare | |||||
<earwigbot.wiki.copyvios.CopyvioMixIn.copyvio_compare>`. | |||||
""" | |||||
class SearchQueryError(CopyvioCheckError): | |||||
"""Some error ocurred while doing a search query. | |||||
Raised by :py:meth:`Page.copyvio_check | |||||
<earwigbot.wiki.copyvios.CopyvioMixIn.copyvio_check>` and | |||||
:py:meth:`Page.copyvio_compare | |||||
<earwigbot.wiki.copyvios.CopyvioMixIn.copyvio_compare>`. | |||||
""" |
@@ -1,137 +0,0 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
""" | |||||
EarwigBot's IRC Frontend Component | |||||
The IRC frontend runs on a normal IRC server and expects users to interact with | |||||
it and give it commands. Commands are stored as "command classes", subclasses | |||||
of BaseCommand in irc/base_command.py. All command classes are automatically | |||||
imported by irc/command_handler.py if they are in irc/commands. | |||||
""" | |||||
import logging | |||||
import re | |||||
from earwigbot import commands | |||||
from earwigbot.classes import Connection, Data, BrokenSocketException | |||||
from earwigbot.config import config | |||||
__all__ = ["get_connection", "startup", "main"] | |||||
connection = None | |||||
logger = logging.getLogger("earwigbot.frontend") | |||||
sender_regex = re.compile(":(.*?)!(.*?)@(.*?)\Z") | |||||
def get_connection(): | |||||
"""Return a new Connection() instance with information about our server | |||||
connection, but don't actually connect yet.""" | |||||
cf = config.irc["frontend"] | |||||
connection = Connection(cf["host"], cf["port"], cf["nick"], cf["ident"], | |||||
cf["realname"], logger) | |||||
return connection | |||||
def startup(conn): | |||||
"""Accept a single arg, a Connection() object, and set our global variable | |||||
'connection' to it. Load all command classes in irc/commands with | |||||
command_handler, and then establish a connection with the IRC server.""" | |||||
global connection | |||||
connection = conn | |||||
commands.load(connection) | |||||
connection.connect() | |||||
def main(): | |||||
"""Main loop for the frontend component. | |||||
get_connection() and startup() should have already been called before this. | |||||
""" | |||||
read_buffer = str() | |||||
while 1: | |||||
try: | |||||
read_buffer = read_buffer + connection.get() | |||||
except BrokenSocketException: | |||||
logger.warn("Socket has broken on front-end; restarting bot") | |||||
return | |||||
lines = read_buffer.split("\n") | |||||
read_buffer = lines.pop() | |||||
for line in lines: | |||||
ret = _process_message(line) | |||||
if ret: | |||||
return | |||||
def _process_message(line): | |||||
"""Process a single message from IRC.""" | |||||
line = line.strip().split() | |||||
data = Data(line) # new Data instance to store info about this line | |||||
if line[1] == "JOIN": | |||||
data.nick, data.ident, data.host = sender_regex.findall(line[0])[0] | |||||
data.chan = line[2] | |||||
# Check for 'join' hooks in our commands: | |||||
commands.check("join", data) | |||||
elif line[1] == "PRIVMSG": | |||||
data.nick, data.ident, data.host = sender_regex.findall(line[0])[0] | |||||
data.msg = ' '.join(line[3:])[1:] | |||||
data.chan = line[2] | |||||
if data.chan == config.irc["frontend"]["nick"]: | |||||
# This is a privmsg to us, so set 'chan' as the nick of the, sender | |||||
# then check for private-only command hooks: | |||||
data.chan = data.nick | |||||
commands.check("msg_private", data) | |||||
else: | |||||
# Check for public-only command hooks: | |||||
commands.check("msg_public", data) | |||||
# Check for command hooks that apply to all messages: | |||||
commands.check("msg", data) | |||||
# Hardcode the !restart command (we can't restart from within an | |||||
# ordinary command): | |||||
if data.msg in ["!restart", ".restart"]: | |||||
if data.host in config.irc["permissions"]["owners"]: | |||||
logger.info("Restarting bot per owner request") | |||||
return True | |||||
# If we are pinged, pong back: | |||||
elif line[0] == "PING": | |||||
msg = " ".join(("PONG", line[1])) | |||||
connection.send(msg) | |||||
# On successful connection to the server: | |||||
elif line[1] == "376": | |||||
# If we're supposed to auth to NickServ, do that: | |||||
try: | |||||
username = config.irc["frontend"]["nickservUsername"] | |||||
password = config.irc["frontend"]["nickservPassword"] | |||||
except KeyError: | |||||
pass | |||||
else: | |||||
msg = " ".join(("IDENTIFY", username, password)) | |||||
connection.say("NickServ", msg) | |||||
# Join all of our startup channels: | |||||
for chan in config.irc["frontend"]["channels"]: | |||||
connection.join(chan) |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -20,8 +20,8 @@ | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||||
# SOFTWARE. | # SOFTWARE. | ||||
from earwigbot.classes.base_command import * | |||||
from earwigbot.classes.base_task import * | |||||
from earwigbot.classes.connection import * | |||||
from earwigbot.classes.data import * | |||||
from earwigbot.classes.rc import * | |||||
from earwigbot.irc.connection import * | |||||
from earwigbot.irc.data import * | |||||
from earwigbot.irc.frontend import * | |||||
from earwigbot.irc.rc import * | |||||
from earwigbot.irc.watcher import * |
@@ -0,0 +1,228 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
import socket | |||||
from threading import Lock | |||||
from time import sleep, time | |||||
from earwigbot.exceptions import BrokenSocketError | |||||
__all__ = ["IRCConnection"] | |||||
class IRCConnection(object): | |||||
"""Interface with an IRC server.""" | |||||
def __init__(self, host, port, nick, ident, realname): | |||||
self._host = host | |||||
self._port = port | |||||
self._nick = nick | |||||
self._ident = ident | |||||
self._realname = realname | |||||
self._is_running = False | |||||
self._send_lock = Lock() | |||||
self._last_recv = time() | |||||
self._last_ping = 0 | |||||
def __repr__(self): | |||||
"""Return the canonical string representation of the IRCConnection.""" | |||||
res = "IRCConnection(host={0!r}, port={1!r}, nick={2!r}, ident={3!r}, realname={4!r})" | |||||
return res.format(self.host, self.port, self.nick, self.ident, | |||||
self.realname) | |||||
def __str__(self): | |||||
"""Return a nice string representation of the IRCConnection.""" | |||||
res = "<IRCConnection {0}!{1} at {2}:{3}>" | |||||
return res.format(self.nick, self.ident, self.host, self.port) | |||||
def _connect(self): | |||||
"""Connect to our IRC server.""" | |||||
self._sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) | |||||
try: | |||||
self._sock.connect((self.host, self.port)) | |||||
except socket.error: | |||||
self.logger.exception("Couldn't connect to IRC server; retrying") | |||||
sleep(8) | |||||
self._connect() | |||||
self._send("NICK {0}".format(self.nick)) | |||||
self._send("USER {0} {1} * :{2}".format(self.ident, self.host, self.realname)) | |||||
def _close(self): | |||||
"""Completely close our connection with the IRC server.""" | |||||
try: | |||||
self._sock.shutdown(socket.SHUT_RDWR) # Shut down connection first | |||||
except socket.error: | |||||
pass # Ignore if the socket is already down | |||||
self._sock.close() | |||||
def _get(self, size=4096): | |||||
"""Receive (i.e. get) data from the server.""" | |||||
data = self._sock.recv(size) | |||||
if not data: | |||||
# Socket isn't giving us any data, so it is dead or broken: | |||||
raise BrokenSocketError() | |||||
return data | |||||
def _send(self, msg, hidelog=False): | |||||
"""Send data to the server.""" | |||||
with self._send_lock: | |||||
try: | |||||
self._sock.sendall(msg + "\r\n") | |||||
except socket.error: | |||||
self._is_running = False | |||||
else: | |||||
if not hidelog: | |||||
self.logger.debug(msg) | |||||
def _quit(self, msg=None): | |||||
"""Issue a quit message to the server. Doesn't close the connection.""" | |||||
if msg: | |||||
self._send("QUIT :{0}".format(msg)) | |||||
else: | |||||
self._send("QUIT") | |||||
def _process_defaults(self, line): | |||||
"""Default process hooks for lines received on IRC.""" | |||||
self._last_recv = time() | |||||
if line[0] == "PING": # If we are pinged, pong back | |||||
self.pong(line[1][1:]) | |||||
@property | |||||
def host(self): | |||||
"""The hostname of the IRC server, like ``"irc.freenode.net"``.""" | |||||
return self._host | |||||
@property | |||||
def port(self): | |||||
"""The port of the IRC server, like ``6667``.""" | |||||
return self._port | |||||
@property | |||||
def nick(self): | |||||
"""Our nickname on the server, like ``"EarwigBot"``.""" | |||||
return self._nick | |||||
@property | |||||
def ident(self): | |||||
"""Our ident on the server, like ``"earwig"``. | |||||
See http://en.wikipedia.org/wiki/Ident. | |||||
""" | |||||
return self._ident | |||||
@property | |||||
def realname(self): | |||||
"""Our realname (gecos field) on the server.""" | |||||
return self._realname | |||||
def say(self, target, msg, hidelog=False): | |||||
"""Send a private message to a target on the server.""" | |||||
msg = "PRIVMSG {0} :{1}".format(target, msg) | |||||
self._send(msg, hidelog) | |||||
def reply(self, data, msg, hidelog=False): | |||||
"""Send a private message as a reply to a user on the server.""" | |||||
msg = "\x02{0}\x0f: {1}".format(data.nick, msg) | |||||
self.say(data.chan, msg, hidelog) | |||||
def action(self, target, msg, hidelog=False): | |||||
"""Send a private message to a target on the server as an action.""" | |||||
msg = "\x01ACTION {0}\x01".format(msg) | |||||
self.say(target, msg, hidelog) | |||||
def notice(self, target, msg, hidelog=False): | |||||
"""Send a notice to a target on the server.""" | |||||
msg = "NOTICE {0} :{1}".format(target, msg) | |||||
self._send(msg, hidelog) | |||||
def join(self, chan, hidelog=False): | |||||
"""Join a channel on the server.""" | |||||
msg = "JOIN {0}".format(chan) | |||||
self._send(msg, hidelog) | |||||
def part(self, chan, msg=None, hidelog=False): | |||||
"""Part from a channel on the server, optionally using an message.""" | |||||
if msg: | |||||
self._send("PART {0} :{1}".format(chan, msg), hidelog) | |||||
else: | |||||
self._send("PART {0}".format(chan), hidelog) | |||||
def mode(self, target, level, msg, hidelog=False): | |||||
"""Send a mode message to the server.""" | |||||
msg = "MODE {0} {1} {2}".format(target, level, msg) | |||||
self._send(msg, hidelog) | |||||
def ping(self, target, hidelog=False): | |||||
"""Ping another entity on the server.""" | |||||
msg = "PING {0}".format(target) | |||||
self._send(msg, hidelog) | |||||
def pong(self, target, hidelog=False): | |||||
"""Pong another entity on the server.""" | |||||
msg = "PONG {0}".format(target) | |||||
self._send(msg, hidelog) | |||||
def loop(self): | |||||
"""Main loop for the IRC connection.""" | |||||
self._is_running = True | |||||
read_buffer = "" | |||||
while 1: | |||||
try: | |||||
read_buffer += self._get() | |||||
except BrokenSocketError: | |||||
self._is_running = False | |||||
break | |||||
lines = read_buffer.split("\n") | |||||
read_buffer = lines.pop() | |||||
for line in lines: | |||||
line = line.strip().split() | |||||
self._process_defaults(line) | |||||
self._process_message(line) | |||||
if self.is_stopped(): | |||||
break | |||||
self._close() | |||||
def keep_alive(self): | |||||
"""Ensure that we stay connected, stopping if the connection breaks.""" | |||||
now = time() | |||||
if now - self._last_recv > 120: | |||||
if self._last_ping < self._last_recv: | |||||
log = "Last message was received over 120 seconds ago. Pinging." | |||||
self.logger.debug(log) | |||||
self.ping(self.host) | |||||
self._last_ping = now | |||||
elif now - self._last_ping > 60: | |||||
self.logger.debug("No ping response in 60 seconds. Stopping.") | |||||
self.stop() | |||||
def stop(self, msg=None): | |||||
"""Request the IRC connection to close at earliest convenience.""" | |||||
if self._is_running: | |||||
self._quit(msg) | |||||
self._is_running = False | |||||
def is_stopped(self): | |||||
"""Return whether the IRC connection has been (or is to be) closed.""" | |||||
return not self._is_running |
@@ -0,0 +1,211 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
import re | |||||
__all__ = ["Data"] | |||||
class Data(object): | |||||
"""Store data from an individual line received on IRC.""" | |||||
def __init__(self, bot, my_nick, line, msgtype): | |||||
self._bot = bot | |||||
self._my_nick = my_nick.lower() | |||||
self._line = line | |||||
self._is_private = self._is_command = False | |||||
self._msg = self._command = self._trigger = None | |||||
self._args = [] | |||||
self._kwargs = {} | |||||
self._parse(msgtype) | |||||
def __repr__(self): | |||||
"""Return the canonical string representation of the Data.""" | |||||
res = "Data(bot={0!r}, my_nick={1!r}, line={2!r})" | |||||
return res.format(self.bot, self.my_nick, self.line) | |||||
def __str__(self): | |||||
"""Return a nice string representation of the Data.""" | |||||
return "<Data of {0!r}>".format(" ".join(self.line)) | |||||
def _parse(self, msgtype): | |||||
"""Parse a line from IRC into its components as instance attributes.""" | |||||
sender = re.findall(":(.*?)!(.*?)@(.*?)\Z", self.line[0])[0] | |||||
self._nick, self._ident, self._host = sender | |||||
self._chan = self.line[2] | |||||
if msgtype == "PRIVMSG": | |||||
if self.chan.lower() == self.my_nick: | |||||
# This is a privmsg to us, so set 'chan' as the nick of the | |||||
# sender instead of the 'channel', which is ourselves: | |||||
self._chan = self._nick | |||||
self._is_private = True | |||||
self._msg = " ".join(self.line[3:])[1:] | |||||
self._parse_args() | |||||
self._parse_kwargs() | |||||
def _parse_args(self): | |||||
"""Parse command arguments from the message. | |||||
self.msg is converted into the string self.command and the argument | |||||
list self.args if the message starts with a "trigger" ("!", ".", or the | |||||
bot's name); self.is_command will be set to True, and self.trigger will | |||||
store the trigger string. Otherwise, is_command will be set to False. | |||||
""" | |||||
self._args = self.msg.strip().split() | |||||
try: | |||||
self._command = self.args.pop(0).lower() | |||||
except IndexError: | |||||
return | |||||
if self.command.startswith("!") or self.command.startswith("."): | |||||
# e.g. "!command arg1 arg2" | |||||
self._is_command = True | |||||
self._trigger = self.command[0] | |||||
self._command = self.command[1:] # Strip the "!" or "." | |||||
elif self.command.startswith(self.my_nick): | |||||
# e.g. "EarwigBot, command arg1 arg2" | |||||
self._is_command = True | |||||
self._trigger = self.my_nick | |||||
try: | |||||
self._command = self.args.pop(0).lower() | |||||
except IndexError: | |||||
self._command = "" | |||||
else: | |||||
try: | |||||
if self.msg[-1] == "." and self.msg[-2] != ".": | |||||
if self.args: | |||||
self.args[-1] = self.args[-1][:-1] | |||||
else: | |||||
self._command = self.command[:-1] | |||||
except IndexError: | |||||
pass | |||||
def _parse_kwargs(self): | |||||
"""Parse keyword arguments embedded in self.args. | |||||
Parse a command given as "!command key1=value1 key2=value2..." into a | |||||
dict, self.kwargs, like {'key1': 'value2', 'key2': 'value2'...}. | |||||
""" | |||||
for arg in self.args: | |||||
try: | |||||
key, value = re.findall("^(.*?)\=(.*?)$", arg)[0] | |||||
except IndexError: | |||||
continue | |||||
if key and value: | |||||
self.kwargs[key] = value | |||||
@property | |||||
def my_nick(self): | |||||
"""Our nickname, *not* the nickname of the sender.""" | |||||
return self._my_nick | |||||
@property | |||||
def line(self): | |||||
"""The full message received on IRC, including escape characters.""" | |||||
return self._line | |||||
@property | |||||
def chan(self): | |||||
"""Channel the message was sent from. | |||||
This will be equal to :py:attr:`nick` if the message is a private | |||||
message. | |||||
""" | |||||
return self._chan | |||||
@property | |||||
def nick(self): | |||||
"""Nickname of the sender.""" | |||||
return self._nick | |||||
@property | |||||
def ident(self): | |||||
"""`Ident <http://en.wikipedia.org/wiki/Ident>`_ of the sender.""" | |||||
return self._ident | |||||
@property | |||||
def host(self): | |||||
"""Hostname of the sender.""" | |||||
return self._host | |||||
@property | |||||
def msg(self): | |||||
"""Text of the sent message, if it is a message, else ``None``.""" | |||||
return self._msg | |||||
@property | |||||
def is_private(self): | |||||
"""``True`` if this message was sent to us *only*, else ``False``.""" | |||||
return self._is_private | |||||
@property | |||||
def is_command(self): | |||||
"""Boolean telling whether or not this message is a bot command. | |||||
A message is considered a command if and only if it begins with the | |||||
character ``"!"``, ``"."``, or the bot's name followed by optional | |||||
punctuation and a space (so ``EarwigBot: do something``, ``EarwigBot, | |||||
do something``, and ``EarwigBot do something`` are all valid). | |||||
""" | |||||
return self._is_command | |||||
@property | |||||
def command(self): | |||||
"""If the message is a command, this is the name of the command used. | |||||
See :py:attr:`is_command <self.is_command>` for when a message is | |||||
considered a command. If it's not a command, this will be set to | |||||
``None``. | |||||
""" | |||||
return self._command | |||||
@property | |||||
def trigger(self): | |||||
"""If this message is a command, this is what triggered it. | |||||
It can be either "!" (``"!help"``), "." (``".help"``), or the bot's | |||||
name (``"EarwigBot: help"``). Otherwise, it will be ``None``.""" | |||||
return self._trigger | |||||
@property | |||||
def args(self): | |||||
"""List of all arguments given to this command. | |||||
For example, the message ``"!command arg1 arg2 arg3=val3"`` will | |||||
produce the args ``["arg1", "arg2", "arg3=val3"]``. This is empty if | |||||
the message was not a command or if it doesn't have arguments. | |||||
""" | |||||
return self._args | |||||
@property | |||||
def kwargs(self): | |||||
"""Dictionary of keyword arguments given to this command. | |||||
For example, the message ``"!command arg1=val1 arg2=val2"`` will | |||||
produce the kwargs ``{"arg1": "val1", "arg2": "val2"}``. This is empty | |||||
if the message was not a command or if it doesn't have keyword | |||||
arguments. | |||||
""" | |||||
return self._kwargs |
@@ -0,0 +1,88 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
from earwigbot.irc import IRCConnection, Data | |||||
__all__ = ["Frontend"] | |||||
class Frontend(IRCConnection): | |||||
""" | |||||
**EarwigBot: IRC Frontend Component** | |||||
The IRC frontend runs on a normal IRC server and expects users to interact | |||||
with it and give it commands. Commands are stored as "command classes", | |||||
subclasses of :py:class:`~earwigbot.commands.Command`. All command classes | |||||
are automatically imported by :py:meth:`commands.load() | |||||
<earwigbot.managers._ResourceManager.load>` if they are in | |||||
:py:mod:`earwigbot.commands` or the bot's custom command directory | |||||
(explained in the :doc:`documentation </customizing>`). | |||||
""" | |||||
def __init__(self, bot): | |||||
self.bot = bot | |||||
self.logger = bot.logger.getChild("frontend") | |||||
cf = bot.config.irc["frontend"] | |||||
base = super(Frontend, self) | |||||
base.__init__(cf["host"], cf["port"], cf["nick"], cf["ident"], | |||||
cf["realname"]) | |||||
self._connect() | |||||
def __repr__(self): | |||||
"""Return the canonical string representation of the Frontend.""" | |||||
res = "Frontend(host={0!r}, port={1!r}, nick={2!r}, ident={3!r}, realname={4!r}, bot={5!r})" | |||||
return res.format(self.host, self.port, self.nick, self.ident, | |||||
self.realname, self.bot) | |||||
def __str__(self): | |||||
"""Return a nice string representation of the Frontend.""" | |||||
res = "<Frontend {0}!{1} at {2}:{3}>" | |||||
return res.format(self.nick, self.ident, self.host, self.port) | |||||
def _process_message(self, line): | |||||
"""Process a single message from IRC.""" | |||||
if line[1] == "JOIN": | |||||
data = Data(self.bot, self.nick, line, msgtype="JOIN") | |||||
self.bot.commands.call("join", data) | |||||
elif line[1] == "PRIVMSG": | |||||
data = Data(self.bot, self.nick, line, msgtype="PRIVMSG") | |||||
if data.is_private: | |||||
self.bot.commands.call("msg_private", data) | |||||
else: | |||||
self.bot.commands.call("msg_public", data) | |||||
self.bot.commands.call("msg", data) | |||||
elif line[1] == "376": # On successful connection to the server | |||||
# If we're supposed to auth to NickServ, do that: | |||||
try: | |||||
username = self.bot.config.irc["frontend"]["nickservUsername"] | |||||
password = self.bot.config.irc["frontend"]["nickservPassword"] | |||||
except KeyError: | |||||
pass | |||||
else: | |||||
msg = "IDENTIFY {0} {1}".format(username, password) | |||||
self.say("NickServ", msg, hidelog=True) | |||||
# Join all of our startup channels: | |||||
for chan in self.bot.config.irc["frontend"]["channels"]: | |||||
self.join(chan) |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -25,14 +25,26 @@ import re | |||||
__all__ = ["RC"] | __all__ = ["RC"] | ||||
class RC(object): | class RC(object): | ||||
"""A class to store data on an event received from our IRC watcher.""" | |||||
"""Store data from an event received from our IRC watcher.""" | |||||
re_color = re.compile("\x03([0-9]{1,2}(,[0-9]{1,2})?)?") | re_color = re.compile("\x03([0-9]{1,2}(,[0-9]{1,2})?)?") | ||||
re_edit = re.compile("\A\[\[(.*?)\]\]\s(.*?)\s(http://.*?)\s\*\s(.*?)\s\*\s(.*?)\Z") | re_edit = re.compile("\A\[\[(.*?)\]\]\s(.*?)\s(http://.*?)\s\*\s(.*?)\s\*\s(.*?)\Z") | ||||
re_log = re.compile("\A\[\[(.*?)\]\]\s(.*?)\s\*\s(.*?)\s\*\s(.*?)\Z") | |||||
re_log = re.compile("\A\[\[(.*?)\]\]\s(.*?)\s\s\*\s(.*?)\s\*\s(.*?)\Z") | |||||
def __init__(self, msg): | |||||
pretty_edit = "\x02New {0}\x0F: \x0314[[\x0307{1}\x0314]]\x0306 * \x0303{2}\x0306 * \x0302{3}\x0306 * \x0310{4}" | |||||
pretty_log = "\x02New {0}\x0F: \x0303{1}\x0306 * \x0302{2}\x0306 * \x0310{3}" | |||||
def __init__(self, chan, msg): | |||||
self.chan = chan | |||||
self.msg = msg | self.msg = msg | ||||
def __repr__(self): | |||||
"""Return the canonical string representation of the RC.""" | |||||
return "RC(chan={0!r}, msg={1!r})".format(self.chan, self.msg) | |||||
def __str__(self): | |||||
"""Return a nice string representation of the RC.""" | |||||
return "<RC of {0!r} on {1}>".format(self.msg, self.chan) | |||||
def parse(self): | def parse(self): | ||||
"""Parse a recent change event into some variables.""" | """Parse a recent change event into some variables.""" | ||||
# Strip IRC color codes; we don't want or need 'em: | # Strip IRC color codes; we don't want or need 'em: | ||||
@@ -48,7 +60,7 @@ class RC(object): | |||||
# We're probably missing the http:// part, because it's a log | # We're probably missing the http:// part, because it's a log | ||||
# entry, which lacks a URL: | # entry, which lacks a URL: | ||||
page, flags, user, comment = self.re_log.findall(msg)[0] | page, flags, user, comment = self.re_log.findall(msg)[0] | ||||
url = "".join(("http://en.wikipedia.org/wiki/", page)) | |||||
url = "http://{0}.org/wiki/{1}".format(self.chan[1:], page) | |||||
self.is_edit = False # This is a log entry, not edit | self.is_edit = False # This is a log entry, not edit | ||||
@@ -61,35 +73,24 @@ class RC(object): | |||||
def prettify(self): | def prettify(self): | ||||
"""Make a nice, colorful message to send back to the IRC front-end.""" | """Make a nice, colorful message to send back to the IRC front-end.""" | ||||
flags = self.flags | flags = self.flags | ||||
# "New <event>:" if we don't know exactly what happened: | |||||
event_type = flags | |||||
if "N" in flags: | |||||
event_type = "page" # "New page:" | |||||
elif flags == "delete": | |||||
event_type = "deletion" # "New deletion:" | |||||
if self.is_edit: | |||||
if "N" in flags: | |||||
event = "page" # "New page:" | |||||
else: | |||||
event = "edit" # "New edit:" | |||||
if "B" in flags: | |||||
event = "bot edit" # "New bot edit:" | |||||
if "M" in flags: | |||||
event = "minor " + event # "New minor (bot)? edit:" | |||||
return self.pretty_edit.format(event, self.page, self.user, | |||||
self.url, self.comment) | |||||
if flags == "delete": | |||||
event = "deletion" # "New deletion:" | |||||
elif flags == "protect": | elif flags == "protect": | ||||
event_type = "protection" # "New protection:" | |||||
event = "protection" # "New protection:" | |||||
elif flags == "create": | elif flags == "create": | ||||
event_type = "user" # "New user:" | |||||
if self.page == "Special:Log/move": | |||||
event_type = "move" # New move: | |||||
event = "user" # "New user:" | |||||
else: | else: | ||||
event_type = "edit" # "New edit:" | |||||
if "B" in flags: | |||||
# "New bot edit:" | |||||
event_type = "bot {}".format(event_type) | |||||
if "M" in flags: | |||||
# "New minor edit:" OR "New minor bot edit:" | |||||
event_type = "minor {}".format(event_type) | |||||
# Example formatting: | |||||
# New edit: [[Page title]] * User name * http://en... * edit summary | |||||
if self.is_edit: | |||||
return "".join(("\x02New ", event_type, "\x0F: \x0314[[\x0307", | |||||
self.page, "\x0314]]\x0306 *\x0303 ", self.user, | |||||
"\x0306 *\x0302 ", self.url, "\x0306 *\x0310 ", | |||||
self.comment)) | |||||
return "".join(("\x02New ", event_type, "\x0F: \x0303", self.user, | |||||
"\x0306 *\x0302 ", self.url, "\x0306 *\x0310 ", | |||||
self.comment)) | |||||
event = flags # Works for "move", "block", etc | |||||
return self.pretty_log.format(event, self.user, self.url, self.comment) |
@@ -0,0 +1,125 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
import imp | |||||
from earwigbot.irc import IRCConnection, RC | |||||
__all__ = ["Watcher"] | |||||
class Watcher(IRCConnection): | |||||
""" | |||||
**EarwigBot: IRC Watcher Component** | |||||
The IRC watcher runs on a wiki recent-changes server and listens for | |||||
edits. Users cannot interact with this part of the bot. When an event | |||||
occurs, we run it through some rules stored in our config, which can result | |||||
in wiki bot tasks being started or messages being sent to channels on the | |||||
IRC frontend. | |||||
""" | |||||
def __init__(self, bot): | |||||
self.bot = bot | |||||
self.logger = bot.logger.getChild("watcher") | |||||
cf = bot.config.irc["watcher"] | |||||
base = super(Watcher, self) | |||||
base.__init__(cf["host"], cf["port"], cf["nick"], cf["ident"], | |||||
cf["realname"]) | |||||
self._prepare_process_hook() | |||||
self._connect() | |||||
def __repr__(self): | |||||
"""Return the canonical string representation of the Watcher.""" | |||||
res = "Watcher(host={0!r}, port={1!r}, nick={2!r}, ident={3!r}, realname={4!r}, bot={5!r})" | |||||
return res.format(self.host, self.port, self.nick, self.ident, | |||||
self.realname, self.bot) | |||||
def __str__(self): | |||||
"""Return a nice string representation of the Watcher.""" | |||||
res = "<Watcher {0}!{1} at {2}:{3}>" | |||||
return res.format(self.nick, self.ident, self.host, self.port) | |||||
def _process_message(self, line): | |||||
"""Process a single message from IRC.""" | |||||
if line[1] == "PRIVMSG": | |||||
chan = line[2] | |||||
# Ignore messages originating from channels not in our list, to | |||||
# prevent someone PMing us false data: | |||||
if chan not in self.bot.config.irc["watcher"]["channels"]: | |||||
return | |||||
msg = " ".join(line[3:])[1:] | |||||
rc = RC(chan, msg) # New RC object to store this event's data | |||||
rc.parse() # Parse a message into pagenames, usernames, etc. | |||||
self._process_rc_event(rc) | |||||
# When we've finished starting up, join all watcher channels: | |||||
elif line[1] == "376": | |||||
for chan in self.bot.config.irc["watcher"]["channels"]: | |||||
self.join(chan) | |||||
def _prepare_process_hook(self): | |||||
"""Create our RC event process hook from information in config. | |||||
This will get put in the function self._process_hook, which takes the | |||||
Bot object and an RC object and returns a list of frontend channels to | |||||
report this event to. | |||||
""" | |||||
# Set a default RC process hook that does nothing: | |||||
self._process_hook = lambda rc: () | |||||
try: | |||||
rules = self.bot.config.data["rules"] | |||||
except KeyError: | |||||
return | |||||
module = imp.new_module("_rc_event_processing_rules") | |||||
path = self.bot.config.path | |||||
try: | |||||
exec compile(rules, path, "exec") in module.__dict__ | |||||
except Exception: | |||||
e = "Could not compile config file's RC event rules:" | |||||
self.logger.exception(e) | |||||
return | |||||
self._process_hook_module = module | |||||
try: | |||||
self._process_hook = module.process | |||||
except AttributeError: | |||||
e = "RC event rules compiled correctly, but no process(bot, rc) function was found" | |||||
self.logger.error(e) | |||||
return | |||||
def _process_rc_event(self, rc): | |||||
"""Process a recent change event from IRC (or, an RC object). | |||||
The actual processing is configurable, so we don't have that hard-coded | |||||
here. We simply call our process hook (self._process_hook), created by | |||||
self._prepare_process_hook() from information in the "rules" section of | |||||
our config. | |||||
""" | |||||
chans = self._process_hook(self.bot, rc) | |||||
with self.bot.component_lock: | |||||
frontend = self.bot.frontend | |||||
if chans and frontend and not frontend.is_stopped(): | |||||
pretty = rc.prettify() | |||||
for chan in chans: | |||||
frontend.say(chan, pretty) |
@@ -1,146 +0,0 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
""" | |||||
EarwigBot's Main Module | |||||
The core is essentially responsible for starting the various bot components | |||||
(irc, scheduler, etc) and making sure they are all happy. An explanation of the | |||||
different components follows: | |||||
EarwigBot has three components that can run independently of each other: an IRC | |||||
front-end, an IRC watcher, and a wiki scheduler. | |||||
* The IRC front-end runs on a normal IRC server and expects users to interact | |||||
with it/give it commands. | |||||
* The IRC watcher runs on a wiki recent-changes server and listens for edits. | |||||
Users cannot interact with this part of the bot. | |||||
* The wiki scheduler runs wiki-editing bot tasks in separate threads at | |||||
user-defined times through a cron-like interface. | |||||
There is a "priority" system here: | |||||
1. If the IRC frontend is enabled, it will run on the main thread, and the IRC | |||||
watcher and wiki scheduler (if enabled) will run on separate threads. | |||||
2. If the wiki scheduler is enabled, it will run on the main thread, and the | |||||
IRC watcher (if enabled) will run on a separate thread. | |||||
3. If the IRC watcher is enabled, it will run on the main (and only) thread. | |||||
Else, the bot will stop, as no components are enabled. | |||||
""" | |||||
import logging | |||||
import threading | |||||
import time | |||||
from earwigbot import frontend | |||||
from earwigbot import tasks | |||||
from earwigbot import watcher | |||||
from earwigbot.config import config | |||||
logger = logging.getLogger("earwigbot") | |||||
f_conn = None | |||||
w_conn = None | |||||
def irc_watcher(f_conn=None): | |||||
"""Function to handle the IRC watcher as another thread (if frontend and/or | |||||
scheduler is enabled), otherwise run as the main thread.""" | |||||
global w_conn | |||||
while 1: # restart the watcher component if it breaks (and nothing else) | |||||
w_conn = watcher.get_connection() | |||||
w_conn.connect() | |||||
try: | |||||
watcher.main(w_conn, f_conn) | |||||
except: | |||||
logger.exception("Watcher had an error") | |||||
time.sleep(5) # sleep a bit before restarting watcher | |||||
logger.warn("Watcher has stopped; restarting component") | |||||
def wiki_scheduler(): | |||||
"""Function to handle the wiki scheduler as another thread, or as the | |||||
primary thread if the IRC frontend is not enabled.""" | |||||
while 1: | |||||
time_start = time.time() | |||||
now = time.gmtime(time_start) | |||||
tasks.schedule(now) | |||||
time_end = time.time() | |||||
time_diff = time_start - time_end | |||||
if time_diff < 60: # sleep until the next minute | |||||
time.sleep(60 - time_diff) | |||||
def irc_frontend(): | |||||
"""If the IRC frontend is enabled, make it run on our primary thread, and | |||||
enable the wiki scheduler and IRC watcher on new threads if they are | |||||
enabled.""" | |||||
global f_conn | |||||
logger.info("Starting IRC frontend") | |||||
f_conn = frontend.get_connection() | |||||
frontend.startup(f_conn) | |||||
if "wiki_schedule" in config.components: | |||||
logger.info("Starting wiki scheduler") | |||||
tasks.load() | |||||
t_scheduler = threading.Thread(target=wiki_scheduler) | |||||
t_scheduler.name = "wiki-scheduler" | |||||
t_scheduler.daemon = True | |||||
t_scheduler.start() | |||||
if "irc_watcher" in config.components: | |||||
logger.info("Starting IRC watcher") | |||||
t_watcher = threading.Thread(target=irc_watcher, args=(f_conn,)) | |||||
t_watcher.name = "irc-watcher" | |||||
t_watcher.daemon = True | |||||
t_watcher.start() | |||||
frontend.main() | |||||
if "irc_watcher" in config.components: | |||||
w_conn.close() | |||||
f_conn.close() | |||||
def main(): | |||||
if "irc_frontend" in config.components: | |||||
# Make the frontend run on our primary thread if enabled, and enable | |||||
# additional components through that function | |||||
irc_frontend() | |||||
elif "wiki_schedule" in config.components: | |||||
# Run the scheduler on the main thread, but also run the IRC watcher on | |||||
# another thread iff it is enabled | |||||
logger.info("Starting wiki scheduler") | |||||
tasks.load() | |||||
if "irc_watcher" in enabled: | |||||
logger.info("Starting IRC watcher") | |||||
t_watcher = threading.Thread(target=irc_watcher) | |||||
t_watcher.name = "irc-watcher" | |||||
t_watcher.daemon = True | |||||
t_watcher.start() | |||||
wiki_scheduler() | |||||
elif "irc_watcher" in config.components: | |||||
# The IRC watcher is our only enabled component, so run its function | |||||
# only and don't worry about anything else: | |||||
logger.info("Starting IRC watcher") | |||||
irc_watcher() | |||||
else: # Nothing is enabled! | |||||
logger.critical("No bot parts are enabled; stopping") | |||||
exit(1) |
@@ -0,0 +1,246 @@ | |||||
#! /usr/bin/env python | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
import imp | |||||
from os import listdir, path | |||||
from re import sub | |||||
from threading import RLock, Thread | |||||
from time import gmtime, strftime | |||||
from earwigbot.commands import Command | |||||
from earwigbot.tasks import Task | |||||
__all__ = ["CommandManager", "TaskManager"] | |||||
class _ResourceManager(object): | |||||
""" | |||||
**EarwigBot: Resource Manager** | |||||
Resources are essentially objects dynamically loaded by the bot, both | |||||
packaged with it (built-in resources) and created by users (plugins, aka | |||||
custom resources). Currently, the only two types of resources are IRC | |||||
commands and bot tasks. These are both loaded from two locations: the | |||||
:py:mod:`earwigbot.commands` and :py:mod:`earwigbot.tasks packages`, and | |||||
the :file:`commands/` and :file:`tasks/` directories within the bot's | |||||
working directory. | |||||
This class handles the low-level tasks of (re)loading resources via | |||||
:py:meth:`load`, retrieving specific resources via :py:meth:`get`, and | |||||
iterating over all resources via :py:meth:`__iter__`. | |||||
""" | |||||
def __init__(self, bot, name, base): | |||||
self.bot = bot | |||||
self.logger = bot.logger.getChild(name) | |||||
self._resources = {} | |||||
self._resource_name = name # e.g. "commands" or "tasks" | |||||
self._resource_base = base # e.g. Command or Task | |||||
self._resource_access_lock = RLock() | |||||
def __repr__(self): | |||||
"""Return the canonical string representation of the manager.""" | |||||
res = "{0}(bot={1!r}, name={2!r}, base={3!r})" | |||||
return res.format(self.__class__.__name__, self.bot, | |||||
self._resource_name, self._resource_base) | |||||
def __str__(self): | |||||
"""Return a nice string representation of the manager.""" | |||||
return "<{0} of {1}>".format(self.__class__.__name__, self.bot) | |||||
def __iter__(self): | |||||
with self.lock: | |||||
for resource in self._resources.itervalues(): | |||||
yield resource | |||||
def _load_resource(self, name, path, klass): | |||||
"""Instantiate a resource class and add it to the dictionary.""" | |||||
res_type = self._resource_name[:-1] # e.g. "command" or "task" | |||||
try: | |||||
resource = klass(self.bot) # Create instance of resource | |||||
except Exception: | |||||
e = "Error instantiating {0} class in {1} (from {2})" | |||||
self.logger.exception(e.format(res_type, name, path)) | |||||
else: | |||||
self._resources[resource.name] = resource | |||||
self.logger.debug("Loaded {0} {1}".format(res_type, resource.name)) | |||||
def _load_module(self, name, path): | |||||
"""Load a specific resource from a module, identified by name and path. | |||||
We'll first try to import it using imp magic, and if that works, make | |||||
instances of any classes inside that are subclasses of the base | |||||
(:py:attr:`self._resource_base <_resource_base>`), add them to the | |||||
resources dictionary with :py:meth:`self._load_resource() | |||||
<_load_resource>`, and finally log the addition. Any problems along | |||||
the way will either be ignored or logged. | |||||
""" | |||||
f, path, desc = imp.find_module(name, [path]) | |||||
try: | |||||
module = imp.load_module(name, f, path, desc) | |||||
except Exception: | |||||
e = "Couldn't load module {0} (from {1})" | |||||
self.logger.exception(e.format(name, path)) | |||||
return | |||||
finally: | |||||
f.close() | |||||
for obj in vars(module).values(): | |||||
if type(obj) is type: | |||||
isresource = issubclass(obj, self._resource_base) | |||||
if isresource and not obj is self._resource_base: | |||||
self._load_resource(name, path, obj) | |||||
def _load_directory(self, dir): | |||||
"""Load all valid resources in a given directory.""" | |||||
self.logger.debug("Loading directory {0}".format(dir)) | |||||
processed = [] | |||||
for name in listdir(dir): | |||||
if not name.endswith(".py") and not name.endswith(".pyc"): | |||||
continue | |||||
if name.startswith("_") or name.startswith("."): | |||||
continue | |||||
modname = sub("\.pyc?$", "", name) # Remove extension | |||||
if modname not in processed: | |||||
self._load_module(modname, dir) | |||||
processed.append(modname) | |||||
@property | |||||
def lock(self): | |||||
"""The resource access/modify lock.""" | |||||
return self._resource_access_lock | |||||
def load(self): | |||||
"""Load (or reload) all valid resources into :py:attr:`_resources`.""" | |||||
name = self._resource_name # e.g. "commands" or "tasks" | |||||
with self.lock: | |||||
self._resources.clear() | |||||
builtin_dir = path.join(path.dirname(__file__), name) | |||||
plugins_dir = path.join(self.bot.config.root_dir, name) | |||||
self._load_directory(builtin_dir) # Built-in resources | |||||
self._load_directory(plugins_dir) # Custom resources, aka plugins | |||||
msg = "Loaded {0} {1}: {2}" | |||||
resources = ", ".join(self._resources.keys()) | |||||
self.logger.info(msg.format(len(self._resources), name, resources)) | |||||
def get(self, key): | |||||
"""Return the class instance associated with a certain resource. | |||||
Will raise :py:exc:`KeyError` if the resource (a command or task) is | |||||
not found. | |||||
""" | |||||
with self.lock: | |||||
return self._resources[key] | |||||
class CommandManager(_ResourceManager): | |||||
""" | |||||
Manages (i.e., loads, reloads, and calls) IRC commands. | |||||
""" | |||||
def __init__(self, bot): | |||||
super(CommandManager, self).__init__(bot, "commands", Command) | |||||
def _wrap_check(self, command, data): | |||||
"""Check whether a command should be called, catching errors.""" | |||||
try: | |||||
return command.check(data) | |||||
except Exception: | |||||
e = "Error checking command '{0}' with data: {1}:" | |||||
self.logger.exception(e.format(command.name, data)) | |||||
def _wrap_process(self, command, data): | |||||
"""process() the message, catching and reporting any errors.""" | |||||
try: | |||||
command.process(data) | |||||
except Exception: | |||||
e = "Error executing command '{0}':" | |||||
self.logger.exception(e.format(command.name)) | |||||
def call(self, hook, data): | |||||
"""Respond to a hook type and a :py:class:`Data` object.""" | |||||
for command in self: | |||||
if hook in command.hooks and self._wrap_check(command, data): | |||||
thread = Thread(target=self._wrap_process, | |||||
args=(command, data)) | |||||
start_time = strftime("%b %d %H:%M:%S") | |||||
thread.name = "irc:{0} ({1})".format(command.name, start_time) | |||||
thread.daemon = True | |||||
thread.start() | |||||
return | |||||
class TaskManager(_ResourceManager): | |||||
""" | |||||
Manages (i.e., loads, reloads, schedules, and runs) wiki bot tasks. | |||||
""" | |||||
def __init__(self, bot): | |||||
super(TaskManager, self).__init__(bot, "tasks", Task) | |||||
def _wrapper(self, task, **kwargs): | |||||
"""Wrapper for task classes: run the task and catch any errors.""" | |||||
try: | |||||
task.run(**kwargs) | |||||
except Exception: | |||||
msg = "Task '{0}' raised an exception and had to stop:" | |||||
self.logger.exception(msg.format(task.name)) | |||||
else: | |||||
msg = "Task '{0}' finished successfully" | |||||
self.logger.info(msg.format(task.name)) | |||||
def start(self, task_name, **kwargs): | |||||
"""Start a given task in a new daemon thread, and return the thread. | |||||
kwargs are passed to :py:meth:`task.run() <earwigbot.tasks.Task.run>`. | |||||
If the task is not found, ``None`` will be returned an an error is | |||||
logged. | |||||
""" | |||||
msg = "Starting task '{0}' in a new thread" | |||||
self.logger.info(msg.format(task_name)) | |||||
try: | |||||
task = self.get(task_name) | |||||
except KeyError: | |||||
e = "Couldn't find task '{0}'" | |||||
self.logger.error(e.format(task_name)) | |||||
return | |||||
task_thread = Thread(target=self._wrapper, args=(task,), kwargs=kwargs) | |||||
start_time = strftime("%b %d %H:%M:%S") | |||||
task_thread.name = "{0} ({1})".format(task_name, start_time) | |||||
task_thread.daemon = True | |||||
task_thread.start() | |||||
return task_thread | |||||
def schedule(self, now=None): | |||||
"""Start all tasks that are supposed to be run at a given time.""" | |||||
if not now: | |||||
now = gmtime() | |||||
# Get list of tasks to run this turn: | |||||
tasks = self.bot.config.schedule(now.tm_min, now.tm_hour, now.tm_mday, | |||||
now.tm_mon, now.tm_wday) | |||||
for task in tasks: | |||||
if isinstance(task, list): # They've specified kwargs, | |||||
self.start(task[0], **task[1]) # so pass those to start | |||||
else: # Otherwise, just pass task_name | |||||
self.start(task) |
@@ -1,85 +0,0 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
""" | |||||
EarwigBot's IRC Watcher Rules | |||||
This file contains (configurable!) rules that EarwigBot's watcher uses after it | |||||
recieves an event from IRC. | |||||
""" | |||||
import re | |||||
from earwigbot import tasks | |||||
afc_prefix = "wikipedia( talk)?:(wikiproject )?articles for creation" | |||||
# compile some regexps used when finding specific events | |||||
r_page = re.compile(afc_prefix) | |||||
r_ffu = re.compile("wikipedia( talk)?:files for upload") | |||||
r_move1 = re.compile("moved \[\[{}".format(afc_prefix)) | |||||
r_move2 = re.compile("moved \[\[(.*?)\]\] to \[\[{}".format(afc_prefix)) | |||||
r_moved_pages = re.compile("^moved \[\[(.*?)\]\] to \[\[(.*?)\]\]") | |||||
r_delete = re.compile("deleted \"\[\[{}".format(afc_prefix)) | |||||
r_deleted_page = re.compile("^deleted \"\[\[(.*?)\]\]") | |||||
r_restore = re.compile("restored \"\[\[{}".format(afc_prefix)) | |||||
r_restored_page = re.compile("^restored \"\[\[(.*?)\]\]") | |||||
r_protect = re.compile("protected \"\[\[{}".format(afc_prefix)) | |||||
def process(rc): | |||||
"""Given an RC() object, return a list of channels to report this event to. | |||||
Also, start any wiki bot tasks within this function if necessary.""" | |||||
chans = set() # channels to report this message to | |||||
page_name = rc.page.lower() | |||||
comment = rc.comment.lower() | |||||
if "!earwigbot" in rc.msg.lower(): | |||||
chans.update(("##earwigbot", "#wikipedia-en-afc-feed")) | |||||
if r_page.search(page_name): | |||||
#tasks.start("afc_copyvios", page=rc.page) | |||||
chans.add("#wikipedia-en-afc-feed") | |||||
elif r_ffu.match(page_name): | |||||
chans.add("#wikipedia-en-afc-feed") | |||||
elif page_name.startswith("template:afc submission"): | |||||
chans.add("#wikipedia-en-afc-feed") | |||||
elif rc.flags == "move" and (r_move1.match(comment) or | |||||
r_move2.match(comment)): | |||||
p = r_moved_pages.findall(rc.comment)[0] | |||||
chans.add("#wikipedia-en-afc-feed") | |||||
elif rc.flags == "delete" and r_delete.match(comment): | |||||
p = r_deleted_page.findall(rc.comment)[0] | |||||
chans.add("#wikipedia-en-afc-feed") | |||||
elif rc.flags == "restore" and r_restore.match(comment): | |||||
p = r_restored_page.findall(rc.comment)[0] | |||||
#tasks.start("afc_copyvios", page=p) | |||||
chans.add("#wikipedia-en-afc-feed") | |||||
elif rc.flags == "protect" and r_protect.match(comment): | |||||
chans.add("#wikipedia-en-afc-feed") | |||||
return chans |
@@ -1,65 +0,0 @@ | |||||
#! /usr/bin/env python | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
""" | |||||
EarwigBot Runner | |||||
This is a very simple script that can be run from anywhere. It will add the | |||||
'earwigbot' package to sys.path if it's not already in there (i.e., it hasn't | |||||
been "installed"), accept a root_dir (the directory in which bot.py is located) | |||||
and a decryption key from raw_input (if passwords are encrypted), then call | |||||
config.load() and decrypt any passwords, and finally call the main() function | |||||
of earwigbot.main. | |||||
""" | |||||
from os import path | |||||
import sys | |||||
def run(): | |||||
pkg_dir = path.split(path.dirname(path.abspath(__file__)))[0] | |||||
if pkg_dir not in sys.path: | |||||
sys.path.insert(0, pkg_dir) | |||||
from earwigbot.config import config | |||||
from earwigbot import main | |||||
root_dir = raw_input() | |||||
config_path = path.join(root_dir, "config.json") | |||||
log_dir = path.join(root_dir, "logs") | |||||
is_encrypted = config.load(config_path, log_dir) | |||||
if is_encrypted: | |||||
config._decryption_key = raw_input() | |||||
config.decrypt(config.wiki, "password") | |||||
config.decrypt(config.wiki, "search", "credentials", "key") | |||||
config.decrypt(config.wiki, "search", "credentials", "secret") | |||||
config.decrypt(config.irc, "frontend", "nickservPassword") | |||||
config.decrypt(config.irc, "watcher", "nickservPassword") | |||||
try: | |||||
main.main() | |||||
except KeyboardInterrupt: | |||||
main.logger.critical("KeyboardInterrupt: stopping main bot loop") | |||||
exit(1) | |||||
if __name__ == "__main__": | |||||
run() |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -20,119 +20,124 @@ | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||||
# SOFTWARE. | # SOFTWARE. | ||||
""" | |||||
EarwigBot's Wiki Task Manager | |||||
This package provides the wiki bot "tasks" EarwigBot runs. Here in __init__, | |||||
you can find some functions used to load and run these tasks. | |||||
""" | |||||
import logging | |||||
import os | |||||
import sys | |||||
import threading | |||||
import time | |||||
from earwigbot.classes import BaseTask | |||||
from earwigbot.config import config | |||||
__all__ = ["load", "schedule", "start", "get", "get_all"] | |||||
# Base directory when searching for tasks: | |||||
base_dir = os.path.dirname(os.path.abspath(__file__)) | |||||
# Store loaded tasks as a dict where the key is the task name and the value is | |||||
# an instance of the task class: | |||||
_tasks = {} | |||||
# Logger for this module: | |||||
logger = logging.getLogger("earwigbot.commands") | |||||
def _load_task(filename): | |||||
"""Try to load a specific task from a module, identified by file name.""" | |||||
global _tasks | |||||
# Strip .py from the end of the filename and join with our package name: | |||||
name = ".".join(("tasks", filename[:-3])) | |||||
try: | |||||
__import__(name) | |||||
except: | |||||
logger.exception("Couldn't load file {0}:".format(filename)) | |||||
return | |||||
task = sys.modules[name].Task() | |||||
task._setup_logger() | |||||
if not isinstance(task, BaseTask): | |||||
return | |||||
_tasks[task.name] = task | |||||
logger.debug("Added task {0}".format(task.name)) | |||||
def _wrapper(task, **kwargs): | |||||
"""Wrapper for task classes: run the task and catch any errors.""" | |||||
try: | |||||
task.run(**kwargs) | |||||
except: | |||||
error = "Task '{0}' raised an exception and had to stop" | |||||
logger.exception(error.format(task.name)) | |||||
else: | |||||
logger.info("Task '{0}' finished without error".format(task.name)) | |||||
def load(): | |||||
"""Load all valid tasks from bot/tasks/, into the _tasks variable.""" | |||||
files = os.listdir(base_dir) | |||||
files.sort() | |||||
for filename in files: | |||||
if filename.startswith("_") or not filename.endswith(".py"): | |||||
continue | |||||
try: | |||||
_load_task(filename) | |||||
except AttributeError: | |||||
pass # The file is doesn't contain a task, so just move on | |||||
logger.info("Found {0} tasks: {1}".format(len(_tasks), ', '.join(_tasks.keys()))) | |||||
def schedule(now=time.gmtime()): | |||||
"""Start all tasks that are supposed to be run at a given time.""" | |||||
# Get list of tasks to run this turn: | |||||
tasks = config.schedule(now.tm_min, now.tm_hour, now.tm_mday, now.tm_mon, | |||||
now.tm_wday) | |||||
for task in tasks: | |||||
if isinstance(task, list): # they've specified kwargs | |||||
start(task[0], **task[1]) # so pass those to start_task | |||||
else: # otherwise, just pass task_name | |||||
start(task) | |||||
from earwigbot import exceptions | |||||
from earwigbot import wiki | |||||
def start(task_name, **kwargs): | |||||
"""Start a given task in a new thread. Pass args to the task's run() | |||||
function.""" | |||||
logger.info("Starting task '{0}' in a new thread".format(task_name)) | |||||
__all__ = ["Task"] | |||||
try: | |||||
task = _tasks[task_name] | |||||
except KeyError: | |||||
error = "Couldn't find task '{0}': bot/tasks/{0}.py does not exist" | |||||
logger.error(error.format(task_name)) | |||||
return | |||||
task_thread = threading.Thread(target=lambda: _wrapper(task, **kwargs)) | |||||
start_time = time.strftime("%b %d %H:%M:%S") | |||||
task_thread.name = "{0} ({1})".format(task_name, start_time) | |||||
# Stop bot task threads automagically if the main bot stops: | |||||
task_thread.daemon = True | |||||
class Task(object): | |||||
""" | |||||
**EarwigBot: Base Bot Task** | |||||
task_thread.start() | |||||
This package provides built-in wiki bot "tasks" EarwigBot runs. Additional | |||||
tasks can be installed as plugins in the bot's working directory. | |||||
def get(task_name): | |||||
"""Return the class instance associated with a certain task name. | |||||
This class (import with ``from earwigbot.tasks import Task``) can be | |||||
subclassed to create custom bot tasks. | |||||
Will raise KeyError if the task is not found. | |||||
To run a task, use :py:meth:`bot.tasks.start(name, **kwargs) | |||||
<earwigbot.managers.TaskManager.start>`. ``**kwargs`` get passed to the | |||||
Task's :meth:`run` method. | |||||
""" | """ | ||||
return _tasks[task_name] | |||||
name = None | |||||
number = 0 | |||||
def __init__(self, bot): | |||||
"""Constructor for new tasks. | |||||
This is called once immediately after the task class is loaded by | |||||
the task manager (in :py:meth:`tasks.load() | |||||
<earwigbot.managers._ResourceManager.load>`). Don't override this | |||||
directly; if you do, remember to place ``super(Task, self).__init()`` | |||||
first. Use :py:meth:`setup` for typical task-init/setup needs. | |||||
""" | |||||
self.bot = bot | |||||
self.config = bot.config | |||||
self.logger = bot.tasks.logger.getChild(self.name) | |||||
self.setup() | |||||
def __repr__(self): | |||||
"""Return the canonical string representation of the Task.""" | |||||
res = "Task(name={0!r}, number={1!r}, bot={2!r})" | |||||
return res.format(self.name, self.number, self.bot) | |||||
def __str__(self): | |||||
"""Return a nice string representation of the Task.""" | |||||
res = "<Task {0} ({1}) of {2}>" | |||||
return res.format(self.name, self.number, self.bot) | |||||
def setup(self): | |||||
"""Hook called immediately after the task is loaded. | |||||
Does nothing by default; feel free to override. | |||||
""" | |||||
pass | |||||
def run(self, **kwargs): | |||||
"""Main entry point to run a given task. | |||||
This is called directly by :py:meth:`tasks.start() | |||||
<earwigbot.managers.TaskManager.start>` and is the main way to make a | |||||
task do stuff. *kwargs* will be any keyword arguments passed to | |||||
:py:meth:`~earwigbot.managers.TaskManager.start`, which are entirely | |||||
optional. | |||||
""" | |||||
pass | |||||
def make_summary(self, comment): | |||||
"""Make an edit summary by filling in variables in a config value. | |||||
:py:attr:`config.wiki["summary"] <earwigbot.config.BotConfig.wiki>` is | |||||
used, where ``$2`` is replaced by the main summary body, given by the | |||||
*comment* argument, and ``$1`` is replaced by the task number. | |||||
If the config value is not found, we'll just return *comment* as-is. | |||||
""" | |||||
try: | |||||
summary = self.bot.config.wiki["summary"] | |||||
except KeyError: | |||||
return comment | |||||
return summary.replace("$1", str(self.number)).replace("$2", comment) | |||||
def shutoff_enabled(self, site=None): | |||||
"""Return whether on-wiki shutoff is enabled for this task. | |||||
We check a certain page for certain content. This is determined by | |||||
our config file: :py:attr:`config.wiki["shutoff"]["page"] | |||||
<earwigbot.config.BotConfig.wiki>` is used as the title, with any | |||||
embedded ``$1`` replaced by our username and ``$2`` replaced by the | |||||
task number; and :py:attr:`config.wiki["shutoff"]["disabled"] | |||||
<earwigbot.config.BotConfig.wiki>` is used as the content. | |||||
If the page has that exact content or the page does not exist, then | |||||
shutoff is "disabled", meaning the bot is supposed to run normally, and | |||||
we return ``False``. If the page's content is something other than | |||||
what we expect, shutoff is enabled, and we return ``True``. | |||||
If a site is not provided, we'll try to use :py:attr:`self.site <site>` | |||||
if it's set. Otherwise, we'll use our default site. | |||||
""" | |||||
if not site: | |||||
if hasattr(self, "site"): | |||||
site = getattr(self, "site") | |||||
else: | |||||
site = self.bot.wiki.get_site() | |||||
def get_all(): | |||||
"""Return our dict of all loaded tasks.""" | |||||
return _tasks | |||||
try: | |||||
cfg = self.config.wiki["shutoff"] | |||||
except KeyError: | |||||
return False | |||||
title = cfg.get("page", "User:$1/Shutoff/Task $2") | |||||
username = site.get_user().name | |||||
title = title.replace("$1", username).replace("$2", str(self.number)) | |||||
page = site.get_page(title) | |||||
try: | |||||
content = page.get() | |||||
except exceptions.PageNotFoundError: | |||||
return False | |||||
if content == cfg.get("disabled", "run"): | |||||
return False | |||||
self.logger.warn("Emergency task shutoff has been enabled!") | |||||
return True |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -20,14 +20,14 @@ | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||||
# SOFTWARE. | # SOFTWARE. | ||||
from earwigbot.classes import BaseTask | |||||
from earwigbot.tasks import Task | |||||
class Task(BaseTask): | |||||
class AFCCatDelink(Task): | |||||
"""A task to delink mainspace categories in declined [[WP:AFC]] | """A task to delink mainspace categories in declined [[WP:AFC]] | ||||
submissions.""" | submissions.""" | ||||
name = "afc_catdelink" | name = "afc_catdelink" | ||||
def __init__(self): | |||||
def setup(self): | |||||
pass | pass | ||||
def run(self, **kwargs): | def run(self, **kwargs): | ||||
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -26,18 +26,16 @@ from threading import Lock | |||||
import oursql | import oursql | ||||
from earwigbot import wiki | |||||
from earwigbot.classes import BaseTask | |||||
from earwigbot.config import config | |||||
from earwigbot.tasks import Task | |||||
class Task(BaseTask): | |||||
class AFCCopyvios(Task): | |||||
"""A task to check newly-edited [[WP:AFC]] submissions for copyright | """A task to check newly-edited [[WP:AFC]] submissions for copyright | ||||
violations.""" | violations.""" | ||||
name = "afc_copyvios" | name = "afc_copyvios" | ||||
number = 1 | number = 1 | ||||
def __init__(self): | |||||
cfg = config.tasks.get(self.name, {}) | |||||
def setup(self): | |||||
cfg = self.config.tasks.get(self.name, {}) | |||||
self.template = cfg.get("template", "AfC suspected copyvio") | self.template = cfg.get("template", "AfC suspected copyvio") | ||||
self.ignore_list = cfg.get("ignoreList", []) | self.ignore_list = cfg.get("ignoreList", []) | ||||
self.min_confidence = cfg.get("minConfidence", 0.5) | self.min_confidence = cfg.get("minConfidence", 0.5) | ||||
@@ -63,20 +61,20 @@ class Task(BaseTask): | |||||
if self.shutoff_enabled(): | if self.shutoff_enabled(): | ||||
return | return | ||||
title = kwargs["page"] | title = kwargs["page"] | ||||
page = wiki.get_site().get_page(title) | |||||
page = self.bot.wiki.get_site().get_page(title) | |||||
with self.db_access_lock: | with self.db_access_lock: | ||||
self.conn = oursql.connect(**self.conn_data) | self.conn = oursql.connect(**self.conn_data) | ||||
self.process(page) | self.process(page) | ||||
def process(self, page): | def process(self, page): | ||||
"""Detect copyvios in 'page' and add a note if any are found.""" | """Detect copyvios in 'page' and add a note if any are found.""" | ||||
title = page.title() | |||||
title = page.title | |||||
if title in self.ignore_list: | if title in self.ignore_list: | ||||
msg = "Skipping page in ignore list: [[{0}]]" | msg = "Skipping page in ignore list: [[{0}]]" | ||||
self.logger.info(msg.format(title)) | self.logger.info(msg.format(title)) | ||||
return | return | ||||
pageid = page.pageid() | |||||
pageid = page.pageid | |||||
if self.has_been_processed(pageid): | if self.has_been_processed(pageid): | ||||
msg = "Skipping check on already processed page [[{0}]]" | msg = "Skipping check on already processed page [[{0}]]" | ||||
self.logger.info(msg.format(title)) | self.logger.info(msg.format(title)) | ||||
@@ -89,9 +87,9 @@ class Task(BaseTask): | |||||
if result.violation: | if result.violation: | ||||
content = page.get() | content = page.get() | ||||
template = "\{\{{0}|url={1}|confidence={2}\}\}" | |||||
template = "\{\{{0}|url={1}|confidence={2}\}\}\n" | |||||
template = template.format(self.template, url, confidence) | template = template.format(self.template, url, confidence) | ||||
newtext = "\n".join((template, content)) | |||||
newtext = template + content | |||||
if "{url}" in self.summary: | if "{url}" in self.summary: | ||||
page.edit(newtext, self.summary.format(url=url)) | page.edit(newtext, self.summary.format(url=url)) | ||||
else: | else: | ||||
@@ -140,10 +138,10 @@ class Task(BaseTask): | |||||
be) retained for one day; this task does not remove old entries (that | be) retained for one day; this task does not remove old entries (that | ||||
is handled by the Toolserver component). | is handled by the Toolserver component). | ||||
This will only be called if "cache_results" == True in the task's, | |||||
This will only be called if "cache_results" == True in the task's | |||||
config, which is False by default. | config, which is False by default. | ||||
""" | """ | ||||
pageid = page.pageid() | |||||
pageid = page.pageid | |||||
hash = sha256(page.get()).hexdigest() | hash = sha256(page.get()).hexdigest() | ||||
query1 = "SELECT 1 FROM cache WHERE cache_id = ?" | query1 = "SELECT 1 FROM cache WHERE cache_id = ?" | ||||
query2 = "DELETE FROM cache WHERE cache_id = ?" | query2 = "DELETE FROM cache WHERE cache_id = ?" | ||||
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -20,14 +20,14 @@ | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||||
# SOFTWARE. | # SOFTWARE. | ||||
from earwigbot.classes import BaseTask | |||||
from earwigbot.tasks import Task | |||||
class Task(BaseTask): | |||||
class AFCDailyCats(Task): | |||||
""" A task to create daily categories for [[WP:AFC]].""" | """ A task to create daily categories for [[WP:AFC]].""" | ||||
name = "afc_dailycats" | name = "afc_dailycats" | ||||
number = 3 | number = 3 | ||||
def __init__(self): | |||||
def setup(self): | |||||
pass | pass | ||||
def run(self, **kwargs): | def run(self, **kwargs): | ||||
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -32,16 +32,9 @@ from numpy import arange | |||||
import oursql | import oursql | ||||
from earwigbot import wiki | from earwigbot import wiki | ||||
from earwigbot.classes import BaseTask | |||||
from earwigbot.config import config | |||||
from earwigbot.tasks import Task | |||||
# Valid submission statuses: | |||||
STATUS_NONE = 0 | |||||
STATUS_PEND = 1 | |||||
STATUS_DECLINE = 2 | |||||
STATUS_ACCEPT = 3 | |||||
class Task(BaseTask): | |||||
class AFCHistory(Task): | |||||
"""A task to generate charts about AfC submissions over time. | """A task to generate charts about AfC submissions over time. | ||||
The main function of the task is to work through the "AfC submissions by | The main function of the task is to work through the "AfC submissions by | ||||
@@ -57,8 +50,14 @@ class Task(BaseTask): | |||||
""" | """ | ||||
name = "afc_history" | name = "afc_history" | ||||
def __init__(self): | |||||
cfg = config.tasks.get(self.name, {}) | |||||
# Valid submission statuses: | |||||
STATUS_NONE = 0 | |||||
STATUS_PEND = 1 | |||||
STATUS_DECLINE = 2 | |||||
STATUS_ACCEPT = 3 | |||||
def setup(self): | |||||
cfg = self.config.tasks.get(self.name, {}) | |||||
self.num_days = cfg.get("days", 90) | self.num_days = cfg.get("days", 90) | ||||
self.categories = cfg.get("categories", {}) | self.categories = cfg.get("categories", {}) | ||||
@@ -73,10 +72,10 @@ class Task(BaseTask): | |||||
self.db_access_lock = Lock() | self.db_access_lock = Lock() | ||||
def run(self, **kwargs): | def run(self, **kwargs): | ||||
self.site = wiki.get_site() | |||||
self.site = self.bot.wiki.get_site() | |||||
with self.db_access_lock: | with self.db_access_lock: | ||||
self.conn = oursql.connect(**self.conn_data) | self.conn = oursql.connect(**self.conn_data) | ||||
action = kwargs.get("action") | action = kwargs.get("action") | ||||
try: | try: | ||||
num_days = int(kwargs.get("days", self.num_days)) | num_days = int(kwargs.get("days", self.num_days)) | ||||
@@ -90,9 +89,9 @@ class Task(BaseTask): | |||||
def update(self, num_days): | def update(self, num_days): | ||||
self.logger.info("Updating past {0} days".format(num_days)) | self.logger.info("Updating past {0} days".format(num_days)) | ||||
generator = self.backwards_cat_iterator() | generator = self.backwards_cat_iterator() | ||||
for d in xrange(num_days): | |||||
for i in xrange(num_days): | |||||
category = generator.next() | category = generator.next() | ||||
date = category.title().split("/")[-1] | |||||
date = category.title.split("/")[-1] | |||||
self.update_date(date, category) | self.update_date(date, category) | ||||
sleep(10) | sleep(10) | ||||
self.logger.info("Update complete") | self.logger.info("Update complete") | ||||
@@ -101,9 +100,9 @@ class Task(BaseTask): | |||||
self.logger.info("Generating chart for past {0} days".format(num_days)) | self.logger.info("Generating chart for past {0} days".format(num_days)) | ||||
data = OrderedDict() | data = OrderedDict() | ||||
generator = self.backwards_cat_iterator() | generator = self.backwards_cat_iterator() | ||||
for d in xrange(num_days): | |||||
for i in xrange(num_days): | |||||
category = generator.next() | category = generator.next() | ||||
date = category.title().split("/")[-1] | |||||
date = category.title.split("/")[-1] | |||||
data[date] = self.get_date_counts(date) | data[date] = self.get_date_counts(date) | ||||
data = OrderedDict(reversed(data.items())) # Oldest to most recent | data = OrderedDict(reversed(data.items())) # Oldest to most recent | ||||
@@ -122,14 +121,14 @@ class Task(BaseTask): | |||||
current -= timedelta(1) # Subtract one day from date | current -= timedelta(1) # Subtract one day from date | ||||
def update_date(self, date, category): | def update_date(self, date, category): | ||||
msg = "Updating {0} ([[{1}]])".format(date, category.title()) | |||||
msg = "Updating {0} ([[{1}]])".format(date, category.title) | |||||
self.logger.debug(msg) | self.logger.debug(msg) | ||||
q_select = "SELECT page_date, page_status FROM page WHERE page_id = ?" | q_select = "SELECT page_date, page_status FROM page WHERE page_id = ?" | ||||
q_delete = "DELETE FROM page WHERE page_id = ?" | q_delete = "DELETE FROM page WHERE page_id = ?" | ||||
q_update = "UPDATE page SET page_date = ?, page_status = ? WHERE page_id = ?" | q_update = "UPDATE page SET page_date = ?, page_status = ? WHERE page_id = ?" | ||||
q_insert = "INSERT INTO page VALUES (?, ?, ?)" | q_insert = "INSERT INTO page VALUES (?, ?, ?)" | ||||
members = category.members(use_sql=True) | |||||
members = category.get_members() | |||||
with self.conn.cursor() as cursor: | with self.conn.cursor() as cursor: | ||||
for title, pageid in members: | for title, pageid in members: | ||||
@@ -137,7 +136,7 @@ class Task(BaseTask): | |||||
stored = cursor.fetchall() | stored = cursor.fetchall() | ||||
status = self.get_status(title, pageid) | status = self.get_status(title, pageid) | ||||
if status == STATUS_NONE: | |||||
if status == self.STATUS_NONE: | |||||
if stored: | if stored: | ||||
cursor.execute(q_delete, (pageid,)) | cursor.execute(q_delete, (pageid,)) | ||||
continue | continue | ||||
@@ -152,17 +151,17 @@ class Task(BaseTask): | |||||
def get_status(self, title, pageid): | def get_status(self, title, pageid): | ||||
page = self.site.get_page(title) | page = self.site.get_page(title) | ||||
ns = page.namespace() | |||||
ns = page.namespace | |||||
if ns == wiki.NS_FILE_TALK: # Ignore accepted FFU requests | if ns == wiki.NS_FILE_TALK: # Ignore accepted FFU requests | ||||
return STATUS_NONE | |||||
return self.STATUS_NONE | |||||
if ns == wiki.NS_TALK: | if ns == wiki.NS_TALK: | ||||
new_page = page.toggle_talk() | new_page = page.toggle_talk() | ||||
sleep(2) | sleep(2) | ||||
if new_page.is_redirect(): | |||||
return STATUS_NONE # Ignore accepted AFC/R requests | |||||
return STATUS_ACCEPT | |||||
if new_page.is_redirect: | |||||
return self.STATUS_NONE # Ignore accepted AFC/R requests | |||||
return self.STATUS_ACCEPT | |||||
cats = self.categories | cats = self.categories | ||||
sq = self.site.sql_query | sq = self.site.sql_query | ||||
@@ -170,16 +169,16 @@ class Task(BaseTask): | |||||
match = lambda cat: list(sq(query, (cat.replace(" ", "_"), pageid))) | match = lambda cat: list(sq(query, (cat.replace(" ", "_"), pageid))) | ||||
if match(cats["pending"]): | if match(cats["pending"]): | ||||
return STATUS_PEND | |||||
return self.STATUS_PEND | |||||
elif match(cats["unsubmitted"]): | elif match(cats["unsubmitted"]): | ||||
return STATUS_NONE | |||||
return self.STATUS_NONE | |||||
elif match(cats["declined"]): | elif match(cats["declined"]): | ||||
return STATUS_DECLINE | |||||
return STATUS_NONE | |||||
return self.STATUS_DECLINE | |||||
return self.STATUS_NONE | |||||
def get_date_counts(self, date): | def get_date_counts(self, date): | ||||
query = "SELECT COUNT(*) FROM page WHERE page_date = ? AND page_status = ?" | query = "SELECT COUNT(*) FROM page WHERE page_date = ? AND page_status = ?" | ||||
statuses = [STATUS_PEND, STATUS_DECLINE, STATUS_ACCEPT] | |||||
statuses = [self.STATUS_PEND, self.STATUS_DECLINE, self.STATUS_ACCEPT] | |||||
counts = {} | counts = {} | ||||
with self.conn.cursor() as cursor: | with self.conn.cursor() as cursor: | ||||
for status in statuses: | for status in statuses: | ||||
@@ -193,9 +192,9 @@ class Task(BaseTask): | |||||
plt.xlabel(self.graph.get("xaxis", "Date")) | plt.xlabel(self.graph.get("xaxis", "Date")) | ||||
plt.ylabel(self.graph.get("yaxis", "Submissions")) | plt.ylabel(self.graph.get("yaxis", "Submissions")) | ||||
pends = [d[STATUS_PEND] for d in data.itervalues()] | |||||
declines = [d[STATUS_DECLINE] for d in data.itervalues()] | |||||
accepts = [d[STATUS_ACCEPT] for d in data.itervalues()] | |||||
pends = [d[self.STATUS_PEND] for d in data.itervalues()] | |||||
declines = [d[self.STATUS_DECLINE] for d in data.itervalues()] | |||||
accepts = [d[self.STATUS_ACCEPT] for d in data.itervalues()] | |||||
pends_declines = [p + d for p, d in zip(pends, declines)] | pends_declines = [p + d for p, d in zip(pends, declines)] | ||||
ind = arange(len(data)) | ind = arange(len(data)) | ||||
xsize = self.graph.get("xsize", 1200) | xsize = self.graph.get("xsize", 1200) | ||||
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -21,7 +21,6 @@ | |||||
# SOFTWARE. | # SOFTWARE. | ||||
from datetime import datetime | from datetime import datetime | ||||
import logging | |||||
import re | import re | ||||
from os.path import expanduser | from os.path import expanduser | ||||
from threading import Lock | from threading import Lock | ||||
@@ -29,20 +28,11 @@ from time import sleep | |||||
import oursql | import oursql | ||||
from earwigbot import exceptions | |||||
from earwigbot import wiki | from earwigbot import wiki | ||||
from earwigbot.classes import BaseTask | |||||
from earwigbot.config import config | |||||
# Chart status number constants: | |||||
CHART_NONE = 0 | |||||
CHART_PEND = 1 | |||||
CHART_DRAFT = 2 | |||||
CHART_REVIEW = 3 | |||||
CHART_ACCEPT = 4 | |||||
CHART_DECLINE = 5 | |||||
CHART_MISPLACE = 6 | |||||
class Task(BaseTask): | |||||
from earwigbot.tasks import Task | |||||
class AFCStatistics(Task): | |||||
"""A task to generate statistics for WikiProject Articles for Creation. | """A task to generate statistics for WikiProject Articles for Creation. | ||||
Statistics are stored in a MySQL database ("u_earwig_afc_statistics") | Statistics are stored in a MySQL database ("u_earwig_afc_statistics") | ||||
@@ -53,8 +43,17 @@ class Task(BaseTask): | |||||
name = "afc_statistics" | name = "afc_statistics" | ||||
number = 2 | number = 2 | ||||
def __init__(self): | |||||
self.cfg = cfg = config.tasks.get(self.name, {}) | |||||
# Chart status number constants: | |||||
CHART_NONE = 0 | |||||
CHART_PEND = 1 | |||||
CHART_DRAFT = 2 | |||||
CHART_REVIEW = 3 | |||||
CHART_ACCEPT = 4 | |||||
CHART_DECLINE = 5 | |||||
CHART_MISPLACE = 6 | |||||
def setup(self): | |||||
self.cfg = cfg = self.config.tasks.get(self.name, {}) | |||||
# Set some wiki-related attributes: | # Set some wiki-related attributes: | ||||
self.pagename = cfg.get("page", "Template:AFC statistics") | self.pagename = cfg.get("page", "Template:AFC statistics") | ||||
@@ -83,22 +82,30 @@ class Task(BaseTask): | |||||
(self.save()). We will additionally create an SQL connection with our | (self.save()). We will additionally create an SQL connection with our | ||||
local database. | local database. | ||||
""" | """ | ||||
self.site = wiki.get_site() | |||||
with self.db_access_lock: | |||||
self.conn = oursql.connect(**self.conn_data) | |||||
action = kwargs.get("action") | |||||
if not self.db_access_lock.acquire(False): # Non-blocking | |||||
if action == "sync": | |||||
self.logger.info("A sync is already ongoing; aborting") | |||||
return | |||||
self.logger.info("Waiting for database access lock") | |||||
self.db_access_lock.acquire() | |||||
action = kwargs.get("action") | |||||
try: | |||||
self.site = self.bot.wiki.get_site() | |||||
self.conn = oursql.connect(**self.conn_data) | |||||
try: | try: | ||||
if action == "save": | if action == "save": | ||||
self.save(**kwargs) | |||||
self.save(kwargs) | |||||
elif action == "sync": | elif action == "sync": | ||||
self.sync(**kwargs) | |||||
self.sync(kwargs) | |||||
elif action == "update": | elif action == "update": | ||||
self.update(**kwargs) | |||||
self.update(kwargs) | |||||
finally: | finally: | ||||
self.conn.close() | self.conn.close() | ||||
finally: | |||||
self.db_access_lock.release() | |||||
def save(self, **kwargs): | |||||
def save(self, kwargs): | |||||
"""Save our local statistics to the wiki. | """Save our local statistics to the wiki. | ||||
After checking for emergency shutoff, the statistics chart is compiled, | After checking for emergency shutoff, the statistics chart is compiled, | ||||
@@ -107,7 +114,7 @@ class Task(BaseTask): | |||||
""" | """ | ||||
self.logger.info("Saving chart") | self.logger.info("Saving chart") | ||||
if kwargs.get("fromIRC"): | if kwargs.get("fromIRC"): | ||||
summary = " ".join((self.summary, "(!earwigbot)")) | |||||
summary = self.summary + " (!earwigbot)" | |||||
else: | else: | ||||
if self.shutoff_enabled(): | if self.shutoff_enabled(): | ||||
return | return | ||||
@@ -117,17 +124,18 @@ class Task(BaseTask): | |||||
page = self.site.get_page(self.pagename) | page = self.site.get_page(self.pagename) | ||||
text = page.get().encode("utf8") | text = page.get().encode("utf8") | ||||
newtext = re.sub("(<!-- stat begin -->)(.*?)(<!-- stat end -->)", | |||||
statistics.join(("\\1\n", "\n\\3")), text, | |||||
flags=re.DOTALL) | |||||
newtext = re.sub("<!-- stat begin -->(.*?)<!-- stat end -->", | |||||
"<!-- stat begin -->\n" + statistics + "\n<!-- stat end -->", | |||||
text, flags=re.DOTALL) | |||||
if newtext == text: | if newtext == text: | ||||
self.logger.info("Chart unchanged; not saving") | self.logger.info("Chart unchanged; not saving") | ||||
return # Don't edit the page if we're not adding anything | return # Don't edit the page if we're not adding anything | ||||
newtext = re.sub("(<!-- sig begin -->)(.*?)(<!-- sig end -->)", | |||||
"\\1~~~ at ~~~~~\\3", newtext) | |||||
newtext = re.sub("<!-- sig begin -->(.*?)<!-- sig end -->", | |||||
"<!-- sig begin -->~~~ at ~~~~~<!-- sig end -->", | |||||
newtext) | |||||
page.edit(newtext, summary, minor=True, bot=True) | page.edit(newtext, summary, minor=True, bot=True) | ||||
self.logger.info("Chart saved to [[{0}]]".format(page.title())) | |||||
self.logger.info(u"Chart saved to [[{0}]]".format(page.title)) | |||||
def compile_charts(self): | def compile_charts(self): | ||||
"""Compile and return all statistics information from our local db.""" | """Compile and return all statistics information from our local db.""" | ||||
@@ -142,10 +150,10 @@ class Task(BaseTask): | |||||
"""Compile and return a single statistics chart.""" | """Compile and return a single statistics chart.""" | ||||
chart_id, chart_title, special_title = chart_info | chart_id, chart_title, special_title = chart_info | ||||
chart = "|".join((self.tl_header, chart_title)) | |||||
chart = self.tl_header + "|" + chart_title | |||||
if special_title: | if special_title: | ||||
chart += "".join(("|", special_title)) | |||||
chart = "".join(("{{", chart, "}}")) | |||||
chart += "|" + special_title | |||||
chart = "{{" + chart + "}}" | |||||
query = "SELECT * FROM page JOIN row ON page_id = row_id WHERE row_chart = ?" | query = "SELECT * FROM page JOIN row ON page_id = row_id WHERE row_chart = ?" | ||||
with self.conn.cursor(oursql.DictCursor) as cursor: | with self.conn.cursor(oursql.DictCursor) as cursor: | ||||
@@ -153,7 +161,7 @@ class Task(BaseTask): | |||||
for page in cursor: | for page in cursor: | ||||
chart += "\n" + self.compile_chart_row(page).decode("utf8") | chart += "\n" + self.compile_chart_row(page).decode("utf8") | ||||
chart += "".join(("\n{{", self.tl_footer, "}}")) | |||||
chart += "\n{{" + self.tl_footer + "}}" | |||||
return chart | return chart | ||||
def compile_chart_row(self, page): | def compile_chart_row(self, page): | ||||
@@ -163,31 +171,22 @@ class Task(BaseTask): | |||||
table, where keys are column names and values are their cell contents. | table, where keys are column names and values are their cell contents. | ||||
""" | """ | ||||
row = "{0}|s={page_status}|t={page_title}|h={page_short}|z={page_size}|" | row = "{0}|s={page_status}|t={page_title}|h={page_short}|z={page_size}|" | ||||
row += "sr={page_special_user}|sh={page_special_hidden}|sd={page_special_time}|si={page_special_oldid}|" | |||||
row += "mr={page_modify_user}|mh={page_modify_hidden}|md={page_modify_time}|mi={page_modify_oldid}" | |||||
row += "sr={page_special_user}|sd={page_special_time}|si={page_special_oldid}|" | |||||
row += "mr={page_modify_user}|md={page_modify_time}|mi={page_modify_oldid}" | |||||
page["page_special_hidden"] = self.format_hidden(page["page_special_time"]) | |||||
page["page_modify_hidden"] = self.format_hidden(page["page_modify_time"]) | |||||
page["page_special_time"] = self.format_time(page["page_special_time"]) | page["page_special_time"] = self.format_time(page["page_special_time"]) | ||||
page["page_modify_time"] = self.format_time(page["page_modify_time"]) | page["page_modify_time"] = self.format_time(page["page_modify_time"]) | ||||
if page["page_notes"]: | if page["page_notes"]: | ||||
row += "|n=1{page_notes}" | row += "|n=1{page_notes}" | ||||
return "".join(("{{", row.format(self.tl_row, **page), "}}")) | |||||
return "{{" + row.format(self.tl_row, **page) + "}}" | |||||
def format_time(self, dt): | def format_time(self, dt): | ||||
"""Format a datetime into the standard MediaWiki timestamp format.""" | """Format a datetime into the standard MediaWiki timestamp format.""" | ||||
return dt.strftime("%H:%M, %d %b %Y") | return dt.strftime("%H:%M, %d %b %Y") | ||||
def format_hidden(self, dt): | |||||
"""Convert a datetime into seconds since the epoch. | |||||
This is used by the template as a hidden sortkey. | |||||
""" | |||||
return int((dt - datetime(1970, 1, 1)).total_seconds()) | |||||
def sync(self, **kwargs): | |||||
def sync(self, kwargs): | |||||
"""Synchronize our local statistics database with the site. | """Synchronize our local statistics database with the site. | ||||
Syncing involves, in order, updating tracked submissions that have | Syncing involves, in order, updating tracked submissions that have | ||||
@@ -205,7 +204,7 @@ class Task(BaseTask): | |||||
replag = self.site.get_replag() | replag = self.site.get_replag() | ||||
self.logger.debug("Server replag is {0}".format(replag)) | self.logger.debug("Server replag is {0}".format(replag)) | ||||
if replag > 600 and not kwargs.get("ignore_replag"): | if replag > 600 and not kwargs.get("ignore_replag"): | ||||
msg = "Sync canceled as replag ({0} secs) is greater than ten minutes." | |||||
msg = "Sync canceled as replag ({0} secs) is greater than ten minutes" | |||||
self.logger.warn(msg.format(replag)) | self.logger.warn(msg.format(replag)) | ||||
return | return | ||||
@@ -239,18 +238,23 @@ class Task(BaseTask): | |||||
self.untrack_page(cursor, pageid) | self.untrack_page(cursor, pageid) | ||||
continue | continue | ||||
title = title.decode("utf8") # SQL gives strings; we want Unicode | |||||
real_oldid = result[0][0] | real_oldid = result[0][0] | ||||
if oldid != real_oldid: | if oldid != real_oldid: | ||||
msg = "Updating page [[{0}]] (id: {1}) @ {2}" | |||||
msg = u"Updating page [[{0}]] (id: {1}) @ {2}" | |||||
self.logger.debug(msg.format(title, pageid, oldid)) | self.logger.debug(msg.format(title, pageid, oldid)) | ||||
self.logger.debug(" {0} -> {1}".format(oldid, real_oldid)) | self.logger.debug(" {0} -> {1}".format(oldid, real_oldid)) | ||||
body = result[0][1].replace("_", " ") | |||||
base = result[0][1].decode("utf8").replace("_", " ") | |||||
ns = self.site.namespace_id_to_name(result[0][2]) | ns = self.site.namespace_id_to_name(result[0][2]) | ||||
if ns: | if ns: | ||||
real_title = ":".join((str(ns), body)) | |||||
real_title = u":".join((ns, base)) | |||||
else: | else: | ||||
real_title = body | |||||
self.update_page(cursor, pageid, real_title) | |||||
real_title = base | |||||
try: | |||||
self.update_page(cursor, pageid, real_title) | |||||
except Exception: | |||||
e = u"Error updating page [[{0}]] (id: {1})" | |||||
self.logger.exception(e.format(real_title, pageid)) | |||||
def add_untracked(self, cursor): | def add_untracked(self, cursor): | ||||
"""Add pending submissions that are not yet tracked. | """Add pending submissions that are not yet tracked. | ||||
@@ -265,15 +269,17 @@ class Task(BaseTask): | |||||
tracked = [i[0] for i in cursor.fetchall()] | tracked = [i[0] for i in cursor.fetchall()] | ||||
category = self.site.get_category(self.pending_cat) | category = self.site.get_category(self.pending_cat) | ||||
pending = category.members(use_sql=True) | |||||
for title, pageid in pending: | |||||
if title.decode("utf8") in self.ignore_list: | |||||
for title, pageid in category.get_members(): | |||||
if title in self.ignore_list: | |||||
continue | continue | ||||
if pageid not in tracked: | if pageid not in tracked: | ||||
msg = "Tracking page [[{0}]] (id: {1})".format(title, pageid) | |||||
msg = u"Tracking page [[{0}]] (id: {1})".format(title, pageid) | |||||
self.logger.debug(msg) | self.logger.debug(msg) | ||||
self.track_page(cursor, pageid, title) | |||||
try: | |||||
self.track_page(cursor, pageid, title) | |||||
except Exception: | |||||
e = u"Error tracking page [[{0}]] (id: {1})" | |||||
self.logger.exception(e.format(title, pageid)) | |||||
def delete_old(self, cursor): | def delete_old(self, cursor): | ||||
"""Remove old submissions from the database. | """Remove old submissions from the database. | ||||
@@ -285,9 +291,9 @@ class Task(BaseTask): | |||||
query = """DELETE FROM page, row USING page JOIN row | query = """DELETE FROM page, row USING page JOIN row | ||||
ON page_id = row_id WHERE row_chart IN (?, ?) | ON page_id = row_id WHERE row_chart IN (?, ?) | ||||
AND ADDTIME(page_special_time, '36:00:00') < NOW()""" | AND ADDTIME(page_special_time, '36:00:00') < NOW()""" | ||||
cursor.execute(query, (CHART_ACCEPT, CHART_DECLINE)) | |||||
cursor.execute(query, (self.CHART_ACCEPT, self.CHART_DECLINE)) | |||||
def update(self, **kwargs): | |||||
def update(self, kwargs): | |||||
"""Update a page by name, regardless of whether anything has changed. | """Update a page by name, regardless of whether anything has changed. | ||||
Mainly intended as a command to be used via IRC, e.g.: | Mainly intended as a command to be used via IRC, e.g.: | ||||
@@ -297,17 +303,17 @@ class Task(BaseTask): | |||||
if not title: | if not title: | ||||
return | return | ||||
title = title.replace("_", " ") | |||||
title = title.replace("_", " ").decode("utf8") | |||||
query = "SELECT page_id, page_modify_oldid FROM page WHERE page_title = ?" | query = "SELECT page_id, page_modify_oldid FROM page WHERE page_title = ?" | ||||
with self.conn.cursor() as cursor: | with self.conn.cursor() as cursor: | ||||
cursor.execute(query, (title,)) | cursor.execute(query, (title,)) | ||||
try: | try: | ||||
pageid, oldid = cursor.fetchall()[0] | pageid, oldid = cursor.fetchall()[0] | ||||
except IndexError: | except IndexError: | ||||
msg = "Page [[{0}]] not found in database".format(title) | |||||
msg = u"Page [[{0}]] not found in database".format(title) | |||||
self.logger.error(msg) | self.logger.error(msg) | ||||
msg = "Updating page [[{0}]] (id: {1}) @ {2}" | |||||
msg = u"Updating page [[{0}]] (id: {1}) @ {2}" | |||||
self.logger.info(msg.format(title, pageid, oldid)) | self.logger.info(msg.format(title, pageid, oldid)) | ||||
self.update_page(cursor, pageid, title) | self.update_page(cursor, pageid, title) | ||||
@@ -326,14 +332,14 @@ class Task(BaseTask): | |||||
""" | """ | ||||
content = self.get_content(title) | content = self.get_content(title) | ||||
if content is None: | if content is None: | ||||
msg = "Could not get page content for [[{0}]]".format(title) | |||||
msg = u"Could not get page content for [[{0}]]".format(title) | |||||
self.logger.error(msg) | self.logger.error(msg) | ||||
return | return | ||||
namespace = self.site.get_page(title).namespace() | |||||
namespace = self.site.get_page(title).namespace | |||||
status, chart = self.get_status_and_chart(content, namespace) | status, chart = self.get_status_and_chart(content, namespace) | ||||
if chart == CHART_NONE: | |||||
msg = "Could not find a status for [[{0}]]".format(title) | |||||
if chart == self.CHART_NONE: | |||||
msg = u"Could not find a status for [[{0}]]".format(title) | |||||
self.logger.warn(msg) | self.logger.warn(msg) | ||||
return | return | ||||
@@ -346,10 +352,8 @@ class Task(BaseTask): | |||||
query1 = "INSERT INTO row VALUES (?, ?)" | query1 = "INSERT INTO row VALUES (?, ?)" | ||||
query2 = "INSERT INTO page VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)" | query2 = "INSERT INTO page VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)" | ||||
cursor.execute(query1, (pageid, chart)) | cursor.execute(query1, (pageid, chart)) | ||||
cursor.execute(query2, (pageid, status, title.decode("utf8"), | |||||
short.decode("utf8"), size, notes, | |||||
m_user.decode("utf8"), m_time, m_id, | |||||
s_user.decode("utf8"), s_time, s_id)) | |||||
cursor.execute(query2, (pageid, status, title, short, size, notes, | |||||
m_user, m_time, m_id, s_user, s_time, s_id)) | |||||
def update_page(self, cursor, pageid, title): | def update_page(self, cursor, pageid, title): | ||||
"""Update hook for when page is already in our database. | """Update hook for when page is already in our database. | ||||
@@ -360,13 +364,13 @@ class Task(BaseTask): | |||||
""" | """ | ||||
content = self.get_content(title) | content = self.get_content(title) | ||||
if content is None: | if content is None: | ||||
msg = "Could not get page content for [[{0}]]".format(title) | |||||
msg = u"Could not get page content for [[{0}]]".format(title) | |||||
self.logger.error(msg) | self.logger.error(msg) | ||||
return | return | ||||
namespace = self.site.get_page(title).namespace() | |||||
namespace = self.site.get_page(title).namespace | |||||
status, chart = self.get_status_and_chart(content, namespace) | status, chart = self.get_status_and_chart(content, namespace) | ||||
if chart == CHART_NONE: | |||||
if chart == self.CHART_NONE: | |||||
self.untrack_page(cursor, pageid) | self.untrack_page(cursor, pageid) | ||||
return | return | ||||
@@ -377,17 +381,25 @@ class Task(BaseTask): | |||||
size = self.get_size(content) | size = self.get_size(content) | ||||
m_user, m_time, m_id = self.get_modify(pageid) | m_user, m_time, m_id = self.get_modify(pageid) | ||||
notes = self.get_notes(chart, content, m_time, result["page_special_user"]) | |||||
if title != result["page_title"]: | |||||
if title != result["page_title"].decode("utf8"): | |||||
self.update_page_title(cursor, result, pageid, title) | self.update_page_title(cursor, result, pageid, title) | ||||
if m_id != result["page_modify_oldid"]: | if m_id != result["page_modify_oldid"]: | ||||
self.update_page_modify(cursor, result, pageid, size, m_user, m_time, m_id) | |||||
self.update_page_modify(cursor, result, pageid, size, m_user, | |||||
m_time, m_id) | |||||
if status != result["page_status"]: | if status != result["page_status"]: | ||||
self.update_page_status(cursor, result, pageid, status, chart) | |||||
special = self.update_page_status(cursor, result, pageid, status, | |||||
chart) | |||||
s_user = special[0] | |||||
else: | |||||
try: | |||||
s_user = result["page_special_user"].decode("utf8") | |||||
except AttributeError: # Happens if page_special_user is None | |||||
s_user = result["page_special_user"] | |||||
notes = self.get_notes(chart, content, m_time, s_user) | |||||
if notes != result["page_notes"]: | if notes != result["page_notes"]: | ||||
self.update_page_notes(cursor, result, pageid, notes) | self.update_page_notes(cursor, result, pageid, notes) | ||||
@@ -395,21 +407,21 @@ class Task(BaseTask): | |||||
"""Update the title and short_title of a page in our database.""" | """Update the title and short_title of a page in our database.""" | ||||
query = "UPDATE page SET page_title = ?, page_short = ? WHERE page_id = ?" | query = "UPDATE page SET page_title = ?, page_short = ? WHERE page_id = ?" | ||||
short = self.get_short_title(title) | short = self.get_short_title(title) | ||||
cursor.execute(query, (title.decode("utf8"), short.decode("utf8"), | |||||
pageid)) | |||||
msg = " {0}: title: {1} -> {2}" | |||||
self.logger.debug(msg.format(pageid, result["page_title"], title)) | |||||
cursor.execute(query, (title, short, pageid)) | |||||
msg = u" {0}: title: {1} -> {2}" | |||||
old_title = result["page_title"].decode("utf8") | |||||
self.logger.debug(msg.format(pageid, old_title, title)) | |||||
def update_page_modify(self, cursor, result, pageid, size, m_user, m_time, m_id): | def update_page_modify(self, cursor, result, pageid, size, m_user, m_time, m_id): | ||||
"""Update the last modified information of a page in our database.""" | """Update the last modified information of a page in our database.""" | ||||
query = """UPDATE page SET page_size = ?, page_modify_user = ?, | query = """UPDATE page SET page_size = ?, page_modify_user = ?, | ||||
page_modify_time = ?, page_modify_oldid = ? | page_modify_time = ?, page_modify_oldid = ? | ||||
WHERE page_id = ?""" | WHERE page_id = ?""" | ||||
cursor.execute(query, (size, m_user.decode("utf8"), m_time, m_id, | |||||
pageid)) | |||||
cursor.execute(query, (size, m_user, m_time, m_id, pageid)) | |||||
msg = " {0}: modify: {1} / {2} / {3} -> {4} / {5} / {6}" | |||||
msg = msg.format(pageid, result["page_modify_user"], | |||||
msg = u" {0}: modify: {1} / {2} / {3} -> {4} / {5} / {6}" | |||||
msg = msg.format(pageid, result["page_modify_user"].decode("utf8"), | |||||
result["page_modify_time"], | result["page_modify_time"], | ||||
result["page_modify_oldid"], m_user, m_time, m_id) | result["page_modify_oldid"], m_user, m_time, m_id) | ||||
self.logger.debug(msg) | self.logger.debug(msg) | ||||
@@ -428,16 +440,17 @@ class Task(BaseTask): | |||||
result["row_chart"], status, chart)) | result["row_chart"], status, chart)) | ||||
s_user, s_time, s_id = self.get_special(pageid, chart) | s_user, s_time, s_id = self.get_special(pageid, chart) | ||||
if s_id != result["page_special_oldid"]: | if s_id != result["page_special_oldid"]: | ||||
cursor.execute(query2, (s_user.decode("utf8"), s_time, s_id, | |||||
pageid)) | |||||
msg = "{0}: special: {1} / {2} / {3} -> {4} / {5} / {6}" | |||||
msg = msg.format(pageid, result["page_special_user"], | |||||
cursor.execute(query2, (s_user, s_time, s_id, pageid)) | |||||
msg = u"{0}: special: {1} / {2} / {3} -> {4} / {5} / {6}" | |||||
msg = msg.format(pageid, | |||||
result["page_special_user"].decode("utf8"), | |||||
result["page_special_time"], | result["page_special_time"], | ||||
result["page_special_oldid"], s_user, s_time, s_id) | result["page_special_oldid"], s_user, s_time, s_id) | ||||
self.logger.debug(msg) | self.logger.debug(msg) | ||||
return s_user, s_time, s_id | |||||
def update_page_notes(self, cursor, result, pageid, notes): | def update_page_notes(self, cursor, result, pageid, notes): | ||||
"""Update the notes (or warnings) of a page in our database.""" | """Update the notes (or warnings) of a page in our database.""" | ||||
query = "UPDATE page SET page_notes = ? WHERE page_id = ?" | query = "UPDATE page SET page_notes = ? WHERE page_id = ?" | ||||
@@ -454,36 +467,34 @@ class Task(BaseTask): | |||||
""" | """ | ||||
query = "SELECT page_latest FROM page WHERE page_title = ? AND page_namespace = ?" | query = "SELECT page_latest FROM page WHERE page_title = ? AND page_namespace = ?" | ||||
try: | try: | ||||
namespace, base = title.decode("utf8").split(":", 1) | |||||
namespace, base = title.split(":", 1) | |||||
except ValueError: | except ValueError: | ||||
base = title.decode("utf8") | |||||
base = title | |||||
ns = wiki.NS_MAIN | ns = wiki.NS_MAIN | ||||
else: | else: | ||||
try: | try: | ||||
ns = self.site.namespace_name_to_id(namespace) | ns = self.site.namespace_name_to_id(namespace) | ||||
except wiki.NamespaceNotFoundError: | |||||
base = title.decode("utf8") | |||||
except exceptions.NamespaceNotFoundError: | |||||
base = title | |||||
ns = wiki.NS_MAIN | ns = wiki.NS_MAIN | ||||
result = self.site.sql_query(query, (base.replace(" ", "_"), ns)) | result = self.site.sql_query(query, (base.replace(" ", "_"), ns)) | ||||
revid = int(list(result)[0][0]) | |||||
try: | |||||
revid = int(list(result)[0][0]) | |||||
except IndexError: | |||||
return None | |||||
return self.get_revision_content(revid) | return self.get_revision_content(revid) | ||||
def get_revision_content(self, revid): | |||||
def get_revision_content(self, revid, tries=1): | |||||
"""Get the content of a revision by ID from the API.""" | """Get the content of a revision by ID from the API.""" | ||||
res = self.site.api_query(action="query", prop="revisions", | res = self.site.api_query(action="query", prop="revisions", | ||||
revids=revid, rvprop="content") | revids=revid, rvprop="content") | ||||
try: | try: | ||||
return res["query"]["pages"].values()[0]["revisions"][0]["*"] | return res["query"]["pages"].values()[0]["revisions"][0]["*"] | ||||
except KeyError: | except KeyError: | ||||
sleep(5) | |||||
res = self.site.api_query(action="query", prop="revisions", | |||||
revids=revid, rvprop="content") | |||||
try: | |||||
return res["query"]["pages"].values()[0]["revisions"][0]["*"] | |||||
except KeyError: | |||||
return None | |||||
if tries > 0: | |||||
sleep(5) | |||||
return self.get_revision_content(revid, tries=tries - 1) | |||||
def get_status_and_chart(self, content, namespace): | def get_status_and_chart(self, content, namespace): | ||||
"""Determine the status and chart number of an AFC submission. | """Determine the status and chart number of an AFC submission. | ||||
@@ -498,23 +509,23 @@ class Task(BaseTask): | |||||
statuses = self.get_statuses(content) | statuses = self.get_statuses(content) | ||||
if "R" in statuses: | if "R" in statuses: | ||||
status, chart = "r", CHART_REVIEW | |||||
status, chart = "r", self.CHART_REVIEW | |||||
elif "H" in statuses: | elif "H" in statuses: | ||||
status, chart = "p", CHART_DRAFT | |||||
status, chart = "p", self.CHART_DRAFT | |||||
elif "P" in statuses: | elif "P" in statuses: | ||||
status, chart = "p", CHART_PEND | |||||
status, chart = "p", self.CHART_PEND | |||||
elif "T" in statuses: | elif "T" in statuses: | ||||
status, chart = None, CHART_NONE | |||||
status, chart = None, self.CHART_NONE | |||||
elif "D" in statuses: | elif "D" in statuses: | ||||
status, chart = "d", CHART_DECLINE | |||||
status, chart = "d", self.CHART_DECLINE | |||||
else: | else: | ||||
status, chart = None, CHART_NONE | |||||
status, chart = None, self.CHART_NONE | |||||
if namespace == wiki.NS_MAIN: | if namespace == wiki.NS_MAIN: | ||||
if not statuses: | if not statuses: | ||||
status, chart = "a", CHART_ACCEPT | |||||
status, chart = "a", self.CHART_ACCEPT | |||||
else: | else: | ||||
status, chart = None, CHART_MISPLACE | |||||
status, chart = None, self.CHART_MISPLACE | |||||
return status, chart | return status, chart | ||||
@@ -579,7 +590,7 @@ class Task(BaseTask): | |||||
""" | """ | ||||
short = re.sub("Wikipedia(\s*talk)?\:Articles\sfor\screation\/", "", title) | short = re.sub("Wikipedia(\s*talk)?\:Articles\sfor\screation\/", "", title) | ||||
if len(short) > 50: | if len(short) > 50: | ||||
short = "".join((short[:47], "...")) | |||||
short = short[:47] + "..." | |||||
return short | return short | ||||
def get_size(self, content): | def get_size(self, content): | ||||
@@ -596,7 +607,8 @@ class Task(BaseTask): | |||||
JOIN page ON rev_id = page_latest WHERE page_id = ?""" | JOIN page ON rev_id = page_latest WHERE page_id = ?""" | ||||
result = self.site.sql_query(query, (pageid,)) | result = self.site.sql_query(query, (pageid,)) | ||||
m_user, m_time, m_id = list(result)[0] | m_user, m_time, m_id = list(result)[0] | ||||
return m_user, datetime.strptime(m_time, "%Y%m%d%H%M%S"), m_id | |||||
timestamp = datetime.strptime(m_time, "%Y%m%d%H%M%S") | |||||
return m_user.decode("utf8"), timestamp, m_id | |||||
def get_special(self, pageid, chart): | def get_special(self, pageid, chart): | ||||
"""Return information about a page's "special" edit. | """Return information about a page's "special" edit. | ||||
@@ -611,25 +623,25 @@ class Task(BaseTask): | |||||
its revision ID. If the page's status is not something that involves | its revision ID. If the page's status is not something that involves | ||||
"special"-ing, we will return None for all three. The same will be | "special"-ing, we will return None for all three. The same will be | ||||
returned if we cannot determine when the page was "special"-ed, or if | returned if we cannot determine when the page was "special"-ed, or if | ||||
it was "special"-ed more than 250 edits ago. | |||||
it was "special"-ed more than 100 edits ago. | |||||
""" | """ | ||||
if chart ==CHART_NONE: | |||||
if chart == self.CHART_NONE: | |||||
return None, None, None | return None, None, None | ||||
elif chart == CHART_MISPLACE: | |||||
elif chart == self.CHART_MISPLACE: | |||||
return self.get_create(pageid) | return self.get_create(pageid) | ||||
elif chart == CHART_ACCEPT: | |||||
elif chart == self.CHART_ACCEPT: | |||||
search_for = None | search_for = None | ||||
search_not = ["R", "H", "P", "T", "D"] | search_not = ["R", "H", "P", "T", "D"] | ||||
elif chart == CHART_DRAFT: | |||||
elif chart == self.CHART_DRAFT: | |||||
search_for = "H" | search_for = "H" | ||||
search_not = [] | search_not = [] | ||||
elif chart == CHART_PEND: | |||||
elif chart == self.CHART_PEND: | |||||
search_for = "P" | search_for = "P" | ||||
search_not = [] | search_not = [] | ||||
elif chart == CHART_REVIEW: | |||||
elif chart == self.CHART_REVIEW: | |||||
search_for = "R" | search_for = "R" | ||||
search_not = [] | search_not = [] | ||||
elif chart == CHART_DECLINE: | |||||
elif chart == self.CHART_DECLINE: | |||||
search_for = "D" | search_for = "D" | ||||
search_not = ["R", "H", "P", "T"] | search_not = ["R", "H", "P", "T"] | ||||
@@ -641,11 +653,16 @@ class Task(BaseTask): | |||||
last = (None, None, None) | last = (None, None, None) | ||||
for user, ts, revid in result: | for user, ts, revid in result: | ||||
counter += 1 | counter += 1 | ||||
if counter > 100: | |||||
msg = "Exceeded 100 content lookups while determining special for page (id: {0}, chart: {1})" | |||||
if counter > 50: | |||||
msg = "Exceeded 50 content lookups while determining special for page (id: {0}, chart: {1})" | |||||
self.logger.warn(msg.format(pageid, chart)) | self.logger.warn(msg.format(pageid, chart)) | ||||
return None, None, None | return None, None, None | ||||
content = self.get_revision_content(revid) | |||||
try: | |||||
content = self.get_revision_content(revid) | |||||
except exceptions.APIError: | |||||
msg = "API error interrupted SQL query in get_special() for page (id: {0}, chart: {1})" | |||||
self.logger.exception(msg.format(pageid, chart)) | |||||
return None, None, None | |||||
statuses = self.get_statuses(content) | statuses = self.get_statuses(content) | ||||
matches = [s in statuses for s in search_not] | matches = [s in statuses for s in search_not] | ||||
if search_for: | if search_for: | ||||
@@ -654,7 +671,8 @@ class Task(BaseTask): | |||||
else: | else: | ||||
if any(matches): | if any(matches): | ||||
return last | return last | ||||
last = (user, datetime.strptime(ts, "%Y%m%d%H%M%S"), revid) | |||||
timestamp = datetime.strptime(ts, "%Y%m%d%H%M%S") | |||||
last = (user.decode("utf8"), timestamp, revid) | |||||
return last | return last | ||||
@@ -669,7 +687,8 @@ class Task(BaseTask): | |||||
(SELECT MIN(rev_id) FROM revision WHERE rev_page = ?)""" | (SELECT MIN(rev_id) FROM revision WHERE rev_page = ?)""" | ||||
result = self.site.sql_query(query, (pageid,)) | result = self.site.sql_query(query, (pageid,)) | ||||
c_user, c_time, c_id = list(result)[0] | c_user, c_time, c_id = list(result)[0] | ||||
return c_user, datetime.strptime(c_time, "%Y%m%d%H%M%S"), c_id | |||||
timestamp = datetime.strptime(c_time, "%Y%m%d%H%M%S") | |||||
return c_user.encode("utf8"), timestamp, c_id | |||||
def get_notes(self, chart, content, m_time, s_user): | def get_notes(self, chart, content, m_time, s_user): | ||||
"""Return any special notes or warnings about this page. | """Return any special notes or warnings about this page. | ||||
@@ -683,19 +702,21 @@ class Task(BaseTask): | |||||
""" | """ | ||||
notes = "" | notes = "" | ||||
ignored_charts = [CHART_NONE, CHART_ACCEPT, CHART_DECLINE] | |||||
ignored_charts = [self.CHART_NONE, self.CHART_ACCEPT, self.CHART_DECLINE] | |||||
if chart in ignored_charts: | if chart in ignored_charts: | ||||
return notes | return notes | ||||
statuses = self.get_statuses(content) | statuses = self.get_statuses(content) | ||||
if "D" in statuses and chart != CHART_MISPLACE: | |||||
if "D" in statuses and chart != self.CHART_MISPLACE: | |||||
notes += "|nr=1" # Submission was resubmitted | notes += "|nr=1" # Submission was resubmitted | ||||
if len(content) < 500: | if len(content) < 500: | ||||
notes += "|ns=1" # Submission is short | notes += "|ns=1" # Submission is short | ||||
if not re.search("\<ref\s*(.*?)\>(.*?)\</ref\>", content, re.I|re.S): | |||||
if re.search("https?:\/\/(.*?)\.", content, re.I|re.S): | |||||
if not re.search("\<ref\s*(.*?)\>(.*?)\</ref\>", content, re.I | re.S): | |||||
regex = "(https?:)|\[//(?!{0})([^ \]\\t\\n\\r\\f\\v]+?)" | |||||
sitedomain = re.escape(self.site.domain) | |||||
if re.search(regex.format(sitedomain), content, re.I | re.S): | |||||
notes += "|ni=1" # Submission has no inline citations | notes += "|ni=1" # Submission has no inline citations | ||||
else: | else: | ||||
notes += "|nu=1" # Submission is completely unsourced | notes += "|nu=1" # Submission is completely unsourced | ||||
@@ -705,12 +726,12 @@ class Task(BaseTask): | |||||
if time_since_modify > max_time: | if time_since_modify > max_time: | ||||
notes += "|no=1" # Submission hasn't been touched in over 4 days | notes += "|no=1" # Submission hasn't been touched in over 4 days | ||||
if chart in [CHART_PEND, CHART_DRAFT]: | |||||
if chart in [self.CHART_PEND, self.CHART_DRAFT] and s_user: | |||||
submitter = self.site.get_user(s_user) | submitter = self.site.get_user(s_user) | ||||
try: | try: | ||||
if submitter.blockinfo(): | |||||
if submitter.blockinfo: | |||||
notes += "|nb=1" # Submitter is blocked | notes += "|nb=1" # Submitter is blocked | ||||
except wiki.UserNotFoundError: # Likely an IP | |||||
except exceptions.UserNotFoundError: # Likely an IP | |||||
pass | pass | ||||
return notes | return notes |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -20,13 +20,13 @@ | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||||
# SOFTWARE. | # SOFTWARE. | ||||
from earwigbot.classes import BaseTask | |||||
from earwigbot.tasks import Task | |||||
class Task(BaseTask): | |||||
class AFCUndated(Task): | |||||
"""A task to clear [[Category:Undated AfC submissions]].""" | """A task to clear [[Category:Undated AfC submissions]].""" | ||||
name = "afc_undated" | name = "afc_undated" | ||||
def __init__(self): | |||||
def setup(self): | |||||
pass | pass | ||||
def run(self, **kwargs): | def run(self, **kwargs): | ||||
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -20,14 +20,14 @@ | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||||
# SOFTWARE. | # SOFTWARE. | ||||
from earwigbot.classes import BaseTask | |||||
from earwigbot.tasks import Task | |||||
class Task(BaseTask): | |||||
"""A task to add |blp=yes to {{WPB}} or {{WPBS}} when it is used along with | |||||
{{WP Biography}}.""" | |||||
name = "blptag" | |||||
class BLPTag(Task): | |||||
"""A task to add |blp=yes to ``{{WPB}}`` or ``{{WPBS}}`` when it is used | |||||
along with ``{{WP Biography}}``.""" | |||||
name = "blp_tag" | |||||
def __init__(self): | |||||
def setup(self): | |||||
pass | pass | ||||
def run(self, **kwargs): | def run(self, **kwargs): |
@@ -0,0 +1,33 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
from earwigbot.tasks import Task | |||||
class ImageDisplayResize(Task): | |||||
"""A task to resize upscaled portraits in infoboxes.""" | |||||
name = "image_display_resize" | |||||
def setup(self): | |||||
pass | |||||
def run(self, **kwargs): | |||||
pass |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -20,13 +20,13 @@ | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||||
# SOFTWARE. | # SOFTWARE. | ||||
from earwigbot.classes import BaseTask | |||||
from earwigbot.tasks import Task | |||||
class Task(BaseTask): | |||||
"""A task to create daily categories for [[WP:FEED]].""" | |||||
name = "feed_dailycats" | |||||
class WikiProjectTagger(Task): | |||||
"""A task to tag talk pages with WikiProject Banners.""" | |||||
name = "wikiproject_tagger" | |||||
def __init__(self): | |||||
def setup(self): | |||||
pass | pass | ||||
def run(self, **kwargs): | def run(self, **kwargs): |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -20,14 +20,14 @@ | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||||
# SOFTWARE. | # SOFTWARE. | ||||
from earwigbot.classes import BaseTask | |||||
from earwigbot.tasks import Task | |||||
class Task(BaseTask): | |||||
class WrongMIME(Task): | |||||
"""A task to tag files whose extensions do not agree with their MIME | """A task to tag files whose extensions do not agree with their MIME | ||||
type.""" | type.""" | ||||
name = "wrongmime" | |||||
name = "wrong_mime" | |||||
def __init__(self): | |||||
def setup(self): | |||||
pass | pass | ||||
def run(self, **kwargs): | def run(self, **kwargs): |
@@ -1,96 +0,0 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
import unittest | |||||
import random | |||||
import string | |||||
from earwigbot import blowfish | |||||
class TestBlowfish(unittest.TestCase): | |||||
def test_key_sizes(self): | |||||
b = blowfish.Blowfish | |||||
e = blowfish.KeyLengthError | |||||
self.assertRaisesRegexp(e, "no key given", b, None) | |||||
self.assertRaisesRegexp(e, "no key given", b, "") | |||||
self.assertRaisesRegexp(e, "at least", b, " " * 3) | |||||
self.assertRaisesRegexp(e, "at least", b, "1234567") | |||||
self.assertRaisesRegexp(e, "less than", b, " " * 57) | |||||
self.assertRaisesRegexp(e, "less than", b, "x" * 60) | |||||
self.assertRaisesRegexp(e, "less than", b, "1" * 128) | |||||
b("These keys should be valid!") | |||||
b("'!\"#$%&\'()*+,-./:;<=>?@[\\]^_`{|}~'") | |||||
b(" " * 8) | |||||
b(" " * 56) | |||||
def test_symmetry(self): | |||||
def _test_symmetry(): | |||||
key_length = random.randint(8, 56) | |||||
msg_length = random.randint(0, 4096) | |||||
key = "".join([random.choice(chars) for i in xrange(key_length)]) | |||||
msg = "".join([random.choice(chars) for i in xrange(msg_length)]) | |||||
enc = blowfish.encrypt(key, msg) | |||||
dec = blowfish.decrypt(key, enc) | |||||
self.assertEqual(dec, msg) | |||||
chars = string.letters + string.digits + string.punctuation | |||||
for i in xrange(8): | |||||
_test_symmetry() | |||||
def test_encryption(self): | |||||
tests = [ | |||||
("example_key", "Hello, I'm a message!", "8411a21574431176cdff9a549d27962c616014a9fe2a1fe3b0c7a823e8a1e635"), | |||||
("another random key", "Another random message! :(", "2cdcdf4e53145897ed9d4cc2433aa4bf59b087b14d0ac76a13eff12dec00e60c40857109da3c7bc4"), | |||||
("HEY LET'S TRY |°|_J|\|C7|_J/-\\710|\|", "Yes, that was my fail attempt at 1337SP33K >_>", "d4901c7c0956da3b9507cd81cd3c880d7cda25ec6c5336deb9280ce67c099eeddf7c7e052f3a946afbd92c32ae0ab8dbdd875bc5a3f0d686") | |||||
] | |||||
for test in tests: | |||||
self.assertEquals(blowfish.encrypt(test[0], test[1]), test[2]) | |||||
def test_decryption(self): | |||||
tests = [ | |||||
("blah blah blah", "ab35274c66bb8b3b03c9bd26ab477f3de06857e1d369ad35", "Blah, blah, blah!"), | |||||
("random key", "eb2fe950c5c12bca9534ffdd27631f33d3e4bcae53a634b4aaa09f9fe14c4386", "Random message as well!"), | |||||
("Okay, now I'm just desperate", "0da74e1cec41e8323da93d0c05bcf3919084130cef93021991da174fd97f8e1c9b125ed5263b41a8", "Unit testing is SO FUN ISN'T IT.") | |||||
] | |||||
for test in tests: | |||||
self.assertEquals(blowfish.decrypt(test[0], test[1]), test[2]) | |||||
def test_decryption_exceptions(self): | |||||
d = blowfish.decrypt | |||||
e = blowfish.BlowfishError | |||||
e1 = "could not be decoded" | |||||
e2 = "cannot be broken into 8-byte blocks" | |||||
e3 = "key is incorrect" | |||||
self.assertRaisesRegexp(e, e1, d, "some_key", "arr!") | |||||
self.assertRaisesRegexp(e, e2, d, "some_key", "abcd") | |||||
self.assertRaisesRegexp(e, e3, d, "some_key", "abcdabcdabcdabcd") | |||||
if __name__ == "__main__": | |||||
unittest.main(verbosity=2) |
@@ -0,0 +1,113 @@ | |||||
#! /usr/bin/env python | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
""" | |||||
usage: :command:`earwigbot [-h] [-v] [-d] [-q] [-t NAME] [PATH]` | |||||
This is EarwigBot's command-line utility, enabling you to easily start the bot | |||||
or run specific tasks. | |||||
.. glossary:: | |||||
``PATH`` | |||||
path to the bot's working directory, which will be created if it doesn't | |||||
exist; current directory assumed if not specified | |||||
``-h``, ``--help`` | |||||
show this help message and exit | |||||
``-v``, ``--version`` | |||||
show program's version number and exit | |||||
``-d``, ``--debug`` | |||||
print all logs, including ``DEBUG``-level messages | |||||
``-q``, ``--quiet`` | |||||
don't print any logs except warnings and errors | |||||
``-t NAME``, ``--task NAME`` | |||||
given the name of a task, the bot will run it instead of the main bot and | |||||
then exit | |||||
""" | |||||
from argparse import ArgumentParser | |||||
import logging | |||||
from os import path | |||||
from time import sleep | |||||
from earwigbot import __version__ | |||||
from earwigbot.bot import Bot | |||||
__all__ = ["main"] | |||||
def main(): | |||||
"""Main entry point for the command-line utility.""" | |||||
version = "EarwigBot v{0}".format(__version__) | |||||
desc = """This is EarwigBot's command-line utility, enabling you to easily | |||||
start the bot or run specific tasks.""" | |||||
parser = ArgumentParser(description=desc) | |||||
parser.add_argument("path", nargs="?", metavar="PATH", default=path.curdir, | |||||
help="""path to the bot's working directory, which will | |||||
be created if it doesn't exist; current | |||||
directory assumed if not specified""") | |||||
parser.add_argument("-v", "--version", action="version", version=version) | |||||
parser.add_argument("-d", "--debug", action="store_true", | |||||
help="print all logs, including DEBUG-level messages") | |||||
parser.add_argument("-q", "--quiet", action="store_true", | |||||
help="don't print any logs except warnings and errors") | |||||
parser.add_argument("-t", "--task", metavar="NAME", | |||||
help="""given the name of a task, the bot will run it | |||||
instead of the main bot and then exit""") | |||||
args = parser.parse_args() | |||||
level = logging.INFO | |||||
if args.debug and args.quiet: | |||||
parser.print_usage() | |||||
print "earwigbot: error: cannot show debug messages and be quiet at the same time" | |||||
return | |||||
if args.debug: | |||||
level = logging.DEBUG | |||||
elif args.quiet: | |||||
level = logging.WARNING | |||||
print version | |||||
bot = Bot(path.abspath(args.path), level=level) | |||||
if args.task: | |||||
thread = bot.tasks.start(args.task) | |||||
if not thread: | |||||
return | |||||
try: | |||||
while thread.is_alive(): # Keep it alive; it's a daemon | |||||
sleep(1) | |||||
except KeyboardInterrupt: | |||||
pass | |||||
finally: | |||||
if thread.is_alive(): | |||||
bot.tasks.logger.warn("The task is will be killed") | |||||
else: | |||||
try: | |||||
bot.run() | |||||
except KeyboardInterrupt: | |||||
pass | |||||
finally: | |||||
if bot._keep_looping: # Indicates bot hasn't already been stopped | |||||
bot.stop() | |||||
if __name__ == "__main__": | |||||
main() |
@@ -1,114 +0,0 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
""" | |||||
EarwigBot's IRC Watcher Component | |||||
The IRC watcher runs on a wiki recent-changes server and listens for edits. | |||||
Users cannot interact with this part of the bot. When an event occurs, we run | |||||
it through rules.py's process() function, which can result in wiki bot tasks | |||||
being started (located in tasks/) or messages being sent to channels on the IRC | |||||
frontend. | |||||
""" | |||||
import logging | |||||
from earwigbot import rules | |||||
from earwigbot.classes import Connection, RC, BrokenSocketException | |||||
from earwigbot.config import config | |||||
frontend_conn = None | |||||
logger = logging.getLogger("earwigbot.watcher") | |||||
def get_connection(): | |||||
"""Return a new Connection() instance with connection information. | |||||
Don't actually connect yet. | |||||
""" | |||||
cf = config.irc["watcher"] | |||||
connection = Connection(cf["host"], cf["port"], cf["nick"], cf["ident"], | |||||
cf["realname"], logger) | |||||
return connection | |||||
def main(connection, f_conn=None): | |||||
"""Main loop for the Watcher IRC Bot component. | |||||
get_connection() should have already been called and the connection should | |||||
have been started with connection.connect(). Accept the frontend connection | |||||
as well as an optional parameter in order to send messages directly to | |||||
frontend IRC channels. | |||||
""" | |||||
global frontend_conn | |||||
frontend_conn = f_conn | |||||
read_buffer = str() | |||||
while 1: | |||||
try: | |||||
read_buffer = read_buffer + connection.get() | |||||
except BrokenSocketException: | |||||
return | |||||
lines = read_buffer.split("\n") | |||||
read_buffer = lines.pop() | |||||
for line in lines: | |||||
_process_message(connection, line) | |||||
def _process_message(connection, line): | |||||
"""Process a single message from IRC.""" | |||||
line = line.strip().split() | |||||
if line[1] == "PRIVMSG": | |||||
chan = line[2] | |||||
# Ignore messages originating from channels not in our list, to prevent | |||||
# someone PMing us false data: | |||||
if chan not in config.irc["watcher"]["channels"]: | |||||
return | |||||
msg = ' '.join(line[3:])[1:] | |||||
rc = RC(msg) # new RC object to store this event's data | |||||
rc.parse() # parse a message into pagenames, usernames, etc. | |||||
process_rc(rc) # report to frontend channels or start tasks | |||||
# If we are pinged, pong back to the server: | |||||
elif line[0] == "PING": | |||||
msg = " ".join(("PONG", line[1])) | |||||
connection.send(msg) | |||||
# When we've finished starting up, join all watcher channels: | |||||
elif line[1] == "376": | |||||
for chan in config.irc["watcher"]["channels"]: | |||||
connection.join(chan) | |||||
def process_rc(rc): | |||||
"""Process a recent change event from IRC (or, an RC object). | |||||
The actual processing is configurable, so we don't have that hard-coded | |||||
here. We simply call rules's process() function and expect a list of | |||||
channels back, which we report the event data to. | |||||
""" | |||||
chans = rules.process(rc) | |||||
if chans and frontend_conn: | |||||
pretty = rc.prettify() | |||||
for chan in chans: | |||||
frontend_conn.say(chan, pretty) |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -21,24 +21,31 @@ | |||||
# SOFTWARE. | # SOFTWARE. | ||||
""" | """ | ||||
EarwigBot's Wiki Toolset | |||||
**EarwigBot: Wiki Toolset** | |||||
This is a collection of classes and functions to read from and write to | This is a collection of classes and functions to read from and write to | ||||
Wikipedia and other wiki sites. No connection whatsoever to python-wikitools | |||||
written by Mr.Z-man, other than a similar purpose. We share no code. | |||||
Wikipedia and other wiki sites. No connection whatsoever to `python-wikitools | |||||
<http://code.google.com/p/python-wikitools/>`_ written by `Mr.Z-man | |||||
<http://en.wikipedia.org/wiki/User:Mr.Z-man>`_, other than a similar purpose. | |||||
We share no code. | |||||
Import the toolset with `from earwigbot import wiki`. | |||||
Import the toolset directly with ``from earwigbot import wiki``. If using the | |||||
built-in integration with the rest of the bot, :py:class:`~earwigbot.bot.Bot` | |||||
objects contain a :py:attr:`~earwigbot.bot.Bot.wiki` attribute, which is a | |||||
:py:class:`~earwigbot.wiki.sitesdb.SitesDB` object tied to the :file:`sites.db` | |||||
file located in the same directory as :file:`config.yml`. That object has the | |||||
principal methods :py:meth:`~earwigbot.wiki.sitesdb.SitesDB.get_site`, | |||||
:py:meth:`~earwigbot.wiki.sitesdb.SitesDB.add_site`, and | |||||
:py:meth:`~earwigbot.wiki.sitesdb.SitesDB.remove_site` that should handle all | |||||
of your :py:class:`~earwigbot.wiki.site.Site` (and thus, | |||||
:py:class:`~earwigbot.wiki.page.Page`, | |||||
:py:class:`~earwigbot.wiki.category.Category`, and | |||||
:py:class:`~earwigbot.wiki.user.User`) needs. | |||||
""" | """ | ||||
import logging as _log | |||||
logger = _log.getLogger("earwigbot.wiki") | |||||
logger.addHandler(_log.NullHandler()) | |||||
from earwigbot.wiki.category import * | |||||
from earwigbot.wiki.constants import * | from earwigbot.wiki.constants import * | ||||
from earwigbot.wiki.exceptions import * | |||||
from earwigbot.wiki.functions import * | |||||
from earwigbot.wiki.category import Category | |||||
from earwigbot.wiki.page import Page | |||||
from earwigbot.wiki.site import Site | |||||
from earwigbot.wiki.user import User | |||||
from earwigbot.wiki.page import * | |||||
from earwigbot.wiki.site import * | |||||
from earwigbot.wiki.sitesdb import * | |||||
from earwigbot.wiki.user import * |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -22,60 +22,184 @@ | |||||
from earwigbot.wiki.page import Page | from earwigbot.wiki.page import Page | ||||
__all__ = ["Category"] | |||||
class Category(Page): | class Category(Page): | ||||
""" | """ | ||||
EarwigBot's Wiki Toolset: Category Class | |||||
**EarwigBot: Wiki Toolset: Category** | |||||
Represents a Category on a given Site, a subclass of Page. Provides | |||||
additional methods, but Page's own methods should work fine on Category | |||||
objects. Site.get_page() will return a Category instead of a Page if the | |||||
given title is in the category namespace; get_category() is shorthand, | |||||
because it accepts category names without the namespace prefix. | |||||
Represents a category on a given :py:class:`~earwigbot.wiki.site.Site`, a | |||||
subclass of :py:class:`~earwigbot.wiki.page.Page`. Provides additional | |||||
methods, but :py:class:`~earwigbot.wiki.page.Page`'s own methods should | |||||
work fine on :py:class:`Category` objects. :py:meth:`site.get_page() | |||||
<earwigbot.wiki.site.Site.get_page>` will return a :py:class:`Category` | |||||
instead of a :py:class:`~earwigbot.wiki.page.Page` if the given title is in | |||||
the category namespace; :py:meth:`~earwigbot.wiki.site.Site.get_category` | |||||
is shorthand, accepting category names without the namespace prefix. | |||||
Public methods: | |||||
members -- returns a list of page titles in the category | |||||
*Attributes:* | |||||
- :py:attr:`size`: the total number of members in the category | |||||
- :py:attr:`pages`: the number of pages in the category | |||||
- :py:attr:`files`: the number of files in the category | |||||
- :py:attr:`subcats`: the number of subcategories in the category | |||||
*Public methods:* | |||||
- :py:meth:`get_members`: iterates over Pages in the category | |||||
""" | """ | ||||
def __repr__(self): | def __repr__(self): | ||||
"""Returns the canonical string representation of the Category.""" | |||||
"""Return the canonical string representation of the Category.""" | |||||
res = "Category(title={0!r}, follow_redirects={1!r}, site={2!r})" | res = "Category(title={0!r}, follow_redirects={1!r}, site={2!r})" | ||||
return res.format(self._title, self._follow_redirects, self._site) | return res.format(self._title, self._follow_redirects, self._site) | ||||
def __str__(self): | def __str__(self): | ||||
"""Returns a nice string representation of the Category.""" | |||||
return '<Category "{0}" of {1}>'.format(self.title(), str(self._site)) | |||||
"""Return a nice string representation of the Category.""" | |||||
return '<Category "{0}" of {1}>'.format(self.title, str(self.site)) | |||||
def members(self, limit=50, use_sql=False): | |||||
"""Returns a list of page titles in the category. | |||||
def _get_members_via_api(self, limit, follow): | |||||
"""Iterate over Pages in the category using the API.""" | |||||
params = {"action": "query", "list": "categorymembers", | |||||
"cmtitle": self.title} | |||||
If `limit` is provided, we will provide this many titles, or less if | |||||
the category is too small. `limit` defaults to 50; normal users can go | |||||
up to 500, and bots can go up to 5,000 on a single API query. | |||||
while 1: | |||||
params["cmlimit"] = limit if limit else "max" | |||||
result = self.site.api_query(**params) | |||||
for member in result["query"]["categorymembers"]: | |||||
title = member["title"] | |||||
yield self.site.get_page(title, follow_redirects=follow) | |||||
If `use_sql` is True, we will use a SQL query instead of the API. The | |||||
limit argument will be ignored, and pages will be returned as tuples | |||||
of (title, pageid) instead of just titles. | |||||
""" | |||||
if use_sql: | |||||
query = """SELECT page_title, page_namespace, page_id FROM page | |||||
JOIN categorylinks ON page_id = cl_from | |||||
WHERE cl_to = ?""" | |||||
title = self.title().replace(" ", "_").split(":", 1)[1] | |||||
result = self._site.sql_query(query, (title,)) | |||||
members = [] | |||||
for row in result: | |||||
body = row[0].replace("_", " ") | |||||
namespace = self._site.namespace_id_to_name(row[1]) | |||||
if namespace: | |||||
title = ":".join((str(namespace), body)) | |||||
else: # Avoid doing a silly (albeit valid) ":Pagename" thing | |||||
title = body | |||||
members.append((title, row[2])) | |||||
return members | |||||
if "query-continue" in result: | |||||
qcontinue = result["query-continue"]["categorymembers"] | |||||
params["cmcontinue"] = qcontinue["cmcontinue"] | |||||
if limit: | |||||
limit -= len(result["query"]["categorymembers"]) | |||||
else: | |||||
break | |||||
def _get_members_via_sql(self, limit, follow): | |||||
"""Iterate over Pages in the category using SQL.""" | |||||
query = """SELECT page_title, page_namespace, page_id FROM page | |||||
JOIN categorylinks ON page_id = cl_from | |||||
WHERE cl_to = ?""" | |||||
title = self.title.replace(" ", "_").split(":", 1)[1] | |||||
if limit: | |||||
query += " LIMIT ?" | |||||
result = self.site.sql_query(query, (title, limit)) | |||||
else: | |||||
result = self.site.sql_query(query, (title,)) | |||||
members = list(result) | |||||
for row in members: | |||||
base = row[0].replace("_", " ").decode("utf8") | |||||
namespace = self.site.namespace_id_to_name(row[1]) | |||||
if namespace: | |||||
title = u":".join((namespace, base)) | |||||
else: # Avoid doing a silly (albeit valid) ":Pagename" thing | |||||
title = base | |||||
yield self.site.get_page(title, follow_redirects=follow, | |||||
pageid=row[2]) | |||||
def _get_size_via_api(self, member_type): | |||||
"""Return the size of the category using the API.""" | |||||
result = self.site.api_query(action="query", prop="categoryinfo", | |||||
titles=self.title) | |||||
info = result["query"]["pages"].values()[0]["categoryinfo"] | |||||
return info[member_type] | |||||
def _get_size_via_sql(self, member_type): | |||||
"""Return the size of the category using SQL.""" | |||||
query = "SELECT COUNT(*) FROM categorylinks WHERE cl_to = ?" | |||||
title = self.title.replace(" ", "_").split(":", 1)[1] | |||||
if member_type == "size": | |||||
result = self.site.sql_query(query, (title,)) | |||||
else: | else: | ||||
params = {"action": "query", "list": "categorymembers", | |||||
"cmlimit": limit, "cmtitle": self._title} | |||||
result = self._site._api_query(params) | |||||
members = result['query']['categorymembers'] | |||||
return [member["title"] for member in members] | |||||
query += " AND cl_type = ?" | |||||
result = self.site.sql_query(query, (title, member_type[:-1])) | |||||
return list(result)[0][0] | |||||
def _get_size(self, member_type): | |||||
"""Return the size of the category.""" | |||||
services = { | |||||
self.site.SERVICE_API: self._get_size_via_api, | |||||
self.site.SERVICE_SQL: self._get_size_via_sql | |||||
} | |||||
return self.site.delegate(services, (member_type,)) | |||||
@property | |||||
def size(self): | |||||
"""The total number of members in the category. | |||||
Includes pages, files, and subcats. Equal to :py:attr:`pages` + | |||||
:py:attr:`files` + :py:attr:`subcats`. This will use either the API or | |||||
SQL depending on which are enabled and the amount of lag on each. This | |||||
is handled by :py:meth:`site.delegate() | |||||
<earwigbot.wiki.site.Site.delegate>`. | |||||
""" | |||||
return self._get_size("size") | |||||
@property | |||||
def pages(self): | |||||
"""The number of pages in the category. | |||||
This will use either the API or SQL depending on which are enabled and | |||||
the amount of lag on each. This is handled by :py:meth:`site.delegate() | |||||
<earwigbot.wiki.site.Site.delegate>`. | |||||
""" | |||||
return self._get_size("pages") | |||||
@property | |||||
def files(self): | |||||
"""The number of files in the category. | |||||
This will use either the API or SQL depending on which are enabled and | |||||
the amount of lag on each. This is handled by :py:meth:`site.delegate() | |||||
<earwigbot.wiki.site.Site.delegate>`. | |||||
""" | |||||
return self._get_size("files") | |||||
@property | |||||
def subcats(self): | |||||
"""The number of subcategories in the category. | |||||
This will use either the API or SQL depending on which are enabled and | |||||
the amount of lag on each. This is handled by :py:meth:`site.delegate() | |||||
<earwigbot.wiki.site.Site.delegate>`. | |||||
""" | |||||
return self._get_size("subcats") | |||||
def get_members(self, limit=None, follow_redirects=None): | |||||
"""Iterate over Pages in the category. | |||||
If *limit* is given, we will provide this many pages, or less if the | |||||
category is smaller. By default, *limit* is ``None``, meaning we will | |||||
keep iterating over members until the category is exhausted. | |||||
*follow_redirects* is passed directly to :py:meth:`site.get_page() | |||||
<earwigbot.wiki.site.Site.get_page>`; it defaults to ``None``, which | |||||
will use the value passed to our :py:meth:`__init__`. | |||||
This will use either the API or SQL depending on which are enabled and | |||||
the amount of lag on each. This is handled by :py:meth:`site.delegate() | |||||
<earwigbot.wiki.site.Site.delegate>`. | |||||
.. note:: | |||||
Be careful when iterating over very large categories with no limit. | |||||
If using the API, at best, you will make one query per 5000 pages, | |||||
which can add up significantly for categories with hundreds of | |||||
thousands of members. As for SQL, note that *all page titles are | |||||
stored internally* as soon as the query is made, so the site-wide | |||||
SQL lock can be freed and unrelated queries can be made without | |||||
requiring a separate connection to be opened. This is generally not | |||||
an issue unless your category's size approaches several hundred | |||||
thousand, in which case the sheer number of titles in memory becomes | |||||
problematic. | |||||
""" | |||||
services = { | |||||
self.site.SERVICE_API: self._get_members_via_api, | |||||
self.site.SERVICE_SQL: self._get_members_via_sql | |||||
} | |||||
if follow_redirects is None: | |||||
follow_redirects = self._follow_redirects | |||||
return self.site.delegate(services, (limit, follow_redirects)) |
@@ -21,19 +21,23 @@ | |||||
# SOFTWARE. | # SOFTWARE. | ||||
""" | """ | ||||
EarwigBot's Wiki Toolset: Constants | |||||
**EarwigBot: Wiki Toolset: Constants** | |||||
This module defines some useful constants: | This module defines some useful constants: | ||||
* USER_AGENT - our default User Agent when making API queries | |||||
* NS_* - default namespace IDs for easy lookup | |||||
Import with `from earwigbot.wiki import constants` or `from earwigbot.wiki.constants import *`. | |||||
- :py:const:`USER_AGENT`: our default User Agent when making API queries | |||||
- :py:const:`NS_*`: default namespace IDs for easy lookup | |||||
Import directly with ``from earwigbot.wiki import constants`` or | |||||
``from earwigbot.wiki.constants import *``. These are also available from | |||||
:py:mod:`earwigbot.wiki` directly (e.g. ``earwigbot.wiki.USER_AGENT``). | |||||
""" | """ | ||||
# Default User Agent when making API queries: | # Default User Agent when making API queries: | ||||
from earwigbot import __version__ as _v | from earwigbot import __version__ as _v | ||||
from platform import python_version as _p | from platform import python_version as _p | ||||
USER_AGENT = "EarwigBot/{0} (Python/{1}; https://github.com/earwig/earwigbot)".format(_v, _p()) | |||||
USER_AGENT = "EarwigBot/{0} (Python/{1}; https://github.com/earwig/earwigbot)" | |||||
USER_AGENT = USER_AGENT.format(_v, _p()) | |||||
del _v, _p | del _v, _p | ||||
# Default namespace IDs: | # Default namespace IDs: | ||||
@@ -30,12 +30,12 @@ try: | |||||
except ImportError: | except ImportError: | ||||
oauth = None | oauth = None | ||||
from earwigbot import exceptions | |||||
from earwigbot.wiki.copyvios.markov import * | from earwigbot.wiki.copyvios.markov import * | ||||
from earwigbot.wiki.copyvios.parsers import * | from earwigbot.wiki.copyvios.parsers import * | ||||
from earwigbot.wiki.copyvios.search import * | from earwigbot.wiki.copyvios.search import * | ||||
from earwigbot.wiki.exceptions import * | |||||
__all__ = ["CopyvioCheckResult", "CopyvioMixin"] | |||||
__all__ = ["CopyvioCheckResult", "CopyvioMixIn"] | |||||
class CopyvioCheckResult(object): | class CopyvioCheckResult(object): | ||||
def __init__(self, violation, confidence, url, queries, article, chains): | def __init__(self, violation, confidence, url, queries, article, chains): | ||||
@@ -52,7 +52,7 @@ class CopyvioCheckResult(object): | |||||
return r.format(self.violation, self.confidence, self.url, self.queries) | return r.format(self.violation, self.confidence, self.url, self.queries) | ||||
class CopyvioMixin(object): | |||||
class CopyvioMixIn(object): | |||||
""" | """ | ||||
EarwigBot's Wiki Toolset: Copyright Violation Mixin | EarwigBot's Wiki Toolset: Copyright Violation Mixin | ||||
@@ -1,124 +0,0 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
""" | |||||
EarwigBot's Wiki Toolset: Exceptions | |||||
This module contains all exceptions used by the wiki package. There are a lot: | |||||
-- SiteNotFoundError | |||||
-- SiteAPIError | |||||
-- LoginError | |||||
-- NamespaceNotFoundError | |||||
-- PageNotFoundError | |||||
-- InvalidPageError | |||||
-- RedirectError | |||||
-- UserNotFoundError | |||||
-- EditError | |||||
-- PermissionsError | |||||
-- EditConflictError | |||||
-- NoContentError | |||||
-- ContentTooBigError | |||||
-- SpamDetectedError | |||||
-- FilteredError | |||||
-- SQLError | |||||
-- CopyvioCheckError | |||||
-- UnknownSearchEngineError | |||||
-- UnsupportedSearchEngineError | |||||
-- SearchQueryError | |||||
""" | |||||
class WikiToolsetError(Exception): | |||||
"""Base exception class for errors in the Wiki Toolset.""" | |||||
class SiteNotFoundError(WikiToolsetError): | |||||
"""A site matching the args given to get_site() could not be found in the | |||||
config file.""" | |||||
class SiteAPIError(WikiToolsetError): | |||||
"""We couldn't connect to a site's API, perhaps because the server doesn't | |||||
exist, our URL is wrong or incomplete, or they're having temporary | |||||
problems.""" | |||||
class LoginError(WikiToolsetError): | |||||
"""An error occured while trying to login. Perhaps the username/password is | |||||
incorrect.""" | |||||
class NamespaceNotFoundError(WikiToolsetError): | |||||
"""A requested namespace name or namespace ID does not exist.""" | |||||
class PageNotFoundError(WikiToolsetError): | |||||
"""Attempting to get certain information about a page that does not | |||||
exist.""" | |||||
class InvalidPageError(WikiToolsetError): | |||||
"""Attempting to get certain information about a page whose title is | |||||
invalid.""" | |||||
class RedirectError(WikiToolsetError): | |||||
"""Page's get_redirect_target() method failed because the page is either | |||||
not a redirect, or it is malformed.""" | |||||
class UserNotFoundError(WikiToolsetError): | |||||
"""Attempting to get certain information about a user that does not | |||||
exist.""" | |||||
class EditError(WikiToolsetError): | |||||
"""We got some error while editing. Sometimes, a subclass of this exception | |||||
will be used, like PermissionsError or EditConflictError.""" | |||||
class PermissionsError(EditError): | |||||
"""We tried to do something we don't have permission to, like a non-admin | |||||
trying to delete a page, or trying to edit a page when no login information | |||||
was provided.""" | |||||
class EditConflictError(EditError): | |||||
"""We've gotten an edit conflict or a (rarer) delete/recreate conflict.""" | |||||
class NoContentError(EditError): | |||||
"""We tried to create a page or new section with no content.""" | |||||
class ContentTooBigError(EditError): | |||||
"""The edit we tried to push exceeded the article size limit.""" | |||||
class SpamDetectedError(EditError): | |||||
"""The spam filter refused our edit.""" | |||||
class FilteredError(EditError): | |||||
"""The edit filter refused our edit.""" | |||||
class SQLError(WikiToolsetError): | |||||
"""Some error involving SQL querying occurred.""" | |||||
class CopyvioCheckError(WikiToolsetError): | |||||
"""An error occured when checking a page for copyright violations.""" | |||||
class UnknownSearchEngineError(CopyvioCheckError): | |||||
"""CopyrightMixin().copyvio_check() called with an unknown engine.""" | |||||
class UnsupportedSearchEngineError(CopyvioCheckError): | |||||
"""The engine requested is not available, e.g., because a required package | |||||
is missing.""" | |||||
class SearchQueryError(CopyvioCheckError): | |||||
"""Some error ocurred while doing a search query.""" |
@@ -1,219 +0,0 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
""" | |||||
EarwigBot's Wiki Toolset: Misc Functions | |||||
This module, a component of the wiki package, contains miscellaneous functions | |||||
that are not methods of any class, like get_site(). | |||||
There's no need to import this module explicitly. All functions here are | |||||
automatically available from earwigbot.wiki. | |||||
""" | |||||
from cookielib import LWPCookieJar, LoadError | |||||
import errno | |||||
from getpass import getpass | |||||
from os import chmod, path | |||||
import platform | |||||
import stat | |||||
import earwigbot | |||||
from earwigbot.config import config | |||||
from earwigbot.wiki.exceptions import SiteNotFoundError | |||||
from earwigbot.wiki.site import Site | |||||
__all__ = ["get_site", "add_site", "del_site"] | |||||
_cookiejar = None | |||||
def _load_config(): | |||||
"""Called by a config-requiring function, such as get_site(), when config | |||||
has not been loaded. This will usually happen only if we're running code | |||||
directly from Python's interpreter and not the bot itself, because | |||||
earwigbot.py or core/main.py will already call these functions. | |||||
""" | |||||
is_encrypted = config.load() | |||||
if is_encrypted: # Passwords in the config file are encrypted | |||||
key = getpass("Enter key to unencrypt bot passwords: ") | |||||
config._decryption_key = key | |||||
config.decrypt(config.wiki, "password") | |||||
def _get_cookiejar(): | |||||
"""Returns a LWPCookieJar object loaded from our .cookies file. The same | |||||
one is returned every time. | |||||
The .cookies file is located in the project root, same directory as | |||||
config.json and earwigbot.py. If it doesn't exist, we will create the file | |||||
and set it to be readable and writeable only by us. If it exists but the | |||||
information inside is bogus, we will ignore it. | |||||
This is normally called by _get_site_object_from_dict() (in turn called by | |||||
get_site()), and the cookiejar is passed to our Site's constructor, used | |||||
when it makes API queries. This way, we can easily preserve cookies between | |||||
sites (e.g., for CentralAuth), making logins easier. | |||||
""" | |||||
global _cookiejar | |||||
if _cookiejar is not None: | |||||
return _cookiejar | |||||
cookie_file = path.join(config.root_dir, ".cookies") | |||||
_cookiejar = LWPCookieJar(cookie_file) | |||||
try: | |||||
_cookiejar.load() | |||||
except LoadError: | |||||
pass # File contains bad data, so ignore it completely | |||||
except IOError as e: | |||||
if e.errno == errno.ENOENT: # "No such file or directory" | |||||
# Create the file and restrict reading/writing only to the owner, | |||||
# so others can't peak at our cookies: | |||||
open(cookie_file, "w").close() | |||||
chmod(cookie_file, stat.S_IRUSR|stat.S_IWUSR) | |||||
else: | |||||
raise | |||||
return _cookiejar | |||||
def _get_site_object_from_dict(name, d): | |||||
"""Return a Site object based on the contents of a dict, probably acquired | |||||
through our config file, and a separate name. | |||||
""" | |||||
project = d.get("project") | |||||
lang = d.get("lang") | |||||
base_url = d.get("baseURL") | |||||
article_path = d.get("articlePath") | |||||
script_path = d.get("scriptPath") | |||||
sql = d.get("sql", {}) | |||||
namespaces = d.get("namespaces", {}) | |||||
login = (config.wiki.get("username"), config.wiki.get("password")) | |||||
cookiejar = _get_cookiejar() | |||||
user_agent = config.wiki.get("userAgent") | |||||
assert_edit = config.wiki.get("assert") | |||||
maxlag = config.wiki.get("maxlag") | |||||
search_config = config.wiki.get("search") | |||||
if user_agent: | |||||
user_agent = user_agent.replace("$1", earwigbot.__version__) | |||||
user_agent = user_agent.replace("$2", platform.python_version()) | |||||
for key, value in namespaces.items(): # Convert string keys to integers | |||||
del namespaces[key] | |||||
try: | |||||
namespaces[int(key)] = value | |||||
except ValueError: # Data is broken, ignore it | |||||
namespaces = None | |||||
break | |||||
return Site(name=name, project=project, lang=lang, base_url=base_url, | |||||
article_path=article_path, script_path=script_path, sql=sql, | |||||
namespaces=namespaces, login=login, cookiejar=cookiejar, | |||||
user_agent=user_agent, assert_edit=assert_edit, maxlag=maxlag, | |||||
search_config=search_config) | |||||
def get_site(name=None, project=None, lang=None): | |||||
"""Returns a Site instance based on information from our config file. | |||||
With no arguments, returns the default site as specified by our config | |||||
file. This is default = config.wiki["defaultSite"]; | |||||
config.wiki["sites"][default]. | |||||
With `name` specified, returns the site specified by | |||||
config.wiki["sites"][name]. | |||||
With `project` and `lang` specified, returns the site specified by the | |||||
member of config.wiki["sites"], `s`, for which s["project"] == project and | |||||
s["lang"] == lang. | |||||
We will attempt to login to the site automatically | |||||
using config.wiki["username"] and config.wiki["password"] if both are | |||||
defined. | |||||
Specifying a project without a lang or a lang without a project will raise | |||||
TypeError. If all three args are specified, `name` will be first tried, | |||||
then `project` and `lang`. If, with any number of args, a site cannot be | |||||
found in the config, SiteNotFoundError is raised. | |||||
""" | |||||
# Check if config has been loaded, and load it if it hasn't: | |||||
if not config.is_loaded(): | |||||
_load_config() | |||||
# Someone specified a project without a lang (or a lang without a project)! | |||||
if (project is None and lang is not None) or (project is not None and | |||||
lang is None): | |||||
e = "Keyword arguments 'lang' and 'project' must be specified together." | |||||
raise TypeError(e) | |||||
# No args given, so return our default site (project is None implies lang | |||||
# is None, so we don't need to add that in): | |||||
if name is None and project is None: | |||||
try: | |||||
default = config.wiki["defaultSite"] | |||||
except KeyError: | |||||
e = "Default site is not specified in config." | |||||
raise SiteNotFoundError(e) | |||||
try: | |||||
site = config.wiki["sites"][default] | |||||
except KeyError: | |||||
e = "Default site specified by config is not in the config's sites list." | |||||
raise SiteNotFoundError(e) | |||||
return _get_site_object_from_dict(default, site) | |||||
# Name arg given, but don't look at others unless `name` isn't found: | |||||
if name is not None: | |||||
try: | |||||
site = config.wiki["sites"][name] | |||||
except KeyError: | |||||
if project is None: # Implies lang is None, so only name was given | |||||
e = "Site '{0}' not found in config.".format(name) | |||||
raise SiteNotFoundError(e) | |||||
for sitename, site in config.wiki["sites"].items(): | |||||
if site["project"] == project and site["lang"] == lang: | |||||
return _get_site_object_from_dict(sitename, site) | |||||
e = "Neither site '{0}' nor site '{1}:{2}' found in config." | |||||
e.format(name, project, lang) | |||||
raise SiteNotFoundError(e) | |||||
else: | |||||
return _get_site_object_from_dict(name, site) | |||||
# If we end up here, then project and lang are both not None: | |||||
for sitename, site in config.wiki["sites"].items(): | |||||
if site["project"] == project and site["lang"] == lang: | |||||
return _get_site_object_from_dict(sitename, site) | |||||
e = "Site '{0}:{1}' not found in config.".format(project, lang) | |||||
raise SiteNotFoundError(e) | |||||
def add_site(): | |||||
"""STUB: config editing is required first. | |||||
Returns True if the site was added successfully or False if the site was | |||||
already in our config. Raises ConfigError if saving the updated file failed | |||||
for some reason.""" | |||||
pass | |||||
def del_site(name): | |||||
"""STUB: config editing is required first. | |||||
Returns True if the site was removed successfully or False if the site was | |||||
not in our config originally. Raises ConfigError if saving the updated file | |||||
failed for some reason.""" | |||||
pass |
@@ -25,57 +25,83 @@ import re | |||||
from time import gmtime, strftime | from time import gmtime, strftime | ||||
from urllib import quote | from urllib import quote | ||||
from earwigbot.wiki.copyvios import CopyvioMixin | |||||
from earwigbot.wiki.exceptions import * | |||||
try: | |||||
import mwparserfromhell | |||||
except ImportError: | |||||
mwparserfromhell = None | |||||
from earwigbot import exceptions | |||||
from earwigbot.wiki.copyvios import CopyvioMixIn | |||||
__all__ = ["Page"] | |||||
class Page(CopyvioMixin): | class Page(CopyvioMixin): | ||||
""" | """ | ||||
EarwigBot's Wiki Toolset: Page Class | |||||
Represents a Page on a given Site. Has methods for getting information | |||||
about the page, getting page content, and so on. Category is a subclass of | |||||
Page with additional methods. | |||||
Public methods: | |||||
title -- returns the page's title, or pagename | |||||
exists -- returns whether the page exists | |||||
pageid -- returns an integer ID representing the page | |||||
url -- returns the page's URL | |||||
namespace -- returns the page's namespace as an integer | |||||
protection -- returns the page's current protection status | |||||
creator -- returns the page's creator (first user to edit) | |||||
is_talkpage -- returns True if the page is a talkpage, else False | |||||
is_redirect -- returns True if the page is a redirect, else False | |||||
toggle_talk -- returns a content page's talk page, or vice versa | |||||
get -- returns page content | |||||
get_redirect_target -- if the page is a redirect, returns its destination | |||||
edit -- replaces the page's content or creates a new page | |||||
add_section -- adds a new section at the bottom of the page | |||||
copyvio_check -- checks the page for copyright violations | |||||
**EarwigBot: Wiki Toolset: Page** | |||||
Represents a page on a given :py:class:`~earwigbot.wiki.site.Site`. Has | |||||
methods for getting information about the page, getting page content, and | |||||
so on. :py:class:`~earwigbot.wiki.category.Category` is a subclass of | |||||
:py:class:`Page` with additional methods. | |||||
*Attributes:* | |||||
- :py:attr:`site`: the page's corresponding Site object | |||||
- :py:attr:`title`: the page's title, or pagename | |||||
- :py:attr:`exists`: whether or not the page exists | |||||
- :py:attr:`pageid`: an integer ID representing the page | |||||
- :py:attr:`url`: the page's URL | |||||
- :py:attr:`namespace`: the page's namespace as an integer | |||||
- :py:attr:`protection`: the page's current protection status | |||||
- :py:attr:`is_talkpage`: ``True`` if this is a talkpage, else ``False`` | |||||
- :py:attr:`is_redirect`: ``True`` if this is a redirect, else ``False`` | |||||
*Public methods:* | |||||
- :py:meth:`reload`: forcibly reloads the page's attributes | |||||
- :py:meth:`toggle_talk`: returns a content page's talk page, or vice versa | |||||
- :py:meth:`get`: returns the page's content | |||||
- :py:meth:`get_redirect_target`: returns the page's destination if it is a | |||||
redirect | |||||
- :py:meth:`get_creator`: returns a User object representing the first | |||||
person to edit the page | |||||
- :py:meth:`parse`: parses the page content for templates, links, etc | |||||
- :py:meth:`edit`: replaces the page's content or creates a new page | |||||
- :py:meth:`add_section`: adds a new section at the bottom of the page | |||||
- :py:meth:`check_exclusion`: checks whether or not we are allowed to edit | |||||
the page, per ``{{bots}}``/``{{nobots}}`` | |||||
- :py:meth:`~earwigbot.wiki.copyvios.CopyrightMixIn.copyvio_check`: | |||||
checks the page for copyright violations | |||||
- :py:meth:`~earwigbot.wiki.copyvios.CopyrightMixIn.copyvio_compare`: | |||||
checks the page like :py:meth:`copyvio_check`, but against a specific URL | |||||
""" | """ | ||||
re_redirect = "^\s*\#\s*redirect\s*\[\[(.*?)\]\]" | |||||
PAGE_UNKNOWN = 0 | |||||
PAGE_INVALID = 1 | |||||
PAGE_MISSING = 2 | |||||
PAGE_EXISTS = 3 | |||||
def __init__(self, site, title, follow_redirects=False): | |||||
def __init__(self, site, title, follow_redirects=False, pageid=None): | |||||
"""Constructor for new Page instances. | """Constructor for new Page instances. | ||||
Takes three arguments: a Site object, the Page's title (or pagename), | |||||
and whether or not to follow redirects (optional, defaults to False). | |||||
Takes four arguments: a Site object, the Page's title (or pagename), | |||||
whether or not to follow redirects (optional, defaults to False), and | |||||
a page ID to supplement the title (optional, defaults to None - i.e., | |||||
we will have to query the API to get it). | |||||
As with User, site.get_page() is preferred. Site's method has support | |||||
for a default `follow_redirects` value in our config, while __init__ | |||||
always defaults to False. | |||||
As with User, site.get_page() is preferred. | |||||
__init__ will not do any API queries, but it will use basic namespace | |||||
__init__() will not do any API queries, but it will use basic namespace | |||||
logic to determine our namespace ID and if we are a talkpage. | logic to determine our namespace ID and if we are a talkpage. | ||||
""" | """ | ||||
super(Page, self).__init__(site) | super(Page, self).__init__(site) | ||||
self._site = site | self._site = site | ||||
self._title = title.strip() | self._title = title.strip() | ||||
self._follow_redirects = self._keep_following = follow_redirects | self._follow_redirects = self._keep_following = follow_redirects | ||||
self._pageid = pageid | |||||
self._exists = 0 | |||||
self._pageid = None | |||||
self._exists = self.PAGE_UNKNOWN | |||||
self._is_redirect = None | self._is_redirect = None | ||||
self._lastrevid = None | self._lastrevid = None | ||||
self._protection = None | self._protection = None | ||||
@@ -93,8 +119,8 @@ class Page(CopyvioMixin): | |||||
prefix = self._title.split(":", 1)[0] | prefix = self._title.split(":", 1)[0] | ||||
if prefix != title: # ignore a page that's titled "Category" or "User" | if prefix != title: # ignore a page that's titled "Category" or "User" | ||||
try: | try: | ||||
self._namespace = self._site.namespace_name_to_id(prefix) | |||||
except NamespaceNotFoundError: | |||||
self._namespace = self.site.namespace_name_to_id(prefix) | |||||
except exceptions.NamespaceNotFoundError: | |||||
self._namespace = 0 | self._namespace = 0 | ||||
else: | else: | ||||
self._namespace = 0 | self._namespace = 0 | ||||
@@ -107,15 +133,15 @@ class Page(CopyvioMixin): | |||||
self._is_talkpage = self._namespace % 2 == 1 | self._is_talkpage = self._namespace % 2 == 1 | ||||
def __repr__(self): | def __repr__(self): | ||||
"""Returns the canonical string representation of the Page.""" | |||||
"""Return the canonical string representation of the Page.""" | |||||
res = "Page(title={0!r}, follow_redirects={1!r}, site={2!r})" | res = "Page(title={0!r}, follow_redirects={1!r}, site={2!r})" | ||||
return res.format(self._title, self._follow_redirects, self._site) | return res.format(self._title, self._follow_redirects, self._site) | ||||
def __str__(self): | def __str__(self): | ||||
"""Returns a nice string representation of the Page.""" | |||||
return '<Page "{0}" of {1}>'.format(self.title(), str(self._site)) | |||||
"""Return a nice string representation of the Page.""" | |||||
return '<Page "{0}" of {1}>'.format(self.title, str(self.site)) | |||||
def _force_validity(self): | |||||
def _assert_validity(self): | |||||
"""Used to ensure that our page's title is valid. | """Used to ensure that our page's title is valid. | ||||
If this method is called when our page is not valid (and after | If this method is called when our page is not valid (and after | ||||
@@ -124,24 +150,24 @@ class Page(CopyvioMixin): | |||||
Note that validity != existence. If a page's title is invalid (e.g, it | Note that validity != existence. If a page's title is invalid (e.g, it | ||||
contains "[") it will always be invalid, and cannot be edited. | contains "[") it will always be invalid, and cannot be edited. | ||||
""" | """ | ||||
if self._exists == 1: | |||||
e = "Page '{0}' is invalid.".format(self._title) | |||||
raise InvalidPageError(e) | |||||
if self._exists == self.PAGE_INVALID: | |||||
e = u"Page '{0}' is invalid.".format(self._title) | |||||
raise exceptions.InvalidPageError(e) | |||||
def _force_existence(self): | |||||
def _assert_existence(self): | |||||
"""Used to ensure that our page exists. | """Used to ensure that our page exists. | ||||
If this method is called when our page doesn't exist (and after | If this method is called when our page doesn't exist (and after | ||||
_load_attributes() has been called), PageNotFoundError will be raised. | _load_attributes() has been called), PageNotFoundError will be raised. | ||||
It will also call _force_validity() beforehand. | |||||
It will also call _assert_validity() beforehand. | |||||
""" | """ | ||||
self._force_validity() | |||||
if self._exists == 2: | |||||
e = "Page '{0}' does not exist.".format(self._title) | |||||
raise PageNotFoundError(e) | |||||
self._assert_validity() | |||||
if self._exists == self.PAGE_MISSING: | |||||
e = u"Page '{0}' does not exist.".format(self._title) | |||||
raise exceptions.PageNotFoundError(e) | |||||
def _load_wrapper(self): | |||||
"""Calls _load_attributes() and follows redirects if we're supposed to. | |||||
def _load(self): | |||||
"""Call _load_attributes() and follows redirects if we're supposed to. | |||||
This method will only follow redirects if follow_redirects=True was | This method will only follow redirects if follow_redirects=True was | ||||
passed to __init__() (perhaps indirectly passed by site.get_page()). | passed to __init__() (perhaps indirectly passed by site.get_page()). | ||||
@@ -164,21 +190,21 @@ class Page(CopyvioMixin): | |||||
self._load_attributes() | self._load_attributes() | ||||
def _load_attributes(self, result=None): | def _load_attributes(self, result=None): | ||||
"""Loads various data from the API in a single query. | |||||
"""Load various data from the API in a single query. | |||||
Loads self._title, ._exists, ._is_redirect, ._pageid, ._fullurl, | Loads self._title, ._exists, ._is_redirect, ._pageid, ._fullurl, | ||||
._protection, ._namespace, ._is_talkpage, ._creator, ._lastrevid, | ._protection, ._namespace, ._is_talkpage, ._creator, ._lastrevid, | ||||
._token, and ._starttimestamp using the API. It will do a query of | ._token, and ._starttimestamp using the API. It will do a query of | ||||
its own unless `result` is provided, in which case we'll pretend | |||||
`result` is what the query returned. | |||||
its own unless *result* is provided, in which case we'll pretend | |||||
*result* is what the query returned. | |||||
Assuming the API is sound, this should not raise any exceptions. | Assuming the API is sound, this should not raise any exceptions. | ||||
""" | """ | ||||
if result is None: | |||||
params = {"action": "query", "rvprop": "user", "intoken": "edit", | |||||
"prop": "info|revisions", "rvlimit": 1, "rvdir": "newer", | |||||
"titles": self._title, "inprop": "protection|url"} | |||||
result = self._site._api_query(params) | |||||
if not result: | |||||
query = self.site.api_query | |||||
result = query(action="query", rvprop="user", intoken="edit", | |||||
prop="info|revisions", rvlimit=1, rvdir="newer", | |||||
titles=self._title, inprop="protection|url") | |||||
res = result["query"]["pages"].values()[0] | res = result["query"]["pages"].values()[0] | ||||
@@ -192,21 +218,19 @@ class Page(CopyvioMixin): | |||||
else: | else: | ||||
self._is_redirect = True | self._is_redirect = True | ||||
self._pageid = result["query"]["pages"].keys()[0] | |||||
if int(self._pageid) < 0: | |||||
try: | |||||
res["missing"] | |||||
except KeyError: | |||||
self._pageid = int(result["query"]["pages"].keys()[0]) | |||||
if self._pageid < 0: | |||||
if "missing" in res: | |||||
# If it has a negative ID and it's missing; we can still get | |||||
# data like the namespace, protection, and URL: | |||||
self._exists = self.PAGE_MISSING | |||||
else: | |||||
# If it has a negative ID and it's invalid, then break here, | # If it has a negative ID and it's invalid, then break here, | ||||
# because there's no other data for us to get: | # because there's no other data for us to get: | ||||
self._exists = 1 | |||||
self._exists = self.PAGE_INVALID | |||||
return | return | ||||
else: | |||||
# If it has a negative ID and it's missing; we can still get | |||||
# data like the namespace, protection, and URL: | |||||
self._exists = 2 | |||||
else: | else: | ||||
self._exists = 3 | |||||
self._exists = self.PAGE_EXISTS | |||||
self._fullurl = res["fullurl"] | self._fullurl = res["fullurl"] | ||||
self._protection = res["protection"] | self._protection = res["protection"] | ||||
@@ -231,19 +255,19 @@ class Page(CopyvioMixin): | |||||
pass | pass | ||||
def _load_content(self, result=None): | def _load_content(self, result=None): | ||||
"""Loads current page content from the API. | |||||
"""Load current page content from the API. | |||||
If `result` is provided, we'll pretend that is the result of an API | |||||
If *result* is provided, we'll pretend that is the result of an API | |||||
query and try to get content from that. Otherwise, we'll do an API | query and try to get content from that. Otherwise, we'll do an API | ||||
query on our own. | query on our own. | ||||
Don't call this directly, ever - use .get(force=True) if you want to | |||||
force content reloading. | |||||
Don't call this directly, ever; use reload() followed by get() if you | |||||
want to force content reloading. | |||||
""" | """ | ||||
if result is None: | |||||
params = {"action": "query", "prop": "revisions", "rvlimit": 1, | |||||
"rvprop": "content|timestamp", "titles": self._title} | |||||
result = self._site._api_query(params) | |||||
if not result: | |||||
query = self.site.api_query | |||||
result = query(action="query", prop="revisions", rvlimit=1, | |||||
rvprop="content|timestamp", titles=self._title) | |||||
res = result["query"]["pages"].values()[0] | res = result["query"]["pages"].values()[0] | ||||
try: | try: | ||||
@@ -254,14 +278,14 @@ class Page(CopyvioMixin): | |||||
# self._load_attributes(). In that case, some of our attributes are | # self._load_attributes(). In that case, some of our attributes are | ||||
# outdated, so force another self._load_attributes(): | # outdated, so force another self._load_attributes(): | ||||
self._load_attributes() | self._load_attributes() | ||||
self._force_existence() | |||||
self._assert_existence() | |||||
def _edit(self, params=None, text=None, summary=None, minor=None, bot=None, | def _edit(self, params=None, text=None, summary=None, minor=None, bot=None, | ||||
force=None, section=None, captcha_id=None, captcha_word=None, | force=None, section=None, captcha_id=None, captcha_word=None, | ||||
tries=0): | tries=0): | ||||
"""Edit the page! | """Edit the page! | ||||
If `params` is given, we'll use it as our API query parameters. | |||||
If *params* is given, we'll use it as our API query parameters. | |||||
Otherwise, we'll build params using the given kwargs via | Otherwise, we'll build params using the given kwargs via | ||||
_build_edit_params(). | _build_edit_params(). | ||||
@@ -274,10 +298,10 @@ class Page(CopyvioMixin): | |||||
self._load_attributes() | self._load_attributes() | ||||
if not self._token: | if not self._token: | ||||
e = "You don't have permission to edit this page." | e = "You don't have permission to edit this page." | ||||
raise PermissionsError(e) | |||||
raise exceptions.PermissionsError(e) | |||||
# Weed out invalid pages before we get too far: | # Weed out invalid pages before we get too far: | ||||
self._force_validity() | |||||
self._assert_validity() | |||||
# Build our API query string: | # Build our API query string: | ||||
if not params: | if not params: | ||||
@@ -288,8 +312,8 @@ class Page(CopyvioMixin): | |||||
# Try the API query, catching most errors with our handler: | # Try the API query, catching most errors with our handler: | ||||
try: | try: | ||||
result = self._site._api_query(params) | |||||
except SiteAPIError as error: | |||||
result = self.site.api_query(**params) | |||||
except exceptions.APIError as error: | |||||
if not hasattr(error, "code"): | if not hasattr(error, "code"): | ||||
raise # We can only handle errors with a code attribute | raise # We can only handle errors with a code attribute | ||||
result = self._handle_edit_errors(error, params, tries) | result = self._handle_edit_errors(error, params, tries) | ||||
@@ -298,7 +322,7 @@ class Page(CopyvioMixin): | |||||
if result["edit"]["result"] == "Success": | if result["edit"]["result"] == "Success": | ||||
self._content = None | self._content = None | ||||
self._basetimestamp = None | self._basetimestamp = None | ||||
self._exists = 0 | |||||
self._exists = self.PAGE_UNKNOWN | |||||
return | return | ||||
# If we're here, then the edit failed. If it's because of AssertEdit, | # If we're here, then the edit failed. If it's because of AssertEdit, | ||||
@@ -306,7 +330,7 @@ class Page(CopyvioMixin): | |||||
try: | try: | ||||
assertion = result["edit"]["assert"] | assertion = result["edit"]["assert"] | ||||
except KeyError: | except KeyError: | ||||
raise EditError(result["edit"]) | |||||
raise exceptions.EditError(result["edit"]) | |||||
self._handle_assert_edit(assertion, params, tries) | self._handle_assert_edit(assertion, params, tries) | ||||
def _build_edit_params(self, text, summary, minor, bot, force, section, | def _build_edit_params(self, text, summary, minor, bot, force, section, | ||||
@@ -332,7 +356,7 @@ class Page(CopyvioMixin): | |||||
params["starttimestamp"] = self._starttimestamp | params["starttimestamp"] = self._starttimestamp | ||||
if self._basetimestamp: | if self._basetimestamp: | ||||
params["basetimestamp"] = self._basetimestamp | params["basetimestamp"] = self._basetimestamp | ||||
if self._exists == 2: | |||||
if self._exists == self.PAGE_MISSING: | |||||
# Page does not exist; don't edit if it already exists: | # Page does not exist; don't edit if it already exists: | ||||
params["createonly"] = "true" | params["createonly"] = "true" | ||||
else: | else: | ||||
@@ -349,43 +373,43 @@ class Page(CopyvioMixin): | |||||
""" | """ | ||||
if error.code in ["noedit", "cantcreate", "protectedtitle", | if error.code in ["noedit", "cantcreate", "protectedtitle", | ||||
"noimageredirect"]: | "noimageredirect"]: | ||||
raise PermissionsError(error.info) | |||||
raise exceptions.PermissionsError(error.info) | |||||
elif error.code in ["noedit-anon", "cantcreate-anon", | elif error.code in ["noedit-anon", "cantcreate-anon", | ||||
"noimageredirect-anon"]: | "noimageredirect-anon"]: | ||||
if not all(self._site._login_info): | |||||
if not all(self.site._login_info): | |||||
# Insufficient login info: | # Insufficient login info: | ||||
raise PermissionsError(error.info) | |||||
raise exceptions.PermissionsError(error.info) | |||||
if tries == 0: | if tries == 0: | ||||
# We have login info; try to login: | # We have login info; try to login: | ||||
self._site._login(self._site._login_info) | |||||
self.site._login(self.site._login_info) | |||||
self._token = None # Need a new token; old one is invalid now | self._token = None # Need a new token; old one is invalid now | ||||
return self._edit(params=params, tries=1) | return self._edit(params=params, tries=1) | ||||
else: | else: | ||||
# We already tried to log in and failed! | # We already tried to log in and failed! | ||||
e = "Although we should be logged in, we are not. This may be a cookie problem or an odd bug." | e = "Although we should be logged in, we are not. This may be a cookie problem or an odd bug." | ||||
raise LoginError(e) | |||||
raise exceptions.LoginError(e) | |||||
elif error.code in ["editconflict", "pagedeleted", "articleexists"]: | elif error.code in ["editconflict", "pagedeleted", "articleexists"]: | ||||
# These attributes are now invalidated: | # These attributes are now invalidated: | ||||
self._content = None | self._content = None | ||||
self._basetimestamp = None | self._basetimestamp = None | ||||
self._exists = 0 | |||||
raise EditConflictError(error.info) | |||||
self._exists = self.PAGE_UNKNOWN | |||||
raise exceptions.EditConflictError(error.info) | |||||
elif error.code in ["emptypage", "emptynewsection"]: | elif error.code in ["emptypage", "emptynewsection"]: | ||||
raise NoContentError(error.info) | |||||
raise exceptions.NoContentError(error.info) | |||||
elif error.code == "contenttoobig": | elif error.code == "contenttoobig": | ||||
raise ContentTooBigError(error.info) | |||||
raise exceptions.ContentTooBigError(error.info) | |||||
elif error.code == "spamdetected": | elif error.code == "spamdetected": | ||||
raise SpamDetectedError(error.info) | |||||
raise exceptions.SpamDetectedError(error.info) | |||||
elif error.code == "filtered": | elif error.code == "filtered": | ||||
raise FilteredError(error.info) | |||||
raise exceptions.FilteredError(error.info) | |||||
raise EditError(": ".join((error.code, error.info))) | |||||
raise exceptions.EditError(": ".join((error.code, error.info))) | |||||
def _handle_assert_edit(self, assertion, params, tries): | def _handle_assert_edit(self, assertion, params, tries): | ||||
"""If we can't edit due to a failed AssertEdit assertion, handle that. | """If we can't edit due to a failed AssertEdit assertion, handle that. | ||||
@@ -394,179 +418,174 @@ class Page(CopyvioMixin): | |||||
log in. Otherwise, raise PermissionsError with details. | log in. Otherwise, raise PermissionsError with details. | ||||
""" | """ | ||||
if assertion == "user": | if assertion == "user": | ||||
if not all(self._site._login_info): | |||||
if not all(self.site._login_info): | |||||
# Insufficient login info: | # Insufficient login info: | ||||
e = "AssertEdit: user assertion failed, and no login info was provided." | e = "AssertEdit: user assertion failed, and no login info was provided." | ||||
raise PermissionsError(e) | |||||
raise exceptions.PermissionsError(e) | |||||
if tries == 0: | if tries == 0: | ||||
# We have login info; try to login: | # We have login info; try to login: | ||||
self._site._login(self._site._login_info) | |||||
self.site._login(self.site._login_info) | |||||
self._token = None # Need a new token; old one is invalid now | self._token = None # Need a new token; old one is invalid now | ||||
return self._edit(params=params, tries=1) | return self._edit(params=params, tries=1) | ||||
else: | else: | ||||
# We already tried to log in and failed! | # We already tried to log in and failed! | ||||
e = "Although we should be logged in, we are not. This may be a cookie problem or an odd bug." | e = "Although we should be logged in, we are not. This may be a cookie problem or an odd bug." | ||||
raise LoginError(e) | |||||
raise exceptions.LoginError(e) | |||||
elif assertion == "bot": | elif assertion == "bot": | ||||
e = "AssertEdit: bot assertion failed; we don't have a bot flag!" | e = "AssertEdit: bot assertion failed; we don't have a bot flag!" | ||||
raise PermissionsError(e) | |||||
raise exceptions.PermissionsError(e) | |||||
# Unknown assertion, maybe "true", "false", or "exists": | # Unknown assertion, maybe "true", "false", or "exists": | ||||
e = "AssertEdit: assertion '{0}' failed.".format(assertion) | e = "AssertEdit: assertion '{0}' failed.".format(assertion) | ||||
raise PermissionsError(e) | |||||
raise exceptions.PermissionsError(e) | |||||
@property | |||||
def site(self): | |||||
"""The page's corresponding Site object.""" | |||||
return self._site | |||||
def title(self, force=False): | |||||
"""Returns the Page's title, or pagename. | |||||
@property | |||||
def title(self): | |||||
"""The page's title, or "pagename". | |||||
This won't do any API queries on its own unless force is True, in which | |||||
case the title will be forcibly reloaded from the API (normalizing it, | |||||
and following redirects if follow_redirects=True was passed to | |||||
__init__()). Any other methods that do API queries will reload title on | |||||
their own, however, like exists() and get(). | |||||
This won't do any API queries on its own. Any other attributes or | |||||
methods that do API queries will reload the title, however, like | |||||
:py:attr:`exists` and :py:meth:`get`, potentially "normalizing" it or | |||||
following redirects if :py:attr:`self._follow_redirects` is ``True``. | |||||
""" | """ | ||||
if force: | |||||
self._load_wrapper() | |||||
return self._title | return self._title | ||||
def exists(self, force=False): | |||||
"""Returns information about whether the Page exists or not. | |||||
@property | |||||
def exists(self): | |||||
"""Whether or not the page exists. | |||||
The returned "information" is a tuple with two items. The first is a | |||||
bool, either True if the page exists or False if it does not. The | |||||
second is a string giving more information, either "invalid", (title | |||||
is invalid, e.g. it contains "["), "missing", or "exists". | |||||
This will be a number; its value does not matter, but it will equal | |||||
one of :py:attr:`self.PAGE_INVALID <PAGE_INVALID>`, | |||||
:py:attr:`self.PAGE_MISSING <PAGE_MISSING>`, or | |||||
:py:attr:`self.PAGE_EXISTS <PAGE_EXISTS>`. | |||||
Makes an API query if force is True or if we haven't already made one. | |||||
Makes an API query only if we haven't already made one. | |||||
""" | """ | ||||
cases = { | |||||
0: (None, "unknown"), | |||||
1: (False, "invalid"), | |||||
2: (False, "missing"), | |||||
3: (True, "exists"), | |||||
} | |||||
if self._exists == 0 or force: | |||||
self._load_wrapper() | |||||
return cases[self._exists] | |||||
def pageid(self, force=False): | |||||
"""Returns an integer ID representing the Page. | |||||
Makes an API query if force is True or if we haven't already made one. | |||||
Raises InvalidPageError or PageNotFoundError if the page name is | |||||
if self._exists == self.PAGE_UNKNOWN: | |||||
self._load() | |||||
return self._exists | |||||
@property | |||||
def pageid(self): | |||||
"""An integer ID representing the page. | |||||
Makes an API query only if we haven't already made one and the *pageid* | |||||
parameter to :py:meth:`__init__` was left as ``None``, which should be | |||||
true for all cases except when pages are returned by an SQL generator | |||||
(like :py:meth:`category.get_members() | |||||
<earwigbot.wiki.category.Category.get_members>`). | |||||
Raises :py:exc:`~earwigbot.exceptions.InvalidPageError` or | |||||
:py:exc:`~earwigbot.exceptions.PageNotFoundError` if the page name is | |||||
invalid or the page does not exist, respectively. | invalid or the page does not exist, respectively. | ||||
""" | """ | ||||
if self._exists == 0 or force: | |||||
self._load_wrapper() | |||||
self._force_existence() # missing pages do not have IDs | |||||
if self._pageid: | |||||
return self._pageid | |||||
if self._exists == self.PAGE_UNKNOWN: | |||||
self._load() | |||||
self._assert_existence() # Missing pages do not have IDs | |||||
return self._pageid | return self._pageid | ||||
def url(self, force=False): | |||||
"""Returns the page's URL. | |||||
@property | |||||
def url(self): | |||||
"""The page's URL. | |||||
Like title(), this won't do any API queries on its own unless force is | |||||
True. If the API was never queried for this page, we will attempt to | |||||
determine the URL ourselves based on the title. | |||||
Like :py:meth:`title`, this won't do any API queries on its own. If the | |||||
API was never queried for this page, we will attempt to determine the | |||||
URL ourselves based on the title. | |||||
""" | """ | ||||
if force: | |||||
self._load_wrapper() | |||||
if self._fullurl is not None: | |||||
if self._fullurl: | |||||
return self._fullurl | return self._fullurl | ||||
else: | else: | ||||
slug = quote(self._title.replace(" ", "_"), safe="/:") | |||||
path = self._site._article_path.replace("$1", slug) | |||||
return ''.join((self._site._base_url, path)) | |||||
def namespace(self, force=False): | |||||
"""Returns the page's namespace ID (an integer). | |||||
Like title(), this won't do any API queries on its own unless force is | |||||
True. If the API was never queried for this page, we will attempt to | |||||
determine the namespace ourselves based on the title. | |||||
encoded = self._title.encode("utf8").replace(" ", "_") | |||||
slug = quote(encoded, safe="/:") | |||||
path = self.site._article_path.replace("$1", slug) | |||||
return ''.join((self.site.url, path)) | |||||
@property | |||||
def namespace(self): | |||||
"""The page's namespace ID (an integer). | |||||
Like :py:meth:`title`, this won't do any API queries on its own. If the | |||||
API was never queried for this page, we will attempt to determine the | |||||
namespace ourselves based on the title. | |||||
""" | """ | ||||
if force: | |||||
self._load_wrapper() | |||||
return self._namespace | return self._namespace | ||||
def protection(self, force=False): | |||||
"""Returns the page's current protection status. | |||||
@property | |||||
def protection(self): | |||||
"""The page's current protection status. | |||||
Makes an API query if force is True or if we haven't already made one. | |||||
Makes an API query only if we haven't already made one. | |||||
Raises InvalidPageError if the page name is invalid. Will not raise an | |||||
error if the page is missing because those can still be protected. | |||||
Raises :py:exc:`~earwigbot.exceptions.InvalidPageError` if the page | |||||
name is invalid. Won't raise an error if the page is missing because | |||||
those can still be create-protected. | |||||
""" | """ | ||||
if self._exists == 0 or force: | |||||
self._load_wrapper() | |||||
self._force_validity() # invalid pages cannot be protected | |||||
if self._exists == self.PAGE_UNKNOWN: | |||||
self._load() | |||||
self._assert_validity() # Invalid pages cannot be protected | |||||
return self._protection | return self._protection | ||||
def creator(self, force=False): | |||||
"""Returns the page's creator (i.e., the first user to edit the page). | |||||
Makes an API query if force is True or if we haven't already made one. | |||||
Normally, we can get the creator along with everything else (except | |||||
content) in self._load_attributes(). However, due to a limitation in | |||||
the API (can't get the editor of one revision and the content of | |||||
another at both ends of the history), if our other attributes were only | |||||
loaded from get(), we'll have to do another API query. This is done | |||||
by calling ourselves again with force=True. | |||||
@property | |||||
def is_talkpage(self): | |||||
"""``True`` if the page is a talkpage, otherwise ``False``. | |||||
Raises InvalidPageError or PageNotFoundError if the page name is | |||||
invalid or the page does not exist, respectively. | |||||
Like :py:meth:`title`, this won't do any API queries on its own. If the | |||||
API was never queried for this page, we will attempt to determine | |||||
whether it is a talkpage ourselves based on its namespace. | |||||
""" | """ | ||||
if self._exists == 0 or force: | |||||
self._load_wrapper() | |||||
self._force_existence() | |||||
if not self._creator and not force: | |||||
self.creator(force=True) | |||||
return self._creator | |||||
def is_talkpage(self, force=False): | |||||
"""Returns True if the page is a talkpage, else False. | |||||
Like title(), this won't do any API queries on its own unless force is | |||||
True. If the API was never queried for this page, we will attempt to | |||||
determine the talkpage status ourselves based on its namespace ID. | |||||
""" | |||||
if force: | |||||
self._load_wrapper() | |||||
return self._is_talkpage | return self._is_talkpage | ||||
def is_redirect(self, force=False): | |||||
"""Returns True if the page is a redirect, else False. | |||||
@property | |||||
def is_redirect(self): | |||||
"""``True`` if the page is a redirect, otherwise ``False``. | |||||
Makes an API query if force is True or if we haven't already made one. | |||||
Makes an API query only if we haven't already made one. | |||||
We will return False even if the page does not exist or is invalid. | |||||
We will return ``False`` even if the page does not exist or is invalid. | |||||
""" | """ | ||||
if self._exists == 0 or force: | |||||
self._load_wrapper() | |||||
if self._exists == self.PAGE_UNKNOWN: | |||||
self._load() | |||||
return self._is_redirect | return self._is_redirect | ||||
def toggle_talk(self, force=False, follow_redirects=None): | |||||
"""Returns a content page's talk page, or vice versa. | |||||
def reload(self): | |||||
"""Forcibly reload the page's attributes. | |||||
Emphasis on *reload*: this is only necessary if there is reason to | |||||
believe they have changed. | |||||
""" | |||||
self._load() | |||||
if self._content is not None: | |||||
# Only reload content if it has already been loaded: | |||||
self._load_content() | |||||
def toggle_talk(self, follow_redirects=None): | |||||
"""Return a content page's talk page, or vice versa. | |||||
The title of the new page is determined by namespace logic, not API | The title of the new page is determined by namespace logic, not API | ||||
queries. We won't make any API queries on our own unless force is True, | |||||
and the only reason then would be to forcibly update the title or | |||||
follow redirects if we haven't already made an API query. | |||||
queries. We won't make any API queries on our own. | |||||
If `follow_redirects` is anything other than None (the default), it | |||||
will be passed to the new Page's __init__(). Otherwise, we'll use the | |||||
value passed to our own __init__(). | |||||
If *follow_redirects* is anything other than ``None`` (the default), it | |||||
will be passed to the new :py:class:`~earwigbot.wiki.page.Page` | |||||
object's :py:meth:`__init__`. Otherwise, we'll use the value passed to | |||||
our own :py:meth:`__init__`. | |||||
Will raise InvalidPageError if we try to get the talk page of a special | |||||
page (in the Special: or Media: namespaces), but we won't raise an | |||||
exception if our page is otherwise missing or invalid. | |||||
Will raise :py:exc:`~earwigbot.exceptions.InvalidPageError` if we try | |||||
to get the talk page of a special page (in the ``Special:`` or | |||||
``Media:`` namespaces), but we won't raise an exception if our page is | |||||
otherwise missing or invalid. | |||||
""" | """ | ||||
if force: | |||||
self._load_wrapper() | |||||
if self._namespace < 0: | if self._namespace < 0: | ||||
ns = self._site.namespace_id_to_name(self._namespace) | |||||
e = "Pages in the {0} namespace can't have talk pages.".format(ns) | |||||
raise InvalidPageError(e) | |||||
ns = self.site.namespace_id_to_name(self._namespace) | |||||
e = u"Pages in the {0} namespace can't have talk pages.".format(ns) | |||||
raise exceptions.InvalidPageError(e) | |||||
if self._is_talkpage: | if self._is_talkpage: | ||||
new_ns = self._namespace - 1 | new_ns = self._namespace - 1 | ||||
@@ -578,85 +597,116 @@ class Page(CopyvioMixin): | |||||
except IndexError: | except IndexError: | ||||
body = self._title | body = self._title | ||||
new_prefix = self._site.namespace_id_to_name(new_ns) | |||||
new_prefix = self.site.namespace_id_to_name(new_ns) | |||||
# If the new page is in namespace 0, don't do ":Title" (it's correct, | # If the new page is in namespace 0, don't do ":Title" (it's correct, | ||||
# but unnecessary), just do "Title": | # but unnecessary), just do "Title": | ||||
if new_prefix: | if new_prefix: | ||||
new_title = ':'.join((new_prefix, body)) | |||||
new_title = u":".join((new_prefix, body)) | |||||
else: | else: | ||||
new_title = body | new_title = body | ||||
if follow_redirects is None: | if follow_redirects is None: | ||||
follow_redirects = self._follow_redirects | follow_redirects = self._follow_redirects | ||||
return Page(self._site, new_title, follow_redirects) | |||||
return Page(self.site, new_title, follow_redirects) | |||||
def get(self, force=False): | |||||
"""Returns page content, which is cached if you try to call get again. | |||||
Use `force` to forcibly reload page content even if we've already | |||||
loaded some. This is good if you want to edit a page multiple times, | |||||
and you want to get updated content before you make your second edit. | |||||
def get(self): | |||||
"""Return page content, which is cached if you try to call get again. | |||||
Raises InvalidPageError or PageNotFoundError if the page name is | Raises InvalidPageError or PageNotFoundError if the page name is | ||||
invalid or the page does not exist, respectively. | invalid or the page does not exist, respectively. | ||||
""" | """ | ||||
if force or self._exists == 0: | |||||
if self._exists == self.PAGE_UNKNOWN: | |||||
# Kill two birds with one stone by doing an API query for both our | # Kill two birds with one stone by doing an API query for both our | ||||
# attributes and our page content: | # attributes and our page content: | ||||
params = {"action": "query", "rvlimit": 1, "titles": self._title, | |||||
"prop": "info|revisions", "inprop": "protection|url", | |||||
"intoken": "edit", "rvprop": "content|timestamp"} | |||||
result = self._site._api_query(params) | |||||
query = self.site.api_query | |||||
result = query(action="query", rvlimit=1, titles=self._title, | |||||
prop="info|revisions", inprop="protection|url", | |||||
intoken="edit", rvprop="content|timestamp") | |||||
self._load_attributes(result=result) | self._load_attributes(result=result) | ||||
self._force_existence() | |||||
self._assert_existence() | |||||
self._load_content(result=result) | self._load_content(result=result) | ||||
# Follow redirects if we're told to: | # Follow redirects if we're told to: | ||||
if self._keep_following and self._is_redirect: | if self._keep_following and self._is_redirect: | ||||
self._title = self.get_redirect_target() | self._title = self.get_redirect_target() | ||||
self._keep_following = False # don't follow double redirects | |||||
self._content = None # reset the content we just loaded | |||||
self.get(force=True) | |||||
self._keep_following = False # Don't follow double redirects | |||||
self._exists = self.PAGE_UNKNOWN # Force another API query | |||||
self.get() | |||||
return self._content | return self._content | ||||
# Make sure we're dealing with a real page here. This may be outdated | # Make sure we're dealing with a real page here. This may be outdated | ||||
# if the page was deleted since we last called self._load_attributes(), | # if the page was deleted since we last called self._load_attributes(), | ||||
# but self._load_content() can handle that: | # but self._load_content() can handle that: | ||||
self._force_existence() | |||||
self._assert_existence() | |||||
if self._content is None: | if self._content is None: | ||||
self._load_content() | self._load_content() | ||||
return self._content | return self._content | ||||
def get_redirect_target(self, force=False): | |||||
"""If the page is a redirect, returns its destination. | |||||
Use `force` to forcibly reload content even if we've already loaded | |||||
some before. Note that this method calls get() for page content. | |||||
def get_redirect_target(self): | |||||
"""If the page is a redirect, return its destination. | |||||
Raises InvalidPageError or PageNotFoundError if the page name is | |||||
invalid or the page does not exist, respectively. Raises RedirectError | |||||
if the page is not a redirect. | |||||
Raises :py:exc:`~earwigbot.exceptions.InvalidPageError` or | |||||
:py:exc:`~earwigbot.exceptions.PageNotFoundError` if the page name is | |||||
invalid or the page does not exist, respectively. Raises | |||||
:py:exc:`~earwigbot.exceptions.RedirectError` if the page is not a | |||||
redirect. | |||||
""" | """ | ||||
content = self.get(force) | |||||
re_redirect = "^\s*\#\s*redirect\s*\[\[(.*?)\]\]" | |||||
content = self.get() | |||||
try: | try: | ||||
return re.findall(self.re_redirect, content, flags=re.I)[0] | |||||
return re.findall(re_redirect, content, flags=re.I)[0] | |||||
except IndexError: | except IndexError: | ||||
e = "The page does not appear to have a redirect target." | e = "The page does not appear to have a redirect target." | ||||
raise RedirectError(e) | |||||
raise exceptions.RedirectError(e) | |||||
def get_creator(self): | |||||
"""Return the User object for the first person to edit the page. | |||||
Makes an API query only if we haven't already made one. Normally, we | |||||
can get the creator along with everything else (except content) in | |||||
:py:meth:`_load_attributes`. However, due to a limitation in the API | |||||
(can't get the editor of one revision and the content of another at | |||||
both ends of the history), if our other attributes were only loaded | |||||
through :py:meth:`get`, we'll have to do another API query. | |||||
Raises :py:exc:`~earwigbot.exceptions.InvalidPageError` or | |||||
:py:exc:`~earwigbot.exceptions.PageNotFoundError` if the page name is | |||||
invalid or the page does not exist, respectively. | |||||
""" | |||||
if self._exists == self.PAGE_UNKNOWN: | |||||
self._load() | |||||
self._assert_existence() | |||||
if not self._creator: | |||||
self._load() | |||||
self._assert_existence() | |||||
return self.site.get_user(self._creator) | |||||
def parse(self): | |||||
"""Parse the page content for templates, links, etc. | |||||
Actual parsing is handled by :py:mod:`mwparserfromhell`. Raises | |||||
:py:exc:`ImportError` if :py:mod:`mwparserfromhell` isn't installed, | |||||
and :py:exc:`~earwigbot.exceptions.InvalidPageError` or | |||||
:py:exc:`~earwigbot.exceptions.PageNotFoundError` if the page name is | |||||
invalid or the page does not exist, respectively. | |||||
""" | |||||
if not mwparserfromhell: | |||||
raise ImportError("mwparserfromhell") | |||||
return mwparserfromhell.parse(self.get()) | |||||
def edit(self, text, summary, minor=False, bot=True, force=False): | def edit(self, text, summary, minor=False, bot=True, force=False): | ||||
"""Replaces the page's content or creates a new page. | |||||
"""Replace the page's content or creates a new page. | |||||
`text` is the new page content, with `summary` as the edit summary. | |||||
If `minor` is True, the edit will be marked as minor. If `bot` is true, | |||||
the edit will be marked as a bot edit, but only if we actually have a | |||||
bot flag. | |||||
*text* is the new page content, with *summary* as the edit summary. | |||||
If *minor* is ``True``, the edit will be marked as minor. If *bot* is | |||||
``True``, the edit will be marked as a bot edit, but only if we | |||||
actually have a bot flag. | |||||
Use `force` to push the new content even if there's an edit conflict or | |||||
Use *force* to push the new content even if there's an edit conflict or | |||||
the page was deleted/recreated between getting our edit token and | the page was deleted/recreated between getting our edit token and | ||||
editing our page. Be careful with this! | editing our page. Be careful with this! | ||||
""" | """ | ||||
@@ -664,15 +714,66 @@ class Page(CopyvioMixin): | |||||
force=force) | force=force) | ||||
def add_section(self, text, title, minor=False, bot=True, force=False): | def add_section(self, text, title, minor=False, bot=True, force=False): | ||||
"""Adds a new section to the bottom of the page. | |||||
The arguments for this are the same as those for edit(), but instead of | |||||
providing a summary, you provide a section title. | |||||
"""Add a new section to the bottom of the page. | |||||
Likewise, raised exceptions are the same as edit()'s. | |||||
The arguments for this are the same as those for :py:meth:`edit`, but | |||||
instead of providing a summary, you provide a section title. Likewise, | |||||
raised exceptions are the same as :py:meth:`edit`'s. | |||||
This should create the page if it does not already exist, with just the | This should create the page if it does not already exist, with just the | ||||
new section as content. | new section as content. | ||||
""" | """ | ||||
self._edit(text=text, summary=title, minor=minor, bot=bot, force=force, | self._edit(text=text, summary=title, minor=minor, bot=bot, force=force, | ||||
section="new") | section="new") | ||||
def check_exclusion(self, username=None, optouts=None): | |||||
"""Check whether or not we are allowed to edit the page. | |||||
Return ``True`` if we *are* allowed to edit this page, and ``False`` if | |||||
we aren't. | |||||
*username* is used to determine whether we are part of a specific list | |||||
of allowed or disallowed bots (e.g. ``{{bots|allow=EarwigBot}}`` or | |||||
``{{bots|deny=FooBot,EarwigBot}}``). It's ``None`` by default, which | |||||
will swipe our username from :py:meth:`site.get_user() | |||||
<earwigbot.wiki.site.Site.get_user>`.\ | |||||
:py:attr:`~earwigbot.wiki.user.User.name`. | |||||
*optouts* is a list of messages to consider this check as part of for | |||||
the purpose of opt-out; it defaults to ``None``, which ignores the | |||||
parameter completely. For example, if *optouts* is ``["nolicense"]``, | |||||
we'll return ``False`` on ``{{bots|optout=nolicense}}`` or | |||||
``{{bots|optout=all}}``, but `True` on | |||||
``{{bots|optout=orfud,norationale,replaceable}}``. | |||||
""" | |||||
def parse_param(template, param): | |||||
value = template.get_param(param).value | |||||
return [item.strip().lower() for item in value.split(",")] | |||||
if not username: | |||||
username = self.site.get_user().name | |||||
# Lowercase everything: | |||||
username = username.lower() | |||||
optouts = [optout.lower() for optout in optouts] if optouts else [] | |||||
re_bots = "\{\{\s*(no)?bots\s*(\||\}\})" | |||||
filter = self.parse().filter_templates(matches=re_bots, recursive=True) | |||||
for template in filter: | |||||
if template.has_param("deny"): | |||||
denies = parse_param(template, "deny") | |||||
if "all" in denies or username in denies: | |||||
return False | |||||
if template.has_param("allow"): | |||||
allows = parse_param(template, "allow") | |||||
if "all" in allows or username in allows: | |||||
continue | |||||
if optouts and template.has_param("optout"): | |||||
tasks = parse_param(template, "optout") | |||||
matches = [optout in tasks for optout in optouts] | |||||
if "all" in tasks or any(matches): | |||||
return False | |||||
if template.name.strip().lower() == "nobots": | |||||
return False | |||||
return True |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -23,11 +23,13 @@ | |||||
from cookielib import CookieJar | from cookielib import CookieJar | ||||
from gzip import GzipFile | from gzip import GzipFile | ||||
from json import loads | from json import loads | ||||
from logging import getLogger, NullHandler | |||||
from os.path import expanduser | from os.path import expanduser | ||||
from re import escape as re_escape, match as re_match | from re import escape as re_escape, match as re_match | ||||
from StringIO import StringIO | from StringIO import StringIO | ||||
from time import sleep | |||||
from urllib import unquote_plus, urlencode | |||||
from threading import Lock | |||||
from time import sleep, time | |||||
from urllib import quote_plus | |||||
from urllib2 import build_opener, HTTPCookieProcessor, URLError | from urllib2 import build_opener, HTTPCookieProcessor, URLError | ||||
from urlparse import urlparse | from urlparse import urlparse | ||||
@@ -36,53 +38,72 @@ try: | |||||
except ImportError: | except ImportError: | ||||
oursql = None | oursql = None | ||||
from earwigbot.wiki import logger | |||||
from earwigbot import exceptions | |||||
from earwigbot.wiki import constants | |||||
from earwigbot.wiki.category import Category | from earwigbot.wiki.category import Category | ||||
from earwigbot.wiki.constants import * | |||||
from earwigbot.wiki.exceptions import * | |||||
from earwigbot.wiki.page import Page | from earwigbot.wiki.page import Page | ||||
from earwigbot.wiki.user import User | from earwigbot.wiki.user import User | ||||
__all__ = ["Site"] | |||||
class Site(object): | class Site(object): | ||||
""" | """ | ||||
EarwigBot's Wiki Toolset: Site Class | |||||
Represents a Site, with support for API queries and returning Pages, Users, | |||||
and Categories. The constructor takes a bunch of arguments and you probably | |||||
won't need to call it directly, rather tools.get_site() for returning Site | |||||
instances, tools.add_site() for adding new ones to config, and | |||||
tools.del_site() for removing old ones from config, should suffice. | |||||
Public methods: | |||||
name -- returns our name (or "wikiid"), like "enwiki" | |||||
project -- returns our project name, like "wikipedia" | |||||
lang -- returns our language code, like "en" | |||||
domain -- returns our web domain, like "en.wikipedia.org" | |||||
api_query -- does an API query with the given kwargs as params | |||||
sql_query -- does an SQL query and yields its results | |||||
get_replag -- returns the estimated database replication lag | |||||
namespace_id_to_name -- given a namespace ID, returns associated name(s) | |||||
namespace_name_to_id -- given a namespace name, returns associated id | |||||
get_page -- returns a Page object for the given title | |||||
get_category -- returns a Category object for the given title | |||||
get_user -- returns a User object for the given username | |||||
**EarwigBot: Wiki Toolset: Site** | |||||
Represents a site, with support for API queries and returning | |||||
:py:class:`~earwigbot.wiki.page.Page`, | |||||
:py:class:`~earwigbot.wiki.user.User`, | |||||
and :py:class:`~earwigbot.wiki.category.Category` objects. The constructor | |||||
takes a bunch of arguments and you probably won't need to call it directly, | |||||
rather :py:meth:`wiki.get_site() <earwigbot.wiki.sitesdb.SitesDB.get_site>` | |||||
for returning :py:class:`Site` | |||||
instances, :py:meth:`wiki.add_site() | |||||
<earwigbot.wiki.sitesdb.SitesDB.add_site>` for adding new ones to our | |||||
database, and :py:meth:`wiki.remove_site() | |||||
<earwigbot.wiki.sitesdb.SitesDB.remove_site>` for removing old ones from | |||||
our database, should suffice. | |||||
*Attributes:* | |||||
- :py:attr:`name`: the site's name (or "wikiid"), like ``"enwiki"`` | |||||
- :py:attr:`project`: the site's project name, like ``"wikipedia"`` | |||||
- :py:attr:`lang`: the site's language code, like ``"en"`` | |||||
- :py:attr:`domain`: the site's web domain, like ``"en.wikipedia.org"`` | |||||
- :py:attr:`url`: the site's URL, like ``"https://en.wikipedia.org"`` | |||||
*Public methods:* | |||||
- :py:meth:`api_query`: does an API query with kwargs as params | |||||
- :py:meth:`sql_query`: does an SQL query and yields its results | |||||
- :py:meth:`get_maxlag`: returns the internal database lag | |||||
- :py:meth:`get_replag`: estimates the external database lag | |||||
- :py:meth:`namespace_id_to_name`: returns names associated with an NS id | |||||
- :py:meth:`namespace_name_to_id`: returns the ID associated with a NS name | |||||
- :py:meth:`get_page`: returns a Page for the given title | |||||
- :py:meth:`get_category`: returns a Category for the given title | |||||
- :py:meth:`get_user`: returns a User object for the given name | |||||
- :py:meth:`delegate`: controls when the API or SQL is used | |||||
""" | """ | ||||
SERVICE_API = 1 | |||||
SERVICE_SQL = 2 | |||||
def __init__(self, name=None, project=None, lang=None, base_url=None, | def __init__(self, name=None, project=None, lang=None, base_url=None, | ||||
article_path=None, script_path=None, sql=None, | article_path=None, script_path=None, sql=None, | ||||
namespaces=None, login=(None, None), cookiejar=None, | namespaces=None, login=(None, None), cookiejar=None, | ||||
user_agent=None, assert_edit=None, maxlag=None, | |||||
user_agent=None, use_https=False, assert_edit=None, | |||||
maxlag=None, wait_between_queries=3, logger=None, | |||||
search_config=(None, None)): | search_config=(None, None)): | ||||
"""Constructor for new Site instances. | """Constructor for new Site instances. | ||||
This probably isn't necessary to call yourself unless you're building a | This probably isn't necessary to call yourself unless you're building a | ||||
Site that's not in your config and you don't want to add it - normally | Site that's not in your config and you don't want to add it - normally | ||||
all you need is tools.get_site(name), which creates the Site for you | |||||
based on your config file. We accept a bunch of kwargs, but the only | |||||
ones you really "need" are `base_url` and `script_path` - this is | |||||
enough to figure out an API url. `login`, a tuple of | |||||
(username, password), is highly recommended. `cookiejar` will be used | |||||
to store cookies, and we'll use a normal CookieJar if none is given. | |||||
all you need is wiki.get_site(name), which creates the Site for you | |||||
based on your config file and the sites database. We accept a bunch of | |||||
kwargs, but the only ones you really "need" are *base_url* and | |||||
*script_path*; this is enough to figure out an API url. *login*, a | |||||
tuple of (username, password), is highly recommended. *cookiejar will | |||||
be used to store cookies, and we'll use a normal CookieJar if none is | |||||
given. | |||||
First, we'll store the given arguments as attributes, then set up our | First, we'll store the given arguments as attributes, then set up our | ||||
URL opener. We'll load any of the attributes that weren't given from | URL opener. We'll load any of the attributes that weren't given from | ||||
@@ -99,25 +120,32 @@ class Site(object): | |||||
self._script_path = script_path | self._script_path = script_path | ||||
self._namespaces = namespaces | self._namespaces = namespaces | ||||
# Attributes used for API queries: | |||||
# Attributes used for API queries: | |||||
self._use_https = use_https | |||||
self._assert_edit = assert_edit | self._assert_edit = assert_edit | ||||
self._maxlag = maxlag | self._maxlag = maxlag | ||||
self._max_retries = 5 | |||||
self._wait_between_queries = wait_between_queries | |||||
self._max_retries = 6 | |||||
self._last_query_time = 0 | |||||
self._api_lock = Lock() | |||||
self._api_info_cache = {"maxlag": 0, "lastcheck": 0} | |||||
# Attributes used for SQL queries: | # Attributes used for SQL queries: | ||||
self._sql_data = sql | self._sql_data = sql | ||||
self._sql_conn = None | self._sql_conn = None | ||||
self._sql_lock = Lock() | |||||
self._sql_info_cache = {"replag": 0, "lastcheck": 0, "usable": None} | |||||
# Attribute used in copyright violation checks (see CopyrightMixin): | |||||
# Attribute used in copyright violation checks (see CopyrightMixIn): | |||||
self._search_config = search_config | self._search_config = search_config | ||||
# Set up cookiejar and URL opener for making API queries: | # Set up cookiejar and URL opener for making API queries: | ||||
if cookiejar is not None: | |||||
if cookiejar: | |||||
self._cookiejar = cookiejar | self._cookiejar = cookiejar | ||||
else: | else: | ||||
self._cookiejar = CookieJar() | self._cookiejar = CookieJar() | ||||
if user_agent is None: | |||||
user_agent = USER_AGENT # Set default UA from wiki.constants | |||||
if not user_agent: | |||||
user_agent = constants.USER_AGENT # Set default UA | |||||
self._opener = build_opener(HTTPCookieProcessor(self._cookiejar)) | self._opener = build_opener(HTTPCookieProcessor(self._cookiejar)) | ||||
self._opener.addheaders = [("User-Agent", user_agent), | self._opener.addheaders = [("User-Agent", user_agent), | ||||
("Accept-Encoding", "gzip")] | ("Accept-Encoding", "gzip")] | ||||
@@ -125,22 +153,29 @@ class Site(object): | |||||
# Get all of the above attributes that were not specified as arguments: | # Get all of the above attributes that were not specified as arguments: | ||||
self._load_attributes() | self._load_attributes() | ||||
# Set up our internal logger: | |||||
if logger: | |||||
self._logger = logger | |||||
else: # Just set up a null logger to eat up our messages: | |||||
self._logger = getLogger("earwigbot.wiki") | |||||
self._logger.addHandler(NullHandler()) | |||||
# If we have a name/pass and the API says we're not logged in, log in: | # If we have a name/pass and the API says we're not logged in, log in: | ||||
self._login_info = name, password = login | self._login_info = name, password = login | ||||
if name is not None and password is not None: | |||||
if name and password: | |||||
logged_in_as = self._get_username_from_cookies() | logged_in_as = self._get_username_from_cookies() | ||||
if logged_in_as is None or name != logged_in_as: | |||||
if not logged_in_as or name != logged_in_as: | |||||
self._login(login) | self._login(login) | ||||
def __repr__(self): | def __repr__(self): | ||||
"""Returns the canonical string representation of the Site.""" | |||||
"""Return the canonical string representation of the Site.""" | |||||
res = ", ".join(( | res = ", ".join(( | ||||
"Site(name={_name!r}", "project={_project!r}", "lang={_lang!r}", | "Site(name={_name!r}", "project={_project!r}", "lang={_lang!r}", | ||||
"base_url={_base_url!r}", "article_path={_article_path!r}", | "base_url={_base_url!r}", "article_path={_article_path!r}", | ||||
"script_path={_script_path!r}", "assert_edit={_assert_edit!r}", | |||||
"maxlag={_maxlag!r}", "sql={_sql!r}", "login={0}", | |||||
"user_agent={2!r}", "cookiejar={1})" | |||||
)) | |||||
"script_path={_script_path!r}", "use_https={_use_https!r}", | |||||
"assert_edit={_assert_edit!r}", "maxlag={_maxlag!r}", | |||||
"sql={_sql_data!r}", "login={0}", "user_agent={2!r}", | |||||
"cookiejar={1})")) | |||||
name, password = self._login_info | name, password = self._login_info | ||||
login = "({0}, {1})".format(repr(name), "hidden" if password else None) | login = "({0}, {1})".format(repr(name), "hidden" if password else None) | ||||
cookies = self._cookiejar.__class__.__name__ | cookies = self._cookiejar.__class__.__name__ | ||||
@@ -152,49 +187,45 @@ class Site(object): | |||||
return res.format(login, cookies, agent, **self.__dict__) | return res.format(login, cookies, agent, **self.__dict__) | ||||
def __str__(self): | def __str__(self): | ||||
"""Returns a nice string representation of the Site.""" | |||||
"""Return a nice string representation of the Site.""" | |||||
res = "<Site {0} ({1}:{2}) at {3}>" | res = "<Site {0} ({1}:{2}) at {3}>" | ||||
return res.format(self.name(), self.project(), self.lang(), | |||||
self.domain()) | |||||
def _api_query(self, params, tries=0, wait=5): | |||||
"""Do an API query with `params` as a dict of parameters. | |||||
This will first attempt to construct an API url from self._base_url and | |||||
self._script_path. We need both of these, or else we'll raise | |||||
SiteAPIError. | |||||
We'll encode the given params, adding format=json along the way, as | |||||
well as &assert= and &maxlag= based on self._assert_edit and _maxlag. | |||||
We make the request through self._opener, which has built-in cookie | |||||
support via self._cookiejar, a User-Agent (wiki.constants.USER_AGENT), | |||||
and Accept-Encoding set to "gzip". | |||||
Assuming everything went well, we'll gunzip the data (if compressed), | |||||
load it as a JSON object, and return it. | |||||
If our request failed for some reason, we'll raise SiteAPIError with | |||||
details. If that reason was due to maxlag, we'll sleep for a bit and | |||||
then repeat the query until we exceed self._max_retries. | |||||
There's helpful MediaWiki API documentation at | |||||
<http://www.mediawiki.org/wiki/API>. | |||||
return res.format(self.name, self.project, self.lang, self.domain) | |||||
def _unicodeify(self, value, encoding="utf8"): | |||||
"""Return input as unicode if it's not unicode to begin with.""" | |||||
if isinstance(value, unicode): | |||||
return value | |||||
return unicode(value, encoding) | |||||
def _urlencode_utf8(self, params): | |||||
"""Implement urllib.urlencode() with support for unicode input.""" | |||||
enc = lambda s: s.encode("utf8") if isinstance(s, unicode) else str(s) | |||||
args = [] | |||||
for key, val in params.iteritems(): | |||||
key = quote_plus(enc(key)) | |||||
val = quote_plus(enc(val)) | |||||
args.append(key + "=" + val) | |||||
return "&".join(args) | |||||
def _api_query(self, params, tries=0, wait=5, ignore_maxlag=False): | |||||
"""Do an API query with *params* as a dict of parameters. | |||||
See the documentation for :py:meth:`api_query` for full implementation | |||||
details. | |||||
""" | """ | ||||
if self._base_url is None or self._script_path is None: | |||||
e = "Tried to do an API query, but no API URL is known." | |||||
raise SiteAPIError(e) | |||||
url = ''.join((self._base_url, self._script_path, "/api.php")) | |||||
params["format"] = "json" # This is the only format we understand | |||||
if self._assert_edit: # If requested, ensure that we're logged in | |||||
params["assert"] = self._assert_edit | |||||
if self._maxlag: # If requested, don't overload the servers | |||||
params["maxlag"] = self._maxlag | |||||
data = urlencode(params) | |||||
logger.debug("{0} -> {1}".format(url, data)) | |||||
since_last_query = time() - self._last_query_time # Throttling support | |||||
if since_last_query < self._wait_between_queries: | |||||
wait_time = self._wait_between_queries - since_last_query | |||||
log = "Throttled: waiting {0} seconds".format(round(wait_time, 2)) | |||||
self._logger.debug(log) | |||||
sleep(wait_time) | |||||
self._last_query_time = time() | |||||
url, data = self._build_api_query(params, ignore_maxlag) | |||||
if "lgpassword" in params: | |||||
self._logger.debug("{0} -> <hidden>".format(url)) | |||||
else: | |||||
self._logger.debug("{0} -> {1}".format(url, data)) | |||||
try: | try: | ||||
response = self._opener.open(url, data) | response = self._opener.open(url, data) | ||||
@@ -206,7 +237,7 @@ class Site(object): | |||||
e = e.format(error.code) | e = e.format(error.code) | ||||
else: | else: | ||||
e = "API query failed." | e = "API query failed." | ||||
raise SiteAPIError(e) | |||||
raise exceptions.APIError(e) | |||||
result = response.read() | result = response.read() | ||||
if response.headers.get("Content-Encoding") == "gzip": | if response.headers.get("Content-Encoding") == "gzip": | ||||
@@ -214,30 +245,51 @@ class Site(object): | |||||
gzipper = GzipFile(fileobj=stream) | gzipper = GzipFile(fileobj=stream) | ||||
result = gzipper.read() | result = gzipper.read() | ||||
return self._handle_api_query_result(result, params, tries, wait) | |||||
def _build_api_query(self, params, ignore_maxlag): | |||||
"""Given API query params, return the URL to query and POST data.""" | |||||
if not self._base_url or self._script_path is None: | |||||
e = "Tried to do an API query, but no API URL is known." | |||||
raise exceptions.APIError(e) | |||||
url = ''.join((self.url, self._script_path, "/api.php")) | |||||
params["format"] = "json" # This is the only format we understand | |||||
if self._assert_edit: # If requested, ensure that we're logged in | |||||
params["assert"] = self._assert_edit | |||||
if self._maxlag and not ignore_maxlag: | |||||
# If requested, don't overload the servers: | |||||
params["maxlag"] = self._maxlag | |||||
data = self._urlencode_utf8(params) | |||||
return url, data | |||||
def _handle_api_query_result(self, result, params, tries, wait): | |||||
"""Given the result of an API query, attempt to return useful data.""" | |||||
try: | try: | ||||
res = loads(result) # Parse as a JSON object | |||||
res = loads(result) # Try to parse as a JSON object | |||||
except ValueError: | except ValueError: | ||||
e = "API query failed: JSON could not be decoded." | e = "API query failed: JSON could not be decoded." | ||||
raise SiteAPIError(e) | |||||
raise exceptions.APIError(e) | |||||
try: | try: | ||||
code = res["error"]["code"] | code = res["error"]["code"] | ||||
info = res["error"]["info"] | info = res["error"]["info"] | ||||
except (TypeError, KeyError): | |||||
return res | |||||
except (TypeError, KeyError): # Having these keys indicates a problem | |||||
return res # All is well; return the decoded JSON | |||||
if code == "maxlag": | |||||
if code == "maxlag": # We've been throttled by the server | |||||
if tries >= self._max_retries: | if tries >= self._max_retries: | ||||
e = "Maximum number of retries reached ({0})." | e = "Maximum number of retries reached ({0})." | ||||
raise SiteAPIError(e.format(self._max_retries)) | |||||
raise exceptions.APIError(e.format(self._max_retries)) | |||||
tries += 1 | tries += 1 | ||||
msg = 'Server says: "{0}". Retrying in {1} seconds ({2}/{3}).' | |||||
logger.info(msg.format(info, wait, tries, self._max_retries)) | |||||
msg = 'Server says "{0}"; retrying in {1} seconds ({2}/{3})' | |||||
self._logger.info(msg.format(info, wait, tries, self._max_retries)) | |||||
sleep(wait) | sleep(wait) | ||||
return self._api_query(params, tries=tries, wait=wait*3) | |||||
else: | |||||
return self._api_query(params, tries=tries, wait=wait*2) | |||||
else: # Some unknown error occurred | |||||
e = 'API query failed: got error "{0}"; server says: "{1}".' | e = 'API query failed: got error "{0}"; server says: "{1}".' | ||||
error = SiteAPIError(e.format(code, info)) | |||||
error = exceptions.APIError(e.format(code, info)) | |||||
error.code, error.info = code, info | error.code, error.info = code, info | ||||
raise error | raise error | ||||
@@ -248,25 +300,24 @@ class Site(object): | |||||
was not given as a keyword argument. We'll do an API query to get the | was not given as a keyword argument. We'll do an API query to get the | ||||
missing data, but only if there actually *is* missing data. | missing data, but only if there actually *is* missing data. | ||||
Additionally, you can call this with `force=True` to forcibly reload | |||||
all attributes. | |||||
Additionally, you can call this with *force* set to True to forcibly | |||||
reload all attributes. | |||||
""" | """ | ||||
# All attributes to be loaded, except _namespaces, which is a special | # All attributes to be loaded, except _namespaces, which is a special | ||||
# case because it requires additional params in the API query: | # case because it requires additional params in the API query: | ||||
attrs = [self._name, self._project, self._lang, self._base_url, | attrs = [self._name, self._project, self._lang, self._base_url, | ||||
self._article_path, self._script_path] | self._article_path, self._script_path] | ||||
params = {"action": "query", "meta": "siteinfo"} | |||||
params = {"action": "query", "meta": "siteinfo", "siprop": "general"} | |||||
if not self._namespaces or force: | if not self._namespaces or force: | ||||
params["siprop"] = "general|namespaces|namespacealiases" | |||||
result = self._api_query(params) | |||||
params["siprop"] += "|namespaces|namespacealiases" | |||||
result = self.api_query(**params) | |||||
self._load_namespaces(result) | self._load_namespaces(result) | ||||
elif all(attrs): # Everything is already specified and we're not told | elif all(attrs): # Everything is already specified and we're not told | ||||
return # to force a reload, so do nothing | return # to force a reload, so do nothing | ||||
else: # We're only loading attributes other than _namespaces | else: # We're only loading attributes other than _namespaces | ||||
params["siprop"] = "general" | |||||
result = self._api_query(params) | |||||
result = self.api_query(**params) | |||||
res = result["query"]["general"] | res = result["query"]["general"] | ||||
self._name = res["wikiid"] | self._name = res["wikiid"] | ||||
@@ -279,7 +330,7 @@ class Site(object): | |||||
def _load_namespaces(self, result): | def _load_namespaces(self, result): | ||||
"""Fill self._namespaces with a dict of namespace IDs and names. | """Fill self._namespaces with a dict of namespace IDs and names. | ||||
Called by _load_attributes() with API data as `result` when | |||||
Called by _load_attributes() with API data as *result* when | |||||
self._namespaces was not given as an kwarg to __init__(). | self._namespaces was not given as an kwarg to __init__(). | ||||
""" | """ | ||||
self._namespaces = {} | self._namespaces = {} | ||||
@@ -326,44 +377,42 @@ class Site(object): | |||||
If we didn't get any matches, we'll return None. Our goal here isn't to | If we didn't get any matches, we'll return None. Our goal here isn't to | ||||
return the most likely username, or what we *want* our username to be | return the most likely username, or what we *want* our username to be | ||||
(for that, we'd do self._login_info[0]), but rather to get our current | (for that, we'd do self._login_info[0]), but rather to get our current | ||||
username without an unnecessary ?action=query&meta=userinfo API query. | |||||
username without an unnecessary ?action=query&meta=userinfo API query. | |||||
""" | """ | ||||
domain = self.domain() | |||||
name = ''.join((self._name, "Token")) | name = ''.join((self._name, "Token")) | ||||
cookie = self._get_cookie(name, domain) | |||||
cookie = self._get_cookie(name, self.domain) | |||||
if cookie is not None: | |||||
if cookie: | |||||
name = ''.join((self._name, "UserName")) | name = ''.join((self._name, "UserName")) | ||||
user_name = self._get_cookie(name, domain) | |||||
if user_name is not None: | |||||
user_name = self._get_cookie(name, self.domain) | |||||
if user_name: | |||||
return user_name.value | return user_name.value | ||||
name = "centralauth_Token" | name = "centralauth_Token" | ||||
for cookie in self._cookiejar: | |||||
if cookie.domain_initial_dot is False or cookie.is_expired(): | |||||
for cookie in self._cookiejar: | |||||
if not cookie.domain_initial_dot or cookie.is_expired(): | |||||
continue | continue | ||||
if cookie.name != name: | if cookie.name != name: | ||||
continue | continue | ||||
# Build a regex that will match domains this cookie affects: | # Build a regex that will match domains this cookie affects: | ||||
search = ''.join(("(.*?)", re_escape(cookie.domain))) | search = ''.join(("(.*?)", re_escape(cookie.domain))) | ||||
if re_match(search, domain): # Test it against our site | |||||
if re_match(search, self.domain): # Test it against our site | |||||
user_name = self._get_cookie("centralauth_User", cookie.domain) | user_name = self._get_cookie("centralauth_User", cookie.domain) | ||||
if user_name is not None: | |||||
if user_name: | |||||
return user_name.value | return user_name.value | ||||
def _get_username_from_api(self): | def _get_username_from_api(self): | ||||
"""Do a simple API query to get our username and return it. | """Do a simple API query to get our username and return it. | ||||
This is a reliable way to make sure we are actually logged in, because | This is a reliable way to make sure we are actually logged in, because | ||||
it doesn't deal with annoying cookie logic, but it results in an API | it doesn't deal with annoying cookie logic, but it results in an API | ||||
query that is unnecessary in some cases. | query that is unnecessary in some cases. | ||||
Called by _get_username() (in turn called by get_user() with no | Called by _get_username() (in turn called by get_user() with no | ||||
username argument) when cookie lookup fails, probably indicating that | username argument) when cookie lookup fails, probably indicating that | ||||
we are logged out. | we are logged out. | ||||
""" | """ | ||||
params = {"action": "query", "meta": "userinfo"} | |||||
result = self._api_query(params) | |||||
result = self.api_query(action="query", meta="userinfo") | |||||
return result["query"]["userinfo"]["name"] | return result["query"]["userinfo"]["name"] | ||||
def _get_username(self): | def _get_username(self): | ||||
@@ -378,7 +427,7 @@ class Site(object): | |||||
single API query for our username (or IP address) and return that. | single API query for our username (or IP address) and return that. | ||||
""" | """ | ||||
name = self._get_username_from_cookies() | name = self._get_username_from_cookies() | ||||
if name is not None: | |||||
if name: | |||||
return name | return name | ||||
return self._get_username_from_api() | return self._get_username_from_api() | ||||
@@ -411,17 +460,19 @@ class Site(object): | |||||
Raises LoginError on login errors (duh), like bad passwords and | Raises LoginError on login errors (duh), like bad passwords and | ||||
nonexistent usernames. | nonexistent usernames. | ||||
`login` is a (username, password) tuple. `token` is the token returned | |||||
from our first request, and `attempt` is to prevent getting stuck in a | |||||
*login* is a (username, password) tuple. *token* is the token returned | |||||
from our first request, and *attempt* is to prevent getting stuck in a | |||||
loop if MediaWiki isn't acting right. | loop if MediaWiki isn't acting right. | ||||
""" | """ | ||||
name, password = login | name, password = login | ||||
params = {"action": "login", "lgname": name, "lgpassword": password} | |||||
if token is not None: | |||||
params["lgtoken"] = token | |||||
result = self._api_query(params) | |||||
res = result["login"]["result"] | |||||
if token: | |||||
result = self.api_query(action="login", lgname=name, | |||||
lgpassword=password, lgtoken=token) | |||||
else: | |||||
result = self.api_query(action="login", lgname=name, | |||||
lgpassword=password) | |||||
res = result["login"]["result"] | |||||
if res == "Success": | if res == "Success": | ||||
self._save_cookiejar() | self._save_cookiejar() | ||||
elif res == "NeedToken" and attempt == 0: | elif res == "NeedToken" and attempt == 0: | ||||
@@ -438,7 +489,7 @@ class Site(object): | |||||
e = "The given password is incorrect." | e = "The given password is incorrect." | ||||
else: | else: | ||||
e = "Couldn't login; server says '{0}'.".format(res) | e = "Couldn't login; server says '{0}'.".format(res) | ||||
raise LoginError(e) | |||||
raise exceptions.LoginError(e) | |||||
def _logout(self): | def _logout(self): | ||||
"""Safely logout through the API. | """Safely logout through the API. | ||||
@@ -447,18 +498,16 @@ class Site(object): | |||||
cookiejar (which probably contains now-invalidated cookies) and try to | cookiejar (which probably contains now-invalidated cookies) and try to | ||||
save it, if it supports that sort of thing. | save it, if it supports that sort of thing. | ||||
""" | """ | ||||
params = {"action": "logout"} | |||||
self._api_query(params) | |||||
self.api_query(action="logout") | |||||
self._cookiejar.clear() | self._cookiejar.clear() | ||||
self._save_cookiejar() | self._save_cookiejar() | ||||
def _sql_connect(self, **kwargs): | def _sql_connect(self, **kwargs): | ||||
"""Attempt to establish a connection with this site's SQL database. | """Attempt to establish a connection with this site's SQL database. | ||||
oursql.connect() will be called with self._sql_data as its kwargs, | |||||
which is usually config.wiki["sites"][self.name()]["sql"]. Any kwargs | |||||
given to this function will be passed to connect() and will have | |||||
precedence over the config file. | |||||
oursql.connect() will be called with self._sql_data as its kwargs. | |||||
Any kwargs given to this function will be passed to connect() and will | |||||
have precedence over the config file. | |||||
Will raise SQLError() if the module "oursql" is not available. oursql | Will raise SQLError() if the module "oursql" is not available. oursql | ||||
may raise its own exceptions (e.g. oursql.InterfaceError) if it cannot | may raise its own exceptions (e.g. oursql.InterfaceError) if it cannot | ||||
@@ -466,7 +515,7 @@ class Site(object): | |||||
""" | """ | ||||
if not oursql: | if not oursql: | ||||
e = "Module 'oursql' is required for SQL queries." | e = "Module 'oursql' is required for SQL queries." | ||||
raise SQLError(e) | |||||
raise exceptions.SQLError(e) | |||||
args = self._sql_data | args = self._sql_data | ||||
for key, value in kwargs.iteritems(): | for key, value in kwargs.iteritems(): | ||||
@@ -475,85 +524,206 @@ class Site(object): | |||||
if "read_default_file" not in args and "user" not in args and "passwd" not in args: | if "read_default_file" not in args and "user" not in args and "passwd" not in args: | ||||
args["read_default_file"] = expanduser("~/.my.cnf") | args["read_default_file"] = expanduser("~/.my.cnf") | ||||
if "autoping" not in args: | |||||
args["autoping"] = True | |||||
if "autoreconnect" not in args: | |||||
args["autoreconnect"] = True | |||||
self._sql_conn = oursql.connect(**args) | self._sql_conn = oursql.connect(**args) | ||||
def _get_service_order(self): | |||||
"""Return a preferred order for using services (e.g. the API and SQL). | |||||
A list is returned, starting with the most preferred service first and | |||||
ending with the least preferred one. Currently, there are only two | |||||
services. SERVICE_API will always be included since the API is expected | |||||
to be always usable. In normal circumstances, self.SERVICE_SQL will be | |||||
first (with the API second), since using SQL directly is easier on the | |||||
servers than making web queries with the API. self.SERVICE_SQL will be | |||||
second if replag is greater than three minutes (a cached value updated | |||||
every two minutes at most), *unless* API lag is also very high. | |||||
self.SERVICE_SQL will not be included in the list if we cannot form a | |||||
proper SQL connection. | |||||
""" | |||||
now = time() | |||||
if now - self._sql_info_cache["lastcheck"] > 120: | |||||
self._sql_info_cache["lastcheck"] = now | |||||
try: | |||||
self._sql_info_cache["replag"] = sqllag = self.get_replag() | |||||
except (exceptions.SQLError, oursql.Error): | |||||
self._sql_info_cache["usable"] = False | |||||
return [self.SERVICE_API] | |||||
self._sql_info_cache["usable"] = True | |||||
else: | |||||
if not self._sql_info_cache["usable"]: | |||||
return [self.SERVICE_API] | |||||
sqllag = self._sql_info_cache["replag"] | |||||
if sqllag > 180: | |||||
if not self._maxlag: | |||||
return [self.SERVICE_API, self.SERVICE_SQL] | |||||
if now - self._api_info_cache["lastcheck"] > 120: | |||||
self._api_info_cache["lastcheck"] = now | |||||
try: | |||||
self._api_info_cache["maxlag"] = apilag = self.get_maxlag() | |||||
except exceptions.APIError: | |||||
self._api_info_cache["maxlag"] = apilag = 0 | |||||
else: | |||||
apilag = self._api_info_cache["maxlag"] | |||||
if sqllag / (180.0 / self._maxlag) < apilag: | |||||
return [self.SERVICE_SQL, self.SERVICE_API] | |||||
return [self.SERVICE_API, self.SERVICE_SQL] | |||||
return [self.SERVICE_SQL, self.SERVICE_API] | |||||
@property | |||||
def name(self): | def name(self): | ||||
"""Returns the Site's name (or "wikiid" in the API), like "enwiki".""" | |||||
"""The Site's name (or "wikiid" in the API), like ``"enwiki"``.""" | |||||
return self._name | return self._name | ||||
@property | |||||
def project(self): | def project(self): | ||||
"""Returns the Site's project name in lowercase, like "wikipedia".""" | |||||
"""The Site's project name in lowercase, like ``"wikipedia"``.""" | |||||
return self._project | return self._project | ||||
@property | |||||
def lang(self): | def lang(self): | ||||
"""Returns the Site's language code, like "en" or "es".""" | |||||
"""The Site's language code, like ``"en"`` or ``"es"``.""" | |||||
return self._lang | return self._lang | ||||
@property | |||||
def domain(self): | def domain(self): | ||||
"""Returns the Site's web domain, like "en.wikipedia.org".""" | |||||
"""The Site's web domain, like ``"en.wikipedia.org"``.""" | |||||
return urlparse(self._base_url).netloc | return urlparse(self._base_url).netloc | ||||
@property | |||||
def url(self): | |||||
"""The Site's full base URL, like ``"https://en.wikipedia.org"``.""" | |||||
url = self._base_url | |||||
if url.startswith("//"): # Protocol-relative URLs from 1.18 | |||||
if self._use_https: | |||||
url = "https:" + url | |||||
else: | |||||
url = "http:" + url | |||||
return url | |||||
def api_query(self, **kwargs): | def api_query(self, **kwargs): | ||||
"""Do an API query with `kwargs` as the parameters. | """Do an API query with `kwargs` as the parameters. | ||||
See _api_query()'s documentation for details. | |||||
This will first attempt to construct an API url from | |||||
:py:attr:`self._base_url` and :py:attr:`self._script_path`. We need | |||||
both of these, or else we'll raise | |||||
:py:exc:`~earwigbot.exceptions.APIError`. If | |||||
:py:attr:`self._base_url` is protocol-relative (introduced in MediaWiki | |||||
1.18), we'll choose HTTPS only if :py:attr:`self._user_https` is | |||||
``True``, otherwise HTTP. | |||||
We'll encode the given params, adding ``format=json`` along the way, as | |||||
well as ``&assert=`` and ``&maxlag=`` based on | |||||
:py:attr:`self._assert_edit` and :py:attr:`_maxlag` respectively. | |||||
Additionally, we'll sleep a bit if the last query was made fewer than | |||||
:py:attr:`self._wait_between_queries` seconds ago. The request is made | |||||
through :py:attr:`self._opener`, which has cookie support | |||||
(:py:attr:`self._cookiejar`), a ``User-Agent`` | |||||
(:py:const:`earwigbot.wiki.constants.USER_AGENT`), and | |||||
``Accept-Encoding`` set to ``"gzip"``. | |||||
Assuming everything went well, we'll gunzip the data (if compressed), | |||||
load it as a JSON object, and return it. | |||||
If our request failed for some reason, we'll raise | |||||
:py:exc:`~earwigbot.exceptions.APIError` with details. If that | |||||
reason was due to maxlag, we'll sleep for a bit and then repeat the | |||||
query until we exceed :py:attr:`self._max_retries`. | |||||
There is helpful MediaWiki API documentation at `MediaWiki.org | |||||
<http://www.mediawiki.org/wiki/API>`_. | |||||
""" | """ | ||||
return self._api_query(kwargs) | |||||
with self._api_lock: | |||||
return self._api_query(kwargs) | |||||
def sql_query(self, query, params=(), plain_query=False, dict_cursor=False, | def sql_query(self, query, params=(), plain_query=False, dict_cursor=False, | ||||
cursor_class=None, show_table=False): | cursor_class=None, show_table=False): | ||||
"""Do an SQL query and yield its results. | """Do an SQL query and yield its results. | ||||
If `plain_query` is True, we will force an unparameterized query. | |||||
Specifying both params and plain_query will cause an error. | |||||
If `dict_cursor` is True, we will use oursql.DictCursor as our cursor, | |||||
otherwise the default oursql.Cursor. If `cursor_class` is given, it | |||||
will override this option. | |||||
If `show_table` is True, the name of the table will be prepended to the | |||||
name of the column. This will mainly affect a DictCursor. | |||||
Example: | |||||
>>> query = "SELECT user_id, user_registration FROM user WHERE user_name = ?" | |||||
>>> params = ("The Earwig",) | |||||
>>> result1 = site.sql_query(query, params) | |||||
>>> result2 = site.sql_query(query, params, dict_cursor=True) | |||||
>>> for row in result1: print row | |||||
(7418060L, '20080703215134') | |||||
>>> for row in result2: print row | |||||
{'user_id': 7418060L, 'user_registration': '20080703215134'} | |||||
See _sql_connect() for information on how a connection is acquired. | |||||
<http://packages.python.org/oursql> has helpful documentation on the | |||||
oursql module. | |||||
This may raise SQLError() or one of oursql's exceptions | |||||
(oursql.ProgrammingError, oursql.InterfaceError, ...) if there were | |||||
problems with the query. | |||||
If *plain_query* is ``True``, we will force an unparameterized query. | |||||
Specifying both *params* and *plain_query* will cause an error. If | |||||
*dict_cursor* is ``True``, we will use :py:class:`oursql.DictCursor` as | |||||
our cursor, otherwise the default :py:class:`oursql.Cursor`. If | |||||
*cursor_class* is given, it will override this option. If *show_table* | |||||
is True, the name of the table will be prepended to the name of the | |||||
column. This will mainly affect an :py:class:`~oursql.DictCursor`. | |||||
Example usage:: | |||||
>>> query = "SELECT user_id, user_registration FROM user WHERE user_name = ?" | |||||
>>> params = ("The Earwig",) | |||||
>>> result1 = site.sql_query(query, params) | |||||
>>> result2 = site.sql_query(query, params, dict_cursor=True) | |||||
>>> for row in result1: print row | |||||
(7418060L, '20080703215134') | |||||
>>> for row in result2: print row | |||||
{'user_id': 7418060L, 'user_registration': '20080703215134'} | |||||
This may raise :py:exc:`~earwigbot.exceptions.SQLError` or one of | |||||
oursql's exceptions (:py:exc:`oursql.ProgrammingError`, | |||||
:py:exc:`oursql.InterfaceError`, ...) if there were problems with the | |||||
query. | |||||
See :py:meth:`_sql_connect` for information on how a connection is | |||||
acquired. Also relevant is `oursql's documentation | |||||
<http://packages.python.org/oursql>`_ for details on that package. | |||||
""" | """ | ||||
if not self._sql_conn: | |||||
self._sql_connect() | |||||
if not cursor_class: | if not cursor_class: | ||||
if dict_cursor: | if dict_cursor: | ||||
cursor_class = oursql.DictCursor | cursor_class = oursql.DictCursor | ||||
else: | else: | ||||
cursor_class = oursql.Cursor | cursor_class = oursql.Cursor | ||||
with self._sql_conn.cursor(cursor_class, show_table=show_table) as cur: | |||||
cur.execute(query, params, plain_query) | |||||
for result in cur: | |||||
yield result | |||||
klass = cursor_class | |||||
with self._sql_lock: | |||||
if not self._sql_conn: | |||||
self._sql_connect() | |||||
with self._sql_conn.cursor(klass, show_table=show_table) as cur: | |||||
cur.execute(query, params, plain_query) | |||||
for result in cur: | |||||
yield result | |||||
def get_maxlag(self, showall=False): | |||||
"""Return the internal database replication lag in seconds. | |||||
In a typical setup, this function returns the replication lag *within* | |||||
the WMF's cluster, *not* external replication lag affecting the | |||||
Toolserver (see :py:meth:`get_replag` for that). This is useful when | |||||
combined with the ``maxlag`` API query param (added by config), in | |||||
which queries will be halted and retried if the lag is too high, | |||||
usually above five seconds. | |||||
With *showall*, will return a list of the lag for all servers in the | |||||
cluster, not just the one with the highest lag. | |||||
""" | |||||
params = {"action": "query", "meta": "siteinfo", "siprop": "dbrepllag"} | |||||
if showall: | |||||
params["sishowalldb"] = 1 | |||||
with self._api_lock: | |||||
result = self._api_query(params, ignore_maxlag=True) | |||||
if showall: | |||||
return [server["lag"] for server in result["query"]["dbrepllag"]] | |||||
return result["query"]["dbrepllag"][0]["lag"] | |||||
def get_replag(self): | def get_replag(self): | ||||
"""Return the estimated database replication lag in seconds. | |||||
"""Return the estimated external database replication lag in seconds. | |||||
Requires SQL access. This function only makes sense on a replicated | Requires SQL access. This function only makes sense on a replicated | ||||
database (e.g. the Wikimedia Toolserver) and on a wiki that receives a | database (e.g. the Wikimedia Toolserver) and on a wiki that receives a | ||||
large number of edits (ideally, at least one per second), or the result | large number of edits (ideally, at least one per second), or the result | ||||
may be larger than expected. | |||||
may be larger than expected, since it works by subtracting the current | |||||
time from the timestamp of the latest recent changes event. | |||||
This may raise :py:exc:`~earwigbot.exceptions.SQLError` or one of | |||||
oursql's exceptions (:py:exc:`oursql.ProgrammingError`, | |||||
:py:exc:`oursql.InterfaceError`, ...) if there were problems. | |||||
""" | """ | ||||
query = """SELECT UNIX_TIMESTAMP() - UNIX_TIMESTAMP(rc_timestamp) FROM | query = """SELECT UNIX_TIMESTAMP() - UNIX_TIMESTAMP(rc_timestamp) FROM | ||||
recentchanges ORDER BY rc_timestamp DESC LIMIT 1""" | recentchanges ORDER BY rc_timestamp DESC LIMIT 1""" | ||||
@@ -563,14 +733,15 @@ class Site(object): | |||||
def namespace_id_to_name(self, ns_id, all=False): | def namespace_id_to_name(self, ns_id, all=False): | ||||
"""Given a namespace ID, returns associated namespace names. | """Given a namespace ID, returns associated namespace names. | ||||
If all is False (default), we'll return the first name in the list, | |||||
which is usually the localized version. Otherwise, we'll return the | |||||
entire list, which includes the canonical name. | |||||
If *all* is ``False`` (default), we'll return the first name in the | |||||
list, which is usually the localized version. Otherwise, we'll return | |||||
the entire list, which includes the canonical name. For example, this | |||||
returns ``u"Wikipedia"`` if *ns_id* = ``4`` and *all* is ``False`` on | |||||
``enwiki``; returns ``[u"Wikipedia", u"Project", u"WP"]`` if *ns_id* = | |||||
``4`` and *all* is ``True``. | |||||
For example, returns u"Wikipedia" if ns_id=4 and all=False on enwiki; | |||||
returns [u"Wikipedia", u"Project", u"WP"] if ns_id=4 and all=True. | |||||
Raises NamespaceNotFoundError if the ID is not found. | |||||
Raises :py:exc:`~earwigbot.exceptions.NamespaceNotFoundError` if the ID | |||||
is not found. | |||||
""" | """ | ||||
try: | try: | ||||
if all: | if all: | ||||
@@ -579,15 +750,16 @@ class Site(object): | |||||
return self._namespaces[ns_id][0] | return self._namespaces[ns_id][0] | ||||
except KeyError: | except KeyError: | ||||
e = "There is no namespace with id {0}.".format(ns_id) | e = "There is no namespace with id {0}.".format(ns_id) | ||||
raise NamespaceNotFoundError(e) | |||||
raise exceptions.NamespaceNotFoundError(e) | |||||
def namespace_name_to_id(self, name): | def namespace_name_to_id(self, name): | ||||
"""Given a namespace name, returns the associated ID. | """Given a namespace name, returns the associated ID. | ||||
Like namespace_id_to_name(), but reversed. Case is ignored, because | |||||
namespaces are assumed to be case-insensitive. | |||||
Like :py:meth:`namespace_id_to_name`, but reversed. Case is ignored, | |||||
because namespaces are assumed to be case-insensitive. | |||||
Raises NamespaceNotFoundError if the name is not found. | |||||
Raises :py:exc:`~earwigbot.exceptions.NamespaceNotFoundError` if the | |||||
name is not found. | |||||
""" | """ | ||||
lname = name.lower() | lname = name.lower() | ||||
for ns_id, names in self._namespaces.items(): | for ns_id, names in self._namespaces.items(): | ||||
@@ -596,41 +768,80 @@ class Site(object): | |||||
return ns_id | return ns_id | ||||
e = "There is no namespace with name '{0}'.".format(name) | e = "There is no namespace with name '{0}'.".format(name) | ||||
raise NamespaceNotFoundError(e) | |||||
raise exceptions.NamespaceNotFoundError(e) | |||||
def get_page(self, title, follow_redirects=False): | |||||
"""Returns a Page object for the given title (pagename). | |||||
def get_page(self, title, follow_redirects=False, pageid=None): | |||||
"""Return a :py:class:`Page` object for the given title. | |||||
Will return a Category object instead if the given title is in the | |||||
category namespace. As Category is a subclass of Page, this should not | |||||
cause problems. | |||||
*follow_redirects* is passed directly to | |||||
:py:class:`~earwigbot.wiki.page.Page`'s constructor. Also, this will | |||||
return a :py:class:`~earwigbot.wiki.category.Category` object instead | |||||
if the given title is in the category namespace. As | |||||
:py:class:`~earwigbot.wiki.category.Category` is a subclass of | |||||
:py:class:`~earwigbot.wiki.page.Page`, this should not cause problems. | |||||
Note that this doesn't do any direct checks for existence or | Note that this doesn't do any direct checks for existence or | ||||
redirect-following - Page's methods provide that. | |||||
redirect-following: :py:class:`~earwigbot.wiki.page.Page`'s methods | |||||
provide that. | |||||
""" | """ | ||||
prefixes = self.namespace_id_to_name(NS_CATEGORY, all=True) | |||||
title = self._unicodeify(title) | |||||
prefixes = self.namespace_id_to_name(constants.NS_CATEGORY, all=True) | |||||
prefix = title.split(":", 1)[0] | prefix = title.split(":", 1)[0] | ||||
if prefix != title: # Avoid a page that is simply "Category" | if prefix != title: # Avoid a page that is simply "Category" | ||||
if prefix in prefixes: | if prefix in prefixes: | ||||
return Category(self, title, follow_redirects) | |||||
return Page(self, title, follow_redirects) | |||||
return Category(self, title, follow_redirects, pageid) | |||||
return Page(self, title, follow_redirects, pageid) | |||||
def get_category(self, catname, follow_redirects=False): | |||||
"""Returns a Category object for the given category name. | |||||
def get_category(self, catname, follow_redirects=False, pageid=None): | |||||
"""Return a :py:class:`Category` object for the given category name. | |||||
`catname` should be given *without* a namespace prefix. This method is | |||||
really just shorthand for get_page("Category:" + catname). | |||||
*catname* should be given *without* a namespace prefix. This method is | |||||
really just shorthand for :py:meth:`get_page("Category:" + catname) | |||||
<get_page>`. | |||||
""" | """ | ||||
prefix = self.namespace_id_to_name(NS_CATEGORY) | |||||
pagename = ':'.join((prefix, catname)) | |||||
return Category(self, pagename, follow_redirects) | |||||
catname = self._unicodeify(catname) | |||||
prefix = self.namespace_id_to_name(constants.NS_CATEGORY) | |||||
pagename = u':'.join((prefix, catname)) | |||||
return Category(self, pagename, follow_redirects, pageid) | |||||
def get_user(self, username=None): | def get_user(self, username=None): | ||||
"""Returns a User object for the given username. | |||||
"""Return a :py:class:`User` object for the given username. | |||||
If `username` is left as None, then a User object representing the | |||||
currently logged-in (or anonymous!) user is returned. | |||||
If *username* is left as ``None``, then a | |||||
:py:class:`~earwigbot.wiki.user.User` object representing the currently | |||||
logged-in (or anonymous!) user is returned. | |||||
""" | """ | ||||
if username is None: | |||||
if username: | |||||
username = self._unicodeify(username) | |||||
else: | |||||
username = self._get_username() | username = self._get_username() | ||||
return User(self, username) | return User(self, username) | ||||
def delegate(self, services, args=None, kwargs=None): | |||||
"""Delegate a task to either the API or SQL depending on conditions. | |||||
*services* should be a dictionary in which the key is the service name | |||||
(:py:attr:`self.SERVICE_API <SERVICE_API>` or | |||||
:py:attr:`self.SERVICE_SQL <SERVICE_SQL>`), and the value is the | |||||
function to call for this service. All functions will be passed the | |||||
same arguments the tuple *args* and the dict **kwargs**, which are both | |||||
empty by default. The service order is determined by | |||||
:py:meth:`_get_service_order`. | |||||
Not every service needs an entry in the dictionary. Will raise | |||||
:py:exc:`~earwigbot.exceptions.NoServiceError` if an appropriate | |||||
service cannot be found. | |||||
""" | |||||
if not args: | |||||
args = () | |||||
if not kwargs: | |||||
kwargs = {} | |||||
order = self._get_service_order() | |||||
for srv in order: | |||||
if srv in services: | |||||
try: | |||||
return services[srv](*args, **kwargs) | |||||
except exceptions.ServiceError: | |||||
continue | |||||
raise exceptions.NoServiceError(services) |
@@ -0,0 +1,405 @@ | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
from cookielib import LWPCookieJar, LoadError | |||||
import errno | |||||
from os import chmod, path | |||||
from platform import python_version | |||||
import stat | |||||
import sqlite3 as sqlite | |||||
from earwigbot import __version__ | |||||
from earwigbot.exceptions import SiteNotFoundError | |||||
from earwigbot.wiki.site import Site | |||||
__all__ = ["SitesDB"] | |||||
class SitesDB(object): | |||||
""" | |||||
**EarwigBot: Wiki Toolset: Sites Database Manager** | |||||
This class controls the :file:`sites.db` file, which stores information | |||||
about all wiki sites known to the bot. Three public methods act as bridges | |||||
between the bot's config files and :py:class:`~earwigbot.wiki.site.Site` | |||||
objects: | |||||
- :py:meth:`get_site`: returns a Site object corresponding to a site | |||||
- :py:meth:`add_site`: stores a site in the database | |||||
- :py:meth:`remove_site`: removes a site from the database | |||||
There's usually no need to use this class directly. All public methods | |||||
here are available as :py:meth:`bot.wiki.get_site`, | |||||
:py:meth:`bot.wiki.add_site`, and :py:meth:`bot.wiki.remove_site`, which | |||||
use a :file:`sites.db` file located in the same directory as our | |||||
:file:`config.yml` file. Lower-level access can be achieved by importing | |||||
the manager class (``from earwigbot.wiki import SitesDB``). | |||||
""" | |||||
def __init__(self, bot): | |||||
"""Set up the manager with an attribute for the base Bot object.""" | |||||
self.config = bot.config | |||||
self._logger = bot.logger.getChild("wiki") | |||||
self._sites = {} # Internal site cache | |||||
self._sitesdb = path.join(bot.config.root_dir, "sites.db") | |||||
self._cookie_file = path.join(bot.config.root_dir, ".cookies") | |||||
self._cookiejar = None | |||||
def __repr__(self): | |||||
"""Return the canonical string representation of the SitesDB.""" | |||||
res = "SitesDB(config={0!r}, sitesdb={1!r}, cookie_file={2!r})" | |||||
return res.format(self.config, self._sitesdb, self._cookie_file) | |||||
def __str__(self): | |||||
"""Return a nice string representation of the SitesDB.""" | |||||
return "<SitesDB at {0}>".format(self._sitesdb) | |||||
def _get_cookiejar(self): | |||||
"""Return a LWPCookieJar object loaded from our .cookies file. | |||||
The same .cookies file is returned every time, located in the project | |||||
root, same directory as config.yml and bot.py. If it doesn't exist, we | |||||
will create the file and set it to be readable and writeable only by | |||||
us. If it exists but the information inside is bogus, we'll ignore it. | |||||
This is normally called by _make_site_object() (in turn called by | |||||
get_site()), and the cookiejar is passed to our Site's constructor, | |||||
used when it makes API queries. This way, we can easily preserve | |||||
cookies between sites (e.g., for CentralAuth), making logins easier. | |||||
""" | |||||
if self._cookiejar: | |||||
return self._cookiejar | |||||
self._cookiejar = LWPCookieJar(self._cookie_file) | |||||
try: | |||||
self._cookiejar.load() | |||||
except LoadError: | |||||
pass # File contains bad data, so ignore it completely | |||||
except IOError as e: | |||||
if e.errno == errno.ENOENT: # "No such file or directory" | |||||
# Create the file and restrict reading/writing only to the | |||||
# owner, so others can't peak at our cookies: | |||||
open(self._cookie_file, "w").close() | |||||
chmod(self._cookie_file, stat.S_IRUSR|stat.S_IWUSR) | |||||
else: | |||||
raise | |||||
return self._cookiejar | |||||
def _create_sitesdb(self): | |||||
"""Initialize the sitesdb file with its three necessary tables.""" | |||||
script = """ | |||||
CREATE TABLE sites (site_name, site_project, site_lang, site_base_url, | |||||
site_article_path, site_script_path); | |||||
CREATE TABLE sql_data (sql_site, sql_data_key, sql_data_value); | |||||
CREATE TABLE namespaces (ns_site, ns_id, ns_name, ns_is_primary_name); | |||||
""" | |||||
with sqlite.connect(self._sitesdb) as conn: | |||||
conn.executescript(script) | |||||
def _get_site_object(self, name): | |||||
"""Return the site from our cache, or create it if it doesn't exist. | |||||
This is essentially just a wrapper around _make_site_object that | |||||
returns the same object each time a specific site is asked for. | |||||
""" | |||||
try: | |||||
return self._sites[name] | |||||
except KeyError: | |||||
site = self._make_site_object(name) | |||||
self._sites[name] = site | |||||
return site | |||||
def _load_site_from_sitesdb(self, name): | |||||
"""Return all information stored in the sitesdb relating to given site. | |||||
The information will be returned as a tuple, containing the site's | |||||
name, project, language, base URL, article path, script path, SQL | |||||
connection data, and namespaces, in that order. If the site is not | |||||
found in the database, SiteNotFoundError will be raised. An empty | |||||
database will be created before the exception is raised if none exists. | |||||
""" | |||||
query1 = "SELECT * FROM sites WHERE site_name = ?" | |||||
query2 = "SELECT sql_data_key, sql_data_value FROM sql_data WHERE sql_site = ?" | |||||
query3 = "SELECT ns_id, ns_name, ns_is_primary_name FROM namespaces WHERE ns_site = ?" | |||||
error = "Site '{0}' not found in the sitesdb.".format(name) | |||||
with sqlite.connect(self._sitesdb) as conn: | |||||
try: | |||||
site_data = conn.execute(query1, (name,)).fetchone() | |||||
except sqlite.OperationalError: | |||||
self._create_sitesdb() | |||||
raise SiteNotFoundError(error) | |||||
if not site_data: | |||||
raise SiteNotFoundError(error) | |||||
sql_data = conn.execute(query2, (name,)).fetchall() | |||||
ns_data = conn.execute(query3, (name,)).fetchall() | |||||
name, project, lang, base_url, article_path, script_path = site_data | |||||
sql = dict(sql_data) | |||||
namespaces = {} | |||||
for ns_id, ns_name, ns_is_primary_name in ns_data: | |||||
try: | |||||
if ns_is_primary_name: # "Primary" name goes first in list | |||||
namespaces[ns_id].insert(0, ns_name) | |||||
else: # Ordering of the aliases doesn't matter | |||||
namespaces[ns_id].append(ns_name) | |||||
except KeyError: | |||||
namespaces[ns_id] = [ns_name] | |||||
return (name, project, lang, base_url, article_path, script_path, sql, | |||||
namespaces) | |||||
def _make_site_object(self, name): | |||||
"""Return a Site object associated with the site *name* in our sitesdb. | |||||
This calls _load_site_from_sitesdb(), so SiteNotFoundError will be | |||||
raised if the site is not in our sitesdb. | |||||
""" | |||||
cookiejar = self._get_cookiejar() | |||||
(name, project, lang, base_url, article_path, script_path, sql, | |||||
namespaces) = self._load_site_from_sitesdb(name) | |||||
config = self.config | |||||
login = (config.wiki.get("username"), config.wiki.get("password")) | |||||
user_agent = config.wiki.get("userAgent") | |||||
use_https = config.wiki.get("useHTTPS", False) | |||||
assert_edit = config.wiki.get("assert") | |||||
maxlag = config.wiki.get("maxlag") | |||||
wait_between_queries = config.wiki.get("waitTime", 3) | |||||
logger = self._logger.getChild(name) | |||||
search_config = config.wiki.get("search") | |||||
if user_agent: | |||||
user_agent = user_agent.replace("$1", __version__) | |||||
user_agent = user_agent.replace("$2", python_version()) | |||||
return Site(name=name, project=project, lang=lang, base_url=base_url, | |||||
article_path=article_path, script_path=script_path, | |||||
sql=sql, namespaces=namespaces, login=login, | |||||
cookiejar=cookiejar, user_agent=user_agent, | |||||
use_https=use_https, assert_edit=assert_edit, | |||||
maxlag=maxlag, wait_between_queries=wait_between_queries, | |||||
logger=logger, search_config=search_config) | |||||
def _get_site_name_from_sitesdb(self, project, lang): | |||||
"""Return the name of the first site with the given project and lang. | |||||
If the site is not found, return None. An empty sitesdb will be created | |||||
if none exists. | |||||
""" | |||||
query = "SELECT site_name FROM sites WHERE site_project = ? and site_lang = ?" | |||||
with sqlite.connect(self._sitesdb) as conn: | |||||
try: | |||||
site = conn.execute(query, (project, lang)).fetchone() | |||||
return site[0] if site else None | |||||
except sqlite.OperationalError: | |||||
self._create_sitesdb() | |||||
def _add_site_to_sitesdb(self, site): | |||||
"""Extract relevant info from a Site object and add it to the sitesdb. | |||||
Works like a reverse _load_site_from_sitesdb(); the site's project, | |||||
language, base URL, article path, script path, SQL connection data, and | |||||
namespaces are extracted from the site and inserted into the sites | |||||
database. If the sitesdb doesn't exist, we'll create it first. | |||||
""" | |||||
name = site.name | |||||
sites_data = (name, site.project, site.lang, site._base_url, | |||||
site._article_path, site._script_path) | |||||
sql_data = [(name, key, val) for key, val in site._sql_data.iteritems()] | |||||
ns_data = [] | |||||
for ns_id, ns_names in site._namespaces.iteritems(): | |||||
ns_data.append((name, ns_id, ns_names.pop(0), True)) | |||||
for ns_name in ns_names: | |||||
ns_data.append((name, ns_id, ns_name, False)) | |||||
with sqlite.connect(self._sitesdb) as conn: | |||||
check_exists = "SELECT 1 FROM sites WHERE site_name = ?" | |||||
try: | |||||
exists = conn.execute(check_exists, (name,)).fetchone() | |||||
except sqlite.OperationalError: | |||||
self._create_sitesdb() | |||||
else: | |||||
if exists: | |||||
conn.execute("DELETE FROM sites WHERE site_name = ?", (name,)) | |||||
conn.execute("DELETE FROM sql_data WHERE sql_site = ?", (name,)) | |||||
conn.execute("DELETE FROM namespaces WHERE ns_site = ?", (name,)) | |||||
conn.execute("INSERT INTO sites VALUES (?, ?, ?, ?, ?, ?)", sites_data) | |||||
conn.executemany("INSERT INTO sql_data VALUES (?, ?, ?)", sql_data) | |||||
conn.executemany("INSERT INTO namespaces VALUES (?, ?, ?, ?)", ns_data) | |||||
def _remove_site_from_sitesdb(self, name): | |||||
"""Remove a site by name from the sitesdb and the internal cache.""" | |||||
try: | |||||
del self._sites[name] | |||||
except KeyError: | |||||
pass | |||||
with sqlite.connect(self._sitesdb) as conn: | |||||
cursor = conn.execute("DELETE FROM sites WHERE site_name = ?", (name,)) | |||||
if cursor.rowcount == 0: | |||||
return False | |||||
else: | |||||
conn.execute("DELETE FROM sql_data WHERE sql_site = ?", (name,)) | |||||
conn.execute("DELETE FROM namespaces WHERE ns_site = ?", (name,)) | |||||
return True | |||||
def get_site(self, name=None, project=None, lang=None): | |||||
"""Return a Site instance based on information from the sitesdb. | |||||
With no arguments, return the default site as specified by our config | |||||
file. This is ``config.wiki["defaultSite"]``. | |||||
With *name* specified, return the site with that name. This is | |||||
equivalent to the site's ``wikiid`` in the API, like *enwiki*. | |||||
With *project* and *lang* specified, return the site whose project and | |||||
language match these values. If there are multiple sites with the same | |||||
values (unlikely), this is not a reliable way of loading a site. Call | |||||
the function with an explicit *name* in that case. | |||||
We will attempt to login to the site automatically using | |||||
``config.wiki["username"]`` and ``config.wiki["password"]`` if both are | |||||
defined. | |||||
Specifying a project without a lang or a lang without a project will | |||||
raise :py:exc:`TypeError`. If all three args are specified, *name* will | |||||
be first tried, then *project* and *lang* if *name* doesn't work. If a | |||||
site cannot be found in the sitesdb, | |||||
:py:exc:`~earwigbot.exceptions.SiteNotFoundError` will be raised. An | |||||
empty sitesdb will be created if none is found. | |||||
""" | |||||
# Someone specified a project without a lang, or vice versa: | |||||
if (project and not lang) or (not project and lang): | |||||
e = "Keyword arguments 'lang' and 'project' must be specified together." | |||||
raise TypeError(e) | |||||
# No args given, so return our default site: | |||||
if not name and not project and not lang: | |||||
try: | |||||
default = self.config.wiki["defaultSite"] | |||||
except KeyError: | |||||
e = "Default site is not specified in config." | |||||
raise SiteNotFoundError(e) | |||||
return self._get_site_object(default) | |||||
# Name arg given, but don't look at others unless `name` isn't found: | |||||
if name: | |||||
try: | |||||
return self._get_site_object(name) | |||||
except SiteNotFoundError: | |||||
if project and lang: | |||||
name = self._get_site_name_from_sitesdb(project, lang) | |||||
if name: | |||||
return self._get_site_object(name) | |||||
raise | |||||
# If we end up here, then project and lang are the only args given: | |||||
name = self._get_site_name_from_sitesdb(project, lang) | |||||
if name: | |||||
return self._get_site_object(name) | |||||
e = "Site '{0}:{1}' not found in the sitesdb.".format(project, lang) | |||||
raise SiteNotFoundError(e) | |||||
def add_site(self, project=None, lang=None, base_url=None, | |||||
script_path="/w", sql=None): | |||||
"""Add a site to the sitesdb so it can be retrieved with get_site(). | |||||
If only a project and a lang are given, we'll guess the *base_url* as | |||||
``"//{lang}.{project}.org"`` (which is protocol-relative, becoming | |||||
``"https"`` if *useHTTPS* is ``True`` in config otherwise ``"http"``). | |||||
If this is wrong, provide the correct *base_url* as an argument (in | |||||
which case project and lang are ignored). Most wikis use ``"/w"`` as | |||||
the script path (meaning the API is located at | |||||
``"{base_url}{script_path}/api.php"`` -> | |||||
``"//{lang}.{project}.org/w/api.php"``), so this is the default. If | |||||
your wiki is different, provide the script_path as an argument. The | |||||
only other argument to :py:class:`~earwigbot.wiki.site.Site` that we | |||||
can't get from config files or by querying the wiki itself is SQL | |||||
connection info, so provide a dict of kwargs as *sql* and Site will | |||||
pass it to :py:func:`oursql.connect(**sql) <oursql.connect>`, allowing | |||||
you to make queries with :py:meth:`site.sql_query | |||||
<earwigbot.wiki.site.Site.sql_query>`. | |||||
Returns ``True`` if the site was added successfully or ``False`` if the | |||||
site is already in our sitesdb (this can be done purposefully to update | |||||
old site info). Raises :py:exc:`~earwigbot.exception.SiteNotFoundError` | |||||
if not enough information has been provided to identify the site (e.g. | |||||
a *project* but not a *lang*). | |||||
""" | |||||
if not base_url: | |||||
if not project or not lang: | |||||
e = "Without a base_url, both a project and a lang must be given." | |||||
raise SiteNotFoundError(e) | |||||
base_url = "//{0}.{1}.org".format(lang, project) | |||||
cookiejar = self._get_cookiejar() | |||||
config = self.config | |||||
login = (config.wiki.get("username"), config.wiki.get("password")) | |||||
user_agent = config.wiki.get("userAgent") | |||||
use_https = config.wiki.get("useHTTPS", False) | |||||
assert_edit = config.wiki.get("assert") | |||||
maxlag = config.wiki.get("maxlag") | |||||
wait_between_queries = config.wiki.get("waitTime", 5) | |||||
search_config = config.wiki.get("search") | |||||
# Create a Site object to log in and load the other attributes: | |||||
site = Site(base_url=base_url, script_path=script_path, sql=sql, | |||||
login=login, cookiejar=cookiejar, user_agent=user_agent, | |||||
use_https=use_https, assert_edit=assert_edit, | |||||
maxlag=maxlag, wait_between_queries=wait_between_queries, | |||||
search_config=search_config) | |||||
self._add_site_to_sitesdb(site) | |||||
self._sites[site.name] = site | |||||
return site | |||||
def remove_site(self, name=None, project=None, lang=None): | |||||
"""Remove a site from the sitesdb. | |||||
Returns ``True`` if the site was removed successfully or ``False`` if | |||||
the site was not in our sitesdb originally. If all three args (*name*, | |||||
*project*, and *lang*) are given, we'll first try *name* and then try | |||||
the latter two if *name* wasn't found in the database. Raises | |||||
:py:exc:`TypeError` if a project was given but not a language, or vice | |||||
versa. Will create an empty sitesdb if none was found. | |||||
""" | |||||
# Someone specified a project without a lang, or vice versa: | |||||
if (project and not lang) or (not project and lang): | |||||
e = "Keyword arguments 'lang' and 'project' must be specified together." | |||||
raise TypeError(e) | |||||
if name: | |||||
was_removed = self._remove_site_from_sitesdb(name) | |||||
if not was_removed: | |||||
if project and lang: | |||||
name = self._get_site_name_from_sitesdb(project, lang) | |||||
if name: | |||||
return self._remove_site_from_sitesdb(name) | |||||
return was_removed | |||||
if project and lang: | |||||
name = self._get_site_name_from_sitesdb(project, lang) | |||||
if name: | |||||
return self._remove_site_from_sitesdb(name) | |||||
return False |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -22,31 +22,42 @@ | |||||
from time import gmtime, strptime | from time import gmtime, strptime | ||||
from earwigbot.wiki.constants import * | |||||
from earwigbot.wiki.exceptions import UserNotFoundError | |||||
from earwigbot.exceptions import UserNotFoundError | |||||
from earwigbot.wiki import constants | |||||
from earwigbot.wiki.page import Page | from earwigbot.wiki.page import Page | ||||
__all__ = ["User"] | |||||
class User(object): | class User(object): | ||||
""" | """ | ||||
EarwigBot's Wiki Toolset: User Class | |||||
Represents a User on a given Site. Has methods for getting a bunch of | |||||
information about the user, such as editcount and user rights, methods for | |||||
returning the user's userpage and talkpage, etc. | |||||
Public methods: | |||||
name -- returns the user's username | |||||
exists -- returns True if the user exists, False if they do not | |||||
userid -- returns an integer ID representing the user | |||||
blockinfo -- returns information about a current block on the user | |||||
groups -- returns a list of the user's groups | |||||
rights -- returns a list of the user's rights | |||||
editcount -- returns the number of edits made by the user | |||||
registration -- returns the time the user registered as a time.struct_time | |||||
emailable -- returns True if you can email the user, False if you cannot | |||||
gender -- returns the user's gender ("male", "female", or "unknown") | |||||
get_userpage -- returns a Page object representing the user's userpage | |||||
get_talkpage -- returns a Page object representing the user's talkpage | |||||
**EarwigBot: Wiki Toolset: User** | |||||
Represents a user on a given :py:class:`~earwigbot.wiki.site.Site`. Has | |||||
methods for getting a bunch of information about the user, such as | |||||
editcount and user rights, methods for returning the user's userpage and | |||||
talkpage, etc. | |||||
*Attributes:* | |||||
- :py:attr:`site`: the user's corresponding Site object | |||||
- :py:attr:`name`: the user's username | |||||
- :py:attr:`exists`: ``True`` if the user exists, else ``False`` | |||||
- :py:attr:`userid`: an integer ID representing the user | |||||
- :py:attr:`blockinfo`: information about any current blocks on the user | |||||
- :py:attr:`groups`: a list of the user's groups | |||||
- :py:attr:`rights`: a list of the user's rights | |||||
- :py:attr:`editcount`: the number of edits made by the user | |||||
- :py:attr:`registration`: the time the user registered | |||||
- :py:attr:`emailable`: ``True`` if you can email the user, or ``False`` | |||||
- :py:attr:`gender`: the user's gender ("male"/"female"/"unknown") | |||||
*Public methods:* | |||||
- :py:meth:`reload`: forcibly reloads the user's attributes | |||||
- :py:meth:`get_userpage`: returns a Page object representing the user's | |||||
userpage | |||||
- :py:meth:`get_talkpage`: returns a Page object representing the user's | |||||
talkpage | |||||
""" | """ | ||||
def __init__(self, site, name): | def __init__(self, site, name): | ||||
@@ -66,27 +77,26 @@ class User(object): | |||||
self._name = name | self._name = name | ||||
def __repr__(self): | def __repr__(self): | ||||
"""Returns the canonical string representation of the User.""" | |||||
"""Return the canonical string representation of the User.""" | |||||
return "User(name={0!r}, site={1!r})".format(self._name, self._site) | return "User(name={0!r}, site={1!r})".format(self._name, self._site) | ||||
def __str__(self): | def __str__(self): | ||||
"""Returns a nice string representation of the User.""" | |||||
return '<User "{0}" of {1}>'.format(self.name(), str(self._site)) | |||||
"""Return a nice string representation of the User.""" | |||||
return '<User "{0}" of {1}>'.format(self.name, str(self.site)) | |||||
def _get_attribute(self, attr, force): | |||||
def _get_attribute(self, attr): | |||||
"""Internally used to get an attribute by name. | """Internally used to get an attribute by name. | ||||
We'll call _load_attributes() to get this (and all other attributes) | We'll call _load_attributes() to get this (and all other attributes) | ||||
from the API if it is not already defined. If `force` is True, we'll | |||||
re-load them even if they've already been loaded. | |||||
from the API if it is not already defined. | |||||
Raises UserNotFoundError if a nonexistant user prevents us from | Raises UserNotFoundError if a nonexistant user prevents us from | ||||
returning a certain attribute. | returning a certain attribute. | ||||
""" | """ | ||||
if not hasattr(self, attr) or force: | |||||
if not hasattr(self, attr): | |||||
self._load_attributes() | self._load_attributes() | ||||
if self._exists is False: | |||||
e = "User '{0}' does not exist.".format(self._name) | |||||
if not self._exists: | |||||
e = u"User '{0}' does not exist.".format(self._name) | |||||
raise UserNotFoundError(e) | raise UserNotFoundError(e) | ||||
return getattr(self, attr) | return getattr(self, attr) | ||||
@@ -96,9 +106,9 @@ class User(object): | |||||
Normally, this is called by _get_attribute() when a requested attribute | Normally, this is called by _get_attribute() when a requested attribute | ||||
is not defined. This defines it. | is not defined. This defines it. | ||||
""" | """ | ||||
params = {"action": "query", "list": "users", "ususers": self._name, | |||||
"usprop": "blockinfo|groups|rights|editcount|registration|emailable|gender"} | |||||
result = self._site._api_query(params) | |||||
props = "blockinfo|groups|rights|editcount|registration|emailable|gender" | |||||
result = self.site.api_query(action="query", list="users", | |||||
ususers=self._name, usprop=props) | |||||
res = result["query"]["users"][0] | res = result["query"]["users"][0] | ||||
# normalize our username in case it was entered oddly | # normalize our username in case it was entered oddly | ||||
@@ -145,118 +155,136 @@ class User(object): | |||||
self._gender = res["gender"] | self._gender = res["gender"] | ||||
def name(self, force=False): | |||||
"""Returns the user's name. | |||||
@property | |||||
def site(self): | |||||
"""The user's corresponding Site object.""" | |||||
return self._site | |||||
If `force` is True, we will load the name from the API and return that. | |||||
This could potentially return a "normalized" version of the name - for | |||||
example, without a "User:" prefix or without underscores. Unlike other | |||||
attribute getters, this will never make an API query without `force`. | |||||
@property | |||||
def name(self): | |||||
"""The user's username. | |||||
Note that if another attribute getter, like exists(), has already been | |||||
called, then the username has already been normalized. | |||||
This will never make an API query on its own, but if one has already | |||||
been made by the time this is retrieved, the username may have been | |||||
"normalized" from the original input to the constructor, converted into | |||||
a Unicode object, with underscores removed, etc. | |||||
""" | """ | ||||
if force: | |||||
self._load_attributes() | |||||
return self._name | return self._name | ||||
def exists(self, force=False): | |||||
"""Returns True if the user exists, or False if they do not. | |||||
@property | |||||
def exists(self): | |||||
"""``True`` if the user exists, or ``False`` if they do not. | |||||
Makes an API query if `force` is True or if we haven't made one | |||||
already. | |||||
Makes an API query only if we haven't made one already. | |||||
""" | """ | ||||
if not hasattr(self, "_exists") or force: | |||||
if not hasattr(self, "_exists"): | |||||
self._load_attributes() | self._load_attributes() | ||||
return self._exists | return self._exists | ||||
def userid(self, force=False): | |||||
"""Returns an integer ID used by MediaWiki to represent the user. | |||||
@property | |||||
def userid(self): | |||||
"""An integer ID used by MediaWiki to represent the user. | |||||
Raises UserNotFoundError if the user does not exist. Makes an API query | |||||
if `force` is True or if we haven't made one already. | |||||
Raises :py:exc:`~earwigbot.exceptions.UserNotFoundError` if the user | |||||
does not exist. Makes an API query only if we haven't made one already. | |||||
""" | """ | ||||
return self._get_attribute("_userid", force) | |||||
return self._get_attribute("_userid") | |||||
def blockinfo(self, force=False): | |||||
"""Returns information about a current block on the user. | |||||
@property | |||||
def blockinfo(self): | |||||
"""Information about any current blocks on the user. | |||||
If the user is not blocked, returns False. If they are, returns a dict | |||||
with three keys: "by" is the blocker's username, "reason" is the reason | |||||
why they were blocked, and "expiry" is when the block expires. | |||||
If the user is not blocked, returns ``False``. If they are, returns a | |||||
dict with three keys: ``"by"`` is the blocker's username, ``"reason"`` | |||||
is the reason why they were blocked, and ``"expiry"`` is when the block | |||||
expires. | |||||
Raises UserNotFoundError if the user does not exist. Makes an API query | |||||
if `force` is True or if we haven't made one already. | |||||
Raises :py:exc:`~earwigbot.exceptions.UserNotFoundError` if the user | |||||
does not exist. Makes an API query only if we haven't made one already. | |||||
""" | """ | ||||
return self._get_attribute("_blockinfo", force) | |||||
return self._get_attribute("_blockinfo") | |||||
def groups(self, force=False): | |||||
"""Returns a list of groups this user is in, including "*". | |||||
@property | |||||
def groups(self): | |||||
"""A list of groups this user is in, including ``"*"``. | |||||
Raises UserNotFoundError if the user does not exist. Makes an API query | |||||
if `force` is True or if we haven't made one already. | |||||
Raises :py:exc:`~earwigbot.exceptions.UserNotFoundError` if the user | |||||
does not exist. Makes an API query only if we haven't made one already. | |||||
""" | """ | ||||
return self._get_attribute("_groups", force) | |||||
return self._get_attribute("_groups") | |||||
def rights(self, force=False): | |||||
"""Returns a list of this user's rights. | |||||
@property | |||||
def rights(self): | |||||
"""A list of this user's rights. | |||||
Raises UserNotFoundError if the user does not exist. Makes an API query | |||||
if `force` is True or if we haven't made one already. | |||||
Raises :py:exc:`~earwigbot.exceptions.UserNotFoundError` if the user | |||||
does not exist. Makes an API query only if we haven't made one already. | |||||
""" | """ | ||||
return self._get_attribute("_rights", force) | |||||
return self._get_attribute("_rights") | |||||
def editcount(self, force=False): | |||||
@property | |||||
def editcount(self): | |||||
"""Returns the number of edits made by the user. | """Returns the number of edits made by the user. | ||||
Raises UserNotFoundError if the user does not exist. Makes an API query | |||||
if `force` is True or if we haven't made one already. | |||||
Raises :py:exc:`~earwigbot.exceptions.UserNotFoundError` if the user | |||||
does not exist. Makes an API query only if we haven't made one already. | |||||
""" | """ | ||||
return self._get_attribute("_editcount", force) | |||||
return self._get_attribute("_editcount") | |||||
def registration(self, force=False): | |||||
"""Returns the time the user registered as a time.struct_time object. | |||||
@property | |||||
def registration(self): | |||||
"""The time the user registered as a :py:class:`time.struct_time`. | |||||
Raises UserNotFoundError if the user does not exist. Makes an API query | |||||
if `force` is True or if we haven't made one already. | |||||
Raises :py:exc:`~earwigbot.exceptions.UserNotFoundError` if the user | |||||
does not exist. Makes an API query only if we haven't made one already. | |||||
""" | """ | ||||
return self._get_attribute("_registration", force) | |||||
return self._get_attribute("_registration") | |||||
def emailable(self, force=False): | |||||
"""Returns True if the user can be emailed, or False if they cannot. | |||||
@property | |||||
def emailable(self): | |||||
"""``True`` if the user can be emailed, or ``False`` if they cannot. | |||||
Raises UserNotFoundError if the user does not exist. Makes an API query | |||||
if `force` is True or if we haven't made one already. | |||||
Raises :py:exc:`~earwigbot.exceptions.UserNotFoundError` if the user | |||||
does not exist. Makes an API query only if we haven't made one already. | |||||
""" | """ | ||||
return self._get_attribute("_emailable", force) | |||||
return self._get_attribute("_emailable") | |||||
def gender(self, force=False): | |||||
"""Returns the user's gender. | |||||
@property | |||||
def gender(self): | |||||
"""The user's gender. | |||||
Can return either "male", "female", or "unknown", if they did not | |||||
specify it. | |||||
Can return either ``"male"``, ``"female"``, or ``"unknown"``, if they | |||||
did not specify it. | |||||
Raises UserNotFoundError if the user does not exist. Makes an API query | |||||
if `force` is True or if we haven't made one already. | |||||
Raises :py:exc:`~earwigbot.exceptions.UserNotFoundError` if the user | |||||
does not exist. Makes an API query only if we haven't made one already. | |||||
""" | """ | ||||
return self._get_attribute("_gender", force) | |||||
return self._get_attribute("_gender") | |||||
def reload(self): | |||||
"""Forcibly reload the user's attributes. | |||||
Emphasis on *reload*: this is only necessary if there is reason to | |||||
believe they have changed. | |||||
""" | |||||
self._load_attributes() | |||||
def get_userpage(self): | def get_userpage(self): | ||||
"""Returns a Page object representing the user's userpage. | |||||
"""Return a Page object representing the user's userpage. | |||||
No checks are made to see if it exists or not. Proper site namespace | No checks are made to see if it exists or not. Proper site namespace | ||||
conventions are followed. | conventions are followed. | ||||
""" | """ | ||||
prefix = self._site.namespace_id_to_name(NS_USER) | |||||
prefix = self.site.namespace_id_to_name(constants.NS_USER) | |||||
pagename = ':'.join((prefix, self._name)) | pagename = ':'.join((prefix, self._name)) | ||||
return Page(self._site, pagename) | |||||
return Page(self.site, pagename) | |||||
def get_talkpage(self): | def get_talkpage(self): | ||||
"""Returns a Page object representing the user's talkpage. | |||||
"""Return a Page object representing the user's talkpage. | |||||
No checks are made to see if it exists or not. Proper site namespace | No checks are made to see if it exists or not. Proper site namespace | ||||
conventions are followed. | conventions are followed. | ||||
""" | """ | ||||
prefix = self._site.namespace_id_to_name(NS_USER_TALK) | |||||
prefix = self.site.namespace_id_to_name(constants.NS_USER_TALK) | |||||
pagename = ':'.join((prefix, self._name)) | pagename = ':'.join((prefix, self._name)) | ||||
return Page(self._site, pagename) | |||||
return Page(self.site, pagename) |
@@ -0,0 +1,65 @@ | |||||
#! /usr/bin/env python | |||||
# -*- coding: utf-8 -*- | |||||
# | |||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | |||||
# of this software and associated documentation files (the "Software"), to deal | |||||
# in the Software without restriction, including without limitation the rights | |||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | |||||
# all copies or substantial portions of the Software. | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |||||
# SOFTWARE. | |||||
from setuptools import setup, find_packages | |||||
from earwigbot import __version__ | |||||
with open("README.rst") as fp: | |||||
long_docs = fp.read() | |||||
setup( | |||||
name = "earwigbot", | |||||
packages = find_packages(exclude=("tests",)), | |||||
entry_points = {"console_scripts": ["earwigbot = earwigbot.util:main"]}, | |||||
install_requires = ["GitPython >= 0.3.2.RC1", # Interfacing with git | |||||
"PyYAML >= 3.10", # Config parsing | |||||
"mwparserfromhell >= 0.1", # Wikicode parsing | |||||
"oursql >= 0.9.3", # Talking with MediaWiki databases | |||||
"oauth2 >= 1.5.211", # Talking with Yahoo BOSS Search | |||||
"py-bcrypt >= 0.2", # Password hashing in config | |||||
"pycrypto >= 2.5", # Storing bot passwords and keys | |||||
"pytz >= 2012c", # Timezone handling | |||||
], | |||||
test_suite = "tests", | |||||
version = __version__, | |||||
author = "Ben Kurtovic", | |||||
author_email = "ben.kurtovic@verizon.net", | |||||
url = "https://github.com/earwig/earwigbot", | |||||
description = "EarwigBot is a Python robot that edits Wikipedia and interacts with people over IRC.", | |||||
long_description = long_docs, | |||||
download_url = "https://github.com/earwig/earwigbot/tarball/v{0}".format(__version__), | |||||
keywords = "earwig earwigbot irc wikipedia wiki mediawiki", | |||||
license = "MIT License", | |||||
classifiers = [ | |||||
"Development Status :: 3 - Alpha", | |||||
"Environment :: Console", | |||||
"Intended Audience :: Developers", | |||||
"License :: OSI Approved :: MIT License", | |||||
"Natural Language :: English", | |||||
"Operating System :: OS Independent", | |||||
"Programming Language :: Python :: 2.7", | |||||
"Topic :: Communications :: Chat :: Internet Relay Chat", | |||||
"Topic :: Internet :: WWW/HTTP" | |||||
], | |||||
) |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -23,29 +23,44 @@ | |||||
""" | """ | ||||
EarwigBot's Unit Tests | EarwigBot's Unit Tests | ||||
This module __init__ file provides some support code for unit tests. | |||||
This __init__ file provides some support code for unit tests. | |||||
Test cases: | |||||
-- CommandTestCase provides setUp() for creating a fake connection, plus | |||||
some other helpful methods for testing IRC commands. | |||||
Fake objects: | |||||
-- FakeBot implements Bot, using the Fake* equivalents of all objects | |||||
whenever possible. | |||||
-- FakeBotConfig implements BotConfig with silent logging. | |||||
-- FakeIRCConnection implements IRCConnection, using an internal string | |||||
buffer for data instead of sending it over a socket. | |||||
CommandTestCase is a subclass of unittest.TestCase that provides setUp() for | |||||
creating a fake connection and some other helpful methods. It uses | |||||
FakeConnection, a subclass of classes.Connection, but with an internal string | |||||
instead of a socket for data. | |||||
""" | """ | ||||
import logging | |||||
from os import path | |||||
import re | import re | ||||
from threading import Lock | |||||
from unittest import TestCase | from unittest import TestCase | ||||
from earwigbot.classes import Connection, Data | |||||
from earwigbot.bot import Bot | |||||
from earwigbot.commands import CommandManager | |||||
from earwigbot.config import BotConfig | |||||
from earwigbot.irc import IRCConnection, Data | |||||
from earwigbot.tasks import TaskManager | |||||
from earwigbot.wiki import SitesDB | |||||
class CommandTestCase(TestCase): | class CommandTestCase(TestCase): | ||||
re_sender = re.compile(":(.*?)!(.*?)@(.*?)\Z") | re_sender = re.compile(":(.*?)!(.*?)@(.*?)\Z") | ||||
def setUp(self, command): | def setUp(self, command): | ||||
self.connection = FakeConnection() | |||||
self.connection.connect() | |||||
self.command = command(self.connection) | |||||
self.bot = FakeBot(path.dirname(__file__)) | |||||
self.command = command(self.bot) | |||||
self.command.connection = self.connection = self.bot.frontend | |||||
def get_single(self): | def get_single(self): | ||||
data = self.connection.get().split("\n") | |||||
data = self.connection._get().split("\n") | |||||
line = data.pop(0) | line = data.pop(0) | ||||
for remaining in data[1:]: | for remaining in data[1:]: | ||||
self.connection.send(remaining) | self.connection.send(remaining) | ||||
@@ -92,16 +107,42 @@ class CommandTestCase(TestCase): | |||||
line = ":Foo!bar@example.com JOIN :#channel".strip().split() | line = ":Foo!bar@example.com JOIN :#channel".strip().split() | ||||
return self.maker(line, line[2][1:]) | return self.maker(line, line[2][1:]) | ||||
class FakeConnection(Connection): | |||||
def connect(self): | |||||
class FakeBot(Bot): | |||||
def __init__(self, root_dir): | |||||
self.config = FakeBotConfig(root_dir) | |||||
self.logger = logging.getLogger("earwigbot") | |||||
self.commands = CommandManager(self) | |||||
self.tasks = TaskManager(self) | |||||
self.wiki = SitesDB(self) | |||||
self.frontend = FakeIRCConnection(self) | |||||
self.watcher = FakeIRCConnection(self) | |||||
self.component_lock = Lock() | |||||
self._keep_looping = True | |||||
class FakeBotConfig(BotConfig): | |||||
def _setup_logging(self): | |||||
logger = logging.getLogger("earwigbot") | |||||
logger.addHandler(logging.NullHandler()) | |||||
class FakeIRCConnection(IRCConnection): | |||||
def __init__(self, bot): | |||||
self.bot = bot | |||||
self._is_running = False | |||||
self._connect() | |||||
def _connect(self): | |||||
self._buffer = "" | self._buffer = "" | ||||
def close(self): | |||||
pass | |||||
def _close(self): | |||||
self._buffer = "" | |||||
def get(self, size=4096): | |||||
def _get(self, size=4096): | |||||
data, self._buffer = self._buffer, "" | data, self._buffer = self._buffer, "" | ||||
return data | return data | ||||
def send(self, msg): | |||||
def _send(self, msg): | |||||
self._buffer += msg + "\n" | self._buffer += msg + "\n" |
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -23,7 +23,7 @@ | |||||
import unittest | import unittest | ||||
from earwigbot.commands.calc import Command | from earwigbot.commands.calc import Command | ||||
from earwigbot.tests import CommandTestCase | |||||
from tests import CommandTestCase | |||||
class TestCalc(CommandTestCase): | class TestCalc(CommandTestCase): | ||||
@@ -1,17 +1,17 @@ | |||||
# -*- coding: utf-8 -*- | # -*- coding: utf-8 -*- | ||||
# | # | ||||
# Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | # Copyright (C) 2009-2012 by Ben Kurtovic <ben.kurtovic@verizon.net> | ||||
# | |||||
# | |||||
# Permission is hereby granted, free of charge, to any person obtaining a copy | # Permission is hereby granted, free of charge, to any person obtaining a copy | ||||
# of this software and associated documentation files (the "Software"), to deal | # of this software and associated documentation files (the "Software"), to deal | ||||
# in the Software without restriction, including without limitation the rights | # in the Software without restriction, including without limitation the rights | ||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# copies of the Software, and to permit persons to whom the Software is | |||||
# furnished to do so, subject to the following conditions: | # furnished to do so, subject to the following conditions: | ||||
# | |||||
# | |||||
# The above copyright notice and this permission notice shall be included in | # The above copyright notice and this permission notice shall be included in | ||||
# all copies or substantial portions of the Software. | # all copies or substantial portions of the Software. | ||||
# | |||||
# | |||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||||
@@ -23,7 +23,7 @@ | |||||
import unittest | import unittest | ||||
from earwigbot.commands.test import Command | from earwigbot.commands.test import Command | ||||
from earwigbot.tests import CommandTestCase | |||||
from tests import CommandTestCase | |||||
class TestTest(CommandTestCase): | class TestTest(CommandTestCase): | ||||
@@ -38,12 +38,12 @@ class TestTest(CommandTestCase): | |||||
self.assertTrue(self.command.check(self.make_msg("TEST", "foo"))) | self.assertTrue(self.command.check(self.make_msg("TEST", "foo"))) | ||||
def test_process(self): | def test_process(self): | ||||
def _test(): | |||||
def test(): | |||||
self.command.process(self.make_msg("test")) | self.command.process(self.make_msg("test")) | ||||
self.assertSaidIn(["Hey \x02Foo\x0F!", "'sup \x02Foo\x0F?"]) | self.assertSaidIn(["Hey \x02Foo\x0F!", "'sup \x02Foo\x0F?"]) | ||||
for i in xrange(64): | for i in xrange(64): | ||||
_test() | |||||
test() | |||||
if __name__ == "__main__": | if __name__ == "__main__": | ||||
unittest.main(verbosity=2) | unittest.main(verbosity=2) |