ben
/
mwparserfromhell
mirror of https://github.com/earwig/mwparserfromhell
1
1
Fork 0
Code Releases 24 Activity
Browse Source

Clean up/revise docs.

tags/v0.4
Ben Kurtovic 9 years ago
parent
bbd4cd97cf
commit
2b1f7e5511
6 changed files with 91 additions and 94 deletions
Unified View
Diff Options
Show Stats Download Patch File Download Diff File
  1. +0
    -1
      docs/api/mwparserfromhell.nodes.rst
  2. +6
    -5
      docs/api/mwparserfromhell.rst
  3. +61
    -64
      docs/changelog.rst
  4. +3
    -3
      docs/index.rst
  5. +5
    -5
      docs/integration.rst
  6. +16
    -16
      docs/usage.rst

+ 0
- 1
docs/api/mwparserfromhell.nodes.rst View File

@@ -87,4 +87,3 @@ Subpackages
.. toctree:: .. toctree::


mwparserfromhell.nodes.extras mwparserfromhell.nodes.extras


+ 6
- 5
docs/api/mwparserfromhell.rst View File

@@ -15,6 +15,12 @@ mwparserfromhell Package
:members: :members:
:undoc-members: :undoc-members:


:mod:`definitions` Module
-------------------------

.. automodule:: mwparserfromhell.definitions
:members:

:mod:`smart_list` Module :mod:`smart_list` Module
------------------------ ------------------------


@@ -30,12 +36,6 @@ mwparserfromhell Package
:members: :members:
:undoc-members: :undoc-members:


:mod:`definitions` Module

.. automodule:: mwparserfromhell.definitions
:members:

:mod:`utils` Module :mod:`utils` Module
------------------- -------------------




+ 61
- 64
docs/changelog.rst View File

@@ -11,28 +11,28 @@ Unreleased
prevented Windows users from using the C tokenizer. prevented Windows users from using the C tokenizer.
- Added a script to test for memory leaks in :file:`scripts/memtest.py`. - Added a script to test for memory leaks in :file:`scripts/memtest.py`.
- Added a script to do releases in :file:`scripts/release.sh`. - Added a script to do releases in :file:`scripts/release.sh`.
- *skip_style_tags* can now be passed to :py:func:`mwparserfromhell.parse()
<.parse_anything>` (previously, only :py:meth:`.Parser.parse` allowed it).
- The *recursive* argument to :py:class:`Wikicode's <.Wikicode>`
:py:meth:`.filter` methods now accepts a third option, ``RECURSE_OTHERS``,
which recurses over all children except instances of *forcetype* (for
example, ``code.filter_templates(code.RECURSE_OTHERS)`` returns all un-nested
- *skip_style_tags* can now be passed to :func:`mwparserfromhell.parse()
<.parse_anything>` (previously, only :meth:`.Parser.parse` allowed it).
- The *recursive* argument to :class:`Wikicode's <.Wikicode>` :meth:`.filter`
methods now accepts a third option, ``RECURSE_OTHERS``, which recurses over
all children except instances of *forcetype* (for example,
``code.filter_templates(code.RECURSE_OTHERS)`` returns all un-nested
templates). templates).
- The parser now understands HTML tag attributes quoted with single quotes. - The parser now understands HTML tag attributes quoted with single quotes.
When setting a tag attribute's value, quotes will be added if necessary. As When setting a tag attribute's value, quotes will be added if necessary. As
part of this, :py:class:`.Attribute`\ 's :py:attr:`~.Attribute.quoted`
attribute has been changed to :py:attr:`~.Attribute.quotes`, and is now
either a string or ``None``.
- Calling :py:meth:`.Template.remove` with a :py:class:`.Parameter` object that
is not part of the template now raises :py:exc:`ValueError` instead of doing
part of this, :class:`.Attribute`\ 's :attr:`~.Attribute.quoted` attribute
has been changed to :attr:`~.Attribute.quotes`, and is now either a string or
``None``.
- Calling :meth:`.Template.remove` with a :class:`.Parameter` object that is
not part of the template now raises :exc:`ValueError` instead of doing
nothing. nothing.
- :py:class:`.Parameter`\ s with non-integer keys can no longer be created with
- :class:`.Parameter`\ s with non-integer keys can no longer be created with
*showkey=False*, nor have the value of this attribute be set to *False* *showkey=False*, nor have the value of this attribute be set to *False*
later. later.
- :py:meth:`._ListProxy.destroy` has been changed to
:py:meth:`._ListProxy.detach`, and now works in a more useful way.
- If something goes wrong while parsing, :py:exc:`.ParserError` will now be
raised. Previously, the parser would produce an unclear :py:exc:`.BadRoute`
- :meth:`._ListProxy.destroy` has been changed to :meth:`._ListProxy.detach`,
and now works in a more useful way.
- If something goes wrong while parsing, :exc:`.ParserError` will now be
raised. Previously, the parser would produce an unclear :exc:`.BadRoute`
exception or allow an incorrect node tree to be build. exception or allow an incorrect node tree to be build.
- Fixed a parser bug involving nested tags, and another involving comments in - Fixed a parser bug involving nested tags, and another involving comments in
template names. template names.
@@ -46,22 +46,21 @@ v0.3.3
(`changes <https://github.com/earwig/mwparserfromhell/compare/v0.3.2...v0.3.3>`__): (`changes <https://github.com/earwig/mwparserfromhell/compare/v0.3.2...v0.3.3>`__):


- Added support for Python 2.6 and 3.4. - Added support for Python 2.6 and 3.4.
- :py:meth:`.Template.has` is now passed *ignore_empty=False* by default
- :meth:`.Template.has` is now passed *ignore_empty=False* by default
instead of *True*. This fixes a bug when adding parameters to templates with instead of *True*. This fixes a bug when adding parameters to templates with
empty fields, **and is a breaking change if you rely on the default empty fields, **and is a breaking change if you rely on the default
behavior.** behavior.**
- The *matches* argument of :py:class:`Wikicode's <.Wikicode>`
:py:meth:`.filter` methods now accepts a function (taking one argument, a
:py:class:`.Node`, and returning a bool) in addition to a regex.
- Re-added *flat* argument to :py:meth:`.Wikicode.get_sections`, fixed the
order in which it returns sections, and made it faster.
- :py:meth:`.Wikicode.matches` now accepts a tuple or list of
strings/:py:class:`.Wikicode` objects instead of just a single string or
:py:class:`.Wikicode`.
- The *matches* argument of :class:`Wikicode's <.Wikicode>` :meth:`.filter`
methods now accepts a function (taking one argument, a :class:`.Node`, and
returning a bool) in addition to a regex.
- Re-added *flat* argument to :meth:`.Wikicode.get_sections`, fixed the order
in which it returns sections, and made it faster.
- :meth:`.Wikicode.matches` now accepts a tuple or list of
strings/:class:`.Wikicode` objects instead of just a single string or
:class:`.Wikicode`.
- Given the frequency of issues with the (admittedly insufficient) tag parser, - Given the frequency of issues with the (admittedly insufficient) tag parser,
there's a temporary *skip_style_tags* argument to
:py:meth:`~.Parser.parse` that ignores ``''`` and ``'''`` until these issues
are corrected.
there's a temporary *skip_style_tags* argument to :meth:`~.Parser.parse` that
ignores ``''`` and ``'''`` until these issues are corrected.
- Fixed a parser bug involving nested wikilinks and external links. - Fixed a parser bug involving nested wikilinks and external links.
- C code cleanup and speed improvements. - C code cleanup and speed improvements.


@@ -72,9 +71,9 @@ v0.3.2
(`changes <https://github.com/earwig/mwparserfromhell/compare/v0.3.1...v0.3.2>`__): (`changes <https://github.com/earwig/mwparserfromhell/compare/v0.3.1...v0.3.2>`__):


- Added support for Python 3.2 (along with current support for 3.3 and 2.7). - Added support for Python 3.2 (along with current support for 3.3 and 2.7).
- Renamed :py:meth:`.Template.remove`\ 's first argument from *name* to
*param*, which now accepts :py:class:`.Parameter` objects in addition to
parameter name strings.
- Renamed :meth:`.Template.remove`\ 's first argument from *name* to *param*,
which now accepts :class:`.Parameter` objects in addition to parameter name
strings.


v0.3.1 v0.3.1
------ ------
@@ -91,24 +90,24 @@ v0.3
`Released August 24, 2013 <https://github.com/earwig/mwparserfromhell/tree/v0.3>`_ `Released August 24, 2013 <https://github.com/earwig/mwparserfromhell/tree/v0.3>`_
(`changes <https://github.com/earwig/mwparserfromhell/compare/v0.2...v0.3>`__): (`changes <https://github.com/earwig/mwparserfromhell/compare/v0.2...v0.3>`__):


- Added complete support for HTML :py:class:`Tags <.Tag>`, including forms like
- Added complete support for HTML :class:`Tags <.Tag>`, including forms like
``<ref>foo</ref>``, ``<ref name="bar"/>``, and wiki-markup tags like bold ``<ref>foo</ref>``, ``<ref name="bar"/>``, and wiki-markup tags like bold
(``'''``), italics (``''``), and lists (``*``, ``#``, ``;`` and ``:``). (``'''``), italics (``''``), and lists (``*``, ``#``, ``;`` and ``:``).
- Added support for :py:class:`.ExternalLink`\ s (``http://example.com/`` and
- Added support for :class:`.ExternalLink`\ s (``http://example.com/`` and
``[http://example.com/ Example]``). ``[http://example.com/ Example]``).
- :py:class:`Wikicode's <.Wikicode>` :py:meth:`.filter` methods are now passed
- :class:`Wikicode's <.Wikicode>` :meth:`.filter` methods are now passed
*recursive=True* by default instead of *False*. **This is a breaking change *recursive=True* by default instead of *False*. **This is a breaking change
if you rely on any filter() methods being non-recursive by default.** if you rely on any filter() methods being non-recursive by default.**
- Added a :py:meth:`.matches` method to :py:class:`~.Wikicode` for
page/template name comparisons.
- The *obj* param of :py:meth:`Wikicode.insert_before() <.insert_before>`,
:py:meth:`~.insert_after`, :py:meth:`~.Wikicode.replace`, and
:py:meth:`~.Wikicode.remove` now accepts :py:class:`~.Wikicode` objects and
strings representing parts of wikitext, instead of just nodes. These methods
also make all possible substitutions instead of just one.
- Renamed :py:meth:`Template.has_param() <.has_param>` to
:py:meth:`~.Template.has` for consistency with :py:class:`~.Template`\ 's
other methods; :py:meth:`~.has_param` is now an alias.
- Added a :meth:`.matches` method to :class:`.Wikicode` for page/template name
comparisons.
- The *obj* param of :meth:`.Wikicode.insert_before`, :meth:`.insert_after`,
:meth:`~.Wikicode.replace`, and :meth:`~.Wikicode.remove` now accepts
:class:`.Wikicode` objects and strings representing parts of wikitext,
instead of just nodes. These methods also make all possible substitutions
instead of just one.
- Renamed :meth:`.Template.has_param` to :meth:`~.Template.has` for consistency
with :class:`.Template`\ 's other methods; :meth:`.has_param` is now an
alias.
- The C tokenizer extension now works on Python 3 in addition to Python 2.7. - The C tokenizer extension now works on Python 3 in addition to Python 2.7.
- Various bugfixes, internal changes, and cleanup. - Various bugfixes, internal changes, and cleanup.


@@ -121,29 +120,27 @@ v0.2
- The parser now fully supports Python 3 in addition to Python 2.7. - The parser now fully supports Python 3 in addition to Python 2.7.
- Added a C tokenizer extension that is significantly faster than its Python - Added a C tokenizer extension that is significantly faster than its Python
equivalent. It is enabled by default (if available) and can be toggled by equivalent. It is enabled by default (if available) and can be toggled by
setting :py:attr:`mwparserfromhell.parser.use_c` to a boolean value.
setting :attr:`mwparserfromhell.parser.use_c` to a boolean value.
- Added a complete set of unit tests covering parsing and wikicode - Added a complete set of unit tests covering parsing and wikicode
manipulation. manipulation.
- Renamed :py:meth:`.filter_links` to :py:meth:`.filter_wikilinks` (applies to
:py:meth:`.ifilter` as well).
- Added filter methods for :py:class:`Arguments <.Argument>`,
:py:class:`Comments <.Comment>`, :py:class:`Headings <.Heading>`, and
:py:class:`HTMLEntities <.HTMLEntity>`.
- Added *before* param to :py:meth:`Template.add() <.Template.add>`; renamed
*force_nonconformity* to *preserve_spacing*.
- Added *include_lead* param to :py:meth:`Wikicode.get_sections()
<.get_sections>`.
- Removed *flat* param from :py:meth:`.get_sections`.
- Removed *force_no_field* param from :py:meth:`Template.remove()
<.Template.remove>`.
- Renamed :meth:`.filter_links` to :meth:`.filter_wikilinks` (applies to
:meth:`.ifilter` as well).
- Added filter methods for :class:`Arguments <.Argument>`,
:class:`Comments <.Comment>`, :class:`Headings <.Heading>`, and
:class:`HTMLEntities <.HTMLEntity>`.
- Added *before* param to :meth:`.Template.add`; renamed *force_nonconformity*
to *preserve_spacing*.
- Added *include_lead* param to :meth:`.Wikicode.get_sections`.
- Removed *flat* param from :meth:`.get_sections`.
- Removed *force_no_field* param from :meth:`.Template.remove`.
- Added support for Travis CI. - Added support for Travis CI.
- Added note about Windows build issue in the README. - Added note about Windows build issue in the README.
- The tokenizer will limit itself to a realistic recursion depth to prevent - The tokenizer will limit itself to a realistic recursion depth to prevent
errors and unreasonably long parse times. errors and unreasonably long parse times.
- Fixed how some nodes' attribute setters handle input. - Fixed how some nodes' attribute setters handle input.
- Fixed multiple bugs in the tokenizer's handling of invalid markup. - Fixed multiple bugs in the tokenizer's handling of invalid markup.
- Fixed bugs in the implementation of :py:class:`.SmartList` and
:py:class:`.StringMixIn`.
- Fixed bugs in the implementation of :class:`.SmartList` and
:class:`.StringMixIn`.
- Fixed some broken example code in the README; other copyedits. - Fixed some broken example code in the README; other copyedits.
- Other bugfixes and code cleanup. - Other bugfixes and code cleanup.


@@ -153,12 +150,12 @@ v0.1.1
`Released September 21, 2012 <https://github.com/earwig/mwparserfromhell/tree/v0.1.1>`_ `Released September 21, 2012 <https://github.com/earwig/mwparserfromhell/tree/v0.1.1>`_
(`changes <https://github.com/earwig/mwparserfromhell/compare/v0.1...v0.1.1>`__): (`changes <https://github.com/earwig/mwparserfromhell/compare/v0.1...v0.1.1>`__):


- Added support for :py:class:`Comments <.Comment>` (``<!-- foo -->``) and
:py:class:`Wikilinks <.Wikilink>` (``[[foo]]``).
- Added corresponding :py:meth:`.ifilter_links` and :py:meth:`.filter_links`
methods to :py:class:`.Wikicode`.
- Added support for :class:`Comments <.Comment>` (``<!-- foo -->``) and
:class:`Wikilinks <.Wikilink>` (``[[foo]]``).
- Added corresponding :meth:`.ifilter_links` and :meth:`.filter_links` methods
to :class:`.Wikicode`.
- Fixed a bug when parsing incomplete templates. - Fixed a bug when parsing incomplete templates.
- Fixed :py:meth:`.strip_code` to affect the contents of headings.
- Fixed :meth:`.strip_code` to affect the contents of headings.
- Various copyedits in documentation and comments. - Various copyedits in documentation and comments.


v0.1 v0.1


+ 3
- 3
docs/index.rst View File

@@ -1,9 +1,9 @@
MWParserFromHell v\ |version| Documentation MWParserFromHell v\ |version| Documentation
=========================================== ===========================================


:py:mod:`mwparserfromhell` (the *MediaWiki Parser from Hell*) is a Python
package that provides an easy-to-use and outrageously powerful parser for
MediaWiki_ wikicode. It supports Python 2 and Python 3.
:mod:`mwparserfromhell` (the *MediaWiki Parser from Hell*) is a Python package
that provides an easy-to-use and outrageously powerful parser for MediaWiki_
wikicode. It supports Python 2 and Python 3.


Developed by Earwig_ with contributions from `Σ`_, Legoktm_, and others. Developed by Earwig_ with contributions from `Σ`_, Legoktm_, and others.
Development occurs on GitHub_. Development occurs on GitHub_.


+ 5
- 5
docs/integration.rst View File

@@ -1,11 +1,11 @@
Integration Integration
=========== ===========


:py:mod:`mwparserfromhell` is used by and originally developed for EarwigBot_;
:py:class:`~earwigbot.wiki.page.Page` objects have a
:py:meth:`~earwigbot.wiki.page.Page.parse` method that essentially calls
:py:func:`mwparserfromhell.parse() <mwparserfromhell.__init__.parse>` on
:py:meth:`~earwigbot.wiki.page.Page.get`.
:mod:`mwparserfromhell` is used by and originally developed for EarwigBot_;
:class:`~earwigbot.wiki.page.Page` objects have a
:meth:`~earwigbot.wiki.page.Page.parse` method that essentially calls
:func:`mwparserfromhell.parse() <mwparserfromhell.__init__.parse>` on
:meth:`~earwigbot.wiki.page.Page.get`.


If you're using Pywikipedia_, your code might look like this:: If you're using Pywikipedia_, your code might look like this::




+ 16
- 16
docs/usage.rst View File

@@ -6,9 +6,9 @@ Normal usage is rather straightforward (where ``text`` is page text)::
>>> import mwparserfromhell >>> import mwparserfromhell
>>> wikicode = mwparserfromhell.parse(text) >>> wikicode = mwparserfromhell.parse(text)


``wikicode`` is a :py:class:`mwparserfromhell.Wikicode <.Wikicode>` object,
which acts like an ordinary ``unicode`` object (or ``str`` in Python 3) with
some extra methods. For example::
``wikicode`` is a :class:`mwparserfromhell.Wikicode <.Wikicode>` object, which
acts like an ordinary ``unicode`` object (or ``str`` in Python 3) with some
extra methods. For example::


>>> text = "I has a template! {{foo|bar|baz|eggs=spam}} See it?" >>> text = "I has a template! {{foo|bar|baz|eggs=spam}} See it?"
>>> wikicode = mwparserfromhell.parse(text) >>> wikicode = mwparserfromhell.parse(text)
@@ -33,9 +33,9 @@ Since nodes can contain other nodes, getting nested templates is trivial::
>>> mwparserfromhell.parse(text).filter_templates() >>> mwparserfromhell.parse(text).filter_templates()
['{{foo|{{bar}}={{baz|{{spam}}}}}}', '{{bar}}', '{{baz|{{spam}}}}', '{{spam}}'] ['{{foo|{{bar}}={{baz|{{spam}}}}}}', '{{bar}}', '{{baz|{{spam}}}}', '{{spam}}']


You can also pass *recursive=False* to :py:meth:`~.filter_templates` and
explore templates manually. This is possible because nodes can contain
additional :py:class:`~.Wikicode` objects::
You can also pass *recursive=False* to :meth:`.filter_templates` and explore
templates manually. This is possible because nodes can contain additional
:class:`.Wikicode` objects::


>>> code = mwparserfromhell.parse("{{foo|this {{includes a|template}}}}") >>> code = mwparserfromhell.parse("{{foo|this {{includes a|template}}}}")
>>> print code.filter_templates(recursive=False) >>> print code.filter_templates(recursive=False)
@@ -49,11 +49,11 @@ additional :py:class:`~.Wikicode` objects::
template template


Templates can be easily modified to add, remove, or alter params. Templates can be easily modified to add, remove, or alter params.
:py:class:`~.Wikicode` objects can be treated like lists, with
:py:meth:`~.Wikicode.append`, :py:meth:`~.Wikicode.insert`,
:py:meth:`~.Wikicode.remove`, :py:meth:`~.Wikicode.replace`, and more. They
also have a :py:meth:`~.Wikicode.matches` method for comparing page or template
names, which takes care of capitalization and whitespace::
:class:`.Wikicode` objects can be treated like lists, with
:meth:`~.Wikicode.append`, :meth:`~.Wikicode.insert`,
:meth:`~.Wikicode.remove`, :meth:`~.Wikicode.replace`, and more. They also have
a :meth:`~.Wikicode.matches` method for comparing page or template names, which
takes care of capitalization and whitespace::


>>> text = "{{cleanup}} '''Foo''' is a [[bar]]. {{uncategorized}}" >>> text = "{{cleanup}} '''Foo''' is a [[bar]]. {{uncategorized}}"
>>> code = mwparserfromhell.parse(text) >>> code = mwparserfromhell.parse(text)
@@ -69,8 +69,8 @@ names, which takes care of capitalization and whitespace::
>>> print code.filter_templates() >>> print code.filter_templates()
['{{cleanup|date=July 2012}}', '{{bar-stub}}'] ['{{cleanup|date=July 2012}}', '{{bar-stub}}']


You can then convert ``code`` back into a regular :py:class:`unicode` object
(for saving the page!) by calling :py:func:`unicode` on it::
You can then convert ``code`` back into a regular :class:`unicode` object (for
saving the page!) by calling :func:`unicode` on it::


>>> text = unicode(code) >>> text = unicode(code)
>>> print text >>> print text
@@ -78,7 +78,7 @@ You can then convert ``code`` back into a regular :py:class:`unicode` object
>>> text == code >>> text == code
True True


(Likewise, use :py:func:`str(code) <str>` in Python 3.)
(Likewise, use :func:`str(code) <str>` in Python 3.)


For more tips, check out :py:class:`Wikicode's full method list <.Wikicode>`
and the :py:mod:`list of Nodes <.nodes>`.
For more tips, check out :class:`Wikicode's full method list <.Wikicode>` and
the :mod:`list of Nodes <.nodes>`.

Write Preview
Loading…
Cancel
Save