A Python parser for MediaWiki wikicode https://mwparserfromhell.readthedocs.io/
25개 이상의 토픽을 선택하실 수 없습니다. Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

README.rst 6.8 KiB

12 년 전
11 년 전
12 년 전
12 년 전
12 년 전
12 년 전
12 년 전
12 년 전
12 년 전
12 년 전
12 년 전
12 년 전
12 년 전
12 년 전
12 년 전
12 년 전
12 년 전
12 년 전
12 년 전
12 년 전
12 년 전
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183
  1. mwparserfromhell
  2. ================
  3. .. image:: https://img.shields.io/travis/earwig/mwparserfromhell/develop.svg
  4. :alt: Build Status
  5. :target: http://travis-ci.org/earwig/mwparserfromhell
  6. .. image:: https://img.shields.io/coveralls/earwig/mwparserfromhell/develop.svg
  7. :alt: Coverage Status
  8. :target: https://coveralls.io/r/earwig/mwparserfromhell
  9. **mwparserfromhell** (the *MediaWiki Parser from Hell*) is a Python package
  10. that provides an easy-to-use and outrageously powerful parser for MediaWiki_
  11. wikicode. It supports Python 2 and Python 3.
  12. Developed by Earwig_ with contributions from `Σ`_, Legoktm_, and others.
  13. Full documentation is available on ReadTheDocs_. Development occurs on GitHub_.
  14. Installation
  15. ------------
  16. The easiest way to install the parser is through the `Python Package Index`_;
  17. you can install the latest release with ``pip install mwparserfromhell``
  18. (`get pip`_). Make sure your pip is up-to-date first, especially on Windows.
  19. Alternatively, get the latest development version::
  20. git clone https://github.com/earwig/mwparserfromhell.git
  21. cd mwparserfromhell
  22. python setup.py install
  23. You can run the comprehensive unit testing suite with
  24. ``python setup.py test -q``.
  25. Usage
  26. -----
  27. Normal usage is rather straightforward (where ``text`` is page text)::
  28. >>> import mwparserfromhell
  29. >>> wikicode = mwparserfromhell.parse(text)
  30. ``wikicode`` is a ``mwparserfromhell.Wikicode`` object, which acts like an
  31. ordinary ``str`` object (or ``unicode`` in Python 2) with some extra methods.
  32. For example::
  33. >>> text = "I has a template! {{foo|bar|baz|eggs=spam}} See it?"
  34. >>> wikicode = mwparserfromhell.parse(text)
  35. >>> print(wikicode)
  36. I has a template! {{foo|bar|baz|eggs=spam}} See it?
  37. >>> templates = wikicode.filter_templates()
  38. >>> print(templates)
  39. ['{{foo|bar|baz|eggs=spam}}']
  40. >>> template = templates[0]
  41. >>> print(template.name)
  42. foo
  43. >>> print(template.params)
  44. ['bar', 'baz', 'eggs=spam']
  45. >>> print(template.get(1).value)
  46. bar
  47. >>> print(template.get("eggs").value)
  48. spam
  49. Since nodes can contain other nodes, getting nested templates is trivial::
  50. >>> text = "{{foo|{{bar}}={{baz|{{spam}}}}}}"
  51. >>> mwparserfromhell.parse(text).filter_templates()
  52. ['{{foo|{{bar}}={{baz|{{spam}}}}}}', '{{bar}}', '{{baz|{{spam}}}}', '{{spam}}']
  53. You can also pass ``recursive=False`` to ``filter_templates()`` and explore
  54. templates manually. This is possible because nodes can contain additional
  55. ``Wikicode`` objects::
  56. >>> code = mwparserfromhell.parse("{{foo|this {{includes a|template}}}}")
  57. >>> print(code.filter_templates(recursive=False))
  58. ['{{foo|this {{includes a|template}}}}']
  59. >>> foo = code.filter_templates(recursive=False)[0]
  60. >>> print(foo.get(1).value)
  61. this {{includes a|template}}
  62. >>> print(foo.get(1).value.filter_templates()[0])
  63. {{includes a|template}}
  64. >>> print(foo.get(1).value.filter_templates()[0].get(1).value)
  65. template
  66. Templates can be easily modified to add, remove, or alter params. ``Wikicode``
  67. objects can be treated like lists, with ``append()``, ``insert()``,
  68. ``remove()``, ``replace()``, and more. They also have a ``matches()`` method
  69. for comparing page or template names, which takes care of capitalization and
  70. whitespace::
  71. >>> text = "{{cleanup}} '''Foo''' is a [[bar]]. {{uncategorized}}"
  72. >>> code = mwparserfromhell.parse(text)
  73. >>> for template in code.filter_templates():
  74. ... if template.name.matches("Cleanup") and not template.has("date"):
  75. ... template.add("date", "July 2012")
  76. ...
  77. >>> print(code)
  78. {{cleanup|date=July 2012}} '''Foo''' is a [[bar]]. {{uncategorized}}
  79. >>> code.replace("{{uncategorized}}", "{{bar-stub}}")
  80. >>> print(code)
  81. {{cleanup|date=July 2012}} '''Foo''' is a [[bar]]. {{bar-stub}}
  82. >>> print(code.filter_templates())
  83. ['{{cleanup|date=July 2012}}', '{{bar-stub}}']
  84. You can then convert ``code`` back into a regular ``str`` object (for
  85. saving the page!) by calling ``str()`` on it::
  86. >>> text = str(code)
  87. >>> print(text)
  88. {{cleanup|date=July 2012}} '''Foo''' is a [[bar]]. {{bar-stub}}
  89. >>> text == code
  90. True
  91. Likewise, use ``unicode(code)`` in Python 2.
  92. You can get the parent of a node using::
  93. wikicode.get(wikicode.index(node, True))
  94. Caveats
  95. -------
  96. An inherent limitation in wikicode prevents us from generating complete parse
  97. trees in certain cases. For example, the string ``{{echo|''Hello}}, world!''``
  98. produces the valid output ``<i>Hello, world!</i>`` in MediaWiki, assuming
  99. ``{{echo}}`` is a template that returns its first parameter. But since
  100. representing this in mwparserfromhell's node tree would be impossible, we
  101. compromise by treating the first node (i.e., the template) as plain text,
  102. parsing only the italics.
  103. The current workaround for cases where you are not interested in text
  104. formatting is to pass ``skip_style_tags=True`` to ``mwparserfromhell.parse()``.
  105. This treats ``''`` and ``'''`` like plain text.
  106. A future version of mwparserfromhell will include multiple parsing modes to get
  107. around this restriction.
  108. Integration
  109. -----------
  110. ``mwparserfromhell`` is used by and originally developed for EarwigBot_;
  111. ``Page`` objects have a ``parse`` method that essentially calls
  112. ``mwparserfromhell.parse()`` on ``page.get()``.
  113. If you're using Pywikibot_, your code might look like this::
  114. import mwparserfromhell
  115. import pywikibot
  116. def parse(title):
  117. site = pywikibot.Site()
  118. page = pywikibot.Page(site, title)
  119. text = page.get()
  120. return mwparserfromhell.parse(text)
  121. If you're not using a library, you can parse any page using the following
  122. Python 3 code (via the API_)::
  123. import json
  124. from urllib.parse import urlencode
  125. from urllib.request import urlopen
  126. import mwparserfromhell
  127. API_URL = "https://en.wikipedia.org/w/api.php"
  128. def parse(title):
  129. data = {"action": "query", "prop": "revisions", "rvlimit": 1,
  130. "rvprop": "content", "format": "json", "titles": title}
  131. raw = urlopen(API_URL, urlencode(data).encode()).read()
  132. res = json.loads(raw)
  133. text = res["query"]["pages"].values()[0]["revisions"][0]["*"]
  134. return mwparserfromhell.parse(text)
  135. .. _MediaWiki: http://mediawiki.org
  136. .. _ReadTheDocs: http://mwparserfromhell.readthedocs.org
  137. .. _Earwig: http://en.wikipedia.org/wiki/User:The_Earwig
  138. .. _Σ: http://en.wikipedia.org/wiki/User:%CE%A3
  139. .. _Legoktm: http://en.wikipedia.org/wiki/User:Legoktm
  140. .. _GitHub: https://github.com/earwig/mwparserfromhell
  141. .. _Python Package Index: http://pypi.python.org
  142. .. _get pip: http://pypi.python.org/pypi/pip
  143. .. _EarwigBot: https://github.com/earwig/earwigbot
  144. .. _Pywikibot: https://www.mediawiki.org/wiki/Manual:Pywikibot
  145. .. _API: http://mediawiki.org/wiki/API