ben
/
mwparserfromhell
mirror of https://github.com/earwig/mwparserfromhell
1
1
Fork 0
Code Releases 24 Activity
A Python parser for MediaWiki wikicode https://mwparserfromhell.readthedocs.io/
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
588 Commits
1 Branch
4.7 MiB
 
 
 
 
Tree: efcd59e097
Branches Tags
${ item.name }
Create branch ${ searchTerm }
from 'efcd59e097'
${ noResults }
mwparserfromhell/tests/tokenizer/integration.mwtest

188 lines
11 KiB
Raw Blame History 

  1. name:   empty

  2. label:  sanity check that parsing an empty string yields nothing

  3. input:  ""

  4. output: []

  5. 

  6. ---

  7. 

  8. name:   template_argument_mix

  9. label:  an ambiguous mix of templates and arguments

  10. input:  "{{{{{{{{foo}}}}}}}}{{{{{{{bar}}baz}}}buz}}"

  11. output: [TemplateOpen(), ArgumentOpen(), ArgumentOpen(), Text(text="foo"), ArgumentClose(), ArgumentClose(), TemplateClose(), TemplateOpen(), ArgumentOpen(), TemplateOpen(), Text(text="bar"), TemplateClose(), Text(text="baz"), ArgumentClose(), Text(text="buz"), TemplateClose()]

  12. 

  13. ---

  14. 

  15. name:   link_in_template_name

  16. label:  a wikilink inside a template name, which breaks the template

  17. input:  "{{foo[[bar]]}}"

  18. output: [Text(text="{{foo"), WikilinkOpen(), Text(text="bar"), WikilinkClose(), Text(text="}}")]

  19. 

  20. ---

  21. 

  22. name:   rich_heading

  23. label:  a heading with templates/wikilinks in it

  24. input:  "== Head{{ing}} [[with]] {{{funky|{{stuf}}}}} =="

  25. output: [HeadingStart(level=2), Text(text=" Head"), TemplateOpen(), Text(text="ing"), TemplateClose(), Text(text=" "), WikilinkOpen(), Text(text="with"), WikilinkClose(), Text(text=" "), ArgumentOpen(), Text(text="funky"), ArgumentSeparator(), TemplateOpen(), Text(text="stuf"), TemplateClose(), ArgumentClose(), Text(text=" "), HeadingEnd()]

  26. 

  27. ---

  28. 

  29. name:   html_entity_with_template

  30. label:  a HTML entity with a template embedded inside

  31. input:  "&n{{bs}}p;"

  32. output: [Text(text="&n"), TemplateOpen(), Text(text="bs"), TemplateClose(), Text(text="p;")]

  33. 

  34. ---

  35. 

  36. name:   html_entity_with_comment

  37. label:  a HTML entity with a comment embedded inside

  38. input:  "&n<!--foo-->bsp;"

  39. output: [Text(text="&n"), CommentStart(), Text(text="foo"), CommentEnd(), Text(text="bsp;")]

  40. 

  41. ---

  42. 

  43. name:   rich_tags

  44. label:  a HTML tag with tons of other things in it

  45. input:  "{{dubious claim}}<ref name={{abc}}   foo="bar {{baz}}" abc={{de}}f ghi=j{{k}}{{l}} \n mno =  "{{p}} [[q]] {{r}}">[[Source]]</ref>"

  46. output: [TemplateOpen(), Text(text="dubious claim"), TemplateClose(), TagOpenOpen(), Text(text="ref"), TagAttrStart(pad_first=" ", pad_before_eq="", pad_after_eq=""), Text(text="name"), TagAttrEquals(), TemplateOpen(), Text(text="abc"), TemplateClose(), TagAttrStart(pad_first="   ", pad_before_eq="", pad_after_eq=""), Text(text="foo"), TagAttrEquals(), TagAttrQuote(), Text(text="bar "), TemplateOpen(), Text(text="baz"), TemplateClose(), TagAttrStart(pad_first=" ", pad_before_eq="", pad_after_eq=""), Text(text="abc"), TagAttrEquals(), TemplateOpen(), Text(text="de"), TemplateClose(), Text(text="f"), TagAttrStart(pad_first=" ", pad_before_eq="", pad_after_eq=""), Text(text="ghi"), TagAttrEquals(), Text(text="j"), TemplateOpen(), Text(text="k"), TemplateClose(), TemplateOpen(), Text(text="l"), TemplateClose(), TagAttrStart(pad_first=" \n ", pad_before_eq=" ", pad_after_eq="  "), Text(text="mno"), TagAttrEquals(), TagAttrQuote(), TemplateOpen(), Text(text="p"), TemplateClose(), Text(text=" "), WikilinkOpen(), Text(text="q"), WikilinkClose(), Text(text=" "), TemplateOpen(), Text(text="r"), TemplateClose(), TagCloseOpen(padding=""), WikilinkOpen(), Text(text="Source"), WikilinkClose(), TagOpenClose(), Text(text="ref"), TagCloseClose()]

  47. 

  48. ---

  49. 

  50. name:   wildcard

  51. label:  a wildcard assortment of various things

  52. input:  "{{{{{{{{foo}}bar|baz=biz}}buzz}}usr|{{bin}}}}"

  53. output: [TemplateOpen(), TemplateOpen(), TemplateOpen(), TemplateOpen(), Text(text="foo"), TemplateClose(), Text(text="bar"), TemplateParamSeparator(), Text(text="baz"), TemplateParamEquals(), Text(text="biz"), TemplateClose(), Text(text="buzz"), TemplateClose(), Text(text="usr"), TemplateParamSeparator(), TemplateOpen(), Text(text="bin"), TemplateClose(), TemplateClose()]

  54. 

  55. ---

  56. 

  57. name:   wildcard_redux

  58. label:  an even wilder assortment of various things

  59. input:  "{{a|b|{{c|[[d]]{{{e}}}}}}}[[f|{{{g}}}<!--h-->]]{{i|j=&nbsp;}}"

  60. output: [TemplateOpen(), Text(text="a"), TemplateParamSeparator(), Text(text="b"), TemplateParamSeparator(), TemplateOpen(), Text(text="c"), TemplateParamSeparator(), WikilinkOpen(), Text(text="d"), WikilinkClose(), ArgumentOpen(), Text(text="e"), ArgumentClose(), TemplateClose(), TemplateClose(), WikilinkOpen(), Text(text="f"), WikilinkSeparator(), ArgumentOpen(), Text(text="g"), ArgumentClose(), CommentStart(), Text(text="h"), CommentEnd(), WikilinkClose(), TemplateOpen(), Text(text="i"), TemplateParamSeparator(), Text(text="j"), TemplateParamEquals(), HTMLEntityStart(), Text(text="nbsp"), HTMLEntityEnd(), TemplateClose()]

  61. 

  62. ---

  63. 

  64. name:   link_inside_dl

  65. label:  an external link inside a def list, such that the external link is parsed

  66. input:  ";;;mailto:example"

  67. output: [TagOpenOpen(wiki_markup=";"), Text(text="dt"), TagCloseSelfclose(), TagOpenOpen(wiki_markup=";"), Text(text="dt"), TagCloseSelfclose(), TagOpenOpen(wiki_markup=";"), Text(text="dt"), TagCloseSelfclose(), ExternalLinkOpen(brackets=False), Text(text="mailto:example"), ExternalLinkClose()]

  68. 

  69. ---

  70. 

  71. name:   link_inside_dl_2

  72. label:  an external link inside a def list, such that the external link is not parsed

  73. input:  ";;;malito:example"

  74. output: [TagOpenOpen(wiki_markup=";"), Text(text="dt"), TagCloseSelfclose(), TagOpenOpen(wiki_markup=";"), Text(text="dt"), TagCloseSelfclose(), TagOpenOpen(wiki_markup=";"), Text(text="dt"), TagCloseSelfclose(), Text(text="malito"), TagOpenOpen(wiki_markup=":"), Text(text="dd"), TagCloseSelfclose(), Text(text="example")]

  75. 

  76. ---

  77. 

  78. name:   link_inside_template

  79. label:  an external link nested inside a template, before the end

  80. input:  "{{URL|http://example.com}}"

  81. output: [TemplateOpen(), Text(text="URL"), TemplateParamSeparator(), ExternalLinkOpen(brackets=False), Text(text="http://example.com"), ExternalLinkClose(), TemplateClose()]

  82. 

  83. ---

  84. 

  85. name:   link_inside_template_2

  86. label:  an external link nested inside a template, before a separator

  87. input:  "{{URL|http://example.com|foobar}}"

  88. output: [TemplateOpen(), Text(text="URL"), TemplateParamSeparator(), ExternalLinkOpen(brackets=False), Text(text="http://example.com"), ExternalLinkClose(), TemplateParamSeparator(), Text(text="foobar"), TemplateClose()]

  89. 

  90. ---

  91. 

  92. name:   link_inside_template_3

  93. label:  an external link nested inside a template, before an equal sign

  94. input:  "{{URL|http://example.com=foobar}}"

  95. output: [TemplateOpen(), Text(text="URL"), TemplateParamSeparator(), ExternalLinkOpen(brackets=False), Text(text="http://example.com"), ExternalLinkClose(), TemplateParamEquals(), Text(text="foobar"), TemplateClose()]

  96. 

  97. ---

  98. 

  99. name:   link_inside_argument

  100. label:  an external link nested inside an argument

  101. input:  "{{{URL|http://example.com}}}"

  102. output: [ArgumentOpen(), Text(text="URL"), ArgumentSeparator(), ExternalLinkOpen(brackets=False), Text(text="http://example.com"), ExternalLinkClose(), ArgumentClose()]

  103. 

  104. ---

  105. 

  106. name:   link_inside_heading

  107. label:  an external link nested inside a heading

  108. input:  "==http://example.com=="

  109. output: [HeadingStart(level=2), ExternalLinkOpen(brackets=False), Text(text="http://example.com"), ExternalLinkClose(), HeadingEnd()]

  110. 

  111. ---

  112. 

  113. name:   link_inside_tag_body

  114. label:  an external link nested inside the body of a tag

  115. input:  "<ref>http://example.com</ref>"

  116. output: [TagOpenOpen(), Text(text="ref"), TagCloseOpen(padding=""), ExternalLinkOpen(brackets=False), Text(text="http://example.com"), ExternalLinkClose(), TagOpenClose(), Text(text="ref"), TagCloseClose()]

  117. 

  118. ---

  119. 

  120. name:   link_inside_tag_style

  121. label:  an external link nested inside style tags

  122. input:  "''http://example.com''"

  123. output: [TagOpenOpen(wiki_markup="''"), Text(text="i"), TagCloseOpen(), ExternalLinkOpen(brackets=False), Text(text="http://example.com"), ExternalLinkClose(), TagOpenClose(), Text(text="i"), TagCloseClose()]

  124. 

  125. ---

  126. 

  127. name:   style_tag_inside_link

  128. label:  style tags disrupting an external link

  129. input:  "http://example.com/foo''bar''"

  130. output: [ExternalLinkOpen(brackets=False), Text(text="http://example.com/foo"), ExternalLinkClose(), TagOpenOpen(wiki_markup="''"), Text(text="i"), TagCloseOpen(), Text(text="bar"), TagOpenClose(), Text(text="i"), TagCloseClose()]

  131. 

  132. ---

  133. 

  134. name:   comment_inside_link

  135. label:  an HTML comment inside an external link

  136. input:  "http://example.com/foo<!--comment-->bar"

  137. output: [ExternalLinkOpen(brackets=False), Text(text="http://example.com/foo"), CommentStart(), Text(text="comment"), CommentEnd(), Text(text="bar"), ExternalLinkClose()]

  138. 

  139. ---

  140. 

  141. name:   bracketed_link_inside_template

  142. label:  a bracketed external link nested inside a template, before the end

  143. input:  "{{URL|[http://example.com}}]"

  144. output: [Text(text="{{URL|"), ExternalLinkOpen(brackets=True), Text(text="http://example.com}}"), ExternalLinkClose()]

  145. 

  146. 

  147. ---

  148. 

  149. name:   comment_inside_bracketed_link

  150. label:  an HTML comment inside a bracketed external link

  151. input:  "[http://example.com/foo<!--comment-->bar]"

  152. output: [ExternalLinkOpen(brackets=True), Text(text="http://example.com/foo"), CommentStart(), Text(text="comment"), CommentEnd(), Text(text="bar"), ExternalLinkClose()]

  153. 

  154. ---

  155. 

  156. name:   wikilink_inside_external_link

  157. label:  a wikilink inside an external link, which the parser considers valid (see issue #61)

  158. input:  "[http://example.com/foo Foo [[Bar]]]"

  159. output: [ExternalLinkOpen(brackets=True), Text(text="http://example.com/foo"), ExternalLinkSeparator(), Text(text="Foo "), WikilinkOpen(), Text(text="Bar"), WikilinkClose(), ExternalLinkClose()]

  160. 

  161. ---

  162. 

  163. name:   external_link_inside_wikilink

  164. label:  an external link inside a wikilink, valid in the case of images (see issue #62)

  165. input:  "[[File:Example.png|thumb|http://example.com]]"

  166. output: [WikilinkOpen(), Text(text="File:Example.png"), WikilinkSeparator(), Text(text="thumb|"), ExternalLinkOpen(brackets=False), Text(text="http://example.com"), ExternalLinkClose(), WikilinkClose()]

  167. 

  168. ---

  169. 

  170. name:   external_link_inside_wikilink_brackets

  171. label:  an external link with brackets inside a wikilink

  172. input:  "[[File:Example.png|thumb|[http://example.com Example]]]"

  173. output: [WikilinkOpen(), Text(text="File:Example.png"), WikilinkSeparator(), Text(text="thumb|"), ExternalLinkOpen(brackets=True), Text(text="http://example.com"), ExternalLinkSeparator(), Text(text="Example"), ExternalLinkClose(), WikilinkClose()]

  174. 

  175. ---

  176. 

  177. name:   external_link_inside_wikilink_title

  178. label:  an external link inside a wikilink title, which is invalid

  179. input:  "[[File:Example.png http://example.com]]"

  180. output: [WikilinkOpen(), Text(text="File:Example.png http://example.com"), WikilinkClose()]

  181. 

  182. ---

  183. 

  184. name:   italics_inside_external_link_inside_incomplete_list

  185. label:  italic text inside an external link inside an incomplete list

  186. input:  "<li>[http://www.example.com ''example'']"

  187. output: [TagOpenOpen(), Text(text="li"), TagCloseSelfclose(padding="", implicit=True), ExternalLinkOpen(brackets=True), Text(text="http://www.example.com"), ExternalLinkSeparator(), TagOpenOpen(wiki_markup="''"), Text(text="i"), TagCloseOpen(), Text(text="example"), TagOpenClose(), Text(text="i"), TagCloseClose(), ExternalLinkClose()]