ben
/
mwparserfromhell
kopia lustrzana https://github.com/earwig/mwparserfromhell
1
1
Forkuj 0
Kod Wydania 24 Aktywność
A Python parser for MediaWiki wikicode https://mwparserfromhell.readthedocs.io/
Nie możesz wybrać więcej, niż 25 tematów Tematy muszą się zaczynać od litery lub cyfry, mogą zawierać myślniki ('-') i mogą mieć do 35 znaków.
773 Commity
1 Gałąź
5.2 MiB
 
 
 
 
Drzewo: aaffb7f66b
Gałęzie Tagi
${ item.name }
Utwórz gałąź ${ searchTerm }
z 'aaffb7f66b'
${ noResults }
mwparserfromhell/tests/tokenizer/integration.mwtest

349 wiersze
17 KiB
Czysty Wina Historia 

  1. name:   empty

  2. label:  sanity check that parsing an empty string yields nothing

  3. input:  ""

  4. output: []

  5. 

  6. ---

  7. 

  8. name:   template_argument_mix

  9. label:  an ambiguous mix of templates and arguments

  10. input:  "{{{{{{{{foo}}}}}}}}{{{{{{{bar}}baz}}}buz}}"

  11. output: [TemplateOpen(), ArgumentOpen(), ArgumentOpen(), Text(text="foo"), ArgumentClose(), ArgumentClose(), TemplateClose(), TemplateOpen(), ArgumentOpen(), TemplateOpen(), Text(text="bar"), TemplateClose(), Text(text="baz"), ArgumentClose(), Text(text="buz"), TemplateClose()]

  12. 

  13. ---

  14. 

  15. name:   link_in_template_name

  16. label:  a wikilink inside a template name, which breaks the template

  17. input:  "{{foo[[bar]]}}"

  18. output: [Text(text="{{foo"), WikilinkOpen(), Text(text="bar"), WikilinkClose(), Text(text="}}")]

  19. 

  20. ---

  21. 

  22. name:   rich_heading

  23. label:  a heading with templates/wikilinks in it

  24. input:  "== Head{{ing}} [[with]] {{{funky|{{stuf}}}}} =="

  25. output: [HeadingStart(level=2), Text(text=" Head"), TemplateOpen(), Text(text="ing"), TemplateClose(), Text(text=" "), WikilinkOpen(), Text(text="with"), WikilinkClose(), Text(text=" "), ArgumentOpen(), Text(text="funky"), ArgumentSeparator(), TemplateOpen(), Text(text="stuf"), TemplateClose(), ArgumentClose(), Text(text=" "), HeadingEnd()]

  26. 

  27. ---

  28. 

  29. name:   html_entity_with_template

  30. label:  a HTML entity with a template embedded inside

  31. input:  "&n{{bs}}p;"

  32. output: [Text(text="&n"), TemplateOpen(), Text(text="bs"), TemplateClose(), Text(text="p;")]

  33. 

  34. ---

  35. 

  36. name:   html_entity_with_comment

  37. label:  a HTML entity with a comment embedded inside

  38. input:  "&n<!--foo-->bsp;"

  39. output: [Text(text="&n"), CommentStart(), Text(text="foo"), CommentEnd(), Text(text="bsp;")]

  40. 

  41. ---

  42. 

  43. name:   rich_tags

  44. label:  a HTML tag with tons of other things in it

  45. input:  "{{dubious claim}}<ref name={{abc}}   foo="bar {{baz}}" abc={{de}}f ghi=j{{k}}{{l}} \n mno =  "{{p}} [[q]] {{r}}">[[Source]]</ref>"

  46. output: [TemplateOpen(), Text(text="dubious claim"), TemplateClose(), TagOpenOpen(), Text(text="ref"), TagAttrStart(pad_first=" ", pad_before_eq="", pad_after_eq=""), Text(text="name"), TagAttrEquals(), TemplateOpen(), Text(text="abc"), TemplateClose(), TagAttrStart(pad_first="   ", pad_before_eq="", pad_after_eq=""), Text(text="foo"), TagAttrEquals(), TagAttrQuote(char="\""), Text(text="bar "), TemplateOpen(), Text(text="baz"), TemplateClose(), TagAttrStart(pad_first=" ", pad_before_eq="", pad_after_eq=""), Text(text="abc"), TagAttrEquals(), TemplateOpen(), Text(text="de"), TemplateClose(), Text(text="f"), TagAttrStart(pad_first=" ", pad_before_eq="", pad_after_eq=""), Text(text="ghi"), TagAttrEquals(), Text(text="j"), TemplateOpen(), Text(text="k"), TemplateClose(), TemplateOpen(), Text(text="l"), TemplateClose(), TagAttrStart(pad_first=" \n ", pad_before_eq=" ", pad_after_eq="  "), Text(text="mno"), TagAttrEquals(), TagAttrQuote(char="\""), TemplateOpen(), Text(text="p"), TemplateClose(), Text(text=" "), WikilinkOpen(), Text(text="q"), WikilinkClose(), Text(text=" "), TemplateOpen(), Text(text="r"), TemplateClose(), TagCloseOpen(padding=""), WikilinkOpen(), Text(text="Source"), WikilinkClose(), TagOpenClose(), Text(text="ref"), TagCloseClose()]

  47. 

  48. ---

  49. 

  50. name:   wildcard

  51. label:  a wildcard assortment of various things

  52. input:  "{{{{{{{{foo}}bar|baz=biz}}buzz}}usr|{{bin}}}}"

  53. output: [TemplateOpen(), TemplateOpen(), TemplateOpen(), TemplateOpen(), Text(text="foo"), TemplateClose(), Text(text="bar"), TemplateParamSeparator(), Text(text="baz"), TemplateParamEquals(), Text(text="biz"), TemplateClose(), Text(text="buzz"), TemplateClose(), Text(text="usr"), TemplateParamSeparator(), TemplateOpen(), Text(text="bin"), TemplateClose(), TemplateClose()]

  54. 

  55. ---

  56. 

  57. name:   wildcard_redux

  58. label:  an even wilder assortment of various things

  59. input:  "{{a|b|{{c|[[d]]{{{e}}}}}}}[[f|{{{g}}}<!--h-->]]{{i|j=&nbsp;}}"

  60. output: [TemplateOpen(), Text(text="a"), TemplateParamSeparator(), Text(text="b"), TemplateParamSeparator(), TemplateOpen(), Text(text="c"), TemplateParamSeparator(), WikilinkOpen(), Text(text="d"), WikilinkClose(), ArgumentOpen(), Text(text="e"), ArgumentClose(), TemplateClose(), TemplateClose(), WikilinkOpen(), Text(text="f"), WikilinkSeparator(), ArgumentOpen(), Text(text="g"), ArgumentClose(), CommentStart(), Text(text="h"), CommentEnd(), WikilinkClose(), TemplateOpen(), Text(text="i"), TemplateParamSeparator(), Text(text="j"), TemplateParamEquals(), HTMLEntityStart(), Text(text="nbsp"), HTMLEntityEnd(), TemplateClose()]

  61. 

  62. ---

  63. 

  64. name:   link_inside_dl

  65. label:  an external link inside a def list, such that the external link is parsed

  66. input:  ";;;mailto:example"

  67. output: [TagOpenOpen(wiki_markup=";"), Text(text="dt"), TagCloseSelfclose(), TagOpenOpen(wiki_markup=";"), Text(text="dt"), TagCloseSelfclose(), TagOpenOpen(wiki_markup=";"), Text(text="dt"), TagCloseSelfclose(), ExternalLinkOpen(brackets=False), Text(text="mailto:example"), ExternalLinkClose()]

  68. 

  69. ---

  70. 

  71. name:   link_inside_dl_2

  72. label:  an external link inside a def list, such that the external link is not parsed

  73. input:  ";;;malito:example"

  74. output: [TagOpenOpen(wiki_markup=";"), Text(text="dt"), TagCloseSelfclose(), TagOpenOpen(wiki_markup=";"), Text(text="dt"), TagCloseSelfclose(), TagOpenOpen(wiki_markup=";"), Text(text="dt"), TagCloseSelfclose(), Text(text="malito"), TagOpenOpen(wiki_markup=":"), Text(text="dd"), TagCloseSelfclose(), Text(text="example")]

  75. 

  76. ---

  77. 

  78. name:   link_inside_template

  79. label:  an external link nested inside a template, before the end

  80. input:  "{{URL|http://example.com}}"

  81. output: [TemplateOpen(), Text(text="URL"), TemplateParamSeparator(), ExternalLinkOpen(brackets=False), Text(text="http://example.com"), ExternalLinkClose(), TemplateClose()]

  82. 

  83. ---

  84. 

  85. name:   link_inside_template_2

  86. label:  an external link nested inside a template, before a separator

  87. input:  "{{URL|http://example.com|foobar}}"

  88. output: [TemplateOpen(), Text(text="URL"), TemplateParamSeparator(), ExternalLinkOpen(brackets=False), Text(text="http://example.com"), ExternalLinkClose(), TemplateParamSeparator(), Text(text="foobar"), TemplateClose()]

  89. 

  90. ---

  91. 

  92. name:   link_inside_template_3

  93. label:  an external link nested inside a template, before an equal sign

  94. input:  "{{URL|http://example.com=foobar}}"

  95. output: [TemplateOpen(), Text(text="URL"), TemplateParamSeparator(), ExternalLinkOpen(brackets=False), Text(text="http://example.com"), ExternalLinkClose(), TemplateParamEquals(), Text(text="foobar"), TemplateClose()]

  96. 

  97. ---

  98. 

  99. name:   link_inside_argument

  100. label:  an external link nested inside an argument

  101. input:  "{{{URL|http://example.com}}}"

  102. output: [ArgumentOpen(), Text(text="URL"), ArgumentSeparator(), ExternalLinkOpen(brackets=False), Text(text="http://example.com"), ExternalLinkClose(), ArgumentClose()]

  103. 

  104. ---

  105. 

  106. name:   link_inside_heading

  107. label:  an external link nested inside a heading

  108. input:  "==http://example.com=="

  109. output: [HeadingStart(level=2), ExternalLinkOpen(brackets=False), Text(text="http://example.com"), ExternalLinkClose(), HeadingEnd()]

  110. 

  111. ---

  112. 

  113. name:   link_inside_tag_body

  114. label:  an external link nested inside the body of a tag

  115. input:  "<ref>http://example.com</ref>"

  116. output: [TagOpenOpen(), Text(text="ref"), TagCloseOpen(padding=""), ExternalLinkOpen(brackets=False), Text(text="http://example.com"), ExternalLinkClose(), TagOpenClose(), Text(text="ref"), TagCloseClose()]

  117. 

  118. ---

  119. 

  120. name:   link_inside_tag_style

  121. label:  an external link nested inside style tags

  122. input:  "''http://example.com''"

  123. output: [TagOpenOpen(wiki_markup="''"), Text(text="i"), TagCloseOpen(), ExternalLinkOpen(brackets=False), Text(text="http://example.com"), ExternalLinkClose(), TagOpenClose(), Text(text="i"), TagCloseClose()]

  124. 

  125. ---

  126. 

  127. name:   style_tag_inside_link

  128. label:  style tags disrupting an external link

  129. input:  "http://example.com/foo''bar''"

  130. output: [ExternalLinkOpen(brackets=False), Text(text="http://example.com/foo"), ExternalLinkClose(), TagOpenOpen(wiki_markup="''"), Text(text="i"), TagCloseOpen(), Text(text="bar"), TagOpenClose(), Text(text="i"), TagCloseClose()]

  131. 

  132. ---

  133. 

  134. name:   comment_inside_link

  135. label:  an HTML comment inside an external link

  136. input:  "http://example.com/foo<!--comment-->bar"

  137. output: [ExternalLinkOpen(brackets=False), Text(text="http://example.com/foo"), CommentStart(), Text(text="comment"), CommentEnd(), Text(text="bar"), ExternalLinkClose()]

  138. 

  139. ---

  140. 

  141. name:   bracketed_link_inside_template

  142. label:  a bracketed external link nested inside a template, before the end

  143. input:  "{{URL|[http://example.com}}]"

  144. output: [Text(text="{{URL|"), ExternalLinkOpen(brackets=True), Text(text="http://example.com}}"), ExternalLinkClose()]

  145. 

  146. 

  147. ---

  148. 

  149. name:   comment_inside_bracketed_link

  150. label:  an HTML comment inside a bracketed external link

  151. input:  "[http://example.com/foo<!--comment-->bar]"

  152. output: [ExternalLinkOpen(brackets=True), Text(text="http://example.com/foo"), CommentStart(), Text(text="comment"), CommentEnd(), Text(text="bar"), ExternalLinkClose()]

  153. 

  154. ---

  155. 

  156. name:   wikilink_inside_external_link

  157. label:  a wikilink inside an external link, which the parser considers valid (see issue #61)

  158. input:  "[http://example.com/foo Foo [[Bar]]]"

  159. output: [ExternalLinkOpen(brackets=True), Text(text="http://example.com/foo"), ExternalLinkSeparator(), Text(text="Foo "), WikilinkOpen(), Text(text="Bar"), WikilinkClose(), ExternalLinkClose()]

  160. 

  161. ---

  162. 

  163. name:   external_link_inside_wikilink

  164. label:  an external link inside a wikilink, valid in the case of images (see issue #62)

  165. input:  "[[File:Example.png|thumb|http://example.com]]"

  166. output: [WikilinkOpen(), Text(text="File:Example.png"), WikilinkSeparator(), Text(text="thumb|"), ExternalLinkOpen(brackets=False), Text(text="http://example.com"), ExternalLinkClose(), WikilinkClose()]

  167. 

  168. ---

  169. 

  170. name:   external_link_inside_wikilink_brackets

  171. label:  an external link with brackets inside a wikilink

  172. input:  "[[File:Example.png|thumb|[http://example.com Example]]]"

  173. output: [WikilinkOpen(), Text(text="File:Example.png"), WikilinkSeparator(), Text(text="thumb|"), ExternalLinkOpen(brackets=True), Text(text="http://example.com"), ExternalLinkSeparator(), Text(text="Example"), ExternalLinkClose(), WikilinkClose()]

  174. 

  175. ---

  176. 

  177. name:   external_link_inside_wikilink_title

  178. label:  an external link inside a wikilink title, which is not parsed

  179. input:  "[[File:Example.png http://example.com]]"

  180. output: [WikilinkOpen(), Text(text="File:Example.png http://example.com"), WikilinkClose()]

  181. 

  182. ---

  183. 

  184. name:   italics_inside_external_link_inside_incomplete_list

  185. label:  italic text inside an external link inside an incomplete list

  186. input:  "<li>[http://www.example.com ''example'']"

  187. output: [TagOpenOpen(), Text(text="li"), TagCloseSelfclose(padding="", implicit=True), ExternalLinkOpen(brackets=True), Text(text="http://www.example.com"), ExternalLinkSeparator(), TagOpenOpen(wiki_markup="''"), Text(text="i"), TagCloseOpen(), Text(text="example"), TagOpenClose(), Text(text="i"), TagCloseClose(), ExternalLinkClose()]

  188. 

  189. ---

  190. 

  191. name:   nodes_inside_external_link_after_punct

  192. label:  various complex nodes inside an external link following punctuation

  193. input:  "http://example.com/foo.{{bar}}baz.&biz;<!--hello-->bingo"

  194. output: [ExternalLinkOpen(brackets=False), Text(text="http://example.com/foo."), TemplateOpen(), Text(text="bar"), TemplateClose(), Text(text="baz.&biz;"), CommentStart(), Text(text="hello"), CommentEnd(), Text(text="bingo"), ExternalLinkClose()]

  195. 

  196. ---

  197. 

  198. name:   newline_and_comment_in_template_name

  199. label:  a template name containing a newline followed by a comment

  200. input:  "{{foobar\n<!-- comment -->}}"

  201. output: [TemplateOpen(), Text(text="foobar\n"), CommentStart(), Text(text=" comment "), CommentEnd(), TemplateClose()]

  202. 

  203. ---

  204. 

  205. name:   newline_and_comment_in_template_name_2

  206. label:  a template name containing a newline followed by a comment

  207. input:  "{{foobar\n<!-- comment -->|key=value}}"

  208. output: [TemplateOpen(), Text(text="foobar\n"), CommentStart(), Text(text=" comment "), CommentEnd(), TemplateParamSeparator(), Text(text="key"), TemplateParamEquals(), Text(text="value"), TemplateClose()]

  209. 

  210. ---

  211. 

  212. name:   newline_and_comment_in_template_name_3

  213. label:  a template name containing a newline followed by a comment

  214. input:  "{{foobar\n<!-- comment -->\n|key=value}}"

  215. output: [TemplateOpen(), Text(text="foobar\n"), CommentStart(), Text(text=" comment "), CommentEnd(), Text(text="\n"), TemplateParamSeparator(), Text(text="key"), TemplateParamEquals(), Text(text="value"), TemplateClose()]

  216. 

  217. ---

  218. 

  219. name:   newline_and_comment_in_template_name_4

  220. label:  a template name containing a newline followed by a comment

  221. input:  "{{foobar\n<!-- comment -->invalid|key=value}}"

  222. output: [Text(text="{{foobar\n"), CommentStart(), Text(text=" comment "), CommentEnd(), Text(text="invalid|key=value}}")]

  223. 

  224. ---

  225. 

  226. name:   newline_and_comment_in_template_name_5

  227. label:  a template name containing a newline followed by a comment

  228. input:  "{{foobar\n<!-- comment -->\ninvalid|key=value}}"

  229. output: [Text(text="{{foobar\n"), CommentStart(), Text(text=" comment "), CommentEnd(), Text(text="\ninvalid|key=value}}")]

  230. 

  231. ---

  232. 

  233. name:   newline_and_comment_in_template_name_6

  234. label:  a template name containing a newline followed by a comment

  235. input:  "{{foobar\n<!--|key=value}}"

  236. output: [Text(text="{{foobar\n<!--|key=value}}")]

  237. 

  238. ---

  239. 

  240. name:   newline_and_comment_in_template_name_7

  241. label:  a template name containing a newline followed by a comment

  242. input:  "{{foobar\n<!|key=value}}"

  243. output: [Text(text="{{foobar\n<!|key=value}}")]

  244. 

  245. ---

  246. 

  247. name:   newline_and_comment_in_template_name_8

  248. label:  a template name containing a newline followed by a comment

  249. input:  "{{<!-- comment -->\nfoobar\n<!-- comment -->}}"

  250. output: [TemplateOpen(), CommentStart(), Text(text=" comment "), CommentEnd(), Text(text="\nfoobar\n"), CommentStart(), Text(text=" comment "), CommentEnd(), TemplateClose()]

  251. 

  252. ---

  253. 

  254. name:   tag_in_link_title

  255. label:  HTML tags are invalid in link titles, even when complete

  256. input:  "[[foo<i>bar</i>baz]]"

  257. output: [Text(text="[[foo"), TagOpenOpen(), Text(text="i"), TagCloseOpen(padding=""), Text(text="bar"), TagOpenClose(), Text(text="i"), TagCloseClose(), Text(text="baz]]")]

  258. 

  259. ---

  260. 

  261. name:   tag_in_template_name

  262. label:  HTML tags are invalid in template names, even when complete

  263. input:  "{{foo<i>bar</i>baz}}"

  264. output: [Text(text="{{foo"), TagOpenOpen(), Text(text="i"), TagCloseOpen(padding=""), Text(text="bar"), TagOpenClose(), Text(text="i"), TagCloseClose(), Text(text="baz}}")]

  265. 

  266. ---

  267. 

  268. name:   tag_in_link_text

  269. label:  HTML tags are valid in link text

  270. input:  "[[foo|<i>bar</i>baz]]"

  271. output: [WikilinkOpen(), Text(text="foo"), WikilinkSeparator(), TagOpenOpen(), Text(text="i"), TagCloseOpen(padding=""), Text(text="bar"), TagOpenClose(), Text(text="i"), TagCloseClose(), Text(text="baz"), WikilinkClose()]

  272. 

  273. ---

  274. 

  275. name:   comment_in_link_title

  276. label:  comments are valid in link titles

  277. input:  "[[foo<!--bar-->baz]]"

  278. output: [WikilinkOpen(), Text(text="foo"), CommentStart(), Text(text="bar"), CommentEnd(), Text(text="baz"), WikilinkClose()]

  279. 

  280. ---

  281. 

  282. name:   incomplete_comment_in_link_title

  283. label:  incomplete comments are invalid in link titles

  284. input:  "[[foo<!--bar--baz]]"

  285. output: [Text(text="[[foo<!--bar--baz]]")]

  286. 

  287. ---

  288. 

  289. name:   incomplete_comment_in_link_title_2

  290. label:  incomplete comments are invalid in link titles

  291. input:  "[[foo<!--barbaz]]"

  292. output: [Text(text="[[foo<!--barbaz]]")]

  293. 

  294. ---

  295. 

  296. name:   incomplete_comment_in_link_title_3

  297. label:  incomplete comments are invalid in link titles

  298. input:  "[[foo<!barbaz]]"

  299. output: [Text(text="[[foo<!barbaz]]")]

  300. 

  301. ---

  302. 

  303. name:   incomplete_comment_in_link_title_4

  304. label:  incomplete comments are invalid in link titles

  305. input:  "[[foo<!--]]"

  306. output: [Text(text="[[foo<!--]]")]

  307. 

  308. ---

  309. 

  310. name:   incomplete_comment_in_link_title_5

  311. label:  incomplete comments are invalid in link titles

  312. input:  "[[foo<!-"

  313. output: [Text(text="[[foo<!-")]

  314. 

  315. ---

  316. 

  317. name:   incomplete_comment_in_link_title_6

  318. label:  incomplete comments are invalid in link titles

  319. input:  "[[foo<!--bar"

  320. output: [Text(text="[[foo<!--bar")]

  321. 

  322. ---

  323. 

  324. name:   wikilink_to_external_link_fallback

  325. label:  an external link enclosed in an extra pair of brackets (see issue #120)

  326. input:  "[[http://example.com foo bar]]"

  327. output: [Text(text="["), ExternalLinkOpen(brackets=True), Text(text="http://example.com"), ExternalLinkSeparator(), Text(text="foo bar"), ExternalLinkClose(), Text(text="]")]

  328. 

  329. ---

  330. 

  331. name:   wikilink_to_external_link_fallback_2

  332. label:  an external link enclosed in an extra pair of brackets (see issue #120)

  333. input:  "[[http://example.com]]"

  334. output: [Text(text="["), ExternalLinkOpen(brackets=True), Text(text="http://example.com"), ExternalLinkClose(), Text(text="]")]

  335. 

  336. ---

  337. 

  338. name:   tables_in_templates

  339. label:  catch error handling mistakes when wikitables are inside templates

  340. input:  "{{hello|test\n{|\n|} }}"

  341. output: [TemplateOpen(), Text(text="hello"), TemplateParamSeparator(), Text(text="test\n"), TagOpenOpen(wiki_markup="{|"), Text(text="table"), TagCloseOpen(padding="\n"), TagOpenClose(wiki_markup="|}"), Text(text="table"), TagCloseClose(), Text(text=" "), TemplateClose()]

  342. 

  343. ---

  344. 

  345. name:   tables_in_templates_2

  346. label:  catch error handling mistakes when wikitables are inside templates

  347. input:  "{{hello|test\n{|\n| }}"

  348. output: [TemplateOpen(), Text(text="hello"), TemplateParamSeparator(), Text(text="test\n{"), TemplateParamSeparator(), Text(text="\n"), TemplateParamSeparator(), Text(text=" "), TemplateClose()]