Ben Kurtovic
debcb6577e
Fix recursion issues by giving up at a certain point ( closes #16 ).
- Stop parsing new templates if the template depth gets above
MAX_DEPTH (40) or if we've already tried to parse over MAX_CYCLES
(100,000) templates.
- Add two tests to ensure recursion works somewhat correctly.
- Fix parsing the string "{{" with the Python tokenizer; add a test.
11 vuotta sitten
Ben Kurtovic
f803269514
Add a USES_C field to the tokenizers; add TestParser.test_use_c()
11 vuotta sitten
Ben Kurtovic
5a0a00ba98
Change the way verify_safe() handles template params ( #25 ).
- Newlines are now allowed in template param names.
- Changes also affect handling of arguments like {{{foo}}}.
- Update unit tests: remove some unnecessary ones, and add some to cover the changes.
- Update StringMixIn tests to actually work for some of the methods.
- Update copyright notices for the C extensions.
11 vuotta sitten
Ben Kurtovic
0803417901
Port CTokenizer's verify_safe method to Python to solve a failing test.
11 vuotta sitten
Ben Kurtovic
acb7e57904
Make mwparserfromhell.parser() be an alias for parse_anything().
Some other changes, including removal of the 'string' import in the tokenizer.
11 vuotta sitten
Ben Kurtovic
d6f2723a06
Fix safety checks on template params in some odd cases ( closes #24 ).
Also, fix parsing of wikilinks in both tokenizers such that newlines
in any location within the title are an automatic failure.
11 vuotta sitten
Ben Kurtovic
cd5cc6a7d0
Update copyright notices for 2013.
11 vuotta sitten
Ben Kurtovic
11cf5def75
Fix handling of sections headers with equal signs ( closes #20 )
11 vuotta sitten
Ben Kurtovic
29ef2a6682
Propogate fix from b447e7bfc1
11 vuotta sitten
Ben Kurtovic
b447e7bfc1
contexts.WIKILINK should also fail
11 vuotta sitten
Ben Kurtovic
ff0b4439f8
Replace Python exception machinery with a global and some macros.
11 vuotta sitten
Ben Kurtovic
28a25d9a52
Bugfix when text ends within param key context.
11 vuotta sitten
Ben Kurtovic
41535992a1
Implement Tokenizer_handle_template_param_value().
11 vuotta sitten
Ben Kurtovic
6edc24037e
Implement Tokenizer_parse_template_or_argument().
11 vuotta sitten
Ben Kurtovic
03120bc97e
Version bump for 0.2.dev; fix a documentation bug.
11 vuotta sitten
Ben Kurtovic
99e466857b
Support wikilinks in tokenizer.
11 vuotta sitten
Ben Kurtovic
65862befdc
Support comments in tokenizer.
11 vuotta sitten
Ben Kurtovic
ef0ed63e50
Properly handle newlines in template names with named parameters.
11 vuotta sitten
Ben Kurtovic
353122105d
Fix tokenizer so we don't destroy our textbuf writing an empty stack
* Also fix a bug in Argument
11 vuotta sitten
Ben Kurtovic
814ce9f290
Fix an assumption regarding templates mistaken as arguments.
11 vuotta sitten
Ben Kurtovic
70cf46ef0d
Simplify / refactor.
11 vuotta sitten
Ben Kurtovic
6b84a87c7d
Far more robust parsing of templates and template arguments.
11 vuotta sitten
Ben Kurtovic
c01f78aee1
Implement arguments in tokenizer - preliminary results are promising.
11 vuotta sitten
Ben Kurtovic
a735efd0bc
Template parameters containing newlines should be invalid ( closes #6 )
11 vuotta sitten
Ben Kurtovic
816207bb5a
Simplify docs massively.
11 vuotta sitten
Ben Kurtovic
bdf5608425
More documentation for various things.
11 vuotta sitten
Ben Kurtovic
1caeb2d570
Build the actual Sphinx documentation, plus a number of typo fixes.
11 vuotta sitten
Ben Kurtovic
e787c97712
Finish documenting all of the main things ( #5 ).
11 vuotta sitten
Ben Kurtovic
391a5bd373
Compat for the parser itself.
11 vuotta sitten
Riamse
2eb76e7de0
Adding Python 3 support
11 vuotta sitten
Ben Kurtovic
e57b6bdd93
Support Headings in tokenizer; handle tokens backwards in builder.
* Some other fixes, additions.
11 vuotta sitten
Ben Kurtovic
40b6480aa7
Split on all characters that we use.
11 vuotta sitten
Ben Kurtovic
664b6e39ec
_write(text=True) -> _write_text(); __eq__ for Tokens
11 vuotta sitten
Ben Kurtovic
278594a8cf
Faster parsing: split the text on sentinels instead of every letter.
11 vuotta sitten
Ben Kurtovic
aec66b0db7
Missed another call (2.0 seconds -> 1.5 seconds for 1,000,000 chars).
11 vuotta sitten
Ben Kurtovic
3fd13100da
Another speedup by reducing calls to _read().
11 vuotta sitten
Ben Kurtovic
889fd316d3
A rogue print statement appears!
11 vuotta sitten
Ben Kurtovic
31103b5891
_push_textbuffer() in _verify_template_name()
11 vuotta sitten
Ben Kurtovic
2f4ce003b4
Massive speedup and refactoring.
11 vuotta sitten
Ben Kurtovic
010bd34653
Support HTMLEntities.
11 vuotta sitten
Ben Kurtovic
02fe3b0833
Improve handling of templates in strange cases.
11 vuotta sitten
Ben Kurtovic
e79c79762f
Pre- and post-stop context verification, plus refactor parse_until().
11 vuotta sitten
Ben Kurtovic
8dd7b6aada
Handle nested contexts correctly; parameter values; bad parse routes.
11 vuotta sitten
Ben Kurtovic
934b1ef016
Fixes and improvements.
11 vuotta sitten
Ben Kurtovic
d3ea962d27
Starting work on token contexts.
11 vuotta sitten
Ben Kurtovic
2209dfc78d
Better handling of template ends, refactor.
11 vuotta sitten
Ben Kurtovic
32d99c3c75
Tokenizer now supports a very, very limited template syntax.
11 vuotta sitten
Ben Kurtovic
81e5ce30af
Working on the framework for the tokenizer, plus some cleanup, fixes.
11 vuotta sitten
Ben Kurtovic
4539859c55
Merge BuildStack into Builder.
11 vuotta sitten
Ben Kurtovic
8f0782f915
Fix some typos/bugs.
11 vuotta sitten