Ben Kurtovic
c20d3f2a6a
handle_list_marker() and handle_list()
пре 11 година
Ben Kurtovic
9b98907751
Add C hooks and prototypes for wiki-markup tags.
пре 11 година
Ben Kurtovic
4663563ce4
Remove unnecessary markers.
пре 11 година
Ben Kurtovic
e83f321340
Rearrange functions; remove useless prototypes.
пре 11 година
Ben Kurtovic
e3fc27c9e3
Refactor TagData code into dedicated functions.
пре 11 година
Ben Kurtovic
d02a6da81e
Implement Tokenizer_handle_tag_space(); refactor textbuffer writing.
- Add a test for very long strings of text.
пре 11 година
Ben Kurtovic
9365fcf6e4
Implement Tokenizer_handle_tag_data(); add a read-backwards macro.
пре 11 година
Ben Kurtovic
e636bf77cf
Implement Tokenizer_push_tag_buffer()
пре 11 година
Ben Kurtovic
653071379b
Finish porting misc changes; add prototypes for remaining functions.
пре 11 година
Ben Kurtovic
aca0f78cd7
Port more Python tokenizer updates to C.
пре 11 година
Ben Kurtovic
f67cf46900
Start C port of tag tokenization; refactor the init func.
пре 11 година
Ben
a689467577
Replace broken log2 function; add a missing comment.
пре 11 година
Ben Kurtovic
9ede1121ba
Fix tokenizer.c on Windows; add another template test ( #25 )
Mostly by @gdooms , with tweaks.
пре 11 година
Ben Kurtovic
debcb6577e
Fix recursion issues by giving up at a certain point ( closes #16 ).
- Stop parsing new templates if the template depth gets above
MAX_DEPTH (40) or if we've already tried to parse over MAX_CYCLES
(100,000) templates.
- Add two tests to ensure recursion works somewhat correctly.
- Fix parsing the string "{{" with the Python tokenizer; add a test.
пре 11 година
Ben Kurtovic
054a84afe0
A bit of misc cleanup.
пре 11 година
Ben Kurtovic
718fcb24c8
Fix eight failing tests; all template parsing tests now passing ( #25 ).
пре 11 година
Ben Kurtovic
5a0a00ba98
Change the way verify_safe() handles template params ( #25 ).
- Newlines are now allowed in template param names.
- Changes also affect handling of arguments like {{{foo}}}.
- Update unit tests: remove some unnecessary ones, and add some to cover the changes.
- Update StringMixIn tests to actually work for some of the methods.
- Update copyright notices for the C extensions.
пре 11 година
Ben Kurtovic
0803417901
Port CTokenizer's verify_safe method to Python to solve a failing test.
пре 11 година
Ben Kurtovic
d6f2723a06
Fix safety checks on template params in some odd cases ( closes #24 ).
Also, fix parsing of wikilinks in both tokenizers such that newlines
in any location within the title are an automatic failure.
пре 11 година
Ben Kurtovic
1f47e10f04
Start some Py3k compatibility stuff.
пре 11 година
Ben Kurtovic
4a725b7ac5
Fix another couple bugs regarding template contexts and verify_safe().
пре 11 година
Ben Kurtovic
06b20dd8c0
More updates.
пре 11 година
Ben Kurtovic
16024494cf
Mostly implement Tokenizer_really_parse_entity().
пре 11 година
Ben Kurtovic
7de34ed5a6
Rewrite verify_safe completely.
пре 11 година
Ben Kurtovic
cdef073a9b
Replace textbuffer with our own datastructure.
пре 11 година
Ben Kurtovic
f0a36f3262
Store tokens in a global variable instead of retrieving anew each time.
пре 11 година
Ben Kurtovic
6d73eeeab1
Replace Python list of stacks with a singly linked list.
пре 11 година
Ben Kurtovic
ff0b4439f8
Replace Python exception machinery with a global and some macros.
пре 11 година
Ben Kurtovic
8e4819ea37
Fix exception implementation; some other cleanup and fixes.
пре 11 година
Ben Kurtovic
9993f1ba76
Fix object calls.
пре 11 година
Ben Kurtovic
0be18bc5b8
Fix Py_UNICODE comparisons.
пре 11 година
Ben Kurtovic
17a09e395a
Fix some bugs.
пре 11 година
Ben Kurtovic
48188bfa99
Implement Tokenizer_parse_heading(), some other stuff.
пре 11 година
Ben Kurtovic
849016f734
Implement Tokenizer_verify_safe() and some others.
пре 11 година
Ben Kurtovic
0d720a7ef1
Implement Tokenizer_parse_template(); NOARGS and NOKWARGS.
пре 11 година
Ben Kurtovic
6edc24037e
Implement Tokenizer_parse_template_or_argument().
пре 11 година
Ben Kurtovic
7fc45783b7
Add a header file; improve context handling.
пре 11 година