Ben Kurtovic
08cafc0576
Raise ParserError for internal problems. Improve coverage. Cleanup.
il y a 10 ans
Ben Kurtovic
5c5fd6b3cb
Fix a bug involving nested links ( closes #61 and #62 ).
il y a 10 ans
Ben Kurtovic
e5f17eea00
Update copyright notices for 2014.
il y a 10 ans
Ben Kurtovic
1946cf621d
Add a temporary skip_style_tags until we resolve some issues.
il y a 10 ans
Ben Kurtovic
066049b46a
Update email address.
il y a 10 ans
Ben Kurtovic
38050f6878
C code cleanup and speed improvements.
il y a 10 ans
Ben Kurtovic
1bf9868753
Proper sentinel handling with free links in the C tokenizer.
il y a 10 ans
Ben Kurtovic
fcdc0abd22
Fix autofail contexts.
il y a 10 ans
Ben Kurtovic
2561cf5b5e
Fix all bugs in C implementation of external links.
il y a 10 ans
Ben Kurtovic
7dcfa3fe92
Implement Tokenizer_really_parse_external_link(), some other fixes
il y a 10 ans
Ben Kurtovic
6ecf15cad4
Tokenizer_parse_external_link()
il y a 10 ans
Ben Kurtovic
a1948b06aa
Tokenizer_parse_bracketed/free_uri_scheme(), other adjustments
il y a 10 ans
Ben Kurtovic
7b84b3f0df
Refactor out C's is_marker(); hooks for ext links.
il y a 10 ans
Ben Kurtovic
d42e05a554
Implement improved wikilink handling.
il y a 10 ans
Ben Kurtovic
5e6e5b6301
tag_defs.py -> definitions.py; more outline stuff
il y a 10 ans
Ben Kurtovic
cbf67c7842
Add hooks for some ext link stuff; add a INVALID_LINK aggregate context.
il y a 10 ans
Ben Kurtovic
8923d96a57
More unification.
il y a 10 ans
Ben Kurtovic
b5ec7f3beb
Fix py3k module importing; stick a bunch of macros in one place.
il y a 10 ans
Ben Kurtovic
25d53cacf8
Begin porting C tokenizer to Python 3.
il y a 10 ans
Ben Kurtovic
ebf99d722c
Combine emit()/emit_first() internally.
il y a 10 ans
Ben Kurtovic
51ac97de04
Make macros out of the failing/unsafe contexts.
il y a 10 ans
Ben Kurtovic
df9f7388b6
emit_FAST(), emit_first_FAST(); update comment parsing
il y a 10 ans
Ben Kurtovic
c20d3f2a6a
handle_list_marker() and handle_list()
il y a 10 ans
Ben Kurtovic
9b98907751
Add C hooks and prototypes for wiki-markup tags.
il y a 10 ans
Ben Kurtovic
4663563ce4
Remove unnecessary markers.
il y a 10 ans
Ben Kurtovic
e83f321340
Rearrange functions; remove useless prototypes.
il y a 10 ans
Ben Kurtovic
e3fc27c9e3
Refactor TagData code into dedicated functions.
il y a 10 ans
Ben Kurtovic
d02a6da81e
Implement Tokenizer_handle_tag_space(); refactor textbuffer writing.
- Add a test for very long strings of text.
il y a 10 ans
Ben Kurtovic
9365fcf6e4
Implement Tokenizer_handle_tag_data(); add a read-backwards macro.
il y a 10 ans
Ben Kurtovic
e636bf77cf
Implement Tokenizer_push_tag_buffer()
il y a 11 ans
Ben Kurtovic
653071379b
Finish porting misc changes; add prototypes for remaining functions.
il y a 11 ans
Ben Kurtovic
aca0f78cd7
Port more Python tokenizer updates to C.
il y a 11 ans
Ben Kurtovic
f67cf46900
Start C port of tag tokenization; refactor the init func.
il y a 11 ans
Ben
a689467577
Replace broken log2 function; add a missing comment.
il y a 11 ans
Ben Kurtovic
9ede1121ba
Fix tokenizer.c on Windows; add another template test ( #25 )
Mostly by @gdooms , with tweaks.
il y a 11 ans
Ben Kurtovic
debcb6577e
Fix recursion issues by giving up at a certain point ( closes #16 ).
- Stop parsing new templates if the template depth gets above
MAX_DEPTH (40) or if we've already tried to parse over MAX_CYCLES
(100,000) templates.
- Add two tests to ensure recursion works somewhat correctly.
- Fix parsing the string "{{" with the Python tokenizer; add a test.
il y a 11 ans
Ben Kurtovic
054a84afe0
A bit of misc cleanup.
il y a 11 ans
Ben Kurtovic
718fcb24c8
Fix eight failing tests; all template parsing tests now passing ( #25 ).
il y a 11 ans
Ben Kurtovic
5a0a00ba98
Change the way verify_safe() handles template params ( #25 ).
- Newlines are now allowed in template param names.
- Changes also affect handling of arguments like {{{foo}}}.
- Update unit tests: remove some unnecessary ones, and add some to cover the changes.
- Update StringMixIn tests to actually work for some of the methods.
- Update copyright notices for the C extensions.
il y a 11 ans
Ben Kurtovic
0803417901
Port CTokenizer's verify_safe method to Python to solve a failing test.
il y a 11 ans
Ben Kurtovic
d6f2723a06
Fix safety checks on template params in some odd cases ( closes #24 ).
Also, fix parsing of wikilinks in both tokenizers such that newlines
in any location within the title are an automatic failure.
il y a 11 ans
Ben Kurtovic
1f47e10f04
Start some Py3k compatibility stuff.
il y a 11 ans
Ben Kurtovic
4a725b7ac5
Fix another couple bugs regarding template contexts and verify_safe().
il y a 11 ans
Ben Kurtovic
06b20dd8c0
More updates.
il y a 11 ans
Ben Kurtovic
16024494cf
Mostly implement Tokenizer_really_parse_entity().
il y a 11 ans
Ben Kurtovic
7de34ed5a6
Rewrite verify_safe completely.
il y a 11 ans
Ben Kurtovic
cdef073a9b
Replace textbuffer with our own datastructure.
il y a 11 ans
Ben Kurtovic
f0a36f3262
Store tokens in a global variable instead of retrieving anew each time.
il y a 11 ans
Ben Kurtovic
6d73eeeab1
Replace Python list of stacks with a singly linked list.
il y a 11 ans
Ben Kurtovic
ff0b4439f8
Replace Python exception machinery with a global and some macros.
il y a 11 ans