Ben Kurtovic
8923d96a57
More unification.
il y a 11 ans
Ben Kurtovic
5e8e050ca3
A few tweaks; py3k support now complete.
il y a 11 ans
Ben Kurtovic
db86176c08
wiki_markup attr should be unicode, not bytes
il y a 11 ans
Ben Kurtovic
b5ec7f3beb
Fix py3k module importing; stick a bunch of macros in one place.
il y a 11 ans
Ben Kurtovic
e02ad8239f
Make load_entitydefs() work on Python 3.
il y a 11 ans
Ben Kurtovic
25d53cacf8
Begin porting C tokenizer to Python 3.
il y a 11 ans
Ben Kurtovic
be5d2cbb07
Support HTML entities inside parser-blacklisted tags ( closes #36 )
il y a 11 ans
Ben Kurtovic
ebf99d722c
Combine emit()/emit_first() internally.
il y a 11 ans
Ben Kurtovic
95efa7dde9
emit_FAST() -> emit(); emit() -> emit_kwargs()
il y a 11 ans
Ben Kurtovic
a07a96d4ba
Finish emit()'s kwargs version.
il y a 11 ans
Ben Kurtovic
6036dc9d62
Finish new emit_first() and emit_first_kwargs()
il y a 11 ans
Ben Kurtovic
51ac97de04
Make macros out of the failing/unsafe contexts.
il y a 11 ans
Ben Kurtovic
df9f7388b6
emit_FAST(), emit_first_FAST(); update comment parsing
il y a 11 ans
Ben Kurtovic
36180a9e47
To clarify usage, emit_text() -> emit_char() and emit_string() -> emit_text()
il y a 11 ans
Ben Kurtovic
c1379d5f21
Add a emit_string() as a shortcut; a bunch of minor cleanup.
il y a 11 ans
Ben Kurtovic
bbcb906f37
handle_dl_term()
il y a 11 ans
Ben Kurtovic
9993ffe8bf
handle_hr()
il y a 11 ans
Ben Kurtovic
c20d3f2a6a
handle_list_marker() and handle_list()
il y a 11 ans
Ben Kurtovic
34b3756285
parse_italics_and_bold() and parse_style()
il y a 11 ans
Ben Kurtovic
22d7ad032a
emit_style_tag, parse_italics, parse_bold
il y a 11 ans
Ben Kurtovic
9b98907751
Add C hooks and prototypes for wiki-markup tags.
il y a 11 ans
Ben Kurtovic
07d49c680a
Fix resetting of fields when doing multiple tokenizations.
il y a 11 ans
Ben Kurtovic
e83f321340
Rearrange functions; remove useless prototypes.
il y a 11 ans
Ben Kurtovic
9a2556b9fa
Actually write to buffer with invalid tag starts.
il y a 11 ans
Ben Kurtovic
41d66a4a7f
Don't decrement refcount with PyList_SetItem().
il y a 11 ans
Ben Kurtovic
bb4cf03e98
Fix unsafe context definitions.
il y a 11 ans
Ben Kurtovic
2f55654f01
Route failure shouldn't be consumed before it reaches Tokenizer_parse_tag().
il y a 11 ans
Ben Kurtovic
90ef3206f3
Properly handle the first space after a tag name.
il y a 11 ans
Ben Kurtovic
e3fc27c9e3
Refactor TagData code into dedicated functions.
il y a 11 ans
Ben Kurtovic
30c3f27b0e
Correctly set TagOpenData's default values.
il y a 11 ans
Ben Kurtovic
e32063ef4a
Reset routes properly.
il y a 11 ans
Ben Kurtovic
fd8a530259
Implement Tokenizer_handle_single_only_tag_end(), Tokenizer_handle_single_tag_end()
il y a 11 ans
Ben Kurtovic
4c5cfe57b8
Implement Tokenizer_handle_invalid_tag_start()
il y a 11 ans
Ben Kurtovic
c0f83fc803
Implement Tokenizer_handle_tag_close_close()
il y a 11 ans
Ben Kurtovic
b900f84420
Implement Tokenizer_handle_tag_open_close()
il y a 11 ans
Ben Kurtovic
e6b5c46d9c
Implement Tokenizer_handle_tag_close_open()
il y a 11 ans
Ben Kurtovic
b3f7728c31
Implement Tokenizer_handle_tag_text(), Tokenizer_handle_blacklisted_tag()
il y a 11 ans
Ben Kurtovic
d02a6da81e
Implement Tokenizer_handle_tag_space(); refactor textbuffer writing.
- Add a test for very long strings of text.
il y a 11 ans
Ben Kurtovic
9365fcf6e4
Implement Tokenizer_handle_tag_data(); add a read-backwards macro.
il y a 11 ans
Ben Kurtovic
e636bf77cf
Implement Tokenizer_push_tag_buffer()
il y a 11 ans
Ben Kurtovic
c42f67ed5e
Implement Tokenizer_parse_tag(), Tokenizer_really_parse_tag()
il y a 11 ans
Ben Kurtovic
653071379b
Finish porting misc changes; add prototypes for remaining functions.
il y a 11 ans
Ben Kurtovic
aca0f78cd7
Port more Python tokenizer updates to C.
il y a 11 ans
Ben Kurtovic
f67cf46900
Start C port of tag tokenization; refactor the init func.
il y a 11 ans
Ben Kurtovic
2596e697ae
Fix a possible compiler warning on some build systems.
il y a 11 ans
Ben
a689467577
Replace broken log2 function; add a missing comment.
il y a 11 ans
Ben Kurtovic
22e869b142
Fix a failing HTML entity test in the C tokenizer.
Remove some extraneous whitespace in string_mixin.py.
il y a 11 ans
Ben Kurtovic
496475c977
Whoops, that should be one larger ( #25 ).
il y a 11 ans
Ben Kurtovic
9ede1121ba
Fix tokenizer.c on Windows; add another template test ( #25 )
Mostly by @gdooms , with tweaks.
il y a 11 ans
Ben Kurtovic
debcb6577e
Fix recursion issues by giving up at a certain point ( closes #16 ).
- Stop parsing new templates if the template depth gets above
MAX_DEPTH (40) or if we've already tried to parse over MAX_CYCLES
(100,000) templates.
- Add two tests to ensure recursion works somewhat correctly.
- Fix parsing the string "{{" with the Python tokenizer; add a test.
il y a 11 ans