Ben Kurtovic
90bd12dd47
Fix a C tokenizer crash when parsing is interrupted ( fixes #97 )
il y a 8 ans
Ben Kurtovic
8e7a600b51
Fix use-after-free bug.
il y a 9 ans
Ben Kurtovic
5eac0ab16f
More PEP 393 work; update Textbuffer interface and usage.
il y a 9 ans
Ben Kurtovic
2072a10b67
More reworking of CTokenizer Unicode support (incomplete)
il y a 9 ans
Ben Kurtovic
7345a3742e
Fix a thread safety issue involving route state.
il y a 9 ans
Ben Kurtovic
2005efd309
Split up C tokenizer into tag_data, tok_parse, tok_support, tokens.
il y a 9 ans
Ben Kurtovic
0e547aa416
Begin splitting up C tokenizer.
il y a 9 ans
Ben Kurtovic
dad042bc2c
Fix C warnings in MSVC.
il y a 9 ans
Ben Kurtovic
1d5bbbe25b
Disallow < and > in wikilink titles/template names ( fixes #104 )
il y a 9 ans
Ben Kurtovic
e71e7b4ece
Update copyright years for 2015; fix whitespace in docs.
il y a 9 ans
Ben Kurtovic
a00c645bd8
Fix handling of tag closes within <nowiki> ( fixes #89 ).
il y a 9 ans
Ben Kurtovic
a15f6172c0
Minor bugfix.
il y a 9 ans
Ben Kurtovic
9fc4b909e1
Refactor a lot of table error recovery code.
il y a 9 ans
Ben Kurtovic
fb261450d8
Port tokenizer updates to C.
il y a 9 ans
Ben Kurtovic
640005dbb2
Tokenizer cleanup; make inline table syntax invalid as it should be.
il y a 9 ans
Ben Kurtovic
913ff590c8
Cleanup; add a missing test.
il y a 9 ans
Ben Kurtovic
5d29bff918
Remove an incorrect usage of Py_XDECREF().
il y a 9 ans
Ben Kurtovic
7489253e32
Break at 80 cols for most lines.
il y a 9 ans
David Winegar
1a4c88e11f
Correctly handle no table endings
Tests were not correctly testing the situations without a table close.
Fixed tests and then fixed tokenizers for failing tests. Also refactored
pytokenizer to more closely match the ctokenizer by only holding the
`_parse` methods in the try blocks and no other code.
il y a 10 ans
David Winegar
c63108039b
Fix C code to make declarations before statements
Python 3.4 compiles C extensions with the
`-Werror=declaration-after-statement` flag that enforces C90 more
strictly than previous versions. Move all statements after declarations
to make sure this extension builds on 3.4.
il y a 10 ans
David Winegar
213c105666
Table tags are no longer self-closing
Table tags no longer self-closing. Rows and cells now contain their
contents. Also refactored out an `emit_table_tag` method.
Note: this will require changes to the Tag node and possibly the builder,
those changes will be in the next commit.
il y a 10 ans
David Winegar
0128b1f78a
Implement CTokenizer for tables
CTokenizer is completely implemented in this commit - it didn't
make much sense to me to split it up. All tests passing, memory test
shows no leaks on Linux.
il y a 10 ans
David Winegar
2d945b30e5
Use uint64_t for context
For the C tokenizer, include `<stdint.h>` and use `uint64_t` instead
of `int` for context. Changes to tables mean that context can be
larger than 32 bits, and it is possible for `int` to only have 16
bits anyways (though this is very unlikely).
il y a 10 ans
Ben Kurtovic
6954480263
Fix template parsing when comments are inside the name ( fixes #59 ).
il y a 10 ans
Ben Kurtovic
ded89fb14e
Add a few unit tests for untested code; remove a useless conditional.
il y a 10 ans
Ben Kurtovic
b997e4cd71
Support attributes quoted with '; add required quotes in value setter.
il y a 10 ans
Ben Kurtovic
a4c2fd023a
Remove some useless code in the tokenizers.
il y a 10 ans
Ben Kurtovic
08cafc0576
Raise ParserError for internal problems. Improve coverage. Cleanup.
il y a 10 ans
Ben Kurtovic
02eff0fc49
Fully fix #74 . Add another tokenizer test.
il y a 10 ans
Ben Kurtovic
0497b54f03
Fix _handle_single_tag_end()'s token search order ( fixes #74 )
il y a 10 ans
Ben Kurtovic
5c5fd6b3cb
Fix a bug involving nested links ( closes #61 and #62 ).
il y a 10 ans
Ben Kurtovic
1312a1fb8a
Some clean up for Python 2.6 support.
* Removed unittest2 dependency on Python >2.6.
* Moved discover_tests.py into tests/.
* tokenizer.c: Fixed errors noted by -Wshorten-64-to-32.
il y a 10 ans
Ben Kurtovic
e5f17eea00
Update copyright notices for 2014.
il y a 10 ans
Ben Kurtovic
1946cf621d
Add a temporary skip_style_tags until we resolve some issues.
il y a 10 ans
Ben Kurtovic
066049b46a
Update email address.
il y a 10 ans
Ben Kurtovic
38050f6878
C code cleanup and speed improvements.
il y a 10 ans
Ben Kurtovic
951a8737a5
Don't pass underlying context if this is a bracketed link.
il y a 10 ans
Ben Kurtovic
287bf71158
Condense code.
il y a 10 ans
Ben Kurtovic
1bf9868753
Proper sentinel handling with free links in the C tokenizer.
il y a 10 ans
Ben Kurtovic
77092e066c
Fix C tokenizer behavior re: some single_only tag edge cases.
il y a 10 ans
Ben Kurtovic
6784ff73bf
Fix an edge case when we recurse too deeply.
il y a 10 ans
Ben Kurtovic
4d04cae780
Fix a segfault with GCC.
il y a 10 ans
Ben Kurtovic
67f1762aa4
Doc updates, and allow passing a starting context to tokenize().
il y a 10 ans
Ben Kurtovic
f1b95758d6
Squash a memory leak.
il y a 10 ans
Ben Kurtovic
2561cf5b5e
Fix all bugs in C implementation of external links.
il y a 10 ans
Ben Kurtovic
c1b502bbe6
Finish external links implementation.
il y a 10 ans
Ben Kurtovic
7dcfa3fe92
Implement Tokenizer_really_parse_external_link(), some other fixes
il y a 10 ans
Ben Kurtovic
6ecf15cad4
Tokenizer_parse_external_link()
il y a 10 ans
Ben Kurtovic
a1948b06aa
Tokenizer_parse_bracketed/free_uri_scheme(), other adjustments
il y a 10 ans
Ben Kurtovic
7b84b3f0df
Refactor out C's is_marker(); hooks for ext links.
il y a 10 ans