Ben Kurtovic
d8814968b7
Applying latest commit from develop
11 lat temu
Ben Kurtovic
fb71f5507e
Support a 'use_c' field to explicitly disable the C tokenizer.
11 lat temu
Ben Kurtovic
054a84afe0
A bit of misc cleanup.
11 lat temu
Ben Kurtovic
718fcb24c8
Fix eight failing tests; all template parsing tests now passing ( #25 ).
11 lat temu
Ben Kurtovic
5a0a00ba98
Change the way verify_safe() handles template params ( #25 ).
- Newlines are now allowed in template param names.
- Changes also affect handling of arguments like {{{foo}}}.
- Update unit tests: remove some unnecessary ones, and add some to cover the changes.
- Update StringMixIn tests to actually work for some of the methods.
- Update copyright notices for the C extensions.
11 lat temu
Ben Kurtovic
9a87329d69
More tests for test_other_methods()
11 lat temu
Ben Kurtovic
6e748004d1
test_types(), test_other_magics(); add range to compat
11 lat temu
Ben Kurtovic
221af8a9d7
Adding some tests to TestStringMixIn
11 lat temu
Ben Kurtovic
0803417901
Port CTokenizer's verify_safe method to Python to solve a failing test.
11 lat temu
Ben Kurtovic
acb7e57904
Make mwparserfromhell.parser() be an alias for parse_anything().
Some other changes, including removal of the 'string' import in the tokenizer.
11 lat temu
Ben Kurtovic
28b124a96c
Restarting work on unit tests with a test covering doc examples.
11 lat temu
Ben Kurtovic
357e421fe8
Text nodes should now appear a bit better in tree form.
11 lat temu
Ben Kurtovic
d6f2723a06
Fix safety checks on template params in some odd cases ( closes #24 ).
Also, fix parsing of wikilinks in both tokenizers such that newlines
in any location within the title are an automatic failure.
11 lat temu
Ben Kurtovic
0ee505b5a5
Docstrings for new tokenizer methods.
11 lat temu
Ben Kurtovic
cd5cc6a7d0
Update copyright notices for 2013.
11 lat temu
Ben Kurtovic
11cf5def75
Fix handling of sections headers with equal signs ( closes #20 )
11 lat temu
Ben Kurtovic
6ea618460f
_get_tag_type_from_stack() makes more sense now
11 lat temu
Ben Kurtovic
6881caf0bd
Fix StringMixIn's methods taking option arguments (thanks Sigma).
11 lat temu
Ben Kurtovic
eed7c918bf
Implement padding support for Tags completely; open_padding->padding.
11 lat temu
Ben Kurtovic
a58c480639
Fix some usage of attrs; shorten a context, fix some behavior I broke.
11 lat temu
Ben Kurtovic
146d1fd006
Fix a bug in rendering Tags; attrs->attributes; update documentation.
11 lat temu
Ben Kurtovic
ca47305074
Fix attribute behavior under certain strange circumstances.
11 lat temu
Ben Kurtovic
26d30f3d1a
Seems to be working for quoted attributes now.
11 lat temu
Ben Kurtovic
d459899649
More attribute stuff.
11 lat temu
Ben Kurtovic
d9f23b8faa
Really basic, messy, and fragile tag attribute support.
11 lat temu
Ben Kurtovic
06d3036de2
Fix a certain bug in SmartList with Py3k ( closes #17 )
11 lat temu
Ben Kurtovic
252cc13a99
Move repeated context checks into one block in Tokenizer._parse().
11 lat temu
Ben Kurtovic
a21c69fa1e
Split off tag definitions into a new file.
11 lat temu
Ben Kurtovic
827c544721
Should correctly handle closing tags with strange spacing.
11 lat temu
Ben Kurtovic
f78bcf832a
Keep .type and .tag synchronized in Tags when using their setters.
11 lat temu
Ben Kurtovic
7e46601b1d
Tags should fully work now in tokenizer and builder.
Still need to do attributes.
11 lat temu
Ben Kurtovic
05ec7a1a92
Improve padding support for Tags; more code for tags in tokenizer.
11 lat temu
Ben Kurtovic
d1a9ba9a34
Starting tag work.
- Translation dict, contexts, parse_* and handle_* hooks in tokenizer.
11 lat temu
Ben Kurtovic
dab6cc39ef
Fix handling of errors when parsing template param keys ( closes #13 )
11 lat temu
Ben Kurtovic
1f47e10f04
Start some Py3k compatibility stuff.
11 lat temu
Ben Kurtovic
ca11d77efd
Shorten some longer lines.
11 lat temu
Ben Kurtovic
4a725b7ac5
Fix another couple bugs regarding template contexts and verify_safe().
11 lat temu
Ben Kurtovic
1abdb478c9
Fix a couple bugs.
11 lat temu
Ben Kurtovic
88a2b95030
Remove a bunch of white space.
11 lat temu
Ben Kurtovic
3a6335d969
Clean up init_tokenizer().
11 lat temu
Ben Kurtovic
06f02b9753
Clean up whitespace / newlines.
11 lat temu
Ben Kurtovic
0e78571672
Forgot to check numeric/hexadecimal entities for string chars.
11 lat temu
Ben Kurtovic
a1e6dfcef6
Ensure that there is null terminator at the end of entitydefs.
11 lat temu
Ben Kurtovic
19e7c3b6ac
Fix type.
11 lat temu
Ben Kurtovic
06b20dd8c0
More updates.
11 lat temu
Ben Kurtovic
16024494cf
Mostly implement Tokenizer_really_parse_entity().
11 lat temu
Ben Kurtovic
087d606097
Recover failing contexts when due to a template or link opening.
11 lat temu
Ben Kurtovic
7de34ed5a6
Rewrite verify_safe completely.
11 lat temu
Ben Kurtovic
b2f933dddc
Fix memory leak.
11 lat temu
Ben Kurtovic
76b299e4fa
Sync from upstream ( 6413ae21da
); some doc updates.
11 lat temu