Ben Kurtovic
e99c9d3038
More tag refactoring; fix some bugs.
11 vuotta sitten
Ben Kurtovic
5e8794da5e
Refactor more of the tag tokenization process.
11 vuotta sitten
Ben Kurtovic
dd6bb1637d
Support tag nesting properly; unit tests; recursion checks for tags.
11 vuotta sitten
Ben Kurtovic
9693b6d5e6
Replace data.literal and data.quoted with a data.CX_QUOTED context
11 vuotta sitten
Ben Kurtovic
e34026dabe
Support templates and wikilinks inside <open> tags (part 2)
11 vuotta sitten
Ben Kurtovic
dfe100ceb7
Support templates and wikilinks inside <open> tags (part 1)
11 vuotta sitten
Ben Kurtovic
f63480bcf3
Update the integration.rich_tags test to use the new tag tokens.
Remove an now-unused import in the tokenizer.
11 vuotta sitten
Ben Kurtovic
82edc93bbb
Pass some tests by simplifying the way tags are read from the stack.
Two still fail because templates aren't implemented yet, but those
are otherwise handled correctly.
11 vuotta sitten
Ben Kurtovic
962adcd62c
Add docstrings for a couple new methods in the tokenizer.
11 vuotta sitten
Ben Kurtovic
5f5a081d91
Rewrite tag parser to be cleaner and safer.
All tag tests passing. Still need to finish backslash support and
support for templates and tags within <open> tags.
11 vuotta sitten
Ben Kurtovic
81e8fdd682
Give Attributes more attributes for padding data.
11 vuotta sitten
Ben Kurtovic
ce27d5d385
Fix six failing tests; add three more (all passing).
11 vuotta sitten
Ben Kurtovic
6450814729
Remove 'type' attribute from tags; rework tag definitions.
11 vuotta sitten
Ben Kurtovic
58d9194a2c
Version bump for v0.3; fix permissions on compat.py.
11 vuotta sitten
Ben Kurtovic
edf6a3a8a6
release/0.2
11 vuotta sitten
Ben Kurtovic
bbaf09dbf8
Fix docstrings of generated filter methods.
11 vuotta sitten
Ben
a689467577
Replace broken log2 function; add a missing comment.
11 vuotta sitten
Ben Kurtovic
3fb8f3214c
Fix StringMixIn.maketrans() on Py3k.
- Make a test in Py3k actually use StringMixIn instead of str.
- Minor cosmetic fix.
11 vuotta sitten
Ben Kurtovic
d2b3954669
Fix remaining broken tests; some refactoring.
11 vuotta sitten
Ben Kurtovic
9ea06c2830
Push the textbuffer to fix a couple broken tests.
11 vuotta sitten
Ben Kurtovic
1b4c01b4c0
Implement assertTagNodeEqual(), start test_tag(), add to tags.mwtest.
11 vuotta sitten
Ben Kurtovic
61fc5b5eab
Fix handling of self-closing tags ( closes #31 )
11 vuotta sitten
Ben Kurtovic
22e869b142
Fix a failing HTML entity test in the C tokenizer.
Remove some extraneous whitespace in string_mixin.py.
11 vuotta sitten
Ben Kurtovic
4ae10eccf5
Fix bugs in get_sections(); finish test_get_sections()
11 vuotta sitten
Ben Kurtovic
496475c977
Whoops, that should be one larger ( #25 ).
11 vuotta sitten
Ben Kurtovic
9ede1121ba
Fix tokenizer.c on Windows; add another template test ( #25 )
Mostly by @gdooms , with tweaks.
11 vuotta sitten
Ben Kurtovic
35acc1b812
Fix a couple bugs.
11 vuotta sitten
Ben Kurtovic
0b56f2e267
Improve Wikicode.get_sections(); implement test_get_tree(); part of test_get_sections()
11 vuotta sitten
Ben Kurtovic
f700914caf
Cleanup Wikicode's filter functions; implement test_filter_family().
11 vuotta sitten
Ben Kurtovic
17ac79e796
Build filter methods dynamically.
11 vuotta sitten
Ben Kurtovic
3095a4203f
Finish tests for Wikicode's list-like methods; fix a bug.
11 vuotta sitten
Ben Kurtovic
ee99e6eceb
Fix a bug in Wikicode.set(); implement test_set() and test_index()
11 vuotta sitten
Ben Kurtovic
852c5ff9af
Start TestWikicode; make Wikicode.nodes's setter handle more inputs.
11 vuotta sitten
Ben Kurtovic
81849013bc
Finishing tests for Templates; some fixes.
11 vuotta sitten
Ben Kurtovic
2ca3b2805e
Implement 'before' parameter for Template.add() ( closes #21 )
11 vuotta sitten
Ben Kurtovic
b46c98b012
Clean up template.add(); add a before param but do not implement yet.
11 vuotta sitten
Ben Kurtovic
786d619274
Drop force_no_field in template.remove(); implement test_remove().
- Also add tests for spacing in param names.
11 vuotta sitten
Ben Kurtovic
debcb6577e
Fix recursion issues by giving up at a certain point ( closes #16 ).
- Stop parsing new templates if the template depth gets above
MAX_DEPTH (40) or if we've already tried to parse over MAX_CYCLES
(100,000) templates.
- Add two tests to ensure recursion works somewhat correctly.
- Fix parsing the string "{{" with the Python tokenizer; add a test.
11 vuotta sitten
Ben Kurtovic
6a385b3921
TestParameter and a fair chunk of TestTemplate.
11 vuotta sitten
Ben Kurtovic
6bf8cfd2ad
Fix tests for Python 3.
11 vuotta sitten
Ben Kurtovic
8db40689ed
Improve a few things about nodes; simply a method in Wikicode.
11 vuotta sitten
Ben Kurtovic
404b4479a2
Implement the remaining asserts in TreeEqualityTestCase.
11 vuotta sitten
Ben Kurtovic
1b69b5e882
Moving compat stuff exclusively for unit tests to its own file.
11 vuotta sitten
Ben Kurtovic
32ac6958e1
Apply some bugfixes to SmartList to fix tests on Python 3.
- Add a _SliceNormalizerMixIn to properly handle slices.
- Use floor division when applying key.step.
- Implement sort() without 'cmp' parameter.
- Fix bytes(list) behavior.
- Children of _ListProxies are now _ListProxies, not regular lists.
11 vuotta sitten
Ben Kurtovic
7f87a1c4b3
Apply bugfixes so that some tests pass on Python 3.
- Skip CTokenizer tests if CTokenizer is not available.
- TestStringMixin: Don't make assumptions about default encoding.
- Add urllib stuff to mwparserfromhell.compat.
- Fix compat.py's line endings.
- gen.next() -> next(gen)
- assert*Equals() -> assert*Equal()
11 vuotta sitten
Ben Kurtovic
f803269514
Add a USES_C field to the tokenizers; add TestParser.test_use_c()
11 vuotta sitten
Ben Kurtovic
6a741db7ce
Applying fb71f5507e
12 vuotta sitten
Ben Kurtovic
34b85a93cd
Cosmetic change: standardize whitespace after class definition.
12 vuotta sitten
Ben Kurtovic
b298a68b37
Squash a bug dealing with extended slices.
12 vuotta sitten
Ben Kurtovic
740db6ddfa
Implement some more tests; squash bugs in SmartList/_ListProxy
12 vuotta sitten