Fix signals getting stuck inside the C tokenizer (#206)

CHANGELOG

@@ -1,6 +1,8 @@
 v0.6 (unreleased):
 
 - Dropped support for end-of-life Python versions 2.6, 3.2, 3.3. (#199, #204)
+- Fixed signals getting stuck inside the C tokenizer until parsing finishes,
+  in pathological cases. (#206)
 - Fixed <wbr> not being considered a single-only tag. (#200)
 
 v0.5.1 (released March 3, 2018):

docs/changelog.rst

@@ -10,6 +10,9 @@ Unreleased
 - Dropped support for end-of-life Python versions 2.6, 3.2, 3.3.
   (`#199 <https://github.com/earwig/mwparserfromhell/issues/199>`,
   `#204 <https://github.com/earwig/mwparserfromhell/pull/204>`)
+- Fixed signals getting stuck inside the C tokenizer until parsing finishes,
+  in pathological cases.
+  (`#206 <https://github.com/earwig/mwparserfromhell/issues/206>`)
 - Fixed `<wbr>` not being considered a single-only tag.
   (`#200 <https://github.com/earwig/mwparserfromhell/pull/200>`)
 

mwparserfromhell/parser/ctokenizer/tok_parse.c

@@ -2603,6 +2603,8 @@ PyObject* Tokenizer_parse(Tokenizer* self, uint64_t context, int push)
         }
         if (!this)
             return Tokenizer_handle_end(self, this_context);
+        if (PyErr_CheckSignals())
+            return NULL;
         next = Tokenizer_read(self, 1);
         last = Tokenizer_read_backwards(self, 1);
         if (this == next && next == '{') {