The C tokenizer was silently truncating input at the first NUL byte (\x00)
because it used '\0' both as a valid input character and as the EOF sentinel.

Root cause:
- Tokenizer_read() returned '\0' for both:
  1. Real NUL bytes in the input
  2. End-of-input (when index >= text.length)
- This made them indistinguishable, causing real NULs to be treated as EOF

Fix:
1. Define TOKENIZER_EOF as 0x110000 (first invalid Unicode code point)
2. Update Tokenizer_read() and Tokenizer_read_backwards() to return
   TOKENIZER_EOF instead of '\0' for out-of-bounds reads
3. Replace all `!this` and `'\0'` checks with explicit `TOKENIZER_EOF` checks
4. Remove '\0' from the MARKERS array (no longer needed as EOF marker)
5. Move EOF check before is_marker() in main parse loop to ensure
   TOKENIZER_EOF doesn't try to emit as a character
6. Fix Tokenizer_has_leading_whitespace() to recognize TOKENIZER_EOF

The Python tokenizer already preserved NUL bytes correctly; this brings
the C tokenizer into parity.

Regression test added: test_nul_byte_preservation() verifies that both
tokenizers now preserve NUL bytes in plain text, templates, and
multiple-NUL scenarios.

pytest is the preferred way to write and run unit tests these days and
it has a cleaner interface - so lets switch to it. The tokenizer tests
especially are much easier to read/understand.

This was mostly done with find/replace regexes and then cleaned up
manually.