Skip to content

Compare a Fail-Open Parser Against a Strict Reader, and Allow-List the Writer's Keys

Problem

board.sh decides which projects appear on the Spec-Board from intake/projects.yaml. It reads the file with pr_parse_registry, a byte-pinned line parser from SPEC-195 that fails open: when it does not attach a board: false to its item, that item counts as public.

Review passes 5 to 10 each found valid YAML that the parser reads differently from YAML. The first fixes added refused shapes (passes 5–7), and the next ones listed accepted shapes (passes 8–9). Neither approach converged. Pass 10 found three findings that no shape list could have named: the text around the projects: block, the caller's locale and a duplicate key. That made five FAIL passes in a row, each fixing the previous pass's examples.

Root cause

A shape list describes the inputs the author has thought of. The parser's actual input space is all valid YAML, which is much larger and has more dimensions. Each fix closed one point in that space and left the dimension it sat on open.

Resolution

1. Compare against a strict reader instead of describing shapes (Decision 22). Parse the file once with the parser the board uses, then read it again with a strict YAML reader (plugins/vt-base/lib/spec_yaml.py). Refuse the whole file (exit 2, naming the line or item) unless both readers return the same items in the same order, with the same compared fields and the same on/off value. The compare runs on the parse the board actually uses, in the caller's locale. Where the readers can differ (yes, on, ~, a trailing comment), YAML decides, and a disagreement stops the board. No shape is ever published on the parser's reading alone.

2. Agreement is blind to a key that both readers skip; allow-list the writer's keys. Passes 11 and 12 found a gap the compare cannot see. Both readers treat a misspelt or lookalike key as absent: Board:, board followed by U+034F or U+3164, a fullwidth ｂ, Path:. So they agree, and the private item is published. Pass 11 tried to normalise (NFKC, drop non-printables, case-fold), and characters Python counts as printable still got through. The fix that held is an allow-list: refuse any item key outside the 11 that registry_write.sh writes.

3. Isolate the checker from its caller. It runs as python3 -I -B, so neither a re.py in the working directory nor PYTHONPATH can replace the modules it imports, and it writes no bytecode.

Prevention

  • When a fail-open reader guards a privacy or permission decision, stop patching shapes after the second shape-level fix. Compare its result with a strict reader and refuse on disagreement.
  • A differential check only covers what both readers can see. For every field the decision depends on, ask what happens when both readers miss it. Close that with an allow-list taken from the writer, not with normalisation.
  • Test the allow-list with lookalike keys from several Unicode classes (zero-width, combining, fullwidth, Hangul filler, variation selectors, Cyrillic), plus a negative control. The negative control is an item carrying all writer keys, which must pass. See also measure-a-hidden-character-filter-against-the-whole-code-point-space.
  • Known residual (PAT-P13-1): the allow-list is a hand copy of the writer's keys. Deriving it from the writer is still open.