Parsers Don't Have to Be Complicated: A Tiny Scanner That Replaced an INI Library
Years of ad-hoc parsing code—manual loops, pointer arithmetic, and repeated edge-case bugs—led the author to create bx::Scanner, a zero-copy, allocation-free scanner that handles the repetitive parts of parsing without a generator or dependency. With a small API built around character classes and a single cursor, it powers line reading, INI parsing, URL parsing, path normalization, and stack trace symbolication, replacing an INI library and three hand-rolled loops. The design emphasizes simplicity, readability in a debugger, and making common bugs unrepresentable.
Bounding the inner scanner to a single line makes “run past the end of a malformed line” unrepresentable rather than merely unlikely.
- zabzonk
FORTH parsers are ultra simple - get the next space-separated token, if it is a number, push it on the stack, otherwise it's a word - look it up in the dictionary and (if it exists there) execute it.
- f311a
Unfortunately, simple URL parsing breaks on so many things. There is a reason on why every URL parsing library is at least a few thousand LOCs.
One common way to test it is just to pass ipv6 url: http://[f021:d981:b487:e57d:193e:550e::]/
- langbn
A parser/compiler could obviously be improved with an LLM (AI!) to suggest improvements to invalid input. That is actually a super good use case of LLM/AI.
Having clang/gcc, or any other parser, implement that is of course impossible, they are too conservative and would rather die than to implement modern helpful tools.
- imoverclocked
The hardest thing about writing a parser is cognitively accepting what is going to be considered valid input. You can make the best parser that is fast and well specified but invariably someone will (ab)use it in an unexpected way.
Famous examples: despite so many initial good intentions, html tags don’t need to be closed, JSON numbers are too often encoded as strings, YAML can look like what most people expect or it can look progressively more like JSON… and on and on.
- mrkeen
If you draw a line from 'ad-hoc byte-wrangling nonsense' to 'parser combinators', this can't be more than 20% along it.
Looking at the linked URL parser, why doesn't it look like
url = do scheme
authority
path
query
fragment
where
scheme = ...
authority = ...
etc.
It looks totally ad-hoc.