How to #
Short guides for the problems that come up while writing a parser. Each one takes a single task, solves it against a running grammar, and says what it costs. They assume you have already parsed something — the quickstart is five minutes if you haven't.
How these are written #
A guide here is task-shaped, not feature-shaped. The title is the thing you are trying to do; the page is how to do it and what it costs. Where a published package already solves the problem, the guide uses it and says so, rather than reimplementing it for the sake of an example — extension is the point of the engine, and it applies to the documentation too.
Every code sample on these pages has been executed against the
published packages, and the values in the // => comments
are what the engine actually returned. Where a package behaves in a way
that will surprise you, the guide says so instead of quietly avoiding
it.
The reference solutions are the real grammars. When a guide points at @tabnas/csv or @tabnas/expr, it is pointing at a grammar under a couple of thousand lines that you can read in one sitting — usually the fastest way to answer a question this section didn't anticipate.
Before you write a grammar #
The question to ask before any of this is what already parses something close to my format? A new grammar is the expensive answer, and usually the wrong one. The packages page lists what exists; the comparisons page is honest about when another tool is the better answer entirely.
Guides #
Composing grammars #
Assemble a language out of pieces that already work — other sources, expression syntax, and plugins that take options.
- Include one source from another Splice a file, a package or an in-memory string into a parse at the point it is referenced. @tabnas/multisource@tabnas/directive
- Parse expressions with precedence Add infix, prefix, suffix and ternary operators to a grammar, with a binding-power scale you control. @tabnas/expr
- Write a parameterised parser One plugin, many dialects — take options and let them decide the tokens, the lexer and the rules. @tabnas/directive@tabnas/csv@tabnas/expr
Shaping the parse #
The rule table itself: how repetition and nesting are expressed, how the engine picks an alternate, and what changes when newlines matter.
- Handle recursion and repetition Repeat without nesting, nest without recursing forever, and get left recursion past a push-down engine. @tabnas/abnf
- Choose between alternates Order, lookahead, conditions, counters and group tags — how the engine picks a branch, and how to make it pick yours.
- Parse a line-oriented format Make newlines significant — records, sections and one-statement-per-line syntax. @tabnas/csv
Feeding the lexer #
Everything that happens before the rules run — the tokens your language needs, and the ones it should throw away.
- Lex a token the engine doesn't know Fixed literals, regex tokens, value literals, and a hand-written matcher for the cases none of those reach.
- Handle comments and whitespace Turn comment styles on, define your own, and decide what the parser is allowed to throw away.
- Handle strings, quotes and escapes Change what quotes a string, which escapes exist, and what happens to the ones that don't. @tabnas/csv
Working on a grammar #
Seeing what a grammar does, telling the reader what went wrong, and keeping it honest.
- Debug a grammar See the rules you actually have, watch a parse step by step, and draw the result. @tabnas/debug@tabnas/railroad
- Give good parse errors Name the file, define your own error codes, and raise them from the alternate that knows what went wrong.
- Test a grammar Assert what parses, what doesn't, and that the grammar is still the shape you think it is. @tabnas/abnf@tabnas/debug
Also in the docs #
Three task-oriented pages live under /docs because they double as the way in to ABNF, actions and extension. They belong to this section as much as to that one.
If none of these fit #
The rule table reference is the whole format in one page, and is short enough to read end to end. The playground runs a grammar in the browser, so a guess can be checked in a few seconds. Failing both, open an issue — a question this section should have answered is a bug in this section.