ABNF grammars

The @tabnas/abnf plugin compiles RFC 5234 ABNF straight into a working grammar. It’s the fastest way to define a language.

One line of ABNF

import { Tabnas } from '@tabnas/parser'import { abnf } from '@tabnas/abnf'const tn = new Tabnas({ plugins: [abnf] })tn.abnf(`greet = "hi" / "hello"`)// One line of ABNF is a working parser. Every parse is the same// { rule, src, kids } node — and anything else is rejected.for (const src of ['hi', 'hello', 'howdy']) {  try {    const n = tn.parse(src) as any    console.log(`${src.padEnd(6)} ${n.rule} ${JSON.stringify(n.src)} kids=${n.kids.length}`)  } catch {    console.log(`${src.padEnd(6)} rejected`)  }}
package mainimport (	"fmt"	abnf "github.com/tabnas/abnf/go"	tabnas "github.com/tabnas/parser/go")func main() {	j := tabnas.Make()	if _, err := abnf.Install(j, `greet = "hi" / "hello"`, nil, nil); err != nil {		panic(err)	}	// One line of ABNF is a working parser. Every parse is the same	// { rule, src, kids } node — and anything else is rejected.	for _, src := range []string{"hi", "hello", "howdy"} {		out, err := j.Parse(src)		if err != nil {			fmt.Printf("%-6s rejected\n", src)			continue		}		n := out.(map[string]any)		fmt.Printf("%-6s %s %q kids=%d\n", src, n["rule"], n["src"], len(n["kids"].([]any)))	}}

One ABNF line compiles to a working parser: greet matches either literal and yields the uniform { rule, src, kids } node. Anything else is a parse error, which is the half of a grammar worth checking.

TypeScript throws on a bad parse; Go returns an error from Parse: the same outcome, reported the way each language reports failure.

output hi greet "hi" kids=0 · hello greet "hello" kids=0 · howdy rejected

Dialect

tabnas uses the RFC 5234 dialect: = for definitions and / for alternatives (not ::= or |).

  • Literals: "+", case-insensitive by default; %s"Hi" is case-sensitive.
  • Optional: [ … ].
  • Repetition: *element (zero or more), 1*element (one or more), 2*4element (bounded).
  • Grouping: ( … ).
  • Char ranges: %x30-39.
  • Built-in tokens by bareword: NR (number), plus core rules like ALPHA and DIGIT.

Left recursion

The compiler rewrites direct (P = P a / b) and indirect left recursion into iterative rules. See the left-recursion elimination algorithm. For example:

P = P a / b     →     P = b *(a)

Because it’s a rewrite, the tree is flat rather than left-nested, and a purely left-recursive rule (no base branch) is an error. See the @tabnas/abnf README for the full details and caveats.

Actions

Attach behaviour with @ref action references, passed as actions and keyed by rule and phase. There are two forms:

  • '@add:o:NR': an alternate action: runs when the add rule opens on an NR token. The trailing mark is the alternate’s leading discriminator.
  • '@add:bo': a rule-phase hook: before-open. Also ao, bc and ac for after-open, before-close and after-close.

Actions by alternate mark

import { Tabnas } from '@tabnas/parser'import { abnf } from '@tabnas/abnf'const tn = new Tabnas({ plugins: [abnf] })tn.abnf(`  val = add  add = NR [ PL add ]  PL  = "+"`, {  actions: {    // `val` holds the running total.    '@val:o:add': (r: any) => { r.node.value = 0 },    // Each number adds to it.    '@add:o:NR': (r: any) => { r.parent.node.value += r.o[0].val },  },})// `r.parent` is `val` for every repetition, so the total lands in one place.for (const src of ['1+2+3', '12+3+45']) {  console.log(src.padEnd(8), '=>', (tn.parse(src) as any).value)}
package mainimport (	"fmt"	abnf "github.com/tabnas/abnf/go"	tabnas "github.com/tabnas/parser/go")func main() {	j := tabnas.Make()	_, err := abnf.Install(j, `  val = add  add = NR [ PL add ]  PL  = "+"`, nil, abnf.ActionsMap{		// `val` holds the running total.		"@val:o:add": {func(r *tabnas.Rule, ctx *tabnas.Context) {			r.Node.(map[string]any)["value"] = float64(0)		}},		// Each number adds to it.		"@add:o:NR": {func(r *tabnas.Rule, ctx *tabnas.Context) {			node := r.Parent.Node.(map[string]any)			node["value"] = node["value"].(float64) + r.O[0].Val.(float64)		}},	})	if err != nil {		panic(err)	}	// `r.Parent` is `val` for every repetition, so the total lands in one place.	for _, src := range []string{"1+2+3", "12+3+45"} {		out, err := j.Parse(src)		if err != nil {			panic(err)		}		fmt.Printf("%-8s => %v\n", src, int(out.(map[string]any)["value"].(float64)))	}}

Actions bind to alternate marks: @val:o:add is val's open alternate that pushes add, @add:o:NR is add's open alternate on a number token. The ABNF text itself stays valid RFC 5234.

The tail self-reference [ PL add ] compiles to a same-depth repeat, so r.parent is val for every number and the total accumulates in one node.

The one real difference: a number token's val is a JS number in TypeScript and a float64 in Go, so the Go version asserts the type before adding.

output 1+2+3 => 6 · 12+3+45 => 60

r.parent is val for every repetition: a tail self-reference like [ PL add ] compiles to a same-depth repeat of add, not a nested push.

r is the rule instance and r.o the tokens matched in the open phase, so r.o[0].val is the value of the first: a real number, courtesy of the lexer.

Finding the marks

An alternate mark comes from the alternate’s leading discriminator, which the compiler assigns, so don’t guess at the name, ask for it:

tabnas-abnf --marks -f grammar.abnf
val  o:add  p:add
val  c:_  (empty)
add  o:NR  s:#NR
add  c:PL  s:#PL
add  c:_  (empty)

markListing(spec) gives the same listing from code. Reading it also tells you the shape the compiler produced: val o:add p:add says val pushes add as a child rule, and add c:PL s:#PL says add repeats itself from its close phase when a + follows: the tail self-reference [ PL add ] compiled to a same-depth repeat, exactly what a hand-written grammar declares as { s: '#PL', r: 'add' }.

Requires @tabnas/abnf 0.3.0 or later (and @tabnas/parser 0.5.0). Earlier versions compiled the tail into generated option/group rules, nesting each repetition; 0.2.x additionally dissolved pure aliases like val = add.

Parents and generated rules

For a tail self-reference, r.parent is the wrapping rule for every repetition: accumulating onto r.parent.node is the intended idiom, the same as in a hand-written rule table.

Other sugar still desugars into generated group rules: inside ( … ) or *( … ) a rule’s parent at runtime may be a _gen* rule you didn’t write. The mark listing shows the compiled rule set when in doubt.

Describes @tabnas/parser 0.12.9 · all pinned versions