Your first grammar#

The quickstart parsed arithmetic. This one builds a grammar that depends on nothing but the engine (no existing language underneath it) and takes it as far as a usable tree. It should take about ten minutes.

We’ll parse a comma-separated list: a,bc,def.

1 · Install#

npm install @tabnas/parser @tabnas/abnf

2 · Write the grammar#

Three rules, in ABNF:

import { Tabnas } from '@tabnas/parser'import { abnf } from '@tabnas/abnf'const tn = new Tabnas({ plugins: [abnf] })tn.abnf(`  list  = item *( COMMA item )  item  = 1*ALPHA  COMMA = ","`)// `ALPHA` is an RFC 5234 core rule, pulled in because the grammar refers to it.// `*( … )` is sugar: it desugars into generated rules you did not write, which// is what to remember when attaching actions later.const rules = Object.keys(tn.rule() as object)console.log('ALPHA pulled in:', rules.includes('ALPHA'))console.log('rules generated by desugaring:', rules.filter((n) => n.startsWith('_gen')).length)

package mainimport (	"fmt"	"strings"	abnf "github.com/tabnas/abnf/go"	tabnas "github.com/tabnas/parser/go")func main() {	j := tabnas.Make()	_, err := abnf.Install(j, `  list  = item *( COMMA item )  item  = 1*ALPHA  COMMA = ","`, nil, nil)	if err != nil {		panic(err)	}	// `ALPHA` is an RFC 5234 core rule, pulled in because the grammar refers to	// it. `*( … )` is sugar: it desugars into generated rules you did not write,	// which is what to remember when attaching actions later.	alpha, generated := false, 0	for name := range j.RSM() {		if "ALPHA" == name {			alpha = true		}		if strings.HasPrefix(name, "_gen") {			generated++		}	}	fmt.Println("ALPHA pulled in:", alpha)	fmt.Println("rules generated by desugaring:", generated)}

Reading it: a list is an item, followed by zero or more (*) groups of a comma and another item. An item is one or more (1*) letters. ALPHA is one of the RFC 5234 core rules, included automatically when you refer to it.

COMMA is declared as its own rule so the comma becomes a named token rather than an anonymous literal, which means it shows up in the tree, and you can attach behaviour to it later.

3 · Parse#

import { Tabnas } from '@tabnas/parser'import { abnf } from '@tabnas/abnf'const tn = new Tabnas({ plugins: [abnf] })tn.abnf(`  list  = item *( COMMA item )  item  = 1*ALPHA  COMMA = ","`)// Whatever the grammar, a parse returns the same { rule, src, kids } node —// which is why a walker written once works for every language you define.for (const src of ['a', 'a,bc,def']) {  const n = tn.parse(src) as any  console.log(`rule=${n.rule} src=${JSON.stringify(n.src)}`)}

package mainimport (	"fmt"	abnf "github.com/tabnas/abnf/go"	tabnas "github.com/tabnas/parser/go")func main() {	j := tabnas.Make()	_, err := abnf.Install(j, `  list  = item *( COMMA item )  item  = 1*ALPHA  COMMA = ","`, nil, nil)	if err != nil {		panic(err)	}	// Whatever the grammar, a parse returns the same { rule, src, kids } node —	// which is why a walker written once works for every language you define.	for _, src := range []string{"a", "a,bc,def"} {		out, err := j.Parse(src)		if err != nil {			panic(err)		}		n := out.(map[string]any)		fmt.Printf("rule=%s src=%q\n", n["rule"], n["src"])	}}

Every parse returns the same shape ({ rule, src, kids }) whatever the grammar. Walk it once and the walker works for every language you define.

4 · Check that it rejects#

This is the step people skip, and it’s the one that catches a broken grammar. A grammar that accepts everything looks exactly like a grammar that works.

import { Tabnas } from '@tabnas/parser'import { abnf } from '@tabnas/abnf'const tn = new Tabnas({ plugins: [abnf] })tn.abnf(`  list  = item *( COMMA item )  item  = 1*ALPHA  COMMA = ","`)// The step people skip. A grammar that accepts everything looks exactly like a// grammar that works, so try the malformed input too.for (const src of ['a', 'a,bc,def', 'a,,b', ',a', 'a,']) {  let verdict = 'accepted'  try { tn.parse(src) } catch { verdict = 'rejected' }  console.log(JSON.stringify(src).padEnd(11), verdict)}

package mainimport (	"fmt"	abnf "github.com/tabnas/abnf/go"	tabnas "github.com/tabnas/parser/go")func main() {	j := tabnas.Make()	_, err := abnf.Install(j, `  list  = item *( COMMA item )  item  = 1*ALPHA  COMMA = ","`, nil, nil)	if err != nil {		panic(err)	}	// The step people skip. A grammar that accepts everything looks exactly like	// a grammar that works, so try the malformed input too.	for _, src := range []string{"a", "a,bc,def", "a,,b", ",a", "a,"} {		verdict := "accepted"		if _, err := j.Parse(src); err != nil {			verdict = "rejected"		}		fmt.Printf("%-11q %s\n", src, verdict)	}}

If a,,b had parsed, the *( COMMA item ) group would be matching a comma without requiring an item after it. Always try the malformed input.

5 · See inside it#

Two packages read a live grammar, which is possible because the grammar is still data at runtime rather than generated code:

npm install @tabnas/debug @tabnas/railroad

@tabnas/debug can describe the grammar and print it back as ABNF: if the round-trip isn’t what you wrote, the grammar isn’t what you meant. @tabnas/railroad draws it as a syntax diagram.

You can also paste the grammar straight into the playground and watch the tree change as you type.

What you skipped#

This grammar depends on two behaviours:

• The lexer. ALPHA and the number token NR are built in, and COMMA is a fixed token. Beyond that you configure or write matchers yourself: tabnas separates lexing from parsing and is mostly about the parsing half.
• Repetition is desugared. *( … ) compiles to generated group rules, so the runtime rule names aren’t only the ones you wrote. That matters when you attach actions: see attaching actions.

Next#

• A grammar with plugins: the other way to build, and usually the cheaper one.
• Attaching actions: turn a tree into a value.
• The rule table: what the ABNF compiled into.