---
title: "Extending a grammar · Docs · tabnas"
description: "Add to a language that already parses, instead of forking it."
source: "https://tabnas.dev/docs/extending/"
---

# Extending a grammar

Extension is the point of the engine, and the cheapest way to get a parser. The question to ask before writing a grammar is always: _what already parses something close to this?_

## Derive, don’t mutate

`make()` produces a fresh instance. Changes to it leave the original alone, so other code using the base parser is unaffected.

**A fresh instance**

TypeScript:

```ts
import { Jsonic } from '@tabnas/jsonic'

// `make()` produces a fresh instance. Changes to it leave the original alone,
// so other code using the base parser is unaffected.
const mine = Jsonic.make()

console.log('derived  ', JSON.stringify(mine('a:1')))    // the derived instance
console.log('original ', JSON.stringify(Jsonic('a:1')))  // still the original
```

Go:

```go
package main

import (
	"encoding/json"
	"fmt"

	jsonic "github.com/tabnas/jsonic/go"
)

func show(v any, err error) string {
	if err != nil {
		panic(err)
	}
	b, _ := json.Marshal(v)
	return string(b)
}

func main() {
	// `Make()` produces a fresh instance. Changes to it leave the shared
	// package-level parser alone, so other code using it is unaffected.
	mine := jsonic.Make()

	fmt.Println("derived  ", show(mine.Parse("a:1")))   // the derived instance
	fmt.Println("original ", show(jsonic.Parse("a:1"))) // still the original
}
```

Explain: Derive, don't mutate. `make()` returns a fresh instance that starts out identical, so anything you add or remove afterwards affects only your copy and never the parser other code is using. That isolation is what makes extension safe enough to be the default way of building with tabnas.

Output: `derived {"a":1} · original {"a":1}`

## Add a plugin

The common case. A plugin is a function that modifies a grammar: adding rules, adding alternates to existing rules, registering tokens.

**Before and after the plugin**

TypeScript:

```ts
import { Jsonic } from '@tabnas/jsonic'
import { Expr } from '@tabnas/expr'

// A plugin is a function that modifies a grammar — adding rules, adding
// alternates to existing rules, registering tokens.
// Cast: jsonic and expr publish separate copies of the Plugin type.
const cfg = Jsonic.make().use(Expr as any)

// The op node's readable part is its `src`; abbreviate the tree to that.
function simplify(n: any): any {
  if (Array.isArray(n)) return [n[0].src, ...n.slice(1).map(simplify)]
  if (n && 'object' === typeof n) {
    return Object.fromEntries(Object.entries(n).map(([k, v]) => [k, simplify(v)]))
  }
  return n
}

console.log('base  ', JSON.stringify(Jsonic('x: 1+2*3')))
console.log('+Expr ', JSON.stringify(simplify(cfg('x: 1+2*3'))))
```

Go:

```go
package main

import (
	"encoding/json"
	"fmt"

	tabnasexpr "github.com/tabnas/expr/go"
	jsonic "github.com/tabnas/jsonic/go"
)

func show(v any) string {
	b, _ := json.Marshal(v)
	return string(b)
}

func main() {
	// A plugin is a function that modifies a grammar — adding rules, adding
	// alternates to existing rules, registering tokens.
	cfg := jsonic.Make()
	if err := cfg.Use(tabnasexpr.Expr); err != nil {
		panic(err)
	}

	base, err := jsonic.Parse("x: 1+2*3")
	if err != nil {
		panic(err)
	}
	out, err := cfg.Parse("x: 1+2*3")
	if err != nil {
		panic(err)
	}

	fmt.Println("base  ", show(base))
	// Simplify abbreviates each op node to its source string.
	fmt.Println("+Expr ", show(tabnasexpr.Simplify(out)))
}
```

Explain: Adding a plugin is the common case of extension. Base jsonic reads `1+2*3` as a string; with `Expr` layered on, the same input becomes an expression tree with precedence already handled, and the base parser is untouched. TypeScript abbreviates the op nodes with a four-line walker; the Go port ships `Simplify`, which does the same thing.

Output: `base {"x":"1+2*3"} · +Expr {"x":["+",1,["*",2,3]]}`

[A grammar with plugins](https://tabnas.dev/docs/grammar-with-plugins/) walks through this in full. The available plugins are on the [packages](https://tabnas.dev/docs/packages/) page.

## Inspect what you started with

Before changing a grammar, look at it. `rule()` with no arguments lists the rules on an instance:

**Ask the instance**

TypeScript:

```ts
import { Jsonic } from '@tabnas/jsonic'

// Before changing a grammar, look at it. `rule()` with no arguments lists the
// rules on an instance — possible because the grammar is still data at runtime
// rather than generated code.
const rules = Object.keys(Jsonic.make().rule())

console.log(rules.sort().join(' '))
console.log('rule count:', rules.length)
```

Go:

```go
package main

import (
	"fmt"
	"sort"
	"strings"

	jsonic "github.com/tabnas/jsonic/go"
)

func main() {
	// Before changing a grammar, look at it. `RSM()` lists the rules on an
	// instance — possible because the grammar is still data at runtime rather
	// than generated code.
	rules := []string{}
	for name := range jsonic.Make().RSM() {
		rules = append(rules, name)
	}
	sort.Strings(rules)

	fmt.Println(strings.Join(rules, " "))
	fmt.Println("rule count:", len(rules))
}
```

Explain: Five rules: that is the whole of JSON's structure, and the reason extending it is tractable. You can ask a live instance what it holds because the grammar never stopped being data. A Go map has no order, so the Go version sorts; the TypeScript listing is sorted to match.

Output: `elem list map pair val · rule count: 5`

Five rules: that is the whole of JSON’s structure, and the reason extending it is tractable. For more, [@tabnas/debug](https://github.com/tabnas/debug) describes a live grammar and prints it back as ABNF, and [@tabnas/railroad](https://github.com/tabnas/railroad) draws it.

## Remove a rule

Passing `null` prunes a rule. This is how a stricter dialect is built from a looser one: the JSON grammar is partly jsonic with the relaxations removed.

**Prune a rule, narrow the language**

TypeScript:

```ts
import { Jsonic } from '@tabnas/jsonic'

// Passing `null` prunes a rule. This is how a stricter dialect is built from a
// looser one.
const strict = Jsonic.make()
strict.rule('list', null)

console.log('rules left:', Object.keys(strict.rule()).sort().join(' '))

for (const src of ['{a:1}', '[1,2]']) {
  let verdict = 'accepted'
  try { strict(src) } catch { verdict = 'rejected' }
  console.log(src.padEnd(7), verdict)
}
```

Go:

```go
package main

import (
	"fmt"
	"sort"
	"strings"

	jsonic "github.com/tabnas/jsonic/go"
	tabnas "github.com/tabnas/parser/go"
)

func main() {
	// A nil rule entry prunes that rule. This is how a stricter dialect is built
	// from a looser one.
	strict := jsonic.Make()
	err := strict.Grammar(&tabnas.GrammarSpec{
		Rule: map[string]*tabnas.GrammarRuleSpec{"list": nil},
	})
	if err != nil {
		panic(err)
	}

	rules := []string{}
	for name := range strict.RSM() {
		rules = append(rules, name)
	}
	sort.Strings(rules)
	fmt.Println("rules left:", strings.Join(rules, " "))

	for _, src := range []string{"{a:1}", "[1,2]"} {
		verdict := "accepted"
		if _, err := strict.Parse(src); err != nil {
			verdict = "rejected"
		}
		fmt.Printf("%-7s %s\n", src, verdict)
	}
}
```

Explain: Pruning a rule narrows a language: drop `list` and the dialect keeps objects but no longer parses arrays. Subtracting is how a stricter grammar is built from a looser one: the JSON grammar is partly jsonic with the relaxations removed. TypeScript spells the removal as `rule(name, null)`; Go passes a nil entry in a `GrammarSpec`, which means the same thing.

Output: `rules left: elem map pair val · {a:1} accepted · [1,2] rejected`

## What extension looks like in practice

The published grammars are the worked examples, and most are short because each builds on the last:

| Package | Extends | By adding |
| --- | --- | --- |
| [json](https://github.com/tabnas/json) | the engine | the five rules of JSON |
| [jsonc](https://github.com/tabnas/jsonc) | json | comments |
| [jsonic](https://github.com/tabnas/jsonic) | json | unquoted keys, implicit structure, trailing commas |
| [csv](https://github.com/tabnas/csv) | jsonic | record and field rules |
| [ini](https://github.com/tabnas/ini) | jsonic | sections and `key=value` |

Reading one of these diffs is the fastest way to understand the mechanism.

## When not to extend

If the thing you’re parsing shares no structure with anything published, start from [your first grammar](https://tabnas.dev/docs/first-grammar/) instead. Extension is cheap when there’s a real relationship and confusing when there isn’t: a grammar pretending to descend from JSON because JSON was nearby will fight you.

## See also

-   [The rule table](https://tabnas.dev/docs/rule-table/): what a plugin manipulates.
-   [Attaching actions](https://tabnas.dev/docs/actions/): adding behaviour to rules you didn’t write.
-   [Examples](https://tabnas.dev/examples/): two languages built this way.