Grammar: Verilog 2001 (synthesizable subset)

[lanserge]

Summary

Add a verilog_2001.rawast grammar covering the synthesizable subset of Verilog 2001 (IEEE 1364-2001) — modules, ports, declarations, continuous assignments, procedural blocks (always / initial), statements, instantiations, and the 17-level expression precedence chain.

This is the foundation for any future SystemVerilog work: SV-2017/2023 extends Verilog 2001 rather than replacing it, so a clean V2001 base unblocks SVA assertions (via subparse), classes, interfaces, packages, and constraints as incremental extensions.

A working sketch of the top-level rule structure lives at https://github.com/edacommons/rawast/discussions (paste from the design conversation) and demonstrates the patterns — module declaration, port-list Choice between ANSI and Verilog-95 styles, fan-out at MODULE_ITEM, the per-level expression precedence pattern, etc.

Why now

Three concrete use cases that exist today and want a bidirectional Verilog parser:

• Source rewriters / formatters — Verible, the open-source SV formatter, took years to build because it shipped its own emitter alongside its parser. rawast's bidirectional walk gives save-direction emit for free; a Verilog formatter on top is a thin layer.
• Lint / static-analysis tools — the AST is already the structured IR; tools written in Python (via the Python binding) can walk it without reimplementing parsing.
• EDA flow integration — netlist round-tripping for tool-chain wrappers; the bidirectional model means you can take a flat netlist, mutate the AST, and save it back without losing comments or whitespace structure.

The combination of "bidirectional" + "grammar as data" + "JSON-uniform AST" is unique to rawast among EDA-targeted parsers. Verilog is the canonical EDA input format; not having it is a real coverage gap.

Scope: in

Area	Notes
Module structure	module … endmodule, parameter port list, port list (ANSI + Verilog-95 styles)
Module items	Net / reg declarations, parameters, continuous assign, always blocks, initial blocks, module instantiations
Statements	Blocking and non-blocking assignments, if/else, case/casex/casez, begin/end blocks, for, while, repeat, disable, wait, force/release, system task calls
Expressions	Full 17-level precedence chain (assignment → ternary → ‖ → && → | → ^ → & → ==/!= → <=/< → shifts → +/- → */// % → ** → unary → primary)
Primaries	Numeric literals (every form), string literals, identifiers, bit/part selects, concatenation, function calls, system task calls ($display, etc.)
Event control	@(posedge clk or negedge rst_n), @*, @(a or b)
Generate blocks	generate / endgenerate, genvar, generate-loop, generate-if, generate-case
Tasks and functions	task / endtask, function / endfunction with their body grammars
Comments and whitespace	// line and /* */ block comments via the verilog parser group

Scope: out (for this issue)

Area	Why deferred
Preprocessor (`define, `include, `ifdef/`else/`endif)	Conceptually a separate language. Best handled as a separate preprocessing pass before parsing — what real SV tools do. Filed as a follow-up issue.
UDP declarations	Out of scope for synthesizable subset; primitives can be added if a use case appears.
Specify blocks	Analog timing arcs; rare in synthesizable code.
defparam	Generally banned in synthesizable RTL anyway.
SystemVerilog features (classes, SVA, interfaces, modports, constraints, covergroups)	Each gets its own issue. SVA is the most natural first SV extension via subparse.
The full IEEE 1364-2001 LRM grammar	~500 production rules; this issue covers the productive synthesizable subset, not the whole spec.

Pre-requisite: verilog parser group

A C++ parser group (src/parsers_verilog.cpp) with four terminals — same pattern as gdsii / lefdef / tcl:

class VerilogIdentifierParser : public Parser {
    // [a-zA-Z_][a-zA-Z_0-9$]* (simple) OR
    // \<chars-until-whitespace> (escaped) OR
    // $<simple-identifier>     (system task / function name)
};

class VerilogNumberParser : public Parser {
    // 8'hFF, 8'sb01, 'd42, 42, 42.5, 42.5ns, 1step — every numeric
    // form Verilog accepts. ~150 lines on its own; the gating piece.
};

class VerilogStringParser : public Parser {
    // "literal" with C-style escapes (\n, \t, \", \\, \<digit>, \xHH)
};

class VerilogCommentParser : public Parser {
    // // to EOL  AND  /* ... */ block (nestable behavior matches LRM)
};

The number parser is the load-bearing piece — Verilog has 7+ numeric literal syntactic forms. Comparable scope to the LEF/DEF specialised identifier parser.

Implementation phasing

Phase	Scope	Effort
0. Parser group	src/parsers_verilog.cpp + unit tests for each terminal	1 week
1. Module + port list + simple module items	Modules, both port-list styles, net/reg decls, continuous assigns. Smoke-test on simple netlists.	1 week
2. Procedural blocks + statements	always / initial, all statement kinds, event controls. Test against PicoRV32 RTL.	1 week
3. Expression chain	All 17 precedence levels. Decision: parens-required for mixed-op chains (Option C) vs left-associative fold step (post-parse). Document in grammar.	0.5 week
4. Module instantiation + generate	Hierarchical instantiation, generate-loop / -if / -case.	1 week
5. Corpus testing + bug fixes	OpenTitan, OpenCores, PicoRV32, RISC-V core RTL. Expect ~50 edge-case fixes.	2 weeks
6. Type-vs-variable disambiguation	Design decision (see below)	1 week

Total: ~6-7 weeks of focused work for production-grade synthesizable V2001 coverage.

Design decision: type-vs-variable identifier disambiguation

The classic Verilog ambiguity: foo bar; could be a variable declaration (foo is a type, bar is the new variable) OR a statement (foo is an expression, bar is something else — actually this specific form is illegal in V2001, but the general type-vs-variable distinction shows up in many places). Real SV parsers maintain a symbol table during parse and look up identifiers.

PEG can't do this naturally. Two viable paths in rawast:

Path A: Permissive parsing + post-AST disambiguation. Grammar accepts both interpretations as the same AST shape (e.g. {type: "ident_followed_by_ident", first: foo, second: bar}). A post-parse pass walks the AST with a symbol table and rewrites the ambiguous nodes into typed declarations vs. expressions.

• Pros: pure grammar, no callbacks, easy to test
• Cons: AST has a brief "ambiguous" phase that downstream tools have to understand

Path B: Mid-parse callbacks register types as they're declared. rawast already supports rule-completion callbacks (used by LEF's DIVIDERCHAR / BUSBITCHARS mechanism). A typedef-rule callback would register the new type name; the identifier parser would then know to dispatch differently for known type names.

• Pros: closer to how real SV parsers work; AST is immediately correct
• Cons: stateful parse (the type-set varies during the parse); test isolation is harder; the callback mechanism gets a serious workout (currently only used for LEF's 2 directives)

Recommendation: start with Path A. Simpler, deterministic, and the post-AST pass is just regular Python / C++ code. Migrate to Path B only if benchmarks show it matters.

Roadmap fit

• M1 (current) — typed Python developer surface; this issue extends the per-format grammar corpus, sits naturally in M1.
• M2 (.jast container) — once V2001 parses cleanly, real RTL designs can be shipped as .jast artifacts — pre-parsed AST + grammar, portable across tooling, no re-parse cost. Significant value for cloud-scale verification pipelines.

Out-of-scope follow-ups (file as separate issues if/when)

• verilog_preprocessor — `define, `include, `ifdef/`else/`endif, conditional compilation. Separate language, separate grammar.
• systemverilog.rawast — extends V2001 with classes, interfaces, packages, generics, etc. Probably 6-8 weeks on top of V2001.
• sva.rawast — SVA assertions and properties; loaded via subparse from SV-2017 grammar's assert property (...) expressions. ~2 weeks.
• Verilog formatter as a rawast.tools.verilog_format Python package, using the bidirectional save with pretty-print attributes on the grammar.

Related

• The sketch (~280 lines covering top-level structure) — paste below or attach.
• FastDRC constraint grammar discussion — established the patterns for direct {op, args} IR emission and operator-precedence handling that this grammar uses.
• LEF/DEF grammar (grammars/lefdef.rawast) — comparable scope; useful precedent for the parser-group + corpus-test pattern.
• Tcl grammar (grammars/tcl.rawast) — precedent for subparse + rule-local ignore, both of which will be useful for SV's embedded sub-languages later.

[lanserge]

Top-level structural sketch

Pasting the sketch referenced in the issue body so it's discoverable inline. ~280 lines covering the structural skeleton without trying to be complete — demonstrates the patterns, names what's missing, defers to the issue's phasing plan.

// verilog_2001.rawast — synthesizable Verilog 2001 grammar (sketch).
//
// Scope: synthesizable subset (modules, ports, declarations, always,
// assign, instances, basic expressions). NOT covered in this sketch:
// generate blocks, specify blocks, full primitive (UDP) declarations,
// the complete 17-level expression precedence chain, full system task
// argument grammar. These are mechanical extensions of the patterns
// shown here.
//
// Assumed parser groups (need to be implemented in C++ alongside the
// grammar — same pattern as `gdsii` / `lefdef` / `tcl`):
//
//   `verilog` group:
//     verilog_identifier — simple `[a-zA-Z_][a-zA-Z_0-9$]*` or escaped
//                          `\<chars-until-whitespace>` or system `$<id>`
//     verilog_number     — handles all literal forms: `42`, `8'hFF`,
//                          `4'b0101`, `'d42`, `1'sb1`, `42.5`, etc.
//     verilog_string     — double-quoted with C-style escapes
//     verilog_comment    — `//...` to EOL and `/*...*/` block

use: std, verilog

start: <SOURCE_TEXT>

// ─── File level ─────────────────────────────────────────────────────────
//
// A Verilog source file is a sequence of top-level descriptions —
// almost always module declarations. UDP and config are out of
// synthesizable scope, but the choice slot makes them easy to add.

SOURCE_TEXT ignore whitespace verilog_comment: sequence dict {
  repeat <DESCRIPTION>:descriptions[]=@
}

DESCRIPTION: choice {
  <MODULE_DECL>
  // future: <UDP_DECL>, <CONFIG_DECL>, <PACKAGE_DECL> (SV)
}

// ─── Module declaration ─────────────────────────────────────────────────
//
// Verilog has two port-list flavours:
//
//   ANSI-style (1364-2001):    module foo (input a, output b); ... endmodule
//   Verilog-95 style:          module foo (a, b);
//                                  input  a;
//                                  output b;
//                                  ...
//                              endmodule
//
// Choice between them is by the first port's shape: if the first
// `(...)` entry starts with a direction keyword (`input`/`output`/
// `inout`) or a type, it's ANSI-style; if it's just an identifier,
// it's old-style. Both work; alt-failure recovery picks the right path.
// Lint flags it as informational shared-prefix; accept it.

MODULE_DECL: sequence dict {
  'module':type="module",
  verilog_identifier:name=@,
  ?<PARAM_PORT_LIST>,
  ?<PORT_LIST>,
  ";" newline,
  repeat <MODULE_ITEM>:items[]=@,
  'endmodule' newline
}

// `#(parameter A = 4, parameter B = 8)`
PARAM_PORT_LIST: sequence dict {
  "#", "(",
  repeat+ <PARAM_DECL>:params[]=@ separator ",",
  ")"
}

PARAM_DECL: sequence dict {
  'parameter':type="parameter",
  ?<TYPE_SPEC>:type_spec=@,
  verilog_identifier:name=@,
  "=", <EXPR>:value=@
}

// Port list — Choice between two flavours.
PORT_LIST: choice {
  <PORT_LIST_ANSI>,
  <PORT_LIST_NONANSI>
}

PORT_LIST_ANSI: sequence dict {
  :style="ansi",
  "(",
  repeat+ <PORT_DECL_ANSI>:ports[]=@ separator ",",
  ")"
}

PORT_LIST_NONANSI: sequence dict {
  :style="nonansi",
  "(",
  repeat+ verilog_identifier:port_names[]=@ separator ",",
  ")"
}

// ANSI port: direction + optional type + name. Direction is a strict
// keyword so `inputA` is identifier "inputA", not "input" + "A".
PORT_DECL_ANSI: sequence dict {
  <PORT_DIRECTION>:direction=@,
  ?<TYPE_SPEC>:type_spec=@,
  ?<RANGE>:range=@,
  verilog_identifier:name=@
}

PORT_DIRECTION: choice {
  'input':@,
  'output':@,
  'inout':@
}

// `[7:0]` — packed dimension. Trivial today; will need broader
// dimension grammar for SV (multi-dim, type-parameterised).
RANGE: sequence dict {
  "[", <EXPR>:msb=@, ":", <EXPR>:lsb=@, "]"
}

TYPE_SPEC: choice {
  'wire':@, 'reg':@, 'logic':@, 'integer':@, 'real':@,
  'tri':@, 'tri0':@, 'tri1':@, 'wand':@, 'wor':@
}

// ─── Module items ───────────────────────────────────────────────────────
//
// Inside a module, items are mostly: declarations, continuous assigns,
// always/initial blocks, and instantiations. The big Choice fans out
// to per-item-kind rules.

MODULE_ITEM: choice {
  <NET_DECL>,
  <REG_DECL>,
  <PARAM_DECL_STMT>,
  <CONT_ASSIGN>,
  <ALWAYS_BLOCK>,
  <INITIAL_BLOCK>,
  <MODULE_INSTANTIATION>
  // future: <GENERATE_BLOCK>, <SPECIFY_BLOCK>, <TASK_DECL>,
  //        <FUNCTION_DECL>, <DEFPARAM>, <SVA_PROPERTY> (SV)
}

NET_DECL: sequence dict {
  :type="net_decl",
  <NET_KIND>:kind=@,
  ?<RANGE>:range=@,
  repeat+ verilog_identifier:names[]=@ separator ",",
  ";" newline
}

NET_KIND: choice {
  'wire':@, 'tri':@, 'tri0':@, 'tri1':@, 'wand':@, 'wor':@
}

REG_DECL: sequence dict {
  :type="reg_decl",
  'reg':kind="reg",
  ?<RANGE>:range=@,
  repeat+ <REG_NAME>:names[]=@ separator ",",
  ";" newline
}

REG_NAME: sequence dict {
  verilog_identifier:name=@,
  ?<RANGE>:packed=@         // for memory arrays: `reg [7:0] mem [0:255];`
}

PARAM_DECL_STMT: sequence dict {
  <PARAM_DECL>,             // reuses the param-port-list form
  ";" newline
}

// Continuous assignment: `assign x = y & z;`
CONT_ASSIGN: sequence dict {
  'assign':type="cont_assign",
  ?<DELAY>:delay=@,
  repeat+ <ASSIGNMENT>:assignments[]=@ separator ",",
  ";" newline
}

ASSIGNMENT: sequence dict {
  <LVALUE>:lhs=@, "=", <EXPR>:rhs=@
}

LVALUE: sequence dict { verilog_identifier:name=@, ?<RANGE>:bit_select=@ }

DELAY: sequence dict { "#", <EXPR>:value=@ }

// always / initial — the entry points to procedural blocks.
ALWAYS_BLOCK: sequence dict {
  'always':type="always",
  ?<EVENT_CONTROL>:sensitivity=@,
  <STATEMENT>:body=@
}

INITIAL_BLOCK: sequence dict {
  'initial':type="initial",
  <STATEMENT>:body=@
}

// `@(posedge clk or negedge rst_n)` or `@*` or `@(a or b)`
EVENT_CONTROL: sequence dict {
  "@",
  choice {
    sequence { "*":wildcard=true },
    sequence { "(", <EVENT_EXPR>:events=@, ")" }
  }
}

EVENT_EXPR: sequence array {
  repeat+ <EVENT_PRIMARY>:@ separator <EVENT_SEP>:@
}

EVENT_PRIMARY: sequence dict {
  ?<EDGE>:edge=@, <EXPR>:signal=@
}

EDGE: choice { 'posedge':@, 'negedge':@ }
EVENT_SEP: choice { ",":@, 'or':@ }

// Module instantiation: `foo #(param1, param2) inst_name(.a(x), .b(y));`
MODULE_INSTANTIATION: sequence dict {
  :type="instance",
  verilog_identifier:module_name=@,
  ?<PARAM_OVERRIDE>:params=@,
  <INSTANCE_LIST>:instances=@,
  ";" newline
}

PARAM_OVERRIDE: sequence dict {
  "#", "(",
  repeat+ <PARAM_BINDING>:@ separator ",",
  ")"
}

PARAM_BINDING: choice {
  // Named: `.PARAM(value)`
  sequence dict {
    :style="named",
    ".", verilog_identifier:name=@, "(", <EXPR>:value=@, ")"
  },
  // Positional: just an expression
  sequence dict { :style="positional", <EXPR>:value=@ }
}

INSTANCE_LIST: sequence array {
  repeat+ <INSTANCE>:@ separator ","
}

INSTANCE: sequence dict {
  verilog_identifier:name=@,
  ?<RANGE>:array_range=@,
  "(",
  repeat <PORT_BINDING>:port_bindings[]=@ separator ",",
  ")"
}

PORT_BINDING: choice {
  // Named: `.port(signal)` or `.port()` for unconnected
  sequence dict {
    :style="named",
    ".", verilog_identifier:port=@,
    "(", ?<EXPR>:signal=@, ")"
  },
  // Positional: just a signal expression
  sequence dict { :style="positional", <EXPR>:signal=@ }
}

// ─── Statements (inside always/initial) ─────────────────────────────────
//
// Sketch — the real grammar fans out to 15+ statement kinds. Shown
// here: blocking/non-blocking assignment, if, case, block.

STATEMENT: choice {
  <BLOCK_STMT>,
  <IF_STMT>,
  <CASE_STMT>,
  <BLOCKING_ASSIGN>,
  <NONBLOCKING_ASSIGN>,
  <PROC_CONT_ASSIGN>,
  <SYSTEM_TASK_CALL>
  // future: <FOR_STMT>, <WHILE_STMT>, <FORK_JOIN>, <REPEAT_STMT>,
  //        <WAIT_STMT>, <DISABLE_STMT>, <FORCE_RELEASE>
}

BLOCK_STMT: sequence dict {
  :type="block",
  'begin':@, ?<BLOCK_LABEL>:label=@,
  repeat <STATEMENT>:stmts[]=@,
  'end':@
}

BLOCK_LABEL: sequence { ":", verilog_identifier:@ }

IF_STMT: sequence dict {
  :type="if",
  'if':@, "(", <EXPR>:cond=@, ")",
  <STATEMENT>:then=@,
  ?<ELSE_CLAUSE>:else=@
}

ELSE_CLAUSE: sequence { 'else', <STATEMENT>:@ }

CASE_STMT: sequence dict {
  :type="case",
  <CASE_KIND>:kind=@,
  "(", <EXPR>:selector=@, ")",
  repeat+ <CASE_ITEM>:items[]=@,
  'endcase'
}

CASE_KIND: choice { 'case':@, 'casex':@, 'casez':@ }

CASE_ITEM: sequence dict {
  choice {
    sequence { 'default':default=true, ?":" },
    sequence { repeat+ <EXPR>:labels[]=@ separator "," , ":" }
  },
  <STATEMENT>:body=@
}

BLOCKING_ASSIGN: sequence dict {
  :type="blocking_assign",
  <LVALUE>:lhs=@, "=", ?<DELAY_OR_EVENT>:delay=@, <EXPR>:rhs=@, ";"
}

NONBLOCKING_ASSIGN: sequence dict {
  :type="nonblocking_assign",
  <LVALUE>:lhs=@, "<=", ?<DELAY_OR_EVENT>:delay=@, <EXPR>:rhs=@, ";"
}

DELAY_OR_EVENT: choice { <DELAY>, <EVENT_CONTROL> }

PROC_CONT_ASSIGN: sequence dict {
  :type="proc_assign",
  'assign':@, <LVALUE>:lhs=@, "=", <EXPR>:rhs=@, ";"
}

SYSTEM_TASK_CALL: sequence dict {
  :type="system_task",
  verilog_identifier:name=@,   // `$display`, `$finish`, etc. — name starts with `$`
  ?<TASK_ARGS>:args=@,
  ";"
}

TASK_ARGS: sequence array {
  "(",
  repeat+ <EXPR>:@ separator ",",
  ")"
}

// ─── Expressions (sketch — the 17-level pattern) ────────────────────────
//
// The real grammar has 17 precedence levels (assignment, ternary, ||,
// &&, |, ^, &, ==/!=, <=/<, shifts, +/-, *///%, ** , unary, primary).
// Each level is its own rule. Lower-precedence rules call higher-
// precedence ones. Mixed-operator chains can't be left-folded in PEG
// — same Option A/B/C choice as the FastDRC constraint grammar. For
// netlists / simple RTL, requiring parens for ambiguous chains (Option
// C) is the cleanest. Real SV needs full left-associativity, which
// means a small post-parse fold step.
//
// Two levels shown to demonstrate the shape:

EXPR: choice {
  <TERNARY>,
  <LOR>
}

TERNARY: sequence dict {
  :op="ternary",
  <LOR>:args[]=@, "?", <EXPR>:args[]=@, ":", <EXPR>:args[]=@
}

LOR: sequence dict {
  <LAND>:args[]=@:op="||",
  repeat <LOR_TAIL>
}
LOR_TAIL: sequence { "||", <LAND>:args[]=@ }

LAND: sequence dict {
  <PRIMARY>:args[]=@:op="&&",
  repeat <LAND_TAIL>
}
LAND_TAIL: sequence { "&&", <PRIMARY>:args[]=@ }

// PRIMARY — the leaves. Real grammar has ~12 kinds (literal, ident,
// bit-select, part-select, concatenation, function call, etc.).

PRIMARY: choice {
  <PAREN_EXPR>,
  <CONCAT>,
  <NUMBER>,
  <STRING_LIT>,
  <PRIMARY_REF>
}

PAREN_EXPR: sequence { "(", <EXPR>, ")" }

CONCAT: sequence dict {
  :type="concat",
  "{",
  repeat+ <EXPR>:elements[]=@ separator ",",
  "}"
}

NUMBER: sequence dict { verilog_number:value=@ }
STRING_LIT: sequence dict { verilog_string:value=@ }

PRIMARY_REF: sequence dict {
  verilog_identifier:name=@,
  ?<RANGE>:bit_select=@
}

This sketch parses today if the verilog parser group (Phase 0) is implemented first — the grammar references verilog_identifier, verilog_number, verilog_string, verilog_comment which don't exist yet. The int / identifier from the std group could be a temporary stand-in for prototyping, but real Verilog numbers (8'hFF, 'd42, etc.) need the custom parser.

The sketch demonstrates: direct {type, ...} IR emission via bindings (no lower() step), strict keywords ('module', 'always', etc.) to prevent identifier collisions, Choice between ANSI and Verilog-95 port-list styles (informational LL(k) warning expected), and the per-level expression precedence pattern (only 2 of 17 shown — same shape for the rest).