OPNsense bug report — FRR 10.6.1 BGP regression

[kk6axf]

OPNsense bug report — FRR 10.6.1 BGP regression

HOW TO FILE (so the bot doesn't flag it low-priority):

1. Close issue os-frr / FRR 10.6.1: bgpd rejects 10.5.x peer OPEN (NOTIFICATION+RST) and staticd fails to start #5514.
2. Go to https://github.com/opnsense/plugins/issues/new/choose and pick the "Bug report" template (do NOT open a blank issue).
3. Put this in the TITLE field:
   os-frr: FRR 10.6.1 (26.1.10) breaks eBGP — bgpd rejects 10.5.x peer OPEN (NOTIFICATION+RST) and staticd fails to start
4. The template body will be pre-filled with the sections below. Replace each section's placeholder text with the matching text below, leaving the bold section headers and the checkbox lines EXACTLY as the template generated them.
5. Tick the three checkboxes by putting an x in the brackets.

Everything below is the body, in the template's exact order. Paste section-by-section over the pre-filled placeholders.

---

Important notices
Before you add a new report, we ask you kindly to acknowledge the following:

• I have read the contributing guide lines at https://github.com/opnsense/plugins/blob/master/CONTRIBUTING.md
• I have searched the existing issues, open and closed, and I'm convinced that mine is new.
• The title contains the plugin to which this issue belongs

Describe the bug

After updating to OPNsense 26.1.10 (which ships FRRouting 10.6.1 on FreeBSD 14.3), eBGP sessions on the updated node fail to establish. Two distinct failure modes were observed, both tied to FRR 10.6.1:

1. OPEN rejection / interop failure vs FRR 10.5.x peers — a 10.6.1 node rejects a peer's BGP OPEN and resets the session. The peer was running FRR 10.5.2 / 10.5.3 with identical config; only the FRR version differed. On the wire: TCP connects, the 10.5.x side sends its 153-byte OPEN, the 10.6.1 side ACKs it and replies with a 21-byte BGP NOTIFICATION followed by FIN/RST — without ever sending its own OPEN.

2. Daemon comes up incomplete after update — on one node, after the update bgpd was running but staticd was NOT started (watchfrr managed only mgmtd zebra bgpd), despite redistribute static being configured, and /var/log/frr/frr.log did not exist. All BGP peers sat Idle/Active with 0 messages received. A full service frr stop + hard kill + service frr start (verified a new bgpd PID) did NOT recover it; only rolling FRR back resolved it.

WireGuard transport was healthy throughout (tunnels up, handshakes current, ICMP across the tunnel fine, TCP/179 three-way handshake completing). This is strictly an FRR/bgpd-layer problem introduced by the 10.6.1 update — it is easy to misdiagnose as a WireGuard/firewall issue because the tunnels stay up while all BGP-learned routes disappear.

Last known working version: FRR 10.5.2 / 10.5.3 (OPNsense 26.1.x prior to 26.1.10). Rolling FRR back to 10.5.x on the affected node restores BGP immediately.

Note on opnsense-revert: opnsense-revert -r 26.1.9 frr10 os-frr did NOT help — the 26.1.9 repo also ships FRR 10.6.1 (only os-frr moved 1.53 -> 1.52). An older release tag that actually shipped FRR 10.5.x, or a boot-environment / snapshot rollback, is required.

To Reproduce
Steps to reproduce the behavior:

1. Two OPNsense nodes peered via eBGP over a WireGuard point-to-point tunnel, both initially on FRR 10.5.x, session Established.
2. Update one node to 26.1.10 (FRR 10.6.1) and reboot.
3. The updated node's BGP session to the 10.5.x peer fails to establish and loops (OpenSent -> NOTIFICATION -> reset). In a subset of cases the updated node's bgpd comes up with staticd not started and no /var/log/frr/frr.log, and all peers stay Idle/Active with MsgRcvd 0.
4. See error.

Expected behavior

FRR 10.6.1 should interoperate with FRR 10.5.x eBGP peers (BGP OPEN / capability negotiation is meant to be backward compatible), all configured daemons including staticd should start, and the session should reach Established.

Screenshots

N/A (CLI evidence provided below under Relevant log files).

Relevant log files

Wire capture on the 10.6.1 node's tunnel interface during session setup (IPs shown as .1 = 10.5.x peer, .2 = 10.6.1 node):

.1 > .2  [S]                       peer (10.5.x) SYN
.2 > .1  [S.]                      10.6.1 node SYN-ACK
.1 > .2  P. seq 1:154 len 153 BGP  peer sends OPEN (153 bytes)
.2 > .1  . ack 154                 10.6.1 node ACKs the OPEN (received OK)
.2 > .1  P. seq 1:22  len 21  BGP  10.6.1 node sends 21-byte NOTIFICATION
.2 > .1  F. / R                    10.6.1 node closes + RST

show bgp neighbor on the 10.6.1 side:

Opens:         Sent N   Rcvd 0
Notifications: Sent N   Rcvd 0
Last reset: Notification sent (Neighbor Events Error/Receive Unexpected Message in OpenSent State)

Failure mode 2 (staticd missing). show ip bgp summary shows all peers Idle/Active with MsgRcvd 0, and:

# pgrep -lf 'bgpd|zebra|staticd|mgmtd'
<watchfrr> ... -d mgmtd zebra bgpd      <-- no staticd in the managed daemon list
<zebra>
<bgpd>
<mgmtd>
# ls /var/log/frr/frr.log
ls: /var/log/frr/frr.log: No such file or directory

service frr stop + pkill -9 -f 'bgpd|zebra|staticd|mgmtd|watchfrr' + service frr start produced a NEW bgpd PID but staticd still did not start and sessions still failed.

Additional context

• Topology: hub-and-spoke eBGP, each session over its own WireGuard point-to-point tunnel. Multiple private ASNs.
• Relevant FRR config common to all nodes: no bgp ebgp-requires-policy, bgp graceful-restart, redistribute static, per-neighbor default-originate on some neighbors, no per-neighbor route-maps.
• Impact: all routes learned/advertised over BGP are lost on the affected node, dropping inter-site prefixes across the mesh even though WireGuard tunnels remain up.
• Possibly related: a separately reported FRR 10.6.1-on-FreeBSD regression where kernel routes are overwritten after interface events — suggests 10.6.1 on FreeBSD has broader issues this cycle.
• Secondary hardening we applied (not the root cause): pinned bgp router-id to stable tunnel IPs on every node; nodes had blank router-id and were latching onto dynamic DHCP WAN IPs, which aggravated connection-collision behavior during recovery.

Questions for maintainers:

1. Is FRR 10.6.1 in 26.1.10 known to have a BGP OPEN-handling / capability-negotiation regression vs 10.5.x peers?
2. Is the missing-staticd / missing-frr.log startup a packaging issue in os-frr for FRR 10.6.1 on FreeBSD 14.3?
3. Which release / plugin version will carry the fix (or a re-pinned FRR), so we know when it is safe to update BGP nodes again?

Environment
Software version used and hardware type if relevant.

OPNsense 26.1.10 (amd64)
FRRouting 10.6.1 on FreeBSD 14.3-RELEASE-p15   (affected node)
Peers on FRRouting 10.5.2 (FreeBSD 14.3-p9) and 10.5.3 (FreeBSD 14.3-p12) — worked fine
os-frr plugin (26.1.10)

[kk6axf]

Following up with root-cause detail that may help triage and other affected users.

The breaking change is the FRR daemon version, and it was not announced. The 26.1.10 release notes list only:

o plugins: os-frr 1.53

That is the plugin (GUI/integration layer) moving 1.52 → 1.53. The actual frr10 daemon moved to 10.6.1 as a FreeBSD port/package update with no changelog entry. Users upgrading to 26.1.10 therefore have no warning that the routing daemon version changed underneath them.

Why a plugin-level revert does not help. The natural recovery attempt:

opnsense-revert -r 26.1.9 frr10 os-frr

reverts os-frr 1.53 → 1.52 but still installs frr10 daemon 10.6.1, because the daemon version is bound to the package repository snapshot, not the plugin version. So opnsense-revert does not get you back to a working daemon. The only ways back to FRR 10.5.x are a ZFS boot environment / VM snapshot taken before the upgrade, or reinstalling from an ISO/img that predates the bump and then freezing the packages (pkg lock frr10 os-frr).

Symptom recap (two distinct failure modes on 10.6.1):

1. A 10.6.1 bgpd, peering eBGP with a 10.5.x neighbor, accepts the TCP connection and the neighbor's OPEN, then immediately replies with a 21-byte NOTIFICATION + RST without sending its own OPEN. Session never leaves Active/Idle; Last reset = "Neighbor Events Error/Receive Unexpected Message in OpenSent State". Confirmed on the wire (TCP and payload delivered both directions — not transport, not MTU).
2. On at least one node, 10.6.1 came up with bgpd running but staticd never started (watchfrr managing only mgmtd/zebra/bgpd) despite redistribute static in the config, and no /var/log/frr/frr.log was created. A plain service frr restart did not relaunch staticd; only a full stop + start did. This box could not establish BGP with any peer.

Both cleared the moment the affected node was returned to FRR 10.5.x (matching the rest of the mesh).

Requests:

1. Please treat frr10 daemon version bumps as changelog-worthy — a silent major-version daemon change in a point release is how this slipped past everyone.
2. Could you clarify whether a corrected frr10 (10.6.2+ or a re-pin to 10.5.x) is planned for a 26.1.x point release, or only in the upcoming 26.7-BETA? That timeline determines whether affected users need to wipe-and-reload to an older image versus simply waiting for the next update.

Environment: OPNsense 26.1.10 (amd64), FreeBSD 14.3-RELEASE, frr10 10.6.1 (broken) vs 10.5.2 / 10.5.3 (last known working), os-frr 1.53.

[Monviech]

Where is the upstream evidence that something changed? Shouldn't this rather be reported here if you are affected by an upstream regression?
https://github.com/frrouting/frr

Marking this upstream for now.

[kk6axf]

Thanks — fair point, agreed the daemon is upstream. There is a documented OPEN-message change in the FRR 10.6.0 notes ("BGP software version capability encoding changes… TLV length field no longer explicitly encoded"), which lines up with the symptom: a 10.6.1 bgpd rejecting a 10.5.x peer's OPEN with a 21-byte NOTIFICATION. I'm reproducing now with debug bgp neighbor-events + a -vvv packet capture to get the exact NOTIFICATION error/subcode and the capability TLV, and will open an issue at frrouting/frr with that decode. I'll also report whether dont-capability-negotiate works around it. Note: the separate "staticd doesn't start / no frr.log" symptom looks packaging-side, so I'll keep that here rather than upstream.

[Monviech]

About the second report:

We can only reasonably define the daemons watchfrr should start:

plugins/net/frr/src/opnsense/service/templates/OPNsense/Quagga/frr

Line 16 in 02e84fb

if helpers.exists('OPNsense.quagga.static.enabled') and OPNsense.quagga.static.enabled == '1' %} staticd{% endif %}"

If static refuses to start or any daemon generally refuses to start for a reason without logs, there is no reasonable chance to debug this.

I think you just called the service reload wrong since watchfrr is used.

https://github.com/opnsense/ports/blob/2238512cd2de1664d52fd146f59e6f64956792ab/net/frr10/files/frr.in#L18