Skip to content

Improve port-forwarding resilience under bursty multi-port load#144

Merged
DavidObando merged 3 commits intomainfrom
fix/forwarder-resilience-followups
May 5, 2026
Merged

Improve port-forwarding resilience under bursty multi-port load#144
DavidObando merged 3 commits intomainfrom
fix/forwarder-resilience-followups

Conversation

@DavidObando
Copy link
Copy Markdown
Member

@DavidObando DavidObando commented May 4, 2026

Context

Follow-up to #138, which fixed unhandled 'error' events on duplex streams inside StreamForwarder that were crashing Node host processes (extension host using the Codespaces extension, Node apps using TunnelRelayTunnelClient, etc.) under bursty multi-port forwarding.

While reviewing #138 we found several related issues that PR did not cover. This PR addresses the ones that are small, tightly related, and don't change public API or wire-protocol behavior. Two larger follow-ups (TCP socket options on forwarded sockets, and SshStream flow-control improvements) are deferred to their own discussions.

Changes

1. Bug: RemotePortForwarder.forwardChannel constructed a StreamForwarder even after the local TCP connect failed

The previous code set request.failureReason = connectFailed but then fell through and built a forwarder around a destroyed socket and a still-open SSH channel. Pre-#138 this was a likely crash trigger; post-#138 it created a self-disposing zombie. Now we return early and destroy() the forwardedStream so the underlying SSH channel is released.

2. Memory leak: pfs.streamForwarders was append-only

PortForwardingService.streamForwarders was an array that everyone pushed to but no one removed from. Disposed forwarders were retained for the lifetime of the session, holding references to a Socket and an SshStream. Under the bursty connect/disconnect workload that motivated #138, this is a real leak.

Now it's a Set<StreamForwarder> and StreamForwarder accepts an optional onDisposed callback that the PFS uses to splice itself out.

3. Pre-construction unhandled 'error' window on LocalPortForwarder.acceptConnection

Between accepting a TCP connection and constructing the StreamForwarder (which now installs the listeners from #138), the code awaits pfs.openChannel(...) and pfs.forwardedPortConnecting(...). A peer reset during that await window emits 'error' on a listener-less socket — same crash class as #138, just earlier in the lifecycle. Fixed by attaching a temporary 'error' listener on accept and removing/transferring it when the forwarder takes over.

Also: when the connecting event handler rejects the connection (returns falsy), the previous code returned without destroying the socket — leaked file descriptor. Now destroyed.

4. Pre-construction unhandled 'error' window on RemotePortForwarder.forwardChannel

Symmetric fix on the SSH-stream side: attach a temporary listener on the SshStream while we await forwardedPortConnecting and the local net.createConnection. A remote channel reset in that window would otherwise emit an unhandled 'error' from the SshStream.

5. SshStream.destroy swallowed channel.close() errors

Was void this.channel.close().catch(); — silently dropped any rejection. Now traces them at Warning so the new error path doesn't lose diagnostic information.

Deferred

  • TCP socket options on forwarded sockets (the literal // TODO: Set socket options? in both forwarders) — setNoDelay/setKeepAlive are clearly desirable for bursty workloads, but choosing keepalive intervals, deciding on configurability vs. hardcoded defaults, and checking C# parity needs its own discussion. Detailed notes have been written up separately.
  • SshStream flow-control improvements — the existing self-aware comment in sshStream.ts about choppy SSH window adjustment is real, but addressing it requires either decoupling window adjust from push() backpressure (lower-risk patch) or replacing Duplex with a custom pull-based implementation (ideal but invasive). Detailed notes have been written up separately.
  • Migrating to stream.pipeline() in StreamForwarder — would simplify the manual error/teardown wiring, but changes teardown semantics (pipeline destroys both streams on error) and warrants its own PR with focused testing.

Behavior

Happy path is unchanged. On the error path: forwarders self-dispose, are removed from the PFS collection, and warning traces include the side (local / remote) and reason. No public API changes, no wire-protocol changes.

Impact on Codespaces clients

Verification

  • npm run compile
  • npm run eslint
  • npm run test-ts

DavidObando and others added 2 commits May 4, 2026 09:54
Follow-up to #138. PR #138 prevented host-process crashes from unhandled
'error' events on duplex streams inside StreamForwarder, but several
related issues remained that either (a) leaked resources, (b) still left
a small unhandled-'error' window outside the forwarder, or (c) wasted
work on doomed connections. This change addresses them.

Changes:

* StreamForwarder: accept an optional onDisposed callback so the
  PortForwardingService can remove disposed forwarders from its tracking
  collection. Add a clarifying comment about pipe()'s end-vs-error
  semantics.

* PortForwardingService: change streamForwarders from an append-only
  array to a Set, with a removeStreamForwarder callback wired into
  every StreamForwarder. Previously disposed forwarders were retained
  for the lifetime of the session, holding references to their socket
  and SshStream — a real leak under bursty connect/disconnect workloads
  (the exact workload PR #138 was reacting to).

* LocalPortForwarder.acceptConnection: attach a temporary 'error'
  handler on the accepted socket *before* awaiting openChannel and the
  forwardedPortConnecting event. Without this, a peer reset during that
  await window emits 'error' on a listener-less socket and crashes the
  host — the same crash class as #138, just earlier in the lifecycle.
  Also destroy the socket when the connecting event handler rejects the
  connection (was previously leaked).

* RemotePortForwarder.forwardChannel:
  - Bug fix: when the local TCP connect fails, return early instead of
    falling through to construct a StreamForwarder around a destroyed
    socket. The forwardedStream is now also destroyed so the underlying
    SSH channel doesn't leak.
  - Attach a temporary 'error' handler on the SshStream during the
    forwardedPortConnecting + connect window for the same reason as the
    local side: a remote channel reset in that window would emit an
    unhandled 'error'.

* SshStream.destroy: trace channel.close() failures instead of silently
  swallowing them with .catch(). Aids diagnosis of teardown issues that
  the new error path may now expose.

Behavior is unchanged on the happy path. On the error path, forwarders
self-dispose, are removed from the PFS collection, and traces include
the side and reason. No public API changes.

Verified locally: npm run compile, npm run eslint, and npm run test-ts
all pass (one pre-existing unrelated failure in VersionTests).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… and tests

Fix several edge cases identified during code review of PR #144:

1. Listener leak if forwardedPortConnecting throws: wrap the call in
   try/catch in both localPortForwarder and remotePortForwarder so the
   temporary error handler is always removed and streams are destroyed
   on unexpected exceptions.

2. SSH channel leak in localPortForwarder.acceptConnection: when the
   forwardedPortConnecting event handler rejects (returns null), the
   already-opened SSH channel was not being closed. Now destroy the
   SshStream so the channel is released. Also remove the temporary
   error listener in the openChannel catch path for consistency.

3. Synchronous-dispose race in StreamForwarder constructor: if an error
   fires during pipe() setup (before the caller can call
   streamForwarders.add()), the onDisposed callback runs on a forwarder
   not yet in the set, then the caller adds a dead forwarder. Fixed by
   adding a public isDisposed getter and guarding add() with it.

4. StreamForwarder tests: add streamForwarderTests.ts with 10 tests
   covering data forwarding, error-triggered dispose, callback
   invocation, idempotent dispose, the synchronous-dispose race, and
   onDisposed error swallowing. Export StreamForwarder from the package
   (constructor made public) to enable testing.

5. Ensure sshStream is destroyed when forwardedStream is user-substituted:
   in remotePortForwarder error paths, if the forwardedStream returned by
   the connecting event is not the original sshStream, explicitly destroy
   sshStream so the underlying SSH channel does not leak.

Verified: npm run compile, npm run eslint, npm run test-ts all pass
(357 passing, 1 pre-existing unrelated VersionTests failure).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@DavidObando DavidObando requested a review from Copilot May 4, 2026 20:06
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Improve resilience and lifecycle management of SSH TCP port-forwarding under bursty connect/disconnect and multi-port load by closing pre-construction unhandled 'error' windows, preventing leaked forwarders, and improving diagnostics.

Changes:

  • Add pre-forwarding temporary 'error' handlers for accepted sockets and SSH streams; ensure resources are destroyed on rejection/failure paths.
  • Replace append-only forwarder tracking with Set<StreamForwarder> and add an onDisposed callback to support removal on disposal.
  • Add unit tests for StreamForwarder data forwarding and error/disposal behavior; add warning trace on SshStream.destroy() close failures.
Show a summary per file
File Description
test/ts/ssh-test/streamForwarderTests.ts Adds StreamForwarder unit tests covering data piping, error-triggered disposal, and disposal callback behavior.
src/ts/ssh/sshStream.ts Traces channel.close() failures during destroy() instead of silently swallowing them.
src/ts/ssh-tcp/services/streamForwarder.ts Adds onDisposed callback + isDisposed and ensures callback errors are handled during disposal.
src/ts/ssh-tcp/services/remotePortForwarder.ts Adds temporary SSH stream error handler during setup, fixes failure-path teardown, and integrates forwarder removal via callback + Set.
src/ts/ssh-tcp/services/portForwardingService.ts Changes streamForwarders to a Set and adds a removal callback; clears set on dispose.
src/ts/ssh-tcp/services/localPortForwarder.ts Adds temporary socket error handler during setup, destroys socket on rejection, and integrates forwarder removal via callback + Set.
src/ts/ssh-tcp/index.ts Exports StreamForwarder from the package entrypoint (public surface change).

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 7/7 changed files
  • Comments generated: 4

Comment thread src/ts/ssh-tcp/index.ts
Comment thread src/ts/ssh-tcp/services/streamForwarder.ts Outdated
Comment thread src/ts/ssh-tcp/services/remotePortForwarder.ts
Comment thread src/ts/ssh/sshStream.ts Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@DavidObando DavidObando merged commit 8aee585 into main May 5, 2026
10 checks passed
@DavidObando DavidObando deleted the fix/forwarder-resilience-followups branch May 5, 2026 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants