Skip to content

feat(dpi): add FTP control-channel protocol detection and metadata extraction#266

Merged
domcyrus merged 2 commits into
domcyrus:mainfrom
0xghost42:feat/dpi-ftp
May 15, 2026
Merged

feat(dpi): add FTP control-channel protocol detection and metadata extraction#266
domcyrus merged 2 commits into
domcyrus:mainfrom
0xghost42:feat/dpi-ftp

Conversation

@0xghost42
Copy link
Copy Markdown
Contributor

Summary

Adds best-effort Deep Packet Inspection for the plaintext FTP control channel (RFC 959, RFC 2389, RFC 2428, RFC 4217). The data channel (port 20 / passive) is deliberately not inspected — payloads are arbitrary file bytes.

Tracks the DPI Enhancements roadmap item for FTP.

What it does

Detection

  • Port 21 (and FTPS-implicit 990 hint), plus a cheap start-line signature so non-standard FTP ports are still caught.
  • Server response: 3-digit reply code followed by space or - (continuation marker).
  • Client request: known command set (RFC 959 / 2389 / 2428 / 3659 / 4217), case-insensitive, length-capped at 6 chars.

Extracted metadata

  • Request method (USER, PASS, RETR, STOR, PASV, EPSV, AUTH, …) and argument.
  • Response code (220, 215, 230, 530, …) and response message.
  • USER is surfaced as username for flow identity.
  • Server software extracted from the 220 greeting / 215 SYST reply (e.g. ProFTPD, vsftpd).

Surfaces

  • New ApplicationProtocol::Ftp(FtpInfo) variant with a Display formatter tuned for the connection-table column (requests show method + args; responses show code + optional software).
  • DPI details panel in the TUI renders message type, command, arguments, response code, response, username, server software.
  • UDP connection state reports FTP_UDP (sentinel — FTP runs over TCP; included only for enum completeness).
  • ConnectionFilter general search matches ftp, command, username, response code, server software.
  • Per-protocol merge keeps stable identity fields (username, server_software) first-wins and dialog state (message_type, command, args, response_code, response_message) latest-wins, mirroring how an FTP session evolves across many request/response pairs.

Why detection runs after HTTP

FTP and HTTP share an ASCII line-based start-line shape, but the FTP grammar is unambiguous in both directions — a 3-digit reply prefix or a 3-4 letter known command. Running after HTTP lets HTTP claim its own payloads cheaply and avoids any chance of grabbing an HTTP request that happens to begin with a token like HELP (not an HTTP method).

Tests

12 new unit tests in `src/network/dpi/ftp.rs`:

  • Server greeting (220 ProFTPD …).
  • Continuation response (220-Welcome … 220 Ready.).
  • USER request — username captured into username.
  • RETR request with path argument.
  • NOOP (no args).
  • Lowercase command (quit\r\n) accepted.
  • SYST 215 response with software extraction (215 UNIX Type: L8UNIX).
  • Unknown request rejected.
  • HTTP payload rejected.
  • Truncated / non-digit / wrong prefix rejected.
  • EPSV (RFC 2428 extended passive).

Full suite locally on macOS / Apple Silicon:

  • ` cargo test --all-features ` → 336 passed, 0 failed.
  • ` cargo clippy --all-features --all-targets -- -D warnings ` → clean.
  • ` cargo fmt --all -- --check ` → clean.
  • ` cargo build --release ` → clean.

Files touched

  • ` src/network/dpi/ftp.rs ` — new module.
  • ` src/network/dpi/mod.rs ` — register module, add `PORT_FTP` / `PORT_FTPS_IMPLICIT`, dispatch FTP after HTTP.
  • ` src/network/types.rs ` — `FtpInfo`, `FtpMessageType`, enum variant, `Display` arm, `sort_key`, dist counter, UDP state sentinel, idle timeout.
  • ` src/network/merge.rs ` — `merge_ftp_info` (stable fields first-wins, dialog state latest-wins).
  • ` src/filter.rs ` — general-search match arm for FTP metadata.
  • ` src/ui.rs ` — DPI details panel rendering for FTP.

No changes to ROADMAP / README / CONTRIBUTING — happy to add a roadmap tick if you'd prefer.

Test plan

  • Unit tests pass locally on macOS (Apple Silicon).
  • Live smoke against an FTP server (recommended for the maintainer; I do not have an FTP endpoint handy on this machine).

@domcyrus
Copy link
Copy Markdown
Owner

@0xghost42 thanks for the PR. This looks good to me and FTP is still used by enough people to be worth supporting. As mentioned on #262, for bigger additions like this it would be great to open an issue first so we can align on direction. A few notes on the current implementation:

  1. `server_software` for code 215 (SYST) is misleading. RFC 959 §4.2 defines 215 as returning the OS / system type (`UNIX`, `Windows_NT`), not the FTP server software. The `parses_system_type_response` test ends up extracting `"UNIX"` as server software, which would then show up under "Server Software" in the TUI. I would only populate `server_software` on 220 greetings and drop the 215 branch. If you want to keep the SYST data, a separate `system_type` field would be cleaner.

  2. `extract_software_token` over-tags continuation-line text. Only the first line of the payload is parsed, so a multi-line greeting like `220-Welcome to the FTP service.\r\n220 ProFTPD ...\r\n` ends up with `server_software = "Welcome"`. Multi-line greetings are the default on vsftpd, ProFTPD, and Pure-FTPd, so this would trigger on most real servers. The `detects_continuation_response` test does not cover the software field. I would skip software extraction when `line[3] == b'-'` and add a test asserting `server_software` is `None` for the continuation prefix.

  3. Port 990 (implicit FTPS) in the dispatch is a no-op. Traffic on 990 is TLS from the very first byte, so `analyze_ftp` always returns `None` and the protocol falls through. It is not harmful, but it suggests we detect FTPS-implicit when we do not. Can we just drop the `PORT_FTPS_IMPLICIT` check? AUTH TLS on port 21 is already handled correctly via the `AUTH` command.

  4. `FtpMessageType::Display` produces `FTP_REQUEST` / `FTP_RESPONSE`, which the UI shows verbatim under "Message Type". The "FTP" prefix is already implicit in context, so I would change these to `Request` / `Response` for the UI.

  5. Could you also add FTP to the protocol list in README.md (the "Deep packet inspection" bullet) and to the DPI protocols list in ARCHITECTURE.md?

One last thing before merging: have you been able to test this against a real FTP server end-to-end and confirm rustnet picks up the flow and renders the details panel as you expected?

Thanks again, nice contribution.

0xghost42 added a commit to 0xghost42/rustnet that referenced this pull request May 14, 2026
…uation skip, drop FTPS-implicit, Display variants, docs

Per maintainer review on PR domcyrus#266:

1. Code 215 (SYST) no longer populates server_software. RFC 959 §4.2 defines
   215 as returning the OS / system type (UNIX, Windows_NT, ...), not the
   FTP server software. Added a dedicated FtpInfo.system_type field for the
   215 case so the TUI can label it correctly (rendered as 'System Type'
   alongside 'Server Software'). Updated parses_system_type_response test
   to assert server_software is None and system_type == "UNIX".

2. extract_software_token no longer over-tags continuation lines. The
   default greeting on vsftpd, ProFTPD, and Pure-FTPd is multi-line:
   '220-Welcome to the FTP service.\\r\\n220 ProFTPD ...\\r\\n'. We only
   see the first line at the DPI layer, so we now skip software extraction
   when line[3] == b'-' (the RFC 959 §4.2 continuation marker). Added
   skips_software_extraction_on_220_continuation test that exercises the
   ProFTPD-style payload and asserts server_software stays None.

3. Dropped PORT_FTPS_IMPLICIT (port 990) from the FTP dispatch in
   src/network/dpi/mod.rs. Implicit FTPS is TLS from the very first byte;
   ftp::analyze_ftp always returned None for that traffic and it falls
   through to the HTTPS/TLS branch correctly. The port-990 check made it
   look like we were detecting implicit FTPS when we were not. AUTH TLS on
   port 21 is still handled via the plaintext AUTH command before the
   TLS upgrade.

4. FtpMessageType::Display now produces 'Request' / 'Response' instead of
   'FTP_REQUEST' / 'FTP_RESPONSE'. The protocol-name prefix is already in
   the surrounding column / panel context, so the variant only needs to
   disambiguate request-vs-response — previously the details panel showed
   'FTP / Message Type: FTP_REQUEST', which read as a duplicated prefix.

5. Added FTP to the README.md 'Deep packet inspection' bullet and to the
   DPI protocols list in ARCHITECTURE.md.

Also threaded the new system_type field through merge_ftp_info (first-wins
identity-like field), through the UI details renderer (new 'System Type'
row), and through filter.rs so '\sys:UNIX' style queries match it.

All 337 lib unit tests pass; cargo fmt / clippy clean.
@0xghost42
Copy link
Copy Markdown
Contributor Author

hey @domcyrus, thanks for the careful review. pushed 970d0fb addressing all five points:

  1. 215 (SYST) → dedicated system_type field. Server-software extraction is now gated to code == 220 only. FtpInfo gains a system_type: Option<String> populated from 215 replies, threaded through merge_ftp_info (first-wins identity field) and rendered in the details panel as a separate "System Type" row. parses_system_type_response now asserts server_software is None AND system_type == Some("UNIX").

  2. Continuation lines skip software extraction. analyze_ftp records is_continuation = line[3] == b'-' and server_software / system_type are only populated when !is_continuation. Added skips_software_extraction_on_220_continuation exercising the ProFTPD-style 220-Welcome...\r\n220 ProFTPD...\r\n payload and asserting server_software stays None.

  3. Dropped PORT_FTPS_IMPLICIT (port 990) from dispatch. You're right — implicit FTPS is TLS from byte 0 so analyze_ftp always returned None for it. The constant + the two || clauses in analyze_tcp_packet are removed and the surrounding comment now records the rationale (port 21 + AUTH command handles AUTH TLS; 990 falls through to the TLS branch).

  4. FtpMessageType::Display now Request / Response. Dropped the FTP_ prefix since the protocol name is already in the column / panel context.

  5. Docs updated. FTP added to the README "Deep packet inspection" bullet and to the DPI protocols list in ARCHITECTURE.md. The ARCHITECTURE entry mentions the four extracted fields (commands, response codes, username, server software) plus the new system_type so the protocol scope is clear.

On the end-to-end question — caveat: I have not yet been able to point rustnet at a real FTP server to watch a live capture render in the details panel. I implemented + unit-tested against the RFC and a handful of crafted payloads. Want me to spin up a local vsftpd or ProFTPD container, exercise it, and post a screenshot of the details panel before you merge? Happy to do that on this PR if it would help close the loop.

Process note: you're right about opening an issue first for non-trivial additions. Saving that rule for the future and dropping feat/dpi-sip / parking feat/dpi-smtp and feat/dpi-imap until there's a direction signal from you (or from users chiming in on #267 / #268).

Thanks again for the thorough review.

0xghost42 added 2 commits May 15, 2026 14:01
…traction

Adds best-effort Deep Packet Inspection for the plaintext FTP control channel (RFC 959, RFC 2389, RFC 2428, RFC 4217). The data channel is left untouched (arbitrary file bytes).

Detection: port 21 / FTPS-implicit 990 hint plus a cheap signature (3-digit reply code or known command set) so non-standard ports are still classified.

Metadata: request command + args (USER surfaced as username), response code + message, server software extracted from 220/215 greetings. Display formatter produces 'FTP RETR /pub/x.iso' for requests and 'FTP 220 (ProFTPD)' for greetings. Per-flow merge keeps identity fields first-wins and dialog state latest-wins.

Tests: 12 unit tests covering request/response/continuation, USER capture, system-type software extraction, lowercase commands, HTTP/short/non-digit rejection, EPSV. cargo test --all-features: 336 passed. cargo clippy --all-features --all-targets -- -D warnings: clean. cargo fmt --all -- --check: clean.

Tracks ROADMAP DPI Enhancements (FTP).
…uation skip, drop FTPS-implicit, Display variants, docs

Per maintainer review on PR domcyrus#266:

1. Code 215 (SYST) no longer populates server_software. RFC 959 §4.2 defines
   215 as returning the OS / system type (UNIX, Windows_NT, ...), not the
   FTP server software. Added a dedicated FtpInfo.system_type field for the
   215 case so the TUI can label it correctly (rendered as 'System Type'
   alongside 'Server Software'). Updated parses_system_type_response test
   to assert server_software is None and system_type == "UNIX".

2. extract_software_token no longer over-tags continuation lines. The
   default greeting on vsftpd, ProFTPD, and Pure-FTPd is multi-line:
   '220-Welcome to the FTP service.\\r\\n220 ProFTPD ...\\r\\n'. We only
   see the first line at the DPI layer, so we now skip software extraction
   when line[3] == b'-' (the RFC 959 §4.2 continuation marker). Added
   skips_software_extraction_on_220_continuation test that exercises the
   ProFTPD-style payload and asserts server_software stays None.

3. Dropped PORT_FTPS_IMPLICIT (port 990) from the FTP dispatch in
   src/network/dpi/mod.rs. Implicit FTPS is TLS from the very first byte;
   ftp::analyze_ftp always returned None for that traffic and it falls
   through to the HTTPS/TLS branch correctly. The port-990 check made it
   look like we were detecting implicit FTPS when we were not. AUTH TLS on
   port 21 is still handled via the plaintext AUTH command before the
   TLS upgrade.

4. FtpMessageType::Display now produces 'Request' / 'Response' instead of
   'FTP_REQUEST' / 'FTP_RESPONSE'. The protocol-name prefix is already in
   the surrounding column / panel context, so the variant only needs to
   disambiguate request-vs-response — previously the details panel showed
   'FTP / Message Type: FTP_REQUEST', which read as a duplicated prefix.

5. Added FTP to the README.md 'Deep packet inspection' bullet and to the
   DPI protocols list in ARCHITECTURE.md.

Also threaded the new system_type field through merge_ftp_info (first-wins
identity-like field), through the UI details renderer (new 'System Type'
row), and through filter.rs so '\sys:UNIX' style queries match it.

All 337 lib unit tests pass; cargo fmt / clippy clean.
@0xghost42 0xghost42 force-pushed the feat/dpi-ftp branch 2 times, most recently from 970d0fb to f537e6d Compare May 15, 2026 08:32
@domcyrus
Copy link
Copy Markdown
Owner

Thanks @0xghost42 , LGTM!

image

@domcyrus domcyrus merged commit 0143c8d into domcyrus:main May 15, 2026
domcyrus added a commit to luojiyin1987/rustnet that referenced this pull request May 15, 2026
Mirror the FTP control-channel addition from domcyrus#266 in the
Chinese README protocol list and ARCHITECTURE DPI bullets.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants