feat(dpi): add FTP control-channel protocol detection and metadata extraction#266
Conversation
|
@0xghost42 thanks for the PR. This looks good to me and FTP is still used by enough people to be worth supporting. As mentioned on #262, for bigger additions like this it would be great to open an issue first so we can align on direction. A few notes on the current implementation:
One last thing before merging: have you been able to test this against a real FTP server end-to-end and confirm rustnet picks up the flow and renders the details panel as you expected? Thanks again, nice contribution. |
…uation skip, drop FTPS-implicit, Display variants, docs Per maintainer review on PR domcyrus#266: 1. Code 215 (SYST) no longer populates server_software. RFC 959 §4.2 defines 215 as returning the OS / system type (UNIX, Windows_NT, ...), not the FTP server software. Added a dedicated FtpInfo.system_type field for the 215 case so the TUI can label it correctly (rendered as 'System Type' alongside 'Server Software'). Updated parses_system_type_response test to assert server_software is None and system_type == "UNIX". 2. extract_software_token no longer over-tags continuation lines. The default greeting on vsftpd, ProFTPD, and Pure-FTPd is multi-line: '220-Welcome to the FTP service.\\r\\n220 ProFTPD ...\\r\\n'. We only see the first line at the DPI layer, so we now skip software extraction when line[3] == b'-' (the RFC 959 §4.2 continuation marker). Added skips_software_extraction_on_220_continuation test that exercises the ProFTPD-style payload and asserts server_software stays None. 3. Dropped PORT_FTPS_IMPLICIT (port 990) from the FTP dispatch in src/network/dpi/mod.rs. Implicit FTPS is TLS from the very first byte; ftp::analyze_ftp always returned None for that traffic and it falls through to the HTTPS/TLS branch correctly. The port-990 check made it look like we were detecting implicit FTPS when we were not. AUTH TLS on port 21 is still handled via the plaintext AUTH command before the TLS upgrade. 4. FtpMessageType::Display now produces 'Request' / 'Response' instead of 'FTP_REQUEST' / 'FTP_RESPONSE'. The protocol-name prefix is already in the surrounding column / panel context, so the variant only needs to disambiguate request-vs-response — previously the details panel showed 'FTP / Message Type: FTP_REQUEST', which read as a duplicated prefix. 5. Added FTP to the README.md 'Deep packet inspection' bullet and to the DPI protocols list in ARCHITECTURE.md. Also threaded the new system_type field through merge_ftp_info (first-wins identity-like field), through the UI details renderer (new 'System Type' row), and through filter.rs so '\sys:UNIX' style queries match it. All 337 lib unit tests pass; cargo fmt / clippy clean.
|
hey @domcyrus, thanks for the careful review. pushed
On the end-to-end question — caveat: I have not yet been able to point rustnet at a real FTP server to watch a live capture render in the details panel. I implemented + unit-tested against the RFC and a handful of crafted payloads. Want me to spin up a local vsftpd or ProFTPD container, exercise it, and post a screenshot of the details panel before you merge? Happy to do that on this PR if it would help close the loop. Process note: you're right about opening an issue first for non-trivial additions. Saving that rule for the future and dropping Thanks again for the thorough review. |
…traction Adds best-effort Deep Packet Inspection for the plaintext FTP control channel (RFC 959, RFC 2389, RFC 2428, RFC 4217). The data channel is left untouched (arbitrary file bytes). Detection: port 21 / FTPS-implicit 990 hint plus a cheap signature (3-digit reply code or known command set) so non-standard ports are still classified. Metadata: request command + args (USER surfaced as username), response code + message, server software extracted from 220/215 greetings. Display formatter produces 'FTP RETR /pub/x.iso' for requests and 'FTP 220 (ProFTPD)' for greetings. Per-flow merge keeps identity fields first-wins and dialog state latest-wins. Tests: 12 unit tests covering request/response/continuation, USER capture, system-type software extraction, lowercase commands, HTTP/short/non-digit rejection, EPSV. cargo test --all-features: 336 passed. cargo clippy --all-features --all-targets -- -D warnings: clean. cargo fmt --all -- --check: clean. Tracks ROADMAP DPI Enhancements (FTP).
…uation skip, drop FTPS-implicit, Display variants, docs Per maintainer review on PR domcyrus#266: 1. Code 215 (SYST) no longer populates server_software. RFC 959 §4.2 defines 215 as returning the OS / system type (UNIX, Windows_NT, ...), not the FTP server software. Added a dedicated FtpInfo.system_type field for the 215 case so the TUI can label it correctly (rendered as 'System Type' alongside 'Server Software'). Updated parses_system_type_response test to assert server_software is None and system_type == "UNIX". 2. extract_software_token no longer over-tags continuation lines. The default greeting on vsftpd, ProFTPD, and Pure-FTPd is multi-line: '220-Welcome to the FTP service.\\r\\n220 ProFTPD ...\\r\\n'. We only see the first line at the DPI layer, so we now skip software extraction when line[3] == b'-' (the RFC 959 §4.2 continuation marker). Added skips_software_extraction_on_220_continuation test that exercises the ProFTPD-style payload and asserts server_software stays None. 3. Dropped PORT_FTPS_IMPLICIT (port 990) from the FTP dispatch in src/network/dpi/mod.rs. Implicit FTPS is TLS from the very first byte; ftp::analyze_ftp always returned None for that traffic and it falls through to the HTTPS/TLS branch correctly. The port-990 check made it look like we were detecting implicit FTPS when we were not. AUTH TLS on port 21 is still handled via the plaintext AUTH command before the TLS upgrade. 4. FtpMessageType::Display now produces 'Request' / 'Response' instead of 'FTP_REQUEST' / 'FTP_RESPONSE'. The protocol-name prefix is already in the surrounding column / panel context, so the variant only needs to disambiguate request-vs-response — previously the details panel showed 'FTP / Message Type: FTP_REQUEST', which read as a duplicated prefix. 5. Added FTP to the README.md 'Deep packet inspection' bullet and to the DPI protocols list in ARCHITECTURE.md. Also threaded the new system_type field through merge_ftp_info (first-wins identity-like field), through the UI details renderer (new 'System Type' row), and through filter.rs so '\sys:UNIX' style queries match it. All 337 lib unit tests pass; cargo fmt / clippy clean.
970d0fb to
f537e6d
Compare
|
Thanks @0xghost42 , LGTM!
|
Mirror the FTP control-channel addition from domcyrus#266 in the Chinese README protocol list and ARCHITECTURE DPI bullets.

Summary
Adds best-effort Deep Packet Inspection for the plaintext FTP control channel (RFC 959, RFC 2389, RFC 2428, RFC 4217). The data channel (port 20 / passive) is deliberately not inspected — payloads are arbitrary file bytes.
Tracks the DPI Enhancements roadmap item for FTP.
What it does
Detection
-(continuation marker).Extracted metadata
USER,PASS,RETR,STOR,PASV,EPSV,AUTH, …) and argument.220,215,230,530, …) and response message.USERis surfaced asusernamefor flow identity.220greeting /215SYST reply (e.g.ProFTPD,vsftpd).Surfaces
ApplicationProtocol::Ftp(FtpInfo)variant with aDisplayformatter tuned for the connection-table column (requests show method + args; responses show code + optional software).FTP_UDP(sentinel — FTP runs over TCP; included only for enum completeness).ConnectionFiltergeneral search matchesftp, command, username, response code, server software.username,server_software) first-wins and dialog state (message_type,command,args,response_code,response_message) latest-wins, mirroring how an FTP session evolves across many request/response pairs.Why detection runs after HTTP
FTP and HTTP share an ASCII line-based start-line shape, but the FTP grammar is unambiguous in both directions — a 3-digit reply prefix or a 3-4 letter known command. Running after HTTP lets HTTP claim its own payloads cheaply and avoids any chance of grabbing an HTTP request that happens to begin with a token like
HELP(not an HTTP method).Tests
12 new unit tests in `src/network/dpi/ftp.rs`:
220 ProFTPD …).220-Welcome … 220 Ready.).username.quit\r\n) accepted.215 UNIX Type: L8→UNIX).Full suite locally on macOS / Apple Silicon:
Files touched
No changes to ROADMAP / README / CONTRIBUTING — happy to add a roadmap tick if you'd prefer.
Test plan