feat(dpi): add SMTP protocol detection and metadata extraction#267
feat(dpi): add SMTP protocol detection and metadata extraction#2670xghost42 wants to merge 1 commit into
Conversation
Adds best-effort Deep Packet Inspection for the plaintext SMTP control channel (RFC 5321, RFC 1869 ESMTP, RFC 3207 STARTTLS, RFC 4954 AUTH). Once STARTTLS upgrades the channel, subsequent bytes are TLS and will be claimed by the HTTPS/TLS analyzer instead. Detection: ports 25 / 465 / 587 / 2525 plus a cheap signature so non-standard SMTP ports are still caught. Server response is a 3-digit reply code (optionally with '-' continuation marker); client request is one of HELO/EHLO/MAIL/RCPT/DATA/RSET/NOOP/QUIT/VRFY/EXPN/HELP/STARTTLS/AUTH/BDAT/ETRN. Metadata: command + args, response code + message, envelope sender from MAIL FROM:<addr>, envelope recipient from RCPT TO:<addr>, server software extracted from 220/250 greetings (e.g. Postfix). Display formatter shows 'SMTP MAIL <alice@example.com>' for envelope hops and 'SMTP 220 (Postfix)' for greetings. Per-flow merge keeps identity fields first-wins and dialog state latest-wins. Tests: 11 unit tests covering greeting parse, EHLO multi-line, MAIL FROM / RCPT TO envelope extraction, STARTTLS, 354 DATA prompt, lowercase verb, HTTP rejection, unknown verb rejection, short-payload rejection. cargo test --all-features: 335 passed. cargo clippy --all-features --all-targets -- -D warnings: clean. cargo fmt --all -- --check: clean. Tracks ROADMAP DPI Enhancements (SMTP).
|
@0xghost42 thanks for the PR. Before merging I want to take a bit more time to think about whether My main hesitation is around how much plaintext So the open question is whether enough RustNet users actually want I appreciate the work, but I want to think it through before merging. |
|
@domcyrus thanks for thinking it through publicly. Adding some concrete use cases in case they shift the cost / benefit: Plaintext SMTP visibility is still useful for:
If the maintenance worry is mainly about the parser surface, happy to keep the scope tight: command + response code + first response message + the sender / recipient hint from I'll let this sit and see if other users chime in. Happy either way — close it if the maintenance surface still doesn't pencil out for you. |
Summary
Adds best-effort Deep Packet Inspection for the plaintext SMTP control channel (RFC 5321, RFC 1869 ESMTP, RFC 3207 STARTTLS, RFC 4954 AUTH). Once STARTTLS upgrades the channel, subsequent bytes are TLS and will be claimed by the HTTPS/TLS analyzer.
Tracks the DPI Enhancements roadmap item for SMTP.
What it does
Detection
-(continuation marker).HELO,EHLO,MAIL,RCPT,DATA,RSET,NOOP,QUIT,VRFY,EXPN,HELP,STARTTLS,AUTH,BDAT,ETRN), case-insensitive, length-capped.Extracted metadata
EHLO client.example.org,AUTH PLAIN ..., …).250-mail.example.com Hello,354 End data with …).MAIL FROM:<addr>— surfaced assender.RCPT TO:<addr>— surfaced asrecipient.220greeting /250EHLO reply (e.g.Postfix).Surfaces
ApplicationProtocol::Smtp(SmtpInfo)variant with aDisplayformatter tuned for the connection-table column (envelope hops showSMTP MAIL <alice@example.com>; greetings showSMTP 220 (Postfix)).ConnectionFiltergeneral search matchessmtp, command, sender, recipient, server software, and response code.sender,recipient,server_software) first-wins and dialog state latest-wins, mirroring how an SMTP session unfolds across many request/response pairs.Tests
11 new unit tests in `src/network/dpi/smtp.rs`:
Full suite locally on macOS / Apple Silicon:
Files touched
Test plan