Skip to content

feat(dpi): add IMAP protocol detection and metadata extraction#268

Open
0xghost42 wants to merge 1 commit into
domcyrus:mainfrom
0xghost42:feat/dpi-imap
Open

feat(dpi): add IMAP protocol detection and metadata extraction#268
0xghost42 wants to merge 1 commit into
domcyrus:mainfrom
0xghost42:feat/dpi-imap

Conversation

@0xghost42
Copy link
Copy Markdown
Contributor

Summary

Adds best-effort Deep Packet Inspection for the plaintext IMAP4rev1 control channel (RFC 3501, RFC 7888 LITERAL+, RFC 2595 STARTTLS, RFC 4978 COMPRESS). Port 993 (IMAPS) is TLS-wrapped and is claimed by the HTTPS/TLS analyzer.

Tracks the DPI Enhancements roadmap item for IMAP.

What it does

Detection

  • Port 143 plus a signature that recognises all three IMAP frame shapes:
    • Tagged client request — ` COMMAND ` (RFC 3501 §2.2.1 tag grammar enforced; printable ASCII, length-capped).
    • Untagged server response — `* OK …`, `* PREAUTH …`, `* 23 EXISTS`, `* CAPABILITY …`.
    • Tagged server completion — ` OK/NO/BAD/BYE …`.
    • Continuation — `+ `.
  • Known command set spans RFC 3501 / 2971 / 5161 / 2177 / 4978 (CAPABILITY, NOOP, LOGOUT, AUTHENTICATE, LOGIN, STARTTLS, SELECT, EXAMINE, CREATE, DELETE, RENAME, SUBSCRIBE, UNSUBSCRIBE, LIST, LSUB, STATUS, APPEND, CHECK, CLOSE, EXPUNGE, SEARCH, FETCH, STORE, COPY, UID, ID, ENABLE, IDLE, DONE, COMPRESS).

Extracted metadata

  • Tag, command + args.
  • Status (`OK`, `NO`, `BAD`, `BYE`, `PREAUTH`) for response frames.
  • Response message text.
  • Username from `LOGIN ` (handles both bare and `"quoted"` user atoms).
  • Server software extracted from the `* OK` / `* PREAUTH` greeting, skipping the bracketed `[CAPABILITY ...]` list so e.g. `* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR] Dovecot ready.` surfaces `Dovecot`.

Surfaces

  • New `ApplicationProtocol::Imap(ImapInfo)` variant with a `Display` formatter tuned for the connection-table column (requests show `IMAP LOGIN (alice)`; greetings show `IMAP OK (Dovecot)`; tagged responses show `IMAP OK`; continuation shows `IMAP +`).
  • DPI details panel in the TUI renders message type, tag, command, arguments, status, response, username, server software.
  • `ConnectionFilter` general search matches `imap`, command, username, status, and server software.
  • Per-protocol merge keeps stable identity fields (`username`, `server_software`) first-wins and dialog state latest-wins. UDP idle timeout is set to 30 minutes (IMAP IDLE sessions are long-lived).

Tests

14 new unit tests in `src/network/dpi/imap.rs`:

  • Server greeting with bracketed CAPABILITY → `Dovecot` extracted past the brackets.
  • PREAUTH greeting.
  • `LOGIN alice secret` → `alice` captured as username.
  • `LOGIN "alice@example.com" secret` → quotes stripped.
  • `SELECT INBOX`.
  • `UID FETCH 1:* (FLAGS RFC822.SIZE)`.
  • Tagged `OK [READ-WRITE] SELECT completed`.
  • `NO [AUTHENTICATIONFAILED]`.
  • Untagged data response (`* 23 EXISTS`).
  • Continuation (`+ Ready for additional command text`).
  • HTTP payload rejected.
  • Bare tag without command rejected.
  • Unknown command rejected.
  • `STARTTLS`.

Full suite locally on macOS / Apple Silicon:

  • `cargo test --all-features` → 338 passed, 0 failed.
  • `cargo clippy --all-features --all-targets -- -D warnings` → clean.
  • `cargo fmt --all -- --check` → clean.

Files touched

  • `src/network/dpi/imap.rs` — new module.
  • `src/network/dpi/mod.rs` — register module, add `PORT_IMAP`, dispatch IMAP after HTTP/SSH.
  • `src/network/types.rs` — `ImapInfo`, `ImapMessageType`, enum variant, `Display` arm, `sort_key`, dist counter, UDP state sentinel, 30-min idle timeout.
  • `src/network/merge.rs` — `merge_imap_info` (stable fields first-wins, dialog state latest-wins).
  • `src/filter.rs` — general-search match arm for IMAP metadata.
  • `src/ui.rs` — DPI details panel rendering for IMAP.

Test plan

  • Unit tests pass locally on macOS (Apple Silicon).
  • Live smoke against an IMAP server (recommended for the maintainer; I do not have an IMAP endpoint handy on this machine).

Adds best-effort Deep Packet Inspection for the plaintext IMAP4rev1 control channel (RFC 3501, RFC 7888 LITERAL+, RFC 2595 STARTTLS, RFC 4978 COMPRESS). Port 993 (IMAPS) is TLS-wrapped and is claimed by the HTTPS/TLS analyzer instead.

Detection: port 143 plus a signature that recognises all three IMAP frame shapes — tagged client request, tagged/untagged server response, command continuation. RFC 3501 tag grammar enforced (printable ASCII, length-capped) to keep parsing cheap on hostile traffic.

Metadata: tag, command + args, status (OK/NO/BAD/BYE/PREAUTH), response message, username from LOGIN <user>, server software extracted from the * OK / * PREAUTH greeting (skips the bracketed CAPABILITY list). Display formatter shows 'IMAP LOGIN (alice)' for requests and 'IMAP OK (Dovecot)' for greetings. Per-flow merge keeps identity fields first-wins and dialog state latest-wins.

Tests: 14 unit tests covering server greeting (Dovecot extracted past the [CAPABILITY ...] list), PREAUTH greeting, LOGIN with bare and quoted usernames, SELECT, UID FETCH, tagged OK/NO responses, untagged 23 EXISTS, command continuation, STARTTLS, HTTP/unknown/bare-tag rejection. cargo test --all-features: 338 passed. cargo clippy --all-features --all-targets -- -D warnings: clean. cargo fmt --all -- --check: clean.

Tracks ROADMAP DPI Enhancements (IMAP).
@domcyrus
Copy link
Copy Markdown
Owner

@0xghost42 thanks for the PR. Also on this one, before merging I want to take a bit more time to think about whether IMAP DPI is the right addition for RustNet, and I would also like to hear from other users.

My main hesitation is basically the same as for SMTP and is around how much plaintext IMAP traffic RustNet would actually see in practice. Modern mail clients almost universally use IMAPS on port 993 (implicit TLS) or IMAP on 143 with STARTTLS, both of which are encrypted from the first or near-first byte. Plaintext IMAP on 143 is essentially gone from real-world traffic outside intentional test setups and internal diagnostics.

So the open question is whether enough RustNet users actually run setups where they would see plaintext IMAP. If you (or anyone reading this) have a concrete use case for this, please leave a note here so I can gauge demand. I will keep the PR open in the meantime.

Therefore also here I would like to mention that I appreciate the work but I want to think it through before merging.

@0xghost42
Copy link
Copy Markdown
Contributor Author

@domcyrus same concern noted. Adding concrete use cases here too in case demand signal matters:

Plaintext IMAP visibility is still useful for:

  • Internal IMAP / submission inside a LAN. Lots of small offices and home labs still run Dovecot / Cyrus / mailcow with plaintext IMAP on 143 inside their network, terminating TLS at a reverse proxy or simply not bothering on a trusted segment. RustNet on those boxes would surface the SELECT / FETCH / UID SEARCH dialog without an external dissector.

  • STARTTLS upgrade visibility on 143. Even when clients ultimately upgrade, the initial * OK ..., CAPABILITY, STARTTLS request, and the OK Begin TLS negotiation now response are plaintext. Surfacing that handshake lets operators confirm "this client actually issues STARTTLS" vs "this client logs in over cleartext IMAP" — which is one of the more common mail-misconfig findings on real audits.

  • Honeypots / detection labs. mailpit, MailHog, opencanary's IMAP module, and various dovecot honeypots all listen on cleartext 143. Anyone running those for IOC collection or training data wants to see the LOGIN attempts and folder probes at the network layer (without trusting only the honeypot's own log).

  • Debugging IMAP clients / IDLE / push setups. Plenty of mobile mail clients, mu4e, isync / mbsync, fdm, OfflineIMAP, and various open-source mail UIs still talk to local dev servers in cleartext. Rendering the command + response (e.g. UID FETCH ranges, IDLE keep-alives, BAD / NO replies) helps debug "why isn't mail syncing" without strace / mitmproxy.

  • Forensics on legacy / embedded. Older CPE, printers with scan-to-email, and some IoT gear still speak plaintext IMAP on 143 to dump scan / status reports. RustNet operators looking at "what's this printer doing on the network" would benefit from seeing the LOGIN / SELECT exchange directly.

If maintenance surface is the worry, happy to keep the parser tight: detect via * OK, LOGIN, SELECT, CAPABILITY, STARTTLS keywords on port 143; extract command + tagged response status + the folder argument from SELECT / EXAMINE. No body / FETCH-content parsing, no AUTHENTICATE payload, no MIME structure walk. Scope comparable to what landed for FTP in #266.

Will let this sit and see if others chime in. Either call works — close if it still doesn't pencil out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants