doctor: deepen DC verification beyond TCP connect

[dolonet]

Following up on #488 (comment) — opening this so we can agree on shape before a PR.

Today Doctor.checkNetworkAddresses (internal/cli/doctor.go:286) marks a DC healthy as soon as ntw.DialContext returns a net.Conn. Any process listening on :443 passes, which is the shallowness you flagged.

What I'd like to do: after the TCP connect, wrap the conn with the existing obfuscated2 framing from mtglib/internal/obfuscation, send req_pq_multi with a random nonce, and expect a resPQ echoing it back. That's unauthenticated (no auth_key, no state introduced), one round-trip, and a generic listener can't fake it. Two TL messages, so I'd hand-roll the (de)serialization rather than pull in gotd/td or similar — but open to the dependency if you'd prefer.

Output would gain an rpc <rtt> next to the connect line, and failures would distinguish "TCP failed" from "TCP ok, RPC failed". Same treatment for both native and chained-proxy paths (the chained-proxy case is exactly where shallow checks mislead). checkFrontingDomain stays as-is. One PR.

Does this match what you had in mind, or did you want something heavier (full DH + authenticated ping)? Happy to put up the PR if the approach looks right.

[9seconds]

I tried 3 options:

1. Using gotd. This works but bloats mtg 10x times
2. Using different implementations like https://github.com/xelaj/mtproto - works but fail
3. Taking a look at different implementations of mtproto proxies - put on hold due to the lack of time

Full libraries like gotd work but they require to have registered application - not sure if it is a good idea to cloak here.

send req_pq_multi with a random nonce, and expect a resPQ echoing it back. That's unauthenticated (no auth_key, no state introduced), one round-trip, and a generic listener can't fake it. Two TL messages, so I'd hand-roll the (de)serialization rather than pull in gotd/td or similar — but open to the dependency if you'd prefer.

Can you share any example or doc? This sound as a good solution.

[dolonet]

Sure — here's the docs I leaned on plus a working PoC.

Docs

• TL schema (constructor IDs, field types): https://core.telegram.org/schema/mtproto
• Auth-key handshake step 1: https://core.telegram.org/mtproto/auth_key
• Unencrypted message envelope (the gotcha that nearly fooled me): https://core.telegram.org/mtproto/description#unencrypted-message
• Padded-intermediate transport: https://core.telegram.org/mtproto/mtproto-transports#padded-intermediate

What's on the wire

Two TL types are needed:

req_pq_multi#be7e8ef1 nonce:int128 = ResPQ;
resPQ#05162463 nonce:int128 server_nonce:int128 pq:string
    server_public_key_fingerprints:Vector<long> = ResPQ;

Wrapped, in order, the bytes we send over the obfuscated2 conn established by mtglib/internal/obfuscation are:

[ padded-intermediate length: uint32_le ]
  [ MTProto unencrypted message:
      auth_key_id  : uint64_le = 0
      message_id   : uint64_le
      msg_data_len : uint32_le = 20
      msg_data     : 0xbe7e8ef1 (uint32_le) | nonce (16 random bytes) ]

The reply is the same shape with 0x05162463 and the nonce echoed back.
Skip the 20-byte envelope, check the constructor, check the nonce — done.

PoC

Pushed as a leaf package at mtglib/dcprobe/ on branch
dolonet:doctor/rpc-probe-poc. Two files, ~250 LOC total, no new
dependencies — reuses mtglib/internal/obfuscation.Obfuscator{}.SendHandshake
exactly the way mtglib/proxy.go does for outbound DC connections.

• https://github.com/dolonet/mtg-multi/blob/doctor/rpc-probe-poc/mtglib/dcprobe/probe.go
• https://github.com/dolonet/mtg-multi/blob/doctor/rpc-probe-poc/mtglib/dcprobe/probe_test.go

To run locally without entangling with the rest of the fork, single-branch
clone:

git clone --branch doctor/rpc-probe-poc --single-branch \
  https://github.com/dolonet/mtg-multi.git mtg-dcprobe-poc
cd mtg-dcprobe-poc
MTG_PROBE_NETWORK=1 go test -v ./mtglib/dcprobe/

Output from a Hetzner Helsinki host:

=== RUN   TestProbeAgainstTelegramDCs/149.154.175.50:443
    DC 1 (149.154.175.50:443): rtt=129.97ms
=== RUN   TestProbeAgainstTelegramDCs/149.154.167.51:443
    DC 2 (149.154.167.51:443): rtt=30.25ms
=== RUN   TestProbeAgainstTelegramDCs/95.161.76.100:443
    DC 2 (95.161.76.100:443): rtt=27.66ms
=== RUN   TestProbeAgainstTelegramDCs/149.154.175.100:443
    DC 3 (149.154.175.100:443): rtt=129.61ms
=== RUN   TestProbeAgainstTelegramDCs/149.154.167.91:443
    DC 4 (149.154.167.91:443): rtt=27.35ms
=== RUN   TestProbeAgainstTelegramDCs/149.154.171.5:443
    DC 5 (149.154.171.5:443): rtt=187.15ms
=== RUN   TestProbeAgainstTelegramDCs/[2001:b28:f23d:f001::a]:443
    DC 1 ([2001:b28:f23d:f001::a]:443): rtt=130.64ms
=== RUN   TestProbeAgainstTelegramDCs/[2001:67c:04e8:f002::a]:443
    DC 2 ([2001:67c:04e8:f002::a]:443): rtt=27.67ms
PASS

Negative test (MTG_PROBE_FAKE_TARGET=127.0.0.1:22 against local sshd):

probe vs 127.0.0.1:22: peer did not respond with a matching resPQ:
  resPQ frame too large (...) (errors.Is(ErrNotTelegram)=true)

On wiring it into doctor

One Go-isms snag: internal/cli/doctor.go can't import
mtglib/internal/obfuscation directly (different internal/ boundary), so a
real doctor integration needs a thin public entry point in mtglib —
something like mtglib.ProbeDC(ctx, conn, dc) (time.Duration, error) that
wraps the package above. I left it as a leaf package in the PoC to keep this
diff readable; happy to rearrange in the PR.

If you're OK with this approach overall, I'll turn it into a single PR that:

1. promotes dcprobe to a small public mtglib API,
2. adds the post-TCP probe to Doctor.checkNetworkAddresses for both native
   and chained-proxy paths,
3. surfaces rpc <rtt> next to the connect line and distinguishes
   "TCP failed" from "TCP ok, RPC failed" in the output.

Tell me if you'd rather see a different shape (e.g. inline the probe in
doctor.go and skip the public API — that's also fine, just slightly
duplicative).

[9seconds]

Great, looks very good! Amazing work. I would definitely like to get it in mtg for sure. Speaking on internal package, I think nothing really prevents us from moving obfuscation somewhere else. It will simplify many things + gonna make it more straightforward.

I'm unfortunately sunk in job-related ASAPs now and can work on that only next week so feel free to do that if you like.

[9seconds]

@dolonet I also added you as a collaborator to this repository. Thank you very much for your help, please consider it as a small sign of my huge gratitude. Please check your invites

[dolonet]

@dolonet I also added you as a collaborator to this repository. Thank you very much for your help, please consider it as a small sign of my huge gratitude. Please check your invites

Thank you for the honour. I don't know how to reach you personally (tried to nudge you on LinkedIn). At first I thought that this is not the right move. Now I see you have not enough time and I can help here at least a little. Not promising to do all the heavy lifting. Not sure if I am even qualified enough. So please forgive me if I ever need your input on some arbitrary decisions, not making them myself

[dolonet]

Small thing I noticed while settling in: master has no branch protection enabled — direct push and force-push are both allowed. Given how much depends on this proxy, a minimal rule on master ("require PR before merging" + "no force pushes") would fool-proof against me (or anyone) misclicking. ~30 seconds in Settings → Branches; happy to send the exact toggles I'd suggest if useful.

[dolonet]

Status update: PR #496 (the doctor probe implementation) is in draft, stacked on top of #495 (which promotes obfuscation out of internal). It will leave draft once #495 lands.

[9seconds]

@dolonet #495 is merged

[dolonet]

Thanks — rebased #496 on master, the #495 commit dropped out of the stack so it's now just the dcprobe package plus the doctor integration. CI green, draft lifted, ready for review whenever convenient.