smp-server: support namespaces by shumvgolove · Pull Request #1784 · simplex-chat/simplexmq

shumvgolove · 2026-05-22T08:00:15Z

No description provided.

… TLD case, shutdown)

…local)

…r, CSV order)

…ngEndpoint timeout)

- Reject IPv6 aliases of 169.254.169.254 (IPv4-compatible / IPv4-mapped / 6to4 / NAT64) via numeric range check on parsed IPv6. - Disable HTTP redirects on the Eth RPC request. - Restrict SimplexName labels to ASCII (Cyrillic/Greek/full-width otherwise hash to different on-chain records and diverge from UTS-46 registrars). - pingEndpoint: only JsonRpcErr means "reachable"; transport/decode failures fail startup. boundedIniInt: readMaybe over partial read. - Add 127.0.0.0/8 and 0.0.0.0 to isLoopback. - Replace hand-rolled hex helpers with Data.ByteArray.Encoding; raise managerConnCount to match rpcMaxConcurrency; hex Show for NameOwner. - Fuse parallel http/https when into unless+case; drop reverse/re-reverse in mkDomain TLDWeb; first AbiInvariantViolated; Nothing <$ decodeAddress; forM_ (eitherToMaybe ...); >>= chain in NameOwner FromJSON. - Drop dead imports/exports/pragmas and two restating comments. - Tests: factor unsafeOwner/unsafeLink, addr1/2/3, testNamesConfig; add non-ASCII label rejection coverage.

The bare-name fallback and bareDomain parser would otherwise consume arbitrarily many non-space bytes via takeWhile1 before any validation or length check. A crafted multi-megabyte token would be decoded as UTF-8 and re-parsed in full before being rejected. Introduce `boundedNonSpace` (scan with 253-byte cap) at the two takeWhile1 sites. Inputs longer than 253 bytes leave residue that parseOnly's implicit endOfInput rejects, so the parser fails fast without ever allocating the full input. The bound is the DNS full-domain limit, chosen for being a familiar ceiling generous enough to cover any realistic SimpleX name (longest plausible @user.subdomain.simplex stays well under 100 bytes). No per-label cap — SimpleX names don't go through DNS label resolution and there's no semantic reason to constrain individual labels.

…out auth) validateUrl gains two operator-friendly relaxations and a regression test: - Allow a path prefix (e.g. https://gw.example.com:443/snrc) for a resolver behind a reverse-proxy sub-path; /resolve/<name> and /health are appended (HttpResolver already strips one trailing slash, so root and sub-path behave identically). Query/fragment/userinfo stay rejected. - Off-loopback, reject only http WITH resolver_auth (the Authorization header would travel in cleartext). http without auth is now allowed (no secret to leak; resolver data is public — also lets dev setups reach a host resolver via http://host.docker.internal). https is always allowed, with or without auth. Plain http has no response integrity; intended for trusted/local networks only. Exports validateUrl and adds validateUrlSpec (11 cases) to SMPNamesTests.

RSLV collapsed every non-hit (no resolver, malformed name, not found, backing-store failure) to ERR AUTH, so a client iterating its configured servers could not tell "this router has no resolver, try the next" from "name not registered, stop", and a transient backend error read as an authoritative miss. Names capability is runtime config, orthogonal to the linear SMP version (a future v21 router without [NAMES] must still advertise v21), so it is signalled by a command-time error like allowSMPProxy, not by the version range: no resolver configured -> ERR CMD PROHIBITED (client skips, tries next) backing-store failure -> ERR INTERNAL (transient: retry/surface) not found / malformed -> ERR AUTH (authoritative "no such name") Update the protocol spec error table and add agent tests for the no-resolver (CMD PROHIBITED) and backend-failure (INTERNAL) paths.

Addresses epoberezkin's review (PR #1784). Name resolution becomes a server role like proxy; the agent owns resolution + server selection; one error type flows through the whole stack. - ServerRoles gains `names`; UserServers gains `nameSrvs` (opt-in list); resolveSimplexName drops the explicit server arg and picks a names-capable server via getNextServer. - RSLV carries SimplexNameDomain (was RslvRequest): no JSON on the wire, contract dropped, name validated at parse (invalid -> CMD SYNTAX). - Version check moves from the encoder to Client.hs (no ERR to server). - ErrorType.NAME {nameErr :: NameErrorType} (+ AgentErrorType.NAME), wire- and JSON-encoded; resolver errors surface with diagnostics. Success response renamed NAME -> RNAME to free the collision. - NameOwner -> EthAddress (record selector); NameRecord derives FromJSON and gains field-ordered Encoding; per-field caps removed. - Remove newEnvWithNames / runSMPServerBlockingWithNames test seams; stub resolver folded into ServerConfig.namesResolverCall_.

NameResolverStatsData adds 6 lines to the server stats backup (the "rslvStats:" header plus the reqs/succ/notFound/resolverErrs/disabled fields), so testRestoreMessages' expected stats-backup line count is 95 -> 101.

epoberezkin · 2026-06-20T08:48:16Z

+-- Wire encoding for the SMP NAME response: field-ordered smpEncode, not embedded
+-- JSON. Field order = record declaration order. EthAddress encodes as its raw
+-- 20 bytes (length-prefixed via the ByteString instance).
+instance Encoding NameRecord where


wire encoding should use JSON

evgeny-simplex · 2026-06-20T09:36:59Z

+  namesEnv <- case namesConfig of
+    Nothing -> pure Nothing
+    Just nc -> case namesResolverCall_ of


Suggested change

namesEnv <- case namesConfig of

Nothing -> pure Nothing

Just nc -> case namesResolverCall_ of

namesEnv <- forM namesConfig $ \nc -> case namesResolverCall_ of

this is a strange way to inject stub. Shouldn't it be the same endpoint?

evgeny-simplex · 2026-06-20T09:38:47Z

+    Nothing -> pure Nothing
+    Just nc -> case namesResolverCall_ of
+      -- test seam: stub resolver, no real HTTP env or startup probe
+      Just call -> Just <$> newNamesEnvWith nc call Nothing


Suggested change

Just call -> Just <$> newNamesEnvWith nc call Nothing

Just call -> newNamesEnvWith nc call Nothing

evgeny-simplex · 2026-06-20T09:38:58Z

+        pingEndpoint env >>= \case
+          Right _ -> logInfo "[NAMES] endpoint probe ok"
+          Left e -> logWarn $ "[NAMES] endpoint probe failed (server will still start, RSLV will return ERR (NAME ...) until reachable): " <> tshow e
+        pure (Just env)


Suggested change

pure (Just env)

pure env

evgeny-simplex · 2026-06-20T09:41:08Z

+      Nothing -> do
+        logInfo $ "[NAMES] resolver enabled, endpoint=" <> scrubUrl (resolverEndpoint nc)
+        when allowSMPProxy $
+          logWarn "[NAMES] enable: on on a proxy-role host: slow RSLV calls can serialise other forwarded commands on the same proxy-relay session. For high-volume deployments, run [NAMES] on a separate host."


why? this is a strange comment. We allow SMP proxy on all hosts, name resolution should be async in the same way all proxy calls are async.

evgeny-simplex · 2026-06-20T09:42:53Z

+        \# Operator runs the resolver alongside smp-server (default port 8000)\n\
+        \# with its own Ethereum JSON-RPC endpoint configured in resolver.toml.\n\
+        \# Co-locating with the proxy role logs a startup advisory: slow RSLV calls can\n\
+        \# serialise other forwarded commands on the same proxy-relay session.\n\
+        \# For high-volume deployments, run [NAMES] on a separate host.\n\
+        \# Restart required to change settings.\n\


this is wrong and should not be the case

also, we don't colocate it with SMP, right?

evgeny-simplex · 2026-06-20T09:44:51Z

+          else case J.eitherDecodeStrict (BL.toStrict bs) of
+            Left e -> pure (Left (InvalidJson e))
+            Right v -> pure (Right v)


Suggested change

else case J.eitherDecodeStrict (BL.toStrict bs) of

Left e -> pure (Left (InvalidJson e))

Right v -> pure (Right v)

else first InvalidJson <$> J.eitherDecodeStrict (BL.toStrict bs)

in general, please review and refactor such trivial cases into forM, mapM, first, second, bimap, maybe, etc.

evgeny-simplex · 2026-06-20T09:50:35Z

+
+doGet :: ResolverEnv -> Text -> IO (Either ResolverError J.Value)
+doGet ResolverEnv {manager, baseUrl, authHdr, timeoutMicro, maxResponseBytes} path = do
+  req0 <- parseRequest (T.unpack (baseUrl <> path))


this T.unpack is a smell that incorrect type is passed from upstream

evgeny-simplex · 2026-06-20T09:51:34Z

+-- | Percent-encode a name component (path-safe). Aggressive: encode every
+-- byte that isn't an unreserved character per RFC 3986. The resolver expects
+-- raw labels (e.g., `alice.simplex`); slashes and other ASCII punctuation
+-- would change the request path semantics if passed through verbatim.
+percentEncode :: Text -> Text
+percentEncode = decodeLatin1 . urlEncode True . encodeUtf8


trivial function that is not needed if you use correct types (String or ByteString)

evgeny-simplex · 2026-06-20T09:53:22Z

+                -- and the smpEncoded NameRecord is <= its JSON body, so capping
+                -- the body here guarantees the response always frames. An
+                -- over-cap body fails as BodyTooLarge -> ERR (NAME (RESOLVER ..)).
+                resolverMaxResponseBytes = boundedIniInt 16000 1024 16000 "resolver_max_response_bytes"


and in some other place it's 64kb and no 16. If we need larger size, we could use zstd compression in SMP protocol.

evgeny-simplex · 2026-06-20T09:54:06Z

+    other -> Left $ "unexpected port syntax: " <> other
+  unless (null (uriQuery uri)) $ Left "query string not allowed (it does not compose with the appended /resolve/<name> path)"
+  unless (null (uriFragment uri)) $ Left "fragment not allowed (fragments are never sent to the server)"
+  -- A path prefix is allowed and used as the base for /resolve/<name> and


this is very complex and unreadable, needs to be simplified

we use library for URI parsing, why we are re-implement it here?

evgeny-simplex · 2026-06-20T09:55:35Z

+--   * link-local hosts (169.254.0.0/16, including the cloud metadata IP
+--     169.254.169.254) are rejected unconditionally
+validateUrl :: Text -> Maybe RpcAuth -> Either String Text
+validateUrl url auth_ = do


this function needs to be fully re-written or used from library. It has to be like 1-3 lines of code, not 50.

evgeny-simplex · 2026-06-20T09:55:59Z

+-- | Parse an rpc_auth INI value. Scheme keyword is case-insensitive so
+-- "Bearer <token>" / "BEARER <token>" (Caddy / RFC 7235 convention) work
+-- as well as the lowercase form.
+parseRpcAuth :: Text -> Either String RpcAuth


same here - it's an existing library function, it should not be in our code.

the library is uri-bytestring, just needs to be added and used

evgeny-simplex · 2026-06-20T10:00:02Z

+  | -- | no name-resolving servers configured (agent-originated only)
+    NO_SERVERS


as it's agent-originated, it must not be in this type - it must be in Agent errors

This type is only for errors returned by the server, it can't contain errors reported upstream

evgeny-simplex · 2026-06-20T10:02:20Z

+  = -- | the names role / resolver is not configured on this server
+    NO_RESOLVER
+  | -- | the name is not registered (resolver returned not-found)
+    NO_NAME


NOT_FOUND would be more typical

There may also be expired error - I don't know how it will be reported by resolver, so probably not needed for now.

evgeny-simplex · 2026-06-20T10:03:17Z

+    -- Name is validated at parse (invalid syntax fails here -> CMD error),
+    -- so the handler only ever sees a valid SimplexNameDomain.


Suggested change

-- Name is validated at parse (invalid syntax fails here -> CMD error),

-- so the handler only ever sees a valid SimplexNameDomain.

redundant

evgeny-simplex · 2026-06-20T10:04:10Z

    PRXY host auth_ -> e (PRXY_, ' ', host, auth_)
    PFWD fwdV pubKey (EncTransmission s) -> e (PFWD_, ' ', fwdV, pubKey, Tail s)
    RFWD (EncFwdTransmission s) -> e (RFWD_, ' ', Tail s)
+    -- Version gating is the client's job (Client.hs), not the encoder's.


Suggested change

-- Version gating is the client's job (Client.hs), not the encoder's.

evgeny-simplex · 2026-06-20T10:06:11Z

    PFWD fwdV pubKey (EncTransmission s) -> e (PFWD_, ' ', fwdV, pubKey, Tail s)
    RFWD (EncFwdTransmission s) -> e (RFWD_, ' ', Tail s)
+    -- Version gating is the client's job (Client.hs), not the encoder's.
+    RSLV d -> e (RSLV_, ' ', Tail (strEncode d))


Suggested change

RSLV d -> e (RSLV_, ' ', Tail (strEncode d))

RSLV d -> e (RSLV_, ' ', d)

maybe I am missing something, but using Tail explicitly here is redundant - all it achieves is preventing length prefixing of the string. Directly using d here achieves the same.

evgeny-simplex · 2026-06-20T10:07:30Z

+    -- so the handler only ever sees a valid SimplexNameDomain.
+    CT SResolver RSLV_ -> do
+      Tail bs <- _smpP
+      either fail (pure . Cmd SResolver . RSLV) (strDecode bs)


Suggested change

either fail (pure . Cmd SResolver . RSLV) (strDecode bs)

Cmd . SResolver . RSLV <$> _smpP

evgeny-simplex · 2026-06-20T10:08:13Z

    PONG -> e PONG_
+    -- Field-ordered Encoding NameRecord (no JSON on the wire); a response that
+    -- arrived is already on a supported version, so no version gate.
+    RNAME rec -> e (RNAME_, ' ', rec)


Suggested change

RNAME rec -> e (RNAME_, ' ', rec)

RNAME rec -> e (RNAME_, ' ', Tail $ toStrict $ J.encode rec)

evgeny-simplex · 2026-06-20T10:09:40Z

    OK_ -> pure OK
    ERR_ -> ERR <$> _smpP
    PONG_ -> pure PONG
+    RNAME_ -> RNAME <$> _smpP


Suggested change

RNAME_ -> RNAME <$> _smpP

RNAME_ -> fmap RNAME . J.decodeStrict . unTail <$?> _smpP

evgeny-simplex · 2026-06-20T10:10:49Z

+  smpEncode = \case
+    NO_RESOLVER -> "NO_RESOLVER"
+    NO_NAME -> "NO_NAME"
+    NO_SERVERS -> "NO_SERVERS"


evgeny-simplex · 2026-06-20T10:12:35Z

+        -- error that reaches the client as ERR (NAME ...).
+        (selector, msg) <- asks namesEnv >>= \case
+          Nothing -> pure (rslvDisabled, ERR $ NAME NO_RESOLVER)
+          Just nenv -> liftIO (resolveName nenv d) <&> \case


that's the crux of the problem. You call resolver inline, blocking further command processing (and that's what all those comments were about). This has to use the same pattern as we use to create proxy session - it must be processed asynchronously, and response send from forked thread. The function simply returns Nothing as command response, deferring response to forked thread.

shumvgolove requested review from epoberezkin and spaced4ndy as code owners May 22, 2026 08:00

shumvgolove force-pushed the sh/smp-namespace branch 3 times, most recently from a6b960b to 236e69d Compare May 22, 2026 11:26

epoberezkin and others added 11 commits May 29, 2026 09:37

Merge branch 'master' into stable

92fb0ea

smp-server: namespaces resolver scaffolding

19db2c8

smp-server: Names resolver hardening + cleanup

3999fac

smp-server: fuse parallel dispatchers

99b1330

smp-server: JSON wire format for NameRecord + Names.hs restructure

c0e14e3

smp-server: redact RpcAuth in Show

6b216ca

smp-server: JSON wire fixups + spec rewrite + small cleanups

c812725

plan: prepend implementation-diverged banner

66bca0c

move SimplexName into shared module

daac3e2

Merge origin/stable

1e2cf23

smp-server: name + contract whitelist on RSLV

c3e7b61

shumvgolove force-pushed the sh/smp-namespace branch from 37764a4 to c3e7b61 Compare May 29, 2026 16:11

shumvgolove added 8 commits June 1, 2026 09:51

smp-server: address audit findings (canonical JSON, INI guards, SSRF,…

3c93489

… TLD case, shutdown)

smp-server: round 2 audit fixes (label case, response cap, ipv6 link-…

f686a94

…local)

smp-server: round 3 audit fixes (SSRF coverage, drop noop closeManage…

ba245e7

…r, CSV order)

smp-server: round 4 audit fixes (0X-hex host, expanded IPv6 forms, pi…

1d394f5

…ngEndpoint timeout)

smp-server: hardcode TldRegistries (drop registry_tld_* INI keys)

b66d973

Merge remote-tracking branch 'origin/master' into sh/smp-namespace

1950634

shumvgolove force-pushed the sh/smp-namespace branch from e90b13e to 9e4d8d9 Compare June 3, 2026 19:04

shumvgolove and others added 4 commits June 8, 2026 11:37

namespace: switch to Python HTTP resolver + agent plumbing (#1796)

92b3d04

namespace: NameRecord links as arrays (multi-link, cap 5)

f555e9a

epoberezkin reviewed Jun 12, 2026

View reviewed changes

Comment thread src/Simplex/Messaging/Names/Record.hs Outdated

epoberezkin requested changes Jun 12, 2026

View reviewed changes

epoberezkin reviewed Jun 12, 2026

View reviewed changes

Comment thread src/Simplex/Messaging/Agent.hs Outdated

shumvgolove requested a review from epoberezkin June 12, 2026 15:56

test(server): update stats backup line count

ce69adf

NameResolverStatsData adds 6 lines to the server stats backup (the "rslvStats:" header plus the reqs/succ/notFound/resolverErrs/disabled fields), so testRestoreMessages' expected stats-backup line count is 95 -> 101.

epoberezkin force-pushed the master branch from 9ddc33e to 8e0b8de Compare June 17, 2026 16:15

epoberezkin reviewed Jun 20, 2026

View reviewed changes

evgeny-simplex reviewed Jun 20, 2026

View reviewed changes

	Just call -> Just <$> newNamesEnvWith nc call Nothing
	Just call -> newNamesEnvWith nc call Nothing

		\| -- \| no name-resolving servers configured (agent-originated only)
		NO_SERVERS

		-- Name is validated at parse (invalid syntax fails here -> CMD error),
		-- so the handler only ever sees a valid SimplexNameDomain.

	RSLV d -> e (RSLV_, ' ', Tail (strEncode d))
	RSLV d -> e (RSLV_, ' ', d)

	either fail (pure . Cmd SResolver . RSLV) (strDecode bs)
	Cmd . SResolver . RSLV <$> _smpP

	RNAME rec -> e (RNAME_, ' ', rec)
	RNAME rec -> e (RNAME_, ' ', Tail $ toStrict $ J.encode rec)

	RNAME_ -> RNAME <$> _smpP
	RNAME_ -> fmap RNAME . J.decodeStrict . unTail <$?> _smpP

Uh oh!

Conversation

shumvgolove commented May 22, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

evgeny-simplex Jun 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

evgeny-simplex Jun 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

evgeny-simplex Jun 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

evgeny-simplex Jun 20, 2026 •

edited

Loading

evgeny-simplex Jun 20, 2026 •

edited

Loading

evgeny-simplex Jun 20, 2026 •

edited

Loading