feat(rpc): add /genesisBlock and /identities GET endpoints#921
Conversation
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (4)
✅ Files skipped from review due to trivial changes (1)
🚧 Files skipped from review as they are similar to previous changes (3)
WalkthroughAdds GET /genesisBlock (full persisted block 0) and GET /identities (keyset-paginated pubkey + identities) endpoints, introduces an in-process ConcurrencyGate with tests, new identity-specific constants and rate-limit wiring, implements GCR.listIdentities, and adds a runbook documenting behavior and DDoS protections. ChangesGenesis and Identities Endpoints
Sequence Diagram(s)sequenceDiagram
participant Client
participant RateLimiter
participant ConcurrencyGate
participant GCR
participant Postgres
Client->>RateLimiter: GET /identities?limit=100&cursor=abc
alt rate limit exceeded
RateLimiter-->>Client: 429 Too Many Requests
else
RateLimiter->>ConcurrencyGate: acquire()
alt queue full or timeout
ConcurrencyGate-->>Client: 503 Service busy + Retry-After
else slot acquired
ConcurrencyGate->>GCR: listIdentities(limit, cursor)
GCR->>Postgres: SELECT pubkey, identities WHERE pubkey > :cursor ORDER BY pubkey ASC LIMIT N
alt query error
Postgres-->>GCR: error
GCR-->>Client: 500 error
else success
Postgres-->>GCR: rows
GCR-->>ConcurrencyGate: RPCResponse { identities, nextCursor }
ConcurrencyGate-->>Client: 200 OK with payload
ConcurrencyGate->>ConcurrencyGate: release()
end
end
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested labels
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
ESLint install failed. For unrecoverable errors, disable the tool in CodeRabbit configuration. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@greptile review |
Add two read-only GET routes to the node RPC HTTP server, plus DDoS hardening for the expensive one. - /genesisBlock: returns the full genesis Blocks record (block 0) via Chain.getGenesisBlock(). Distinct from the existing /genesis, which returns only the embedded genesisData. Blocks has no bigint columns so it serializes cleanly. - /identities: paginated listing of every account's linked identities, returning only pubkey + the identities jsonb blob per row (never the bigint balance, which JSON.stringify cannot serialize). New GCR.listIdentities(limit, cursor) uses keyset pagination on the pubkey primary key and column projection so a single request cannot pull the whole gcr_main table into memory. limit is clamped to [1, 1000]. DDoS hardening on /identities (three independent brakes): - Per-IP rate limit: 30 req/60s (vs 2000 default GET allowance), via the rate limiter pathMethodMap + methodLimits config. - Global concurrency gate: at most 3 /identities handlers run at once across ALL callers (catches distributed many-IP bursts the per-IP limit cannot). Overflow queues up to 12 deep and waits at most 2s for a slot, then returns 503 + Retry-After. New ConcurrencyGate util: a pure in-process counting semaphore with a bounded FIFO wait queue + per-acquire timeout, covered by 15 unit tests. - Bounded work per request: hard limit cap + keyset pagination + column projection. Bun workers were considered and deliberately not used: /identities is I/O-bound (a Postgres query), not CPU-bound, so workers would add IPC overhead without reducing DB load. The gate caps the actual bottleneck. Additive only. New rate-limit/gate tunables live in src/utilities/constants.ts. Runbook added at docs/runbooks/genesis-and-identities-endpoints.md.
tcsenpai
force-pushed
the
feat/genesis-block-and-identities-endpoints
branch
from
0f4c306 to
d14a14b
Compare
|
@coderabbitai full review |
✅ Action performedFull review finished. |
Greptile SummaryAdds two new GET endpoints to the RPC HTTP server: Confidence Score: 5/5Safe to merge — purely additive, all DDoS brakes correctly wired, previous rate-limit bypass for /genesisBlock addressed. The ConcurrencyGate logic is sound and well-tested. Rate-limiter mapping, concurrency gate wiring, error handling, and keyset pagination are all implemented correctly. The single finding is a maintenance nit in the parameter default. No files require special attention. Important Files Changed
Reviews (4): Last reviewed commit: "fix(rpc): rate-limit /genesisBlock + hoi..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/libs/blockchain/gcr/gcr.ts`:
- Around line 1146-1148: The computed pageSize can become 0 for fractional
positive parsedLimit (e.g., 0.5) causing an empty-page dereference later; clamp
the computed value to a minimum of 1 when setting pageSize (use Math.max(1,
Math.min(Math.floor(parsedLimit), MAX_LIMIT)) or fall back to DEFAULT_LIMIT) and
also add a defensive check before dereferencing page[0] (in the code paths
around the existing page and cursor handling at lines handling page length) to
avoid accessing undefined when page is empty; update references to parsedLimit,
pageSize, MAX_LIMIT, DEFAULT_LIMIT and the page/cursor handling accordingly.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 21dd19ee-b852-4a75-a9e5-eee70280dd2d
📒 Files selected for processing (8)
docs/runbooks/genesis-and-identities-endpoints.mdsrc/libs/blockchain/gcr/gcr.tssrc/libs/network/middleware/rateLimiter.tssrc/libs/network/server_rpc.tssrc/libs/network/utils/concurrencyGate.test.tssrc/libs/network/utils/concurrencyGate.tssrc/utilities/constants.tssrc/utilities/sharedState.ts
A fractional positive limit (e.g. ?limit=0.5) floored to 0, making pageSize 0; the full-page check (identities.length === pageSize) then matched an empty result and dereferenced identities[-1], returning 500. Clamp pageSize to a minimum of 1 and guard nextCursor on length > 0. Addresses CodeRabbit review feedback on PR #921.
…onstants Address Greptile review on PR #921: - /genesisBlock had no pathMethodMap entry, so GET requests fell through to the lax POST bucket (200k/24h) instead of the default GET limit. Map it to the 'genesisblock' method -> default per-IP GET allowance (2000/60s). - Move /identities DEFAULT_LIMIT (100) and MAX_LIMIT (1000) out of listIdentities() into constants.ts (IDENTITIES_DEFAULT_LIMIT / IDENTITIES_MAX_LIMIT), alongside the other /identities tunables, so the page-size bounds are configurable without touching GCR code.
|
@greptile review |
A fractional positive limit (e.g. ?limit=0.5) floored to 0, making pageSize 0; the full-page check (identities.length === pageSize) then matched an empty result and dereferenced identities[-1], returning 500. Clamp pageSize to a minimum of 1 and guard nextCursor on length > 0. Addresses CodeRabbit review feedback on PR #921.
1 participant
What your code does
The Demos node runs an HTTP RPC server (default port 53550) that exposes a
set of read-only GET routes — things like
/health,/peerlist,/version, and/genesis. Each route reads some piece of node or chainstate and returns it as JSON. They all sit behind one shared middleware
stack: CORS, a per-IP rate limiter, and JSON formatting.
Two things were missing from that set:
a
/genesisroute, but it only returns the embedded genesis data(chain params, balances, validators) parsed out of the block — not the
block record itself (hash, number, signatures, the raw content envelope).
GCR (
gcr_maintable); each row has apubkeyand anidentitiesjsonbblob holding the user's linked cross-chain / web2 / pqc identities. There
was no endpoint that enumerated them.
What this enables
/genesisBlock— dump the whole genesis block (block 0) as stored, fortooling/debugging that needs block-level fields, not just the chain
params.
/identities— paginate through every account's linked identities(
pubkey+identitiesonly), so an operator or indexer can walk thefull identity set without a bespoke DB query.
How this lands
/genesisBlockreturnsChain.getGenesisBlock()(block 0) verbatim.503 if the block isn't available yet (node still booting).
Blockshas nobigint columns, so it serializes safely.
/identitiesis backed by a newGCR.listIdentities(limit, cursor):pubkeyprimary key(
WHERE pubkey > :cursor ORDER BY pubkey ASC LIMIT n) — O(log n) at anydepth, unlike
OFFSET.pubkey+identitiesonly — keeps thepayload focused and never touches the
bigintbalancecolumn (whichJSON.stringifycannot serialize).limitclamped to[1, 1000]; response carriesnextCursor(null atend of table).
/identities(it's the most expensive route — apaginated full-table read over jsonb-heavy rows). Three independent brakes:
rate limiter's
pathMethodMap+methodLimits./identitieshandlers runat once across all callers, catching distributed (many-IP) bursts the
per-IP limit can't. Overflow queues up to 12 deep, waits ≤2s for a slot,
then returns 503 +
Retry-After. Implemented by a newConcurrencyGateutil: a pure in-process counting semaphore with abounded FIFO wait queue and per-acquire timeout (15 unit tests).
limitcap + keyset pagination +column projection.
txValidatorPool) offload CPU-bound crypto off the event loop./identitiesis I/O-bound (a Postgres query) — the event loop is alreadyfree while awaiting the DB, so a worker would add IPC copy overhead, would
not reduce DB load, and would give an attacker more threads/memory to
exhaust. The gate caps the actual bottleneck (DB load) directly.
Why this matters
The dangerous combination would have been shipping
/identitieswithoutthe gate: a single full-table read endpoint over a large jsonb table, behind
only the default 2000 req/min per-IP limit, is a trivial amplification
target — a handful of distributed clients could pin the DB. Layering a tight
per-IP limit, a global in-flight cap with fast-503 backpressure, and a hard
page-size bound means no single caller and no distributed burst can drive the
node's DB load past a fixed ceiling. The endpoints stay useful for legitimate
paginated reads while being safe to expose on the same public surface as the
existing GET routes.
Technical notes
Files (additive only — 921 insertions, 0 deletions):
src/libs/blockchain/gcr/gcr.ts—GCR.listIdentities(limit, cursor)src/libs/network/server_rpc.ts—/genesisBlock+/identitiesroutes, gate wiringsrc/libs/network/middleware/rateLimiter.ts—/identities→identitiesmethod inpathMethodMapsrc/utilities/sharedState.ts—identitiesentry inmethodLimitssrc/utilities/constants.ts— rate-limit + gate tunablessrc/libs/network/utils/concurrencyGate.ts(+.test.ts) — new semaphore utildocs/runbooks/genesis-and-identities-endpoints.md— runbook/identitiesresponse shape:{ "success": true, "identities": [ { "pubkey": "0x…", "identities": { "xm": {…}, "web2": {…}, "pqc": {…}, "ud": [] } } ], "count": 100, "limit": 100, "nextCursor": "0x…" // null = end of table }Tunables (
src/utilities/constants.ts):RATE_LIMIT_IDENTITIES_MAX_REQUESTS=30,RATE_LIMIT_IDENTITIES_WINDOW_MS=60000,IDENTITIES_MAX_CONCURRENT=3,IDENTITIES_MAX_QUEUE=12,IDENTITIES_ACQUIRE_TIMEOUT_MS=2000.Verification:
bun test src/libs/network/utils/concurrencyGate.test.ts→ 15 pass, 0 fail.npm run type-check-ts→ no new errors from this change (the two remainingerrors near the touched files —
/genesis's pre-existing.content.extratyping and a
Server<WebSocketData>generic inrateLimiter.ts— arepre-existing, confirmed via
git stashbaseline).rateLimiter.test.ts"trusted internal requests" failures areunrelated (fail identically on baseline).
Not done: no live end-to-end run against a booted node + Postgres. Query
mirrors the proven
GCR.getTopAccountsByPoints; gate is unit-tested inisolation.
Note:
/identitiesand/genesisBlockare public/unauthenticated, likethe other GET routes (
/genesis,/peerlist). Left that way intentionally.Summary by CodeRabbit
New Features
Documentation