fix(node): timeout, retry, context, and atomicity on nodejs.org manifest fetch#82
Merged
Merged
Conversation
…est fetch
The nodejs.org manifest was fetched via `http.Get(url)` against
`http.DefaultClient`, which has no timeout. A hung nodejs.org (or a
corporate proxy that silently drops the connection) would block
`nodeup upgrade` / `nodeup check` indefinitely, with no recovery
path — and even Ctrl-C had no effect because no context was
threaded into the HTTP call. The cache write path was likewise
non-atomic: two `os.WriteFile` calls for data + meta could leave
the cache in a mismatched state if the meta write failed.
Replace with `FetchManifestCtx(ctx)` (the existing `FetchManifest()`
becomes a `context.Background()` wrapper) which:
- Builds the request via `http.NewRequestWithContext`, so a
caller-supplied `cmd.Context()` from the CLI layer propagates and
Ctrl-C cancels an in-flight fetch.
- Uses a package-level `http.Client{Timeout: 30s}`, so the worst-
case floor for an unresponsive host is 30s even when the caller
has no parent deadline.
- Retries up to 3 times with exponential backoff (200ms / 400ms /
800ms, capped at 2s) for transient errors — 5xx responses and
network errors (DNS failures, connection refused, ...) — and the
explicitly retryable 4xx (408 / 429). Permanent 4xx (404 etc.)
short-circuit immediately.
- Honors context cancellation between attempts; no backoff sleep
fires if the caller's ctx is already done.
`saveToCache` is now atomic via temp-file + rename on both data and
meta, so two concurrent `nodeup` invocations refreshing the cache
can no longer tear the on-disk state. The cache helpers take an
explicit `cachePaths` so tests can drive them hermetically.
Both CLI callers (`upgrade.go`, `check.go`) now pass `cmd.Context()`
through. Tests cover: success path, retry-on-503, no-retry-on-404,
context cancellation mid-flight (regression pin for the Ctrl-C
behavior), retry budget exhaustion, the cache round-trip, the
freshness gate (expired meta is rejected), and the no-tmp-leak
invariant after a successful save.
Closes #48.
Co-Authored-By: puku-ai-2.8 <noreply@puku.sh>
Follows golangci-lint v1's misspell / gosimple / gocritic checks
exercised in CI:
- "behaviour" → "behavior", "cancelled" → "canceled" in comments
(US spelling per the project default).
- http.NewRequestWithContext now passes http.NoBody instead of nil
for the GET request body (gocritic httpNoBody).
- The test's single-case `select { case <-r.Context().Done(): }`
block is replaced with a direct channel receive (gosimple S1000).
No behavior change. Just the lint cleanups.
Co-Authored-By: puku-ai-2.8 <noreply@puku.sh>
|
✔️ 95ede99...1bb5042 - Conventional commits check succeeded. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The nodejs.org manifest was fetched via
http.Get(url)againsthttp.DefaultClient, which has no timeout. A hung nodejs.org (or a corporate proxy that silently drops the connection) would blocknodeup upgrade/nodeup checkindefinitely, with no recovery path — and even Ctrl-C had no effect because no context was threaded into the HTTP call. The cache write path was likewise non-atomic: twoos.WriteFilecalls (data + meta) could leave the cache in a mismatched state if the meta write failed.Replace with
FetchManifestCtx(ctx)(the existingFetchManifest()becomes acontext.Background()wrapper) which:http.NewRequestWithContext, so a caller-suppliedcmd.Context()from the CLI layer propagates and Ctrl-C cancels an in-flight fetch.http.Client{Timeout: 30s}, so the worst-case floor for an unresponsive host is 30s even when the caller has no parent deadline.saveToCacheis now atomic via temp-file + rename on both data and meta, so two concurrentnodeupinvocations refreshing the cache can no longer tear the on-disk state. The cache helpers take an explicitcachePathsstruct so tests can drive them hermetically.Both CLI callers (
upgrade.go,check.go) now passcmd.Context()through to the newFetchManifestCtx. New tests ininternal/node/dist_test.gocover: success path, retry-on-503, no-retry-on-404, context cancellation mid-flight (regression pin for the Ctrl-C behavior), retry budget exhaustion, the cache round-trip, the freshness gate (expired meta is rejected), and the no-tmp-leak invariant after a successful save.Closes #48.
Linked issues
Type of change
fix— bug fix (PATCH bump)Checklist
fix(scope): subject)make cilocally and it passes (fmt, vet, test pass locally;make lintfails on the v2→v1 schema mismatch tracked in build(lint): .golangci.yml v1 schema incompatible with golangci-lint v2 (brew installs v2, make lint fails to even parse config) #62 — pre-existing onmain, independent of this fix; CI runs golangci-lint v1.64.8 per.github/workflows/ci.ymland parses the config fine)internal/node/dist_test.go: TestCacheLoadSaveRoundTrip, TestCacheLoadFromCache_ExpiredMetaIsRejected, TestSaveToCache_NoTempFileLeakOnSuccess, TestFetchManifestCtx_Success, TestFetchManifestCtx_RetriesOn5xx, TestFetchManifestCtx_Permanent4xxReturnsFast, TestFetchManifestCtx_ContextCancelMidFlight, TestFetchManifestCtx_RequestRetriesAreBounded; existing TestLatestLTS/LatestCurrent/ManifestUnmarshalUnion/ParseVersion/CacheExpiry are preserved)docs/, inline godoc)FetchManifest()andFetchManifestForce()are kept as thincontext.Background()wrappers around the new_Ctxvariants so all current call sites compile unchanged)Scope notes
httpClientis a single instance with a 30sTimeout. We don't open a fresh client per call — a*http.Clientis documented as safe for concurrent use and the timeout is the floor we care about, not a per-attempt budget.manifestURLis exposed as avar(not const) so tests can redirect it to anhttptest.Server.URL. The pre-fixhttp.Get(const string)had no seam; the new code'shttpClient.Do(req)does, by virtue of one indirection through avarinstead of a literal.withEmptyCachein the test file redirectsos.UserCacheDir()by settingXDG_CACHE_HOME(Linux),HOME(macOS), andAPP/LOCALAPPDATA/USERPROFILE(Windows). Without this, the developer's real cache would short-circuit every fetch test with a stale-but-fresh manifest read — exactly the test pollutionTestCacheExpiryused to dodge by not actually callingloadFromCache.writeFileAtomicwrites the data file first, then the meta. Readers that race us during the window between the data rename and the meta rename see the new data paired with the old (expired) meta and refuse to serve the cache — the safe failure mode (treated as stale).nodeupinvocations racing refresh-cache is rare in practice (the upgrade command takes a process-wide lock viaplatform.AcquireLockper bug(platform): lock file implemented but never wired into upgrade/config — concurrent runs not actually blocked #44, andnodeup checkruns read-only against an externally-writable cache). If we wanted to harden that further we'd put a cache-level flock around saveToCache; not worth the complexity here.manifestURLthrough anOPTIONSenv var (NODEUP_NODEJS_INDEX_URL) for users behind a self-hosted mirror. Out of scope; issue bug(node): no HTTP timeout/retry/context on nodejs.org manifest fetch; cache write not atomic #48 is specifically about resilience, not configurability.Out of scope (separate issues)
internal/cli/packages.gorunSnapshot/runRestorestill usecontext.Background()instead ofcmd.Context()— this is the body of bug(packages): interrupted-upgrade sentinel lifecycle broken (deleted on failure, never cleared after manual restore) #47 but a title-mismatch means it lives filed as fix(cli): context.Background() used instead of cmd.Context() in packages.go (contextcheck violation) #49.--cleanup/--yesprecedence conflict — bug(cli): --yes silently overrides --no-cleanup, auto-deletes with zero confirmation #57.packages restore/diffvia unsanitized manager name — fix(packages): path traversal via unsanitized manager name in packages restore/diff #51.asdf ASDF_DIR vs ASDF_DATA_DIRinconsistency — fix(detector): asdf uses ASDF_DIR vs ASDF_DATA_DIR inconsistently between docs, detector, and system-node classifier #50.Current()failure leaves active Node unprotected from cleanup — bug(cli): Current() failure leaves the active Node version unprotected from cleanup deletion #58.Screenshots / output
User-visible behavior change:
nodeup upgradeagainst a hung nodejs.org blocks indefinitely. Ctrl-C does nothing (no context threaded through the HTTP call). User has no recovery path exceptkill -9.context canceledwithin ~ms and the upgrade command fails with a clean error message.Timeoutfires and the fetch fails the same way.--offline, no network is touched and the cached manifest is used immediately.Verification path:
go test -race ./internal/node/... -v -run TestFetchManifestCtxcovers all five fetch scenarios; the cache tests exercise the round-trip and freshness-gate paths.