feat(scan): P-14 plug-in scanner auto-discovery (v1.69.0)#28
Open
vignyl wants to merge 1 commit into
Open
Conversation
Pre-v1.69 the source list in `server/lib/sources/registry.mjs` was a
static hand-maintained array — adding a new adapter required editing
both `<id>.mjs` AND `registry.mjs`. This closes the `partial` half of
the roadmap item P-14 (`docs/ROADMAP.md`).
Now every `*.mjs` in `server/lib/sources/` is auto-loaded at module
boot; each adapter contributes its identity via a self-describing
`export const meta = { value, label, region, configKey? }` block. The
12 shipped adapters (ashby / greenhouse / lever / rss /
smartrecruiters / workable / workday + geekjob / getmatch / habr /
hh / trudvsem) each grew a `meta` export; `registry.mjs` now uses
`readdirSync` + dynamic `import()` resolved at module-eval via
top-level await (Node 18+ ESM standard).
The public API (`SOURCES`, `SOURCES_BY_REGION`, `RU_CONFIG_KEYS`,
`getRegionalSources`) is unchanged — every existing import keeps
working without edits. Validation rejects malformed `meta` (missing
`value`/`label`/`region`, RU without `configKey`, region outside
`'en'|'ru'`) and logs a single `console.warn` per offending file so
half-migrated branches stay diagnostic-friendly. The bundled
`registry.mjs` is excluded from self-import.
New `tests/sources-registry-discovery.test.mjs` adds 14 cases
covering shipped-adapter coverage, drop-in adapter discovery,
helper-module skip, malformed-meta rejection, self-import exclusion,
missing-directory tolerance, and deterministic ordering. CHANGELOG
parity ×9 locales. Suite 1065 → 1079.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes the
partialhalf of roadmap item P-14 (docs/ROADMAP.md):Pre-v1.69 the source list in
server/lib/sources/registry.mjswas a static hand-maintained array — adding a new adapter required editing both<id>.mjsANDregistry.mjs. After this PR, dropping a.mjsinserver/lib/sources/with ametaexport is sufficient to register a new source.What changed
ashby,greenhouse,lever,rss,smartrecruiters,workable,workday+geekjob,getmatch,habr,hh,trudvsem) grew a self-describing block:server/lib/sources/registry.mjsrewritten toreaddirSync+ dynamicimport()resolved at module-eval via top-level await (Node 18+ ESM standard).meta(missingvalue/label/region, RU withoutconfigKey, region outside'en'|'ru') and logs oneconsole.warnper offending file so half-migrated branches stay diagnostic-friendly.registry.mjsitself is excluded from self-import.discoverSources(dir)exposed so tests can validate drop-in behaviour against a temp directory.Backwards compatibility
SOURCES,SOURCES_BY_REGION,RU_CONFIG_KEYS,getRegionalSources) is unchanged. Every existing import (en-scanner.mjs,ru-scanner.mjs,routes/scan.mjs, SPAviews/scan.js) keeps working with no edit.'en'region first then'ru', alphabetical by label inside each region) — the SPA filter dropdown stays visually stable for end users.Tests
New
tests/sources-registry-discovery.test.mjs(14 cases):ashby/greenhouse/lever/rss/smartrecruiters/workable/workday)configKeyRU_CONFIG_KEYSmatches every RUconfigKey, no dupesgetRegionalSources()returns exactly the RU rowsconfigKeythroughmeta) are silently skipped withconsole.warnconfigKeyrejected'en'|'ru'rejectedregistry.mjsexcluded from self-importLocal results:
tests/sources-registry-discovery.test.mjs— 14/14 ✓tests/en-scanner.test.mjs— green ✓tests/ru-scanner.test.mjs— green ✓node scripts/check-changelog-parity.mjs—✓ CHANGELOG parity: all 8 locales at v1.69.0Note:
tests/sh-files.test.mjsfails locally because it shells out to bash and my environment is plain Windows without WSL. CI on Linux is unaffected.CHANGELOG
Added a
## [1.69.0] — 2026-06-09entry to all 9 locales (EN, ES, FR, JA, KO-KR, PT-BR, RU, ZH-CN, ZH-TW). The FR/EN/ES/PT-BR/RU bodies are full native paragraphs; the JA/KO-KR/ZH-CN/ZH-TW bodies are concise native translations — feel free to polish them in a follow-up commit if you'd like a richer phrasing.Out of scope (kept deliberately small)
server/lib/portals/registry.mjs(the ATS-adapter registry) is unchanged. It has the same hand-maintained pattern and could benefit from the same auto-discovery treatment — happy to follow up in a separate PR if you'd like.docs/architecture/SERVER.mdupdate in this PR — the existing description ofsources/registry.mjs("single source of truth") still reads correctly, and adding a "How to add a portal" walkthrough feels like a separatedocs:PR.Test plan
npm testgreen on CI (Linux)npm run test:cigreen (changelog parity gate)server/lib/sources/fake.mjswith ametaexport, restart server, confirm the new source surfaces inGET /api/scan/sourcesand the#/scanfilter dropdownmetaexport from one adapter, confirm the boot log emits one[sources/registry] <file> has no valid \export const meta` — skipped` warning and that source disappears from the dropdown