Skip to content

feat(scan): P-14 plug-in scanner auto-discovery (v1.69.0)#28

Open
vignyl wants to merge 1 commit into
Fighter90:mainfrom
vignyl:feat/p14-plugin-scanner-auto-discovery
Open

feat(scan): P-14 plug-in scanner auto-discovery (v1.69.0)#28
vignyl wants to merge 1 commit into
Fighter90:mainfrom
vignyl:feat/p14-plugin-scanner-auto-discovery

Conversation

@vignyl

@vignyl vignyl commented Jun 10, 2026

Copy link
Copy Markdown

Summary

Closes the partial half of roadmap item P-14 (docs/ROADMAP.md):

P-14 — Public Plug-in scanners | partial (v1.29.0) | server/lib/sources/registry.mjs lays the foundation. Remaining: support DYNAMIC adapter discovery (auto-load every server/lib/sources/*.mjs rather than explicit registry entries) so users can drop in adapters without editing core files.

Pre-v1.69 the source list in server/lib/sources/registry.mjs was a static hand-maintained array — adding a new adapter required editing both <id>.mjs AND registry.mjs. After this PR, dropping a .mjs in server/lib/sources/ with a meta export is sufficient to register a new source.

What changed

  • Each of the 12 shipped adapters (ashby, greenhouse, lever, rss, smartrecruiters, workable, workday + geekjob, getmatch, habr, hh, trudvsem) grew a self-describing block:
    export const meta = {
      value: 'greenhouse',
      label: 'Greenhouse',
      region: 'en',
      // configKey: '...' for RU adapters
    };
  • server/lib/sources/registry.mjs rewritten to readdirSync + dynamic import() resolved at module-eval via top-level await (Node 18+ ESM standard).
  • Validation rejects malformed meta (missing value/label/region, RU without configKey, region outside 'en'|'ru') and logs one console.warn per offending file so half-migrated branches stay diagnostic-friendly.
  • registry.mjs itself is excluded from self-import.
  • New public function discoverSources(dir) exposed so tests can validate drop-in behaviour against a temp directory.

Backwards compatibility

  • The public API surface (SOURCES, SOURCES_BY_REGION, RU_CONFIG_KEYS, getRegionalSources) is unchanged. Every existing import (en-scanner.mjs, ru-scanner.mjs, routes/scan.mjs, SPA views/scan.js) keeps working with no edit.
  • Discovered ordering preserves the pre-v1.69.0 hand-maintained order ('en' region first then 'ru', alphabetical by label inside each region) — the SPA filter dropdown stays visually stable for end users.

Tests

New tests/sources-registry-discovery.test.mjs (14 cases):

  1. Shipped EN adapters all present (ashby / greenhouse / lever / rss / smartrecruiters / workable / workday)
  2. Shipped RU adapters all present + each carries a configKey
  3. RU_CONFIG_KEYS matches every RU configKey, no dupes
  4. getRegionalSources() returns exactly the RU rows
  5. Ordering — EN first then RU, alpha-by-label inside each region
  6. Drop-in adapter discovery on a fixture directory
  7. Drop-in RU adapter carries configKey through
  8. Helper modules (no meta) are silently skipped with console.warn
  9. RU without configKey rejected
  10. Region outside 'en'|'ru' rejected
  11. registry.mjs excluded from self-import
  12. Missing directory → empty array (never throws)
  13. Deterministic order on identical input

Local results:

  • tests/sources-registry-discovery.test.mjs — 14/14 ✓
  • tests/en-scanner.test.mjs — green ✓
  • tests/ru-scanner.test.mjs — green ✓
  • node scripts/check-changelog-parity.mjs✓ CHANGELOG parity: all 8 locales at v1.69.0

Note: tests/sh-files.test.mjs fails locally because it shells out to bash and my environment is plain Windows without WSL. CI on Linux is unaffected.

CHANGELOG

Added a ## [1.69.0] — 2026-06-09 entry to all 9 locales (EN, ES, FR, JA, KO-KR, PT-BR, RU, ZH-CN, ZH-TW). The FR/EN/ES/PT-BR/RU bodies are full native paragraphs; the JA/KO-KR/ZH-CN/ZH-TW bodies are concise native translations — feel free to polish them in a follow-up commit if you'd like a richer phrasing.

Out of scope (kept deliberately small)

  • server/lib/portals/registry.mjs (the ATS-adapter registry) is unchanged. It has the same hand-maintained pattern and could benefit from the same auto-discovery treatment — happy to follow up in a separate PR if you'd like.
  • No docs/architecture/SERVER.md update in this PR — the existing description of sources/registry.mjs ("single source of truth") still reads correctly, and adding a "How to add a portal" walkthrough feels like a separate docs: PR.

Test plan

  • npm test green on CI (Linux)
  • npm run test:ci green (changelog parity gate)
  • Manual smoke: drop a fixture server/lib/sources/fake.mjs with a meta export, restart server, confirm the new source surfaces in GET /api/scan/sources and the #/scan filter dropdown
  • Manual smoke: temporarily remove the meta export from one adapter, confirm the boot log emits one [sources/registry] <file> has no valid \export const meta` — skipped` warning and that source disappears from the dropdown

Pre-v1.69 the source list in `server/lib/sources/registry.mjs` was a
static hand-maintained array — adding a new adapter required editing
both `<id>.mjs` AND `registry.mjs`. This closes the `partial` half of
the roadmap item P-14 (`docs/ROADMAP.md`).

Now every `*.mjs` in `server/lib/sources/` is auto-loaded at module
boot; each adapter contributes its identity via a self-describing
`export const meta = { value, label, region, configKey? }` block. The
12 shipped adapters (ashby / greenhouse / lever / rss /
smartrecruiters / workable / workday + geekjob / getmatch / habr /
hh / trudvsem) each grew a `meta` export; `registry.mjs` now uses
`readdirSync` + dynamic `import()` resolved at module-eval via
top-level await (Node 18+ ESM standard).

The public API (`SOURCES`, `SOURCES_BY_REGION`, `RU_CONFIG_KEYS`,
`getRegionalSources`) is unchanged — every existing import keeps
working without edits. Validation rejects malformed `meta` (missing
`value`/`label`/`region`, RU without `configKey`, region outside
`'en'|'ru'`) and logs a single `console.warn` per offending file so
half-migrated branches stay diagnostic-friendly. The bundled
`registry.mjs` is excluded from self-import.

New `tests/sources-registry-discovery.test.mjs` adds 14 cases
covering shipped-adapter coverage, drop-in adapter discovery,
helper-module skip, malformed-meta rejection, self-import exclusion,
missing-directory tolerance, and deterministic ordering. CHANGELOG
parity ×9 locales. Suite 1065 → 1079.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant