Skip to content

feat(iii-directory): migrate runtime config to the configuration worker#267

Open
andersonleal wants to merge 1 commit into
mainfrom
feat/iii-directory-configuration-worker
Open

feat(iii-directory): migrate runtime config to the configuration worker#267
andersonleal wants to merge 1 commit into
mainfrom
feat/iii-directory-configuration-worker

Conversation

@andersonleal

@andersonleal andersonleal commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

iii-directory was the last common Rust worker still loading its runtime config from a local config.yaml parsed once at boot. This migrates it to the engine-built-in configuration worker (entry id iii-directory), following the database/storage pattern: register a JSON Schema + seed at boot, read the authoritative (env-expanded) value via configuration::get, and bind a configuration:updated trigger for hot reload.

The change uses full hot-reload parity with database:

  • Tunable (hot-reload live): registry_url, download_timeout_ms, registry_cache_ttl_ms, filter_unregistered
  • Topology (refused with a "restart required" log): skills_folder, local_skills_folder, auto_download — these define on-disk roots and boot-time task wiring.

What changed

  • src/configuration.rs (new) — port of database/src/configuration.rs. CONFIG_ID = "iii-directory", CONFIG_FN_ID = "directory::on-config-change". A SharedState bundles the live config handle, shared cache-TTL cell, both caches, and the boot topology. on_config_change re-fetches the authoritative value (ignores the trigger payload — same security rationale as database), refuses topology changes, otherwise swaps the live config + updates TTL + clears both caches.
  • src/config.rsSkillsConfig derives JsonSchema; adds json_schema/from_json/to_json/from_yaml/from_file/topology/into_shared and the SharedConfig alias (Arc<ArcSwap<SkillsConfig>>).
  • Live-config refactor — every handler in functions/{mod,skills,registry,prompts,download}.rs snapshots the live config per call via load_full() (lock-free). RegistryCache and RegisteredWorkersCache read a shared Arc<AtomicU64> TTL so registry_cache_ttl_ms also hot-reloads.
  • src/main.rs--config is now an optional seed; boot reorders to connect → register_config(seed)fetch_config → build state/caches → register functions → register_config_trigger.
  • Docs/seedconfig.yamlconfig.yaml.example (seed header), README "Configuration" section rewritten (tunable vs topology), manifest default_config completed.
  • Adds the small arc-swap dependency (lock-free per-call reads; one-line change at each handler site).

Verification

  • cargo build, cargo clippy --all-targets (clean), cargo fmt --check (clean)
  • cargo test --lib249 passed
  • cargo test --no-run — all test targets compile + link

BDD/e2e suites require a live engine so they were not run here; the e2e harness is engine-config-driven and stays compatible (the engine delivers the inlined worker config as the --config seed).

Notes

  • The configuration dependency is implicit via runtime iii.trigger (no iii.worker.yaml deps entry), matching database.
  • Pre-existing, unrelated Cargo.lock version bumps in other workers were intentionally left out of this branch.

Summary by CodeRabbit

New Features

  • Configuration is now managed by a dedicated configuration worker with hot-reload support for runtime settings (certain topology changes require restart)
  • The --config flag is now optional; configuration can be provided via seed file or stored values
  • Environment variable expansion is supported in configuration values

Documentation

  • Updated configuration documentation with details on hot-reload semantics, field behavior, and initialization options

iii-directory was the last common Rust worker still loading its runtime
config from a local config.yaml parsed once at boot. Move it to the
engine-built-in `configuration` worker (id `iii-directory`), mirroring the
`database`/`storage` pattern: register a JSON Schema + seed at boot, read the
authoritative env-expanded value via `configuration::get`, and bind a
`configuration:updated` trigger for hot reload.

Full hot-reload parity with `database`:
- Tunable fields apply live: registry_url, download_timeout_ms,
  registry_cache_ttl_ms, filter_unregistered.
- Topology changes are refused with a "restart required" log:
  skills_folder, local_skills_folder, auto_download.

Implementation:
- New src/configuration.rs (port of database/src/configuration.rs) with a
  SharedState bundling the live config, shared cache TTL, both caches, and the
  boot topology. on_config_change re-fetches the authoritative value (ignores
  the trigger payload), refuses topology changes, else swaps config + updates
  TTL + clears caches.
- SkillsConfig: derive JsonSchema; add json_schema/from_json/to_json/from_file/
  topology/into_shared and the SharedConfig alias (Arc<ArcSwap<SkillsConfig>>).
- Handlers snapshot the live config per call via load_full(); RegistryCache and
  RegisteredWorkersCache read a shared Arc<AtomicU64> TTL.
- main.rs: --config is now an optional seed; connect -> register_config ->
  fetch_config -> build state/caches -> register fns -> register_config_trigger.
- config.yaml -> config.yaml.example (seed); README Configure section + manifest
  default_config updated.

Verified: cargo build, clippy --all-targets, fmt --check, 249 lib tests, and all
test targets compile + link.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented Jun 15, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
workers Ready Ready Preview, Comment Jun 15, 2026 7:39pm

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

skill-check — worker

0 verified, 19 skipped (no docs/).

Layer Result
structure
vale
ai
render

Four for four. Nicely done.

@coderabbitai

coderabbitai Bot commented Jun 15, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

iii-directory gains a hot-reloadable configuration system: a SharedConfig type alias (Arc<ArcSwap<SkillsConfig>>) is introduced, a new configuration.rs module integrates with the configuration worker for schema registration, env-expanded fetches, and live reload via trigger, all registered function handlers are updated to load config via load_full(), and caches adopt shared atomic TTLs.

Changes

Hot-reloadable configuration system for iii-directory

Layer / File(s) Summary
SharedConfig type, Topology, and serialization utilities
iii-directory/Cargo.toml, iii-directory/src/config.rs
Adds arc-swap dependency; defines SharedConfig type alias, derives JsonSchema on SkillsConfig, introduces Topology struct projecting restart-requiring fields, and adds json_schema, from_yaml, from_file, from_json, to_json, into_shared with expanded unit tests covering schema shape, round-trips, and topology equality.
Configuration worker integration layer
iii-directory/src/lib.rs, iii-directory/src/configuration.rs
New configuration.rs defines SharedState (live config snapshot, shared TTL cell, caches, boot topology), register_config (schema + optional seed), fetch_config (authoritative env-expanded value with NOT_FOUND fallback), apply_config (atomic swap + cache invalidation), register_config_trigger (directory::on-config-change bound to configuration:updated), on_config_change (topology guard + reload), and trigger_with_retry.
Live atomic TTL in skills and registry caches
iii-directory/src/functions/skills.rs, iii-directory/src/functions/registry.rs
RegisteredWorkersCache and RegistryCache store Arc<AtomicU64> TTL; new_shared constructors added; freshness checks load from the atomic on every access; register_with_cache added to registry; unit test added for live TTL shrink behavior.
SharedConfig propagation across all function registrations
iii-directory/src/functions/mod.rs, iii-directory/src/functions/skills.rs, iii-directory/src/functions/registry.rs, iii-directory/src/functions/prompts.rs, iii-directory/src/functions/download.rs
All register / register_with_cache / register_all / register_all_with_cache signatures updated to accept &SharedConfig; async handler closures call cfg_inner.load_full() instead of cloning a static config; register_all_with_cache gains a registry_cache parameter wired to register_with_cache.
Boot flow refactor and optional --config seed
iii-directory/src/main.rs
Startup now connects to engine, optionally loads seed from --config (Option<String>), calls register_config + fetch_config, builds SharedState, wires triggers; setup_auto_download and spawn_boot_reconcile accept SharedConfig and snapshot via load_full().
Docs, seed config, manifest defaults, and test helper
iii-directory/README.md, iii-directory/config.yaml.example, iii-directory/config.yaml, iii-directory/src/manifest.rs, iii-directory/tests/common/workers.rs
README expanded with configuration worker semantics and hot-reload field taxonomy; config.yaml.example added as documented seed file; config.yaml cleared; build_manifest gains local_skills_folder, filter_unregistered, auto_download defaults; test helper converts config to shared handle via into_shared.

Sequence Diagram(s)

sequenceDiagram
  participant CLI as iii-directory CLI
  participant Main as main.rs
  participant ConfMod as configuration.rs
  participant ConfWorker as configuration worker
  participant SharedState as SharedState / ArcSwap

  CLI->>Main: startup (--config optional seed path)
  Main->>ConfWorker: connect to engine
  Main->>ConfMod: register_config(iii, seed?)
  ConfMod->>ConfWorker: trigger_with_retry (schema + seed value)
  Main->>ConfMod: fetch_config(iii)
  ConfMod->>ConfWorker: configuration::get (env-expanded)
  ConfWorker-->>ConfMod: stored SkillsConfig JSON
  ConfMod-->>Main: SkillsConfig
  Main->>SharedState: SharedState::new(config, ttl_ms, caches, boot_topology)
  Main->>ConfMod: register_config_trigger(iii, state)
  note over ConfMod: registers directory::on-config-change → configuration:updated

  rect rgba(100, 149, 237, 0.5)
    note over ConfWorker, SharedState: Hot-reload path (at runtime)
    ConfWorker-->>ConfMod: configuration:updated trigger fires
    ConfMod->>ConfWorker: fetch_config (re-fetch authoritative)
    ConfWorker-->>ConfMod: new SkillsConfig
    ConfMod->>ConfMod: compare Topology vs boot_topology
    alt topology changed
      ConfMod-->>ConfMod: log restart required, retain prior config
    else tunables only
      ConfMod->>SharedState: apply_config (ArcSwap store, update AtomicU64 TTL, clear caches)
    end
  end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • iii-hq/workers#211: Modifies iii-directory configuration defaults and function wiring (home/local folders, auto-download, filtering) that this PR builds on top of with the hot-reload mechanism.

Suggested reviewers

  • sergiofilhowz

Poem

🐇 Hippity hop, the config can swap,
No restart needed — just load and unwrap!
ArcSwap spins, topology guards the gate,
Tunables reload while the rabbit won't wait.
Schema seeds planted, the worker obeys,
Hot config magic through atomic delays! ✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically describes the primary change: migrating iii-directory's runtime configuration to use the configuration worker instead of local file parsing.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/iii-directory-configuration-worker

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
iii-directory/src/manifest.rs (1)

41-62: ⚡ Quick win

Extend test to assert on the newly added fields.

The test currently verifies only four of the seven fields in default_config. Adding assertions for local_skills_folder, filter_unregistered, and auto_download would provide parity with the README documentation and catch future regressions if any of these fields are accidentally removed or misconfigured.

✅ Suggested test additions
         assert_eq!(parsed["default_config"]["download_timeout_ms"], 60_000);
         assert_eq!(parsed["default_config"]["registry_cache_ttl_ms"], 60_000);
+        assert_eq!(
+            parsed["default_config"]["local_skills_folder"],
+            DEFAULT_LOCAL_SKILLS_FOLDER
+        );
+        assert_eq!(parsed["default_config"]["filter_unregistered"], true);
+        assert_eq!(parsed["default_config"]["auto_download"], true);
         assert!(!parsed["supported_targets"].as_array().unwrap().is_empty());
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@iii-directory/src/manifest.rs` around lines 41 - 62, The test function
json_roundtrip_has_required_fields is incomplete as it only asserts on four of
the seven fields in default_config. Add three additional assert statements after
the existing default_config assertions to verify the values of the newly added
fields: local_skills_folder, filter_unregistered, and auto_download. These
assertions should validate that these fields are present in the parsed JSON and
contain the expected values as documented in the README, ensuring parity with
documentation and catching potential regressions if these fields are
accidentally removed or misconfigured.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@iii-directory/src/manifest.rs`:
- Around line 41-62: The test function json_roundtrip_has_required_fields is
incomplete as it only asserts on four of the seven fields in default_config. Add
three additional assert statements after the existing default_config assertions
to verify the values of the newly added fields: local_skills_folder,
filter_unregistered, and auto_download. These assertions should validate that
these fields are present in the parsed JSON and contain the expected values as
documented in the README, ensuring parity with documentation and catching
potential regressions if these fields are accidentally removed or misconfigured.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 9f887e71-e40b-4a9a-ae73-4e32fa6408f4

📥 Commits

Reviewing files that changed from the base of the PR and between 1ae7497 and 87f1974.

⛔ Files ignored due to path filters (1)
  • iii-directory/Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (15)
  • iii-directory/Cargo.toml
  • iii-directory/README.md
  • iii-directory/config.yaml
  • iii-directory/config.yaml.example
  • iii-directory/src/config.rs
  • iii-directory/src/configuration.rs
  • iii-directory/src/functions/download.rs
  • iii-directory/src/functions/mod.rs
  • iii-directory/src/functions/prompts.rs
  • iii-directory/src/functions/registry.rs
  • iii-directory/src/functions/skills.rs
  • iii-directory/src/lib.rs
  • iii-directory/src/main.rs
  • iii-directory/src/manifest.rs
  • iii-directory/tests/common/workers.rs
💤 Files with no reviewable changes (1)
  • iii-directory/config.yaml

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant