From a2e714ff5f641f5c128e491c6d98bfb39ddc7923 Mon Sep 17 00:00:00 2001 From: Gale W Date: Tue, 28 Apr 2026 18:49:47 -0400 Subject: [PATCH 1/4] tests: run SearchKit in default validation --- ROADMAP.md | 53 ++++++++++++++----- .../SearchKitFetchIndexTests.swift | 8 --- docs/maintainers/fetchkit-product-plan.md | 8 +-- .../repo-maintenance/run-searchkit-tests.sh | 4 +- scripts/repo-maintenance/version-bump.sh | 4 +- 5 files changed, 49 insertions(+), 28 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 39b35f6..150a492 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -12,6 +12,7 @@ Use this roadmap to track milestone-level delivery through checklist sections. - [Milestone 2: Post-v0.1.0 Refinement](#milestone-2-post-v010-refinement) - [Milestone 3: FetchKit Foundation](#milestone-3-fetchkit-foundation) - [Milestone 4: FetchKit Refinement](#milestone-4-fetchkit-refinement) +- [Milestone 5: FetchKit Platform And CI Decisions](#milestone-5-fetchkit-platform-and-ci-decisions) - [Backlog Candidates](#backlog-candidates) - [History](#history) @@ -31,7 +32,8 @@ Use this roadmap to track milestone-level delivery through checklist sections. - Milestone 1: Retrieval Defaults - Completed - Milestone 2: Post-v0.1.0 Refinement - Completed - Milestone 3: FetchKit Foundation - Completed -- Milestone 4: FetchKit Refinement - Planned +- Milestone 4: FetchKit Refinement - In Progress +- Milestone 5: FetchKit Platform And CI Decisions - Planned ## Milestone 0: Foundation @@ -138,7 +140,7 @@ Completed ### Tickets -- [ ] Add maintainer-facing `FetchKit` architecture guidance that explains the Core Data plus SearchKit model and its relationship to `RAGKit`. +- [x] Add maintainer-facing `FetchKit` architecture guidance that explains the Core Data plus SearchKit model and its relationship to `RAGKit`. - [x] Define the first `FetchCore` model and protocol candidates for document records, queries, search results, snippets, and index synchronization. - [x] Add the first explicit `FetchCore` indexing changeset boundary so later `FetchKit` work can sync Core Data updates into a full-text index without ad hoc write paths. - [x] Define the first durable `FetchDocumentRecord` shape so Core Data-backed corpus storage can stay distinct from derived full-text indexing views. @@ -154,7 +156,7 @@ Completed - [x] Add the first explicit store-write to indexing-changeset seam so future index backends can consume real store mutations instead of facade-reconstructed writes. - [x] Add a persisted pending index-sync queue so failed index applies can be retried after the original write call returns or the process restarts. - [x] Add the first thin Search Kit-backed `FetchIndex` implementation on macOS, with direct Search Kit tests kept opt-in on a dedicated local macOS lane. -- [x] Add a small repo-maintenance helper so the local opt-in Search Kit lane is one obvious command instead of a hand-written `xcodebuild` invocation. +- [x] Add a small repo-maintenance helper so the focused Search Kit lane is one obvious command instead of a hand-written `xcodebuild` invocation. - [x] Tighten the persistent `FetchKitLibrary` construction surface so store and index locations feel polished and Cocoa-style for real app callers. - [x] Audit the first Core Data-backed store path on GitHub-hosted macOS, record the Swift Testing executor-assumption failure, move the Core Data verification lane to XCTest, and align the store implementation with a private-queue Core Data context plus async `perform`. @@ -168,33 +170,57 @@ Completed ### Status -Planned +In Progress ### Scope -- [ ] Refine conventional-search ranking and snippet behavior now that the first SearchKit backend works end to end. - [x] Refine conventional-search ranking and snippet behavior now that the first SearchKit backend works end to end. -- [ ] Decide whether the SearchKit-backed path needs a dedicated CI lane beyond the current local-only verification helper. +- [ ] Validate whether the current refinement pass is enough for ordinary app callers or whether another real-corpus quality pass is needed. - [ ] Keep the public `FetchKitLibrary` surface polished as the conventional-search side moves from foundation into quality work. ### Tickets - [x] Refine ranking behavior for conventional search so the first SearchKit backend feels less like a raw index adapter and more like a library product. - [x] Improve snippet behavior and result presentation without bloating `FetchCore` into a larger query or rendering DSL. -- [ ] Revisit the local-only SearchKit verification decision once the opt-in macOS lane has stayed stable long enough to justify a dedicated CI experiment. +- [ ] Audit real-corpus result quality now that field-aware ranking, phrase weighting, truncation cues, and multi-term snippets are in place. +- [ ] Decide whether title-only hits should suppress body snippets or use a different presentation policy in the public facade. +- [ ] Keep the persistent `FetchKitLibrary` construction and search API surface under review as real callers exercise the current design. ### Exit Criteria - [x] Conventional-search results feel intentionally ranked and include useful snippet behavior for ordinary app callers. -- [ ] The team has a clear answer on whether the SearchKit lane should stay local-only, move to a dedicated CI path, or remain intentionally manual. +- [x] The SearchKit-backed path runs in normal local validation and the default GitHub CI lane. - [ ] `FetchKitLibrary` still reads like a small Swift-native facade instead of exposing backend detail drift. +## Milestone 5: FetchKit Platform And CI Decisions + +### Status + +Planned + +### Scope + +- [ ] Decide the long-term verification story for the Apple-asset Natural Language lane. +- [ ] Decide what first-class iOS support means for conventional search beyond the current portable `FetchCore` surface. +- [ ] Keep the product-family docs honest about what is already shippable versus what is still macOS-first or local-only. + +### Tickets + +- [ ] Revisit whether Natural Language asset-backed verification should stay local-only, move to self-hosted CI, or remain intentionally opt-in. +- [ ] Explore the first real iOS conventional-search backend direction, with Core Spotlight as the most likely Apple-native candidate. +- [ ] Decide whether future platform/backend work belongs in `FetchKit` directly or should earn a clearer backend seam only once the next real implementation lands. + +### Exit Criteria + +- [ ] The repo has a stable, explicitly chosen verification story for its remaining framework-heavy optional lane. +- [ ] The family docs describe macOS-first and iOS-first-class support without implying a backend that does not exist yet. +- [ ] The next backend or CI experiment has been narrowed to one concrete path instead of an open-ended backlog note. + ## Backlog Candidates - [ ] If parser-backed markdown chunking still leaves retrieval-quality gaps, add retrieval-specific chunking heuristics on top of the chosen markdown parser instead of rebuilding markdown parsing rules locally. - [ ] If asset-backed automation becomes important again, evaluate a self-hosted macOS runner with prewarmed assets before retrying a hosted GitHub Actions lane. -- [ ] Explore dedicated CI options for the opt-in Search Kit lane once the local macOS path has stayed stable long enough to justify automation. -- [ ] When `FetchKit` moves from docs into code, decide whether the first backend should live behind a SearchKit-specific module seam immediately or only after the first macOS implementation proves the stable `FetchCore` shape. +- [ ] Consider a follow-on conventional-search quality pass only if real corpora show ranking, snippet, or result-presentation gaps beyond the current field-aware heuristics. ## History @@ -229,9 +255,12 @@ Planned - Added the first store-produced indexing-changeset seam and surfaced pending index updates when the apply step fails. - Added a persisted pending index-sync queue plus retry APIs so failed index applies can be acknowledged later instead of being recoverable only from the immediate thrown error. - Added the first thin macOS Search Kit index backend and moved the direct Search Kit suite onto XCTest-style opt-in gating. -- Fixed Search Kit index ownership during teardown so the opt-in Search Kit verification lane is green again under both `swift test` and `xcodebuild test`. -- Added a dedicated repo-maintenance helper for the local opt-in Search Kit test lane and recorded persistent-surface polish plus ranking/snippet refinement as the next FetchKit work. +- Fixed Search Kit index ownership during teardown so the Search Kit verification lane is green again under both `swift test` and `xcodebuild test`. +- Added a dedicated repo-maintenance helper for the focused Search Kit test lane and recorded persistent-surface polish plus ranking/snippet refinement as the next FetchKit work. - Tightened the persistent `FetchKitLibrary` surface around one resolved storage location, with Application Support defaults plus a direct directory override for local callers. - Recorded that the GitHub-hosted `macos-15` Natural Language verification attempt timed out, so Apple-asset coverage stays local-only for now. - Audited the Core Data-backed `FetchKit` store after a GitHub-hosted Swift Testing crash, recorded the executor-assumption findings, moved Core Data verification onto XCTest, and switched the durable store over to a private-queue Core Data context with the framework's async `perform` path. - Refined conventional-search result quality with modest field-aware ranking plus query-aware multi-term snippets across the in-memory and SearchKit-backed `FetchKit` paths. +- Polished conventional-search result presentation with stronger phrase weighting and visible snippet truncation cues, then shipped that refinement as `v0.1.2`. +- Promoted the SearchKit-backed test suite from a local opt-in lane into normal XCTest validation and the default GitHub CI path once the lane proved fast and stable enough. +- Opened the next roadmap phase around SearchKit/Natural Language verification strategy, iOS conventional-search backend direction, and another caller-driven `FetchKitLibrary` polish pass if real usage shows it is needed. diff --git a/Tests/FetchKitTests/SearchKitFetchIndexTests.swift b/Tests/FetchKitTests/SearchKitFetchIndexTests.swift index a246f0e..0a1ddda 100644 --- a/Tests/FetchKitTests/SearchKitFetchIndexTests.swift +++ b/Tests/FetchKitTests/SearchKitFetchIndexTests.swift @@ -5,14 +5,6 @@ import FetchCore @testable import FetchKit final class SearchKitFetchIndexTests: XCTestCase { - override func setUpWithError() throws { - try super.setUpWithError() - try XCTSkipUnless( - ProcessInfo.processInfo.environment["RUN_SEARCHKIT_TESTS"] == "1", - "Enable Search Kit tests with RUN_SEARCHKIT_TESTS=1." - ) - } - func testSearchKitFetchIndexIndexesAndSearchesText() async throws { let index = try SearchKitFetchIndex( configuration: .init( diff --git a/docs/maintainers/fetchkit-product-plan.md b/docs/maintainers/fetchkit-product-plan.md index 3749cb3..d84241a 100644 --- a/docs/maintainers/fetchkit-product-plan.md +++ b/docs/maintainers/fetchkit-product-plan.md @@ -78,9 +78,9 @@ Current status: - `FetchKit` now has its first Core Data-backed `FetchDocumentStore` implementation, built from a programmatic Core Data model that matches the current durable record shape - the store side now produces explicit `FetchIndexingChangeset` values through `FetchStoreMutationResult`, so the index-sync boundary is derived from real store writes instead of being reconstructed ad hoc in the facade - pending index-sync work is now persisted by the store itself and can be retried later through the `FetchKitLibrary` facade, so a failed index apply no longer relies only on an in-memory thrown error for recovery -- the first thin Search Kit backend now exists behind `FetchIndex`, and its direct tests now sit behind XCTest-style opt-in gating so the default package path can stay clean -- the Search Kit crash isolation pass found that `SKIndex` teardown needed unretained adoption on create/open, and the direct opt-in Search Kit verification lane is green again under both `swift test` and `xcodebuild test` -- that Search Kit verification lane is still local-only for now, while the repo defers any dedicated CI story for it +- the first thin Search Kit backend now exists behind `FetchIndex`, and its direct tests now run through the normal XCTest validation path while the focused helper script remains available for faster local iteration +- the Search Kit crash isolation pass found that `SKIndex` teardown needed unretained adoption on create/open, and the Search Kit verification lane is now green under both targeted local runs and the normal XCTest validation path +- because the Search Kit tests now finish quickly and pass reliably, the repo now runs them in ordinary local validation and the default GitHub macOS CI lane instead of keeping them behind a local-only opt-in gate - the persistent `FetchKitLibrary` construction path is now intentionally caller-shaped around one storage location, with an Application Support default plus a direct directory override, instead of asking app code to assemble separate Core Data and Search Kit URLs itself - the first refinement pass on conventional-search result quality is now in place: SearchKit scores are normalized per field, title hits get a modest weight bump, cross-field matches accumulate instead of collapsing to the single best field, and snippets now highlight multiple query terms instead of showing only the first term in a fixed-width window - the CI investigation on GitHub-hosted macOS found that the Core Data-backed store path could abort under Swift Testing with `Incorrect actor executor assumption`, even after global test parallelism was disabled @@ -159,7 +159,7 @@ That pass landed: The next work is refinement, not first architecture: - keep the persistent `FetchKitLibrary` surface polished as real callers exercise it -- decide later whether the local-only SearchKit verification lane deserves dedicated CI +- keep the SearchKit-backed path inside ordinary validation unless a future framework regression forces it back out - decide whether the current ranking and snippet heuristics are already enough for ordinary callers or whether real corpora show a need for another refinement pass ## First Core Data Entity Shape diff --git a/scripts/repo-maintenance/run-searchkit-tests.sh b/scripts/repo-maintenance/run-searchkit-tests.sh index 3381151..4740790 100755 --- a/scripts/repo-maintenance/run-searchkit-tests.sh +++ b/scripts/repo-maintenance/run-searchkit-tests.sh @@ -12,7 +12,7 @@ DESTINATION="${SEARCHKIT_TEST_DESTINATION:-platform=macOS}" SCHEME="${SEARCHKIT_TEST_SCHEME:-SwiftlyFetch-Package}" TEST_TARGET="${SEARCHKIT_TEST_ONLY:-SearchKitFetchIndexTests}" -log "Running the local opt-in Search Kit test lane for $TEST_TARGET from $REPO_ROOT." +log "Running the focused Search Kit test lane for $TEST_TARGET from $REPO_ROOT." cd "$REPO_ROOT" -RUN_SEARCHKIT_TESTS=1 swift test --filter "$TEST_TARGET" +swift test --filter "$TEST_TARGET" diff --git a/scripts/repo-maintenance/version-bump.sh b/scripts/repo-maintenance/version-bump.sh index dc28e2e..c0bfa10 100755 --- a/scripts/repo-maintenance/version-bump.sh +++ b/scripts/repo-maintenance/version-bump.sh @@ -45,7 +45,7 @@ cat >"$release_notes_path" <"$release_notes_path" < Date: Tue, 28 Apr 2026 18:51:24 -0400 Subject: [PATCH 2/4] tests: run NatLang integration in local validation --- .../repo-maintenance/run-nl-integration-tests.sh | 16 ++++++++++++++++ .../validations/40-swift-package.sh | 3 +++ 2 files changed, 19 insertions(+) create mode 100644 scripts/repo-maintenance/run-nl-integration-tests.sh diff --git a/scripts/repo-maintenance/run-nl-integration-tests.sh b/scripts/repo-maintenance/run-nl-integration-tests.sh new file mode 100644 index 0000000..7838247 --- /dev/null +++ b/scripts/repo-maintenance/run-nl-integration-tests.sh @@ -0,0 +1,16 @@ +#!/usr/bin/env sh +set -eu + +SELF_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd) +export REPO_MAINTENANCE_COMMON_DIR="$SELF_DIR/lib" +. "$SELF_DIR/lib/common.sh" + +load_profile_env +ensure_git_repo + +TEST_TARGET="${NL_INTEGRATION_TEST_ONLY:-NaturalLanguageEmbedderIntegrationTests}" + +log "Running the Natural Language integration lane for $TEST_TARGET from $REPO_ROOT." + +cd "$REPO_ROOT" +RUN_NL_INTEGRATION_TESTS=1 swift test --filter "$TEST_TARGET" diff --git a/scripts/repo-maintenance/validations/40-swift-package.sh b/scripts/repo-maintenance/validations/40-swift-package.sh index bfb86f8..c430f20 100755 --- a/scripts/repo-maintenance/validations/40-swift-package.sh +++ b/scripts/repo-maintenance/validations/40-swift-package.sh @@ -17,3 +17,6 @@ swift test --enable-xctest --disable-swift-testing log "Testing Swift Testing targets from $REPO_ROOT with global test parallelism disabled." swift test --disable-xctest --enable-swift-testing --no-parallel + +log "Testing Natural Language integration from $REPO_ROOT." +sh "$REPO_MAINTENANCE_ROOT/run-nl-integration-tests.sh" From 1d53f94e23b1034a2b4c77436913857c52015507 Mon Sep 17 00:00:00 2001 From: Gale W Date: Tue, 28 Apr 2026 18:59:18 -0400 Subject: [PATCH 3/4] tests: keep NatLang local in maintainer validation --- README.md | 2 +- ROADMAP.md | 1 + docs/maintainers/retrieval-package-plan.md | 9 +++++---- scripts/repo-maintenance/validations/40-swift-package.sh | 8 ++++++-- scripts/repo-maintenance/version-bump.sh | 2 +- 5 files changed, 14 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 07b02e8..45ee7ab 100644 --- a/README.md +++ b/README.md @@ -167,7 +167,7 @@ Good current fits: Current constraints: - the SearchKit backend is macOS-first -- Natural Language asset-backed verification stays opt-in and local-only +- Natural Language asset-backed verification runs in local maintainer validation by default, but stays out of the default GitHub-hosted CI lane because hosted macOS still stalls in the asset-backed step - the package family direction is broader than the currently shipped polished surface, especially on the `FetchKit` side If you want to contribute to the package itself, use [CONTRIBUTING.md](./CONTRIBUTING.md). Maintainer planning and architecture notes live under [docs/maintainers/](./docs/maintainers/). diff --git a/ROADMAP.md b/ROADMAP.md index 150a492..d8ed8fc 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -263,4 +263,5 @@ Planned - Refined conventional-search result quality with modest field-aware ranking plus query-aware multi-term snippets across the in-memory and SearchKit-backed `FetchKit` paths. - Polished conventional-search result presentation with stronger phrase weighting and visible snippet truncation cues, then shipped that refinement as `v0.1.2`. - Promoted the SearchKit-backed test suite from a local opt-in lane into normal XCTest validation and the default GitHub CI path once the lane proved fast and stable enough. +- Promoted the Natural Language integration lane into default local maintainer validation, but kept it out of GitHub-hosted CI after another hosted experiment remained stuck in the asset-backed step for minutes while the local path completed in seconds. - Opened the next roadmap phase around SearchKit/Natural Language verification strategy, iOS conventional-search backend direction, and another caller-driven `FetchKitLibrary` polish pass if real usage shows it is needed. diff --git a/docs/maintainers/retrieval-package-plan.md b/docs/maintainers/retrieval-package-plan.md index 78a8715..8caa8a6 100644 --- a/docs/maintainers/retrieval-package-plan.md +++ b/docs/maintainers/retrieval-package-plan.md @@ -168,7 +168,7 @@ Implemented today: - markdown tables now produce one retrieval chunk per body row with header-aware text and table-row metadata - inline links and reference links now default to visible anchor text in chunk text, while raw destinations and reference definitions stay secondary and do not become standalone retrieval chunks unless a caller explicitly opts into chunk metadata for destinations - deterministic tests cover the main retrieval flow and the Natural Language wrapper seam -- an opt-in integration test target exists for real Natural Language embedding coverage and remains a local-only verification path for now because the GitHub-hosted `macos-15` attempt timed out in the Natural Language step +- a real Natural Language integration test target exists, now runs in default local maintainer validation, and remains out of the default GitHub-hosted lane because the hosted `macos-15` path stalled in the Natural Language step Still intentionally incomplete: @@ -376,15 +376,16 @@ That means: Current status: - the deterministic wrapper and knowledge-base tests are in place -- the opt-in Natural Language integration target exists +- the Natural Language integration target exists - real Apple-backed integration coverage now checks semantic retrieval behavior rather than only non-empty normalized vectors - default CI proves `swift build` and `swift test` on the ordinary macOS path, while the asset-backed Natural Language integration target stays outside GitHub-hosted CI for now Current CI position: - keep the default validation job free of Apple asset requirements -- keep `RUN_NL_INTEGRATION_TESTS=1` as explicit local-only verification for now +- keep the Natural Language integration lane in default local maintainer validation, but do not make GitHub-hosted CI run it unless an explicit hosted experiment is being performed - record that the GitHub-hosted `macos-15` attempt started successfully but remained stuck in the Natural Language integration step until the job timeout +- record that a later hosted GitHub experiment still remained stuck in that asset-backed step for several minutes even though the same local lane completed in only a few seconds - if asset-backed automation becomes important later, prefer a self-hosted macOS runner with known asset state over retrying the same hosted setup unchanged ## Package Structure Target @@ -445,7 +446,7 @@ The first concrete implementation pass should happen in this order: 9. Add a heading-aware markdown chunker as the first major retrieval-quality improvement. Completed. 10. Strengthen the real Natural Language integration assertions so asset-enabled runs prove useful similarity behavior, not just vector-shape correctness. Completed. 11. Tighten retrieval defaults around metadata filtering and context assembly without widening the package into chat or generation concerns. Completed with explicit exclusion filters, ordered typed comparisons, grouped annotated output, smarter duplicate suppression, and refined budget handling. -12. Keep default CI focused on `swift build` and `swift test`, and keep Apple-asset integration coverage as explicit local-only verification until a better runner strategy exists. +12. Keep default GitHub-hosted CI focused on `swift build` and `swift test`, while running Apple-asset integration coverage in local maintainer validation until a better runner strategy exists. That sequence matters because it gets a fully testable retrieval loop working before the repo takes on Apple asset-management complexity. diff --git a/scripts/repo-maintenance/validations/40-swift-package.sh b/scripts/repo-maintenance/validations/40-swift-package.sh index c430f20..99e4f54 100755 --- a/scripts/repo-maintenance/validations/40-swift-package.sh +++ b/scripts/repo-maintenance/validations/40-swift-package.sh @@ -18,5 +18,9 @@ swift test --enable-xctest --disable-swift-testing log "Testing Swift Testing targets from $REPO_ROOT with global test parallelism disabled." swift test --disable-xctest --enable-swift-testing --no-parallel -log "Testing Natural Language integration from $REPO_ROOT." -sh "$REPO_MAINTENANCE_ROOT/run-nl-integration-tests.sh" +if [ "${GITHUB_ACTIONS:-}" = "true" ] && [ "${RUN_NL_INTEGRATION_TESTS_IN_GITHUB_CI:-}" != "1" ]; then + log "Skipping Natural Language integration on GitHub Actions; use RUN_NL_INTEGRATION_TESTS_IN_GITHUB_CI=1 for an explicit hosted experiment." +else + log "Testing Natural Language integration from $REPO_ROOT." + sh "$REPO_MAINTENANCE_ROOT/run-nl-integration-tests.sh" +fi diff --git a/scripts/repo-maintenance/version-bump.sh b/scripts/repo-maintenance/version-bump.sh index c0bfa10..aa81e72 100755 --- a/scripts/repo-maintenance/version-bump.sh +++ b/scripts/repo-maintenance/version-bump.sh @@ -56,7 +56,7 @@ cat >"$release_notes_path" < Date: Tue, 28 Apr 2026 19:02:18 -0400 Subject: [PATCH 4/4] docs: clarify NatLang and SearchKit verification --- CONTRIBUTING.md | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9a18991..7e76778 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -44,8 +44,7 @@ This repository does not require secrets or `.env` files for ordinary package de ```bash RUN_NL_INTEGRATION_TESTS=1 -RUN_SEARCHKIT_TESTS=1 -TEST_RUNNER_RUN_SEARCHKIT_TESTS=1 +RUN_NL_INTEGRATION_TESTS_IN_GITHUB_CI=1 ``` ### Runtime Behavior @@ -64,12 +63,23 @@ For the local SearchKit-only verification lane on macOS, use: scripts/repo-maintenance/run-searchkit-tests.sh ``` -If you are touching the opt-in Natural Language asset path, use: +`scripts/repo-maintenance/validate-all.sh` now includes both framework-backed local verification lanes by default on macOS: + +- the SearchKit XCTest lane runs in ordinary local validation and in the default GitHub macOS CI job +- the Natural Language asset-backed integration lane also runs in ordinary local validation, but it is intentionally skipped in the default GitHub-hosted CI job because hosted macOS still stalls in that asset-backed step + +For a focused Natural Language integration pass, use: ```bash RUN_NL_INTEGRATION_TESTS=1 swift test --filter NaturalLanguageEmbedderIntegrationTests ``` +Only use this hosted-CI override when deliberately re-running the GitHub experiment path: + +```bash +RUN_NL_INTEGRATION_TESTS_IN_GITHUB_CI=1 +``` + ## Development Expectations ### Naming Conventions @@ -91,7 +101,7 @@ scripts/repo-maintenance/validate-all.sh scripts/repo-maintenance/run-searchkit-tests.sh ``` -Not every change needs every optional lane, but package, maintainer, and release-affecting work should leave the default validation path green before review. +Not every change needs every focused helper lane, but package, maintainer, and release-affecting work should leave the default validation path green before review. If you are changing Natural Language embedding behavior, asset handling, or the NatLang harness itself, run the focused Natural Language integration command explicitly in addition to the normal repo-maintenance path. ## Pull Request Expectations