feat(agent): drift-guard slash commands + trim skill catalog when active#98
Merged
Conversation
Two grounding improvements surfaced by the self-knowledge review: 1. Drift guard — the slash-command set is hand-maintained twice, in palette.builtinItems (what the operator can run) and in basePreamble's "## cloudy slash commands" prose (what the LLM is told exists). The new TestPalette_CommandsDocumentedInPreamble asserts every palette command is documented in the preamble (via the new agent.BasePreamble accessor). It immediately caught /use (switch kubeconfig context) missing from the preamble, so the agent would have claimed not to know a command cloudy actually offers — fixed by documenting /use. 2. Token diet — buildSystemPrompt now omits the "## Available skills" catalog once a skill is active: that skill's full body is already injected, so listing every other skill is redundant per-request tokens that matter on small models. The catalog still appears when no skill is active so the model can answer "what skills do you have?". Signed-off-by: rlaope <piyrw9754@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two grounding improvements from the self-knowledge review:
palette.builtinItems(what the operator can run) andbasePreamble's prose (what the LLM is told exists). NewTestPalette_CommandsDocumentedInPreambleasserts every palette command is documented in the preamble (via the newagent.BasePreamble()accessor). It immediately caught/usemissing from the preamble — the agent would have denied knowing a command cloudy offers. Fixed by documenting/use.buildSystemPromptomits the## Available skillscatalog once a skill is active (its full body is already injected; listing the others is redundant per-request tokens that matter on small models). The catalog still appears when no skill is active.Test plan
go test -race -count=1 ./...greengolangci-lint run ./...— 0 issuesTestPalette_CommandsDocumentedInPreamble(drift),TestSystemPreamble_DropsCatalogWhenSkillActive(diet); existingTestSystemPreamble_*still green