diff --git a/CHANGELOG.md b/CHANGELOG.md index b9d8950..673d0c2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,8 +12,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - **Learning**: `devflow learn --purge` command to remove invalid entries from learning log - **Learning**: debug logging mode (`devflow learn --configure`) — logs to `~/.devflow/logs/` +### Changed +- **Learning**: Moved from Stop → SessionEnd hook with 3-session batching (adaptive: 5 at 15+ observations) +- **Learning**: Raised procedural thresholds from 2 to 3 observations with 24h+ temporal spread for both types +- **Learning**: Reduced default `max_daily_runs` from 10 to 5 +- **Learning**: Renamed artifact paths: `commands/learned/` → `commands/self-learning/`, `skills/learned-{name}/` → `skills/{name}/` +- **Learning**: Skill artifacts now include `user-invocable: false`, Iron Law section, and `self-learning:` name prefix + ### Fixed - **Learning**: reject observations with empty id/type/pattern fields (validation + auto-purge on migration) +- **Learning**: Handle string-typed `.message.content` in transcript extraction (was only handling arrays) +- **Learning**: Eliminate empty-array loop noise when Sonnet returns no observations +- **Learning**: Race condition in batch file handoff (atomic `mv` replaces `cp`+`rm`) +- **Learning**: `--enable` now auto-upgrades legacy Stop hook to SessionEnd +- **Learning**: `--status` detects legacy hook and shows upgrade instructions --- diff --git a/CLAUDE.md b/CLAUDE.md index 11c103b..b130cac 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -40,7 +40,7 @@ Commands with Teams Variant ship as `{name}.md` (parallel subagents) and `{name} **Working Memory**: Three shell-script hooks (`scripts/hooks/`) provide automatic session continuity. Toggleable via `devflow memory --enable/--disable/--status` or `devflow init --memory/--no-memory`. Stop hook → reads last turn from session transcript (`~/.claude/projects/{encoded-cwd}/{session_id}.jsonl`), spawns background `claude -p --model haiku` to update `.memory/WORKING-MEMORY.md` with structured sections (`## Now`, `## Progress`, `## Decisions`, `## Modified Files`, `## Context`, `## Session Log`; throttled: skips if triggered <2min ago; concurrent sessions serialize via mkdir-based lock). SessionStart hook → injects previous memory + git state as `additionalContext` on `/clear`, startup, or compact (warns if >1h stale; injects pre-compact memory snapshot when compaction happened mid-session). PreCompact hook → saves git state + WORKING-MEMORY.md snapshot + bootstraps minimal WORKING-MEMORY.md if none exists. Zero-ceremony context preservation. -**Self-Learning**: A Stop hook (`stop-update-learning`) spawns a background `claude -p --model sonnet` to detect repeated workflows and procedural knowledge from session transcripts. Observations accumulate in `.memory/learning-log.jsonl` with confidence scores, temporal decay, and daily run caps. When confidence thresholds are met (3 observations for workflows with 24h+ temporal spread, 2 for procedural), artifacts are auto-created as slash commands (`.claude/commands/learned/`) or skills (`.claude/skills/learned-*/`). Toggleable via `devflow learn --enable/--disable/--status` or `devflow init --learn/--no-learn`. Configurable model/throttle/caps/debug via `devflow learn --configure`. Debug logs stored at `~/.devflow/logs/{project-slug}/`. Use `devflow learn --purge` to remove invalid observations. +**Self-Learning**: A SessionEnd hook (`session-end-learning`) accumulates session IDs and triggers a background `claude -p --model sonnet` every 3 sessions (5 at 15+ observations) to detect repeated workflows and procedural knowledge from batch transcripts. Observations accumulate in `.memory/learning-log.jsonl` with confidence scores, temporal decay, and daily run caps. When confidence thresholds are met (3 observations with 24h+ temporal spread for both workflow and procedural types), artifacts are auto-created as slash commands (`.claude/commands/self-learning/`) or skills (`.claude/skills/{slug}/`). Loaded artifacts are reinforced locally (no LLM) on each session end. Toggleable via `devflow learn --enable/--disable/--status` or `devflow init --learn/--no-learn`. Configurable model/throttle/caps/debug via `devflow learn --configure`. Debug logs stored at `~/.devflow/logs/{project-slug}/`. Use `devflow learn --purge` to remove invalid observations. ## Project Structure @@ -51,7 +51,7 @@ devflow/ ├── plugins/devflow-*/ # 17 plugins (8 core + 9 optional language/ecosystem) ├── docs/reference/ # Detailed reference documentation ├── scripts/ # Helper scripts (statusline, docs-helpers) -│ └── hooks/ # Working Memory + ambient + learning hooks (stop, session-start, pre-compact, ambient-prompt, stop-update-learning, background-learning) +│ └── hooks/ # Working Memory + ambient + learning hooks (stop, session-start, pre-compact, ambient-prompt, session-end-learning, stop-update-learning [deprecated], background-learning) ├── src/cli/ # TypeScript CLI (init, list, uninstall, ambient, learn) ├── .claude-plugin/ # Marketplace registry ├── .docs/ # Project docs (reviews, design) — per-project @@ -99,7 +99,8 @@ Working memory files live in a dedicated `.memory/` directory: ├── learning-log.jsonl # Learning observations (JSONL, one entry per line) ├── learning.json # Project-level learning config (max runs, throttle, model, debug) ├── .learning-runs-today # Daily run counter (date + count) -├── .learning-last-trigger # Throttle marker (epoch timestamp) +├── .learning-session-count # Session IDs pending batch (one per line) +├── .learning-batch-ids # Session IDs for current batch run ├── .learning-notified-at # New artifact notification marker (epoch timestamp) └── knowledge/ ├── decisions.md # Architectural decisions (ADR-NNN, append-only) diff --git a/README.md b/README.md index 0e742db..5b47513 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# DevFlow: The Most Advanced Agentic Development Toolkit +# DevFlow: The Most Advanced Agentic Development Toolkit for Production-Grade Code [![npm version](https://img.shields.io/npm/v/devflow-kit)](https://www.npmjs.com/package/devflow-kit) [![CI](https://github.com/dean0x/devflow/actions/workflows/ci.yml/badge.svg)](https://github.com/dean0x/devflow/actions/workflows/ci.yml) @@ -36,6 +36,7 @@ Claude Code is powerful. DevFlow makes it extraordinary. - **Parallel debugging** with competing hypotheses investigated simultaneously - **35 quality skills** with 9 auto-activating core, 8 language/ecosystem, plus specialized review and orchestration skills - **Ambient mode** that classifies intent and loads proportional skill sets automatically +- **Fully composable plugin system** where every feature is a plugin. Install only what you need. No bloat, no take-it-or-leave-it bundles. ## Quick Start @@ -203,14 +204,14 @@ Working memory is **per-project** — scoped to each repo's `.memory/` directory DevFlow detects repeated workflows and procedural knowledge across your sessions and automatically creates slash commands and skills. -A background agent runs on session stop (same as Working Memory) and analyzes your session transcript for patterns. When a pattern is observed enough times (3 for workflows with 24h+ temporal spread, 2 for procedural knowledge), it creates an artifact: +A background agent runs on session end, batching every 3 sessions (5 at 15+ observations) to analyze transcripts for patterns. When a pattern is observed enough times (3 observations with 24h+ temporal spread for both types), it creates an artifact: -- **Workflow patterns** become slash commands at `.claude/commands/learned/` -- **Procedural patterns** become skills at `.claude/skills/learned-*/` +- **Workflow patterns** become slash commands at `.claude/commands/self-learning/` +- **Procedural patterns** become skills at `.claude/skills/{slug}/` | Command | Description | |---------|-------------| -| `devflow learn --enable` | Register the learning Stop hook | +| `devflow learn --enable` | Register the learning SessionEnd hook | | `devflow learn --disable` | Remove the learning hook | | `devflow learn --status` | Show learning status and observation counts | | `devflow learn --list` | Show all observations sorted by confidence | @@ -235,7 +236,8 @@ DevFlow creates project documentation in `.docs/` and working memory in `.memory ├── learning-log.jsonl # Learning observations (JSONL) ├── learning.json # Project-level learning config ├── .learning-runs-today # Daily run counter -├── .learning-last-trigger # Throttle marker +├── .learning-session-count # Session IDs pending batch (one per line) +├── .learning-batch-ids # Session IDs for current batch run ├── .learning-notified-at # New artifact notification marker └── knowledge/ ├── decisions.md # Architectural decisions (ADR-NNN, append-only) @@ -339,8 +341,8 @@ npx devflow-kit skills unshadow core-patterns | Tool | Role | What It Does | |------|------|-------------| -| **[Skim](https://github.com/dean0x/skim)** | Context Optimization | Compresses code, test output, build output, and git output for optimal LLM reasoning | -| **DevFlow** | Quality Orchestration | 18 parallel reviewers, working memory, self-learning, production-grade lifecycle workflows | +| **[Skim](https://github.com/dean0x/skim)** | Context Optimization | Code-aware AST parsing across 12 languages, command rewriting, test/build/git output compression | +| **DevFlow** | Quality Orchestration | 18 parallel reviewers, working memory, self-learning, composable plugin system | | **[Backbeat](https://github.com/dean0x/backbeat)** | Agent Orchestration | Orchestration at scale. Karpathy optimization loops, multi-agent pipelines, DAG dependencies, autoscaling | Skim optimizes what your AI sees. DevFlow enforces how it works. Backbeat scales everything across agents. No other stack covers all three. diff --git a/docs/reference/file-organization.md b/docs/reference/file-organization.md index b2805e7..21bc418 100644 --- a/docs/reference/file-organization.md +++ b/docs/reference/file-organization.md @@ -47,7 +47,8 @@ devflow/ │ ├── session-start-memory # SessionStart hook: injects memory + git state │ ├── pre-compact-memory # PreCompact hook: saves git state backup │ ├── ambient-prompt # UserPromptSubmit hook: ambient skill injection -│ ├── stop-update-learning # Stop hook: triggers background learning +│ ├── session-end-learning # SessionEnd hook: batched learning trigger +│ ├── stop-update-learning # Stop hook: deprecated stub (upgrade via devflow learn) │ ├── background-learning # Background: pattern detection via Sonnet │ ├── json-helper.cjs # Node.js jq-equivalent operations │ └── json-parse # Shell wrapper: jq with node fallback @@ -154,7 +155,7 @@ Included settings: Three hooks in `scripts/hooks/` provide automatic session continuity. Toggleable via `devflow memory --enable/--disable/--status` or `devflow init --memory/--no-memory`. -A fourth hook (`stop-update-learning`) provides self-learning. Toggleable via `devflow learn --enable/--disable/--status` or `devflow init --learn/--no-learn`: +A fourth hook (`session-end-learning`) provides self-learning. Toggleable via `devflow learn --enable/--disable/--status` or `devflow init --learn/--no-learn`: | Hook | Event | File | Purpose | |------|-------|------|---------| diff --git a/scripts/hooks/background-learning b/scripts/hooks/background-learning index 18b1d19..800b476 100755 --- a/scripts/hooks/background-learning +++ b/scripts/hooks/background-learning @@ -1,16 +1,16 @@ #!/bin/bash # Background Learning Agent -# Called by stop-update-learning as a detached background process. -# Reads user messages from the session transcript, then uses a fresh `claude -p` +# Called by session-end-learning as a detached background process. +# Reads user messages from session transcripts (batch mode), then uses a fresh `claude -p` # invocation with Sonnet to detect patterns and update learning-log.jsonl. # On failure: logs error, does nothing (missing patterns are better than fake data). set -e CWD="$1" -SESSION_ID="$2" -CLAUDE_BIN="$3" +MODE="${2:---batch}" +CLAUDE_BIN="${3:-claude}" # Source JSON parsing helpers (jq with node fallback) SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" @@ -20,6 +20,7 @@ source "$SCRIPT_DIR/log-paths" LOG_FILE="$(devflow_log_dir "$CWD")/.learning-update.log" LOCK_DIR="$CWD/.memory/.learning.lock" LEARNING_LOG="$CWD/.memory/learning-log.jsonl" +RESPONSE_FILE="$CWD/.memory/.learning-response.tmp" # --- Logging --- @@ -81,6 +82,8 @@ acquire_lock() { cleanup() { rmdir "$LOCK_DIR" 2>/dev/null || true + rm -f "$CWD/.memory/learning-log.jsonl.tmp" 2>/dev/null || true + rm -f "$RESPONSE_FILE" 2>/dev/null || true } trap cleanup EXIT @@ -92,13 +95,13 @@ load_config() { GLOBAL_CONFIG="$HOME/.devflow/learning.json" PROJECT_CONFIG="$CWD/.memory/learning.json" # Defaults - MAX_DAILY_RUNS=10 + MAX_DAILY_RUNS=5 THROTTLE_MINUTES=5 MODEL="sonnet" DEBUG="false" # Apply global if [ -f "$GLOBAL_CONFIG" ]; then - MAX_DAILY_RUNS=$(json_field_file "$GLOBAL_CONFIG" "max_daily_runs" "10") + MAX_DAILY_RUNS=$(json_field_file "$GLOBAL_CONFIG" "max_daily_runs" "5") THROTTLE_MINUTES=$(json_field_file "$GLOBAL_CONFIG" "throttle_minutes" "5") MODEL=$(json_field_file "$GLOBAL_CONFIG" "model" "sonnet") DEBUG=$(json_field_file "$GLOBAL_CONFIG" "debug" "false") @@ -128,49 +131,81 @@ check_daily_cap() { return 0 } -increment_daily_counter() { - TODAY=$(date +%Y-%m-%d) - COUNT=1 - if [ -f "$COUNTER_FILE" ] && [ "$(cut -f1 "$COUNTER_FILE")" = "$TODAY" ]; then - COUNT=$(( $(cut -f2 "$COUNTER_FILE") + 1 )) - fi - printf '%s\t%d\n' "$TODAY" "$COUNT" > "$COUNTER_FILE" -} - -# --- Temporal Decay --- - -decay_factor() { - case $1 in - 0) echo "100";; 1) echo "90";; 2) echo "81";; - 3) echo "73";; 4) echo "66";; 5) echo "59";; - *) echo "53";; # floor for 6+ periods - esac -} +# --- Batch Transcript Extraction --- -# --- Transcript Extraction --- - -extract_user_messages() { +extract_batch_messages() { local encoded_cwd encoded_cwd=$(echo "$CWD" | sed 's|^/||' | tr '/' '-') - local transcript="$HOME/.claude/projects/-${encoded_cwd}/${SESSION_ID}.jsonl" + local projects_dir="$HOME/.claude/projects/-${encoded_cwd}" + local batch_file="$CWD/.memory/.learning-batch-ids" - if [ ! -f "$transcript" ]; then - log "Transcript not found at $transcript" + if [ ! -f "$batch_file" ]; then + log "No batch IDs file found" return 1 fi - # Extract ALL user text messages, skip tool_result blocks - USER_MESSAGES=$(grep '"type":"user"' "$transcript" 2>/dev/null \ - | while IFS= read -r line; do echo "$line" | json_extract_messages; done \ - | grep -v '^$') + USER_MESSAGES="" + local session_count=0 - # Truncate to 12,000 chars - if [ ${#USER_MESSAGES} -gt 12000 ]; then - USER_MESSAGES="${USER_MESSAGES:0:12000}... [truncated]" - fi + while IFS= read -r sid; do + [ -z "$sid" ] && continue + local transcript="${projects_dir}/${sid}.jsonl" + if [ ! -f "$transcript" ]; then + log "Transcript not found for session $sid" + continue + fi + + # Single-pass extraction: pipe all user-type lines through one jq/node process + local session_msgs + if [ "$_HAS_JQ" = "true" ]; then + session_msgs=$(grep '"type":"user"' "$transcript" 2>/dev/null \ + | jq -r 'if .message.content then + if (.message.content | type) == "string" then .message.content + else [.message.content[] | select(.type == "text") | .text] | join("\n") + end + else "" end' 2>/dev/null \ + | grep -v '^$' || true) + else + session_msgs=$(grep '"type":"user"' "$transcript" 2>/dev/null \ + | node -e " + const lines = require('fs').readFileSync('/dev/stdin','utf8').trim().split('\n'); + for (const line of lines) { + try { + const d = JSON.parse(line); + const c = d && d.message && d.message.content; + if (typeof c === 'string') { if (c) console.log(c); } + else if (Array.isArray(c)) { + const t = c.filter(x=>x.type==='text').map(x=>x.text).join('\n'); + if (t) console.log(t); + } + } catch {} + } + " 2>/dev/null \ + | grep -v '^$' || true) + fi + + if [ -n "$session_msgs" ]; then + # Per-session cap: 8,000 chars ensures each session contributes proportionally + if [ ${#session_msgs} -gt 8000 ]; then + session_msgs="${session_msgs:0:8000}... [truncated]" + fi + if [ -n "$USER_MESSAGES" ]; then + USER_MESSAGES="${USER_MESSAGES} +--- Session ${sid} --- +${session_msgs}" + else + USER_MESSAGES="--- Session ${sid} --- +${session_msgs}" + fi + session_count=$((session_count + 1)) + fi + done < "$batch_file" + + # Clean up batch file after reading + rm -f "$batch_file" if [ -z "$USER_MESSAGES" ]; then - log "No user text content found in transcript" + log "No user text content found in batch transcripts" return 1 fi @@ -179,49 +214,17 @@ extract_user_messages() { return 1 fi + log "Extracted messages from $session_count session(s)" return 0 } -# --- Temporal Decay Pass --- +# --- Temporal Decay --- apply_temporal_decay() { - if [ ! -f "$LEARNING_LOG" ]; then return; fi - - NOW_EPOCH=$(date +%s) - TEMP_FILE="$LEARNING_LOG.tmp" - > "$TEMP_FILE" - - while IFS= read -r line; do - if ! echo "$line" | json_valid; then continue; fi - - LAST_SEEN=$(echo "$line" | json_field "last_seen" "") - CONF=$(echo "$line" | json_field "confidence" "0") - - if [ -n "$LAST_SEEN" ]; then - LAST_EPOCH=$(date -j -f "%Y-%m-%dT%H:%M:%SZ" "$LAST_SEEN" +%s 2>/dev/null \ - || date -d "$LAST_SEEN" +%s 2>/dev/null \ - || echo "$NOW_EPOCH") - DAYS_ELAPSED=$(( (NOW_EPOCH - LAST_EPOCH) / 86400 )) - PERIODS=$(( DAYS_ELAPSED / 30 )) - - if [ "$PERIODS" -gt 0 ]; then - FACTOR=$(decay_factor "$PERIODS") - CONF_INT=$(echo "$CONF" | awk '{printf "%d", $1 * 100}') - NEW_CONF_INT=$(( CONF_INT * FACTOR / 100 )) - NEW_CONF=$(echo "$NEW_CONF_INT" | awk '{printf "%.2f", $1 / 100}') - - if [ "$NEW_CONF_INT" -lt 10 ]; then - log "Pruned observation (confidence: $NEW_CONF)" - continue - fi - - line=$(echo "$line" | json_update_field_json "confidence" "$NEW_CONF") - fi - fi - - echo "$line" >> "$TEMP_FILE" - done < "$LEARNING_LOG" - mv "$TEMP_FILE" "$LEARNING_LOG" + [ ! -f "$LEARNING_LOG" ] && return + local result + result=$(node "$_JSON_HELPER" temporal-decay "$LEARNING_LOG" 2>> "$LOG_FILE") || return + [ "$DEBUG" = "true" ] && log "Decay: $result" } # --- Entry Cap --- @@ -240,28 +243,21 @@ cap_entries() { # --- Prompt Construction --- build_sonnet_prompt() { - EXISTING_OBS="" - if [ -f "$LEARNING_LOG" ]; then - EXISTING_OBS=$(json_slurp_sort "$LEARNING_LOG" "confidence" 30 || echo "[]") - fi - if [ -z "$EXISTING_OBS" ]; then - EXISTING_OBS="[]" - fi - # Filter out contaminated entries with empty required fields - if [ "$_HAS_JQ" = "true" ]; then - EXISTING_OBS=$(echo "$EXISTING_OBS" | jq -c '[.[] | select(.id != "" and (.id | startswith("obs_")) and (.type == "workflow" or .type == "procedural") and .pattern != "")]') - else - EXISTING_OBS=$(echo "$EXISTING_OBS" | node -e "const d=JSON.parse(require('fs').readFileSync('/dev/stdin','utf8'));console.log(JSON.stringify(d.filter(o=>o.id&&o.id.startsWith('obs_')&&(o.type==='workflow'||o.type==='procedural')&&o.pattern)))") - fi + EXISTING_OBS=$(node "$_JSON_HELPER" filter-observations "$LEARNING_LOG" confidence 30 2>> "$LOG_FILE" || echo "[]") + [ -z "$EXISTING_OBS" ] && EXISTING_OBS="[]" PROMPT="You are a pattern detection agent. Analyze the user's session messages to identify repeated workflows and procedural knowledge. +# === CONTEXT === + EXISTING OBSERVATIONS (for deduplication — reuse IDs for matching patterns): $EXISTING_OBS -USER MESSAGES FROM THIS SESSION: +USER MESSAGES FROM RECENT SESSIONS: $USER_MESSAGES +# === OBSERVATION RULES === + Detect two types of patterns: 1. WORKFLOW patterns: Multi-step sequences the user instructs repeatedly (e.g., \"squash merge PR, pull main, delete branch\"). These become slash commands. @@ -269,8 +265,8 @@ Detect two types of patterns: - Temporal spread requirement: first_seen and last_seen must be 24h+ apart 2. PROCEDURAL patterns: Knowledge about how to accomplish specific tasks (e.g., debugging hook failures, configuring specific tools). These become skills. - - Required observations for artifact creation: 2 - - No temporal spread requirement + - Required observations for artifact creation: 3 (same as workflows) + - Temporal spread requirement: first_seen and last_seen must be 24h+ apart (same as workflows) Rules: - If an existing observation matches a pattern from this session, report it with the SAME id so the count can be incremented @@ -281,6 +277,53 @@ Rules: - Only report patterns that are clearly distinct — do not create near-duplicate observations - If no patterns detected, return {\"observations\": [], \"artifacts\": []} +# === SKILL TEMPLATE === + +SKILL TEMPLATE (required structure when creating skill artifacts): + +--- +name: self-learning:{slug} +description: \"This skill should be used when {specific trigger context}\" +user-invocable: false +allowed-tools: Read, Grep, Glob +--- + +# {Title} + +{One-line summary.} + +## Iron Law + +> **{SINGLE RULE IN ALL CAPS}** +> +> {2-3 sentence core principle.} + +--- + +## When This Skill Activates + +- {Trigger condition 1} +- {Trigger condition 2} + +## {Pattern Section} + +{Practical patterns, rules, or procedures.} + +# === COMMAND TEMPLATE === + +COMMAND TEMPLATE (when creating command artifacts): +Standard markdown with description frontmatter. + +# === NAMING RULES === + +NAMING RULES: +- Skill names: self-learning:{slug} (e.g., self-learning:debug-hooks) +- Skill descriptions MUST start with \"This skill should be used when...\" +- Do NOT include project-specific prefixes in the slug +- Keep slugs short and descriptive (2-3 words kebab-case) + +# === OUTPUT FORMAT === + Output ONLY the JSON object. No markdown fences, no explanation. { @@ -311,7 +354,6 @@ run_sonnet_analysis() { log "--- Sending to claude -p --model $MODEL ---" TIMEOUT=180 - RESPONSE_FILE="$CWD/.memory/.learning-response.tmp" DEVFLOW_BG_UPDATER=1 DEVFLOW_BG_LEARNER=1 "$CLAUDE_BIN" -p \ --model "$MODEL" \ @@ -328,9 +370,9 @@ run_sonnet_analysis() { if ! wait "$CLAUDE_PID" 2>/dev/null; then EXIT_CODE=$? if [ "$EXIT_CODE" -gt 128 ]; then - log "Analysis timed out (killed after ${TIMEOUT}s) for session $SESSION_ID" + log "Analysis timed out (killed after ${TIMEOUT}s)" else - log "Analysis failed for session $SESSION_ID (exit code $EXIT_CODE)" + log "Analysis failed (exit code $EXIT_CODE)" fi rm -f "$RESPONSE_FILE" kill "$WATCHDOG_PID" 2>/dev/null || true @@ -347,253 +389,45 @@ run_sonnet_analysis() { return 1 fi - RESPONSE=$(cat "$RESPONSE_FILE") - rm -f "$RESPONSE_FILE" - # Debug: log raw response if [ "$DEBUG" = "true" ]; then log "--- DEBUG: Raw model response ---" - log "$RESPONSE" + cat "$RESPONSE_FILE" >> "$LOG_FILE" log "--- DEBUG: End raw response ---" fi - # Strip markdown fences if present + # Read response, strip markdown fences, validate, write back for Node operations + local RESPONSE + RESPONSE=$(cat "$RESPONSE_FILE") RESPONSE=$(echo "$RESPONSE" | sed '1s/^```json$//' | sed '1s/^```$//' | sed '$s/^```$//') - # Validate JSON if ! echo "$RESPONSE" | json_valid; then log "Invalid JSON response from model — skipping" log "--- Raw response ---" log "$RESPONSE" log "--- End raw response ---" + rm -f "$RESPONSE_FILE" return 1 fi + echo "$RESPONSE" > "$RESPONSE_FILE" return 0 } # --- Process Observations --- process_observations() { - log "--- Processing response ---" - - OBS_COUNT=$(echo "$RESPONSE" | json_array_length "observations") - NOW_ISO=$(date -u '+%Y-%m-%dT%H:%M:%SZ') - - for i in $(seq 0 $((OBS_COUNT - 1))); do - OBS=$(echo "$RESPONSE" | json_array_item "observations" "$i") - OBS_ID=$(echo "$OBS" | json_field "id" "") - OBS_TYPE=$(echo "$OBS" | json_field "type" "") - OBS_PATTERN=$(echo "$OBS" | json_field "pattern" "") - # Evidence needs to stay as JSON array for merging later - if [ "$_HAS_JQ" = "true" ]; then - OBS_EVIDENCE=$(echo "$OBS" | jq -c '.evidence' 2>/dev/null) - else - OBS_EVIDENCE=$(echo "$OBS" | node -e "const d=JSON.parse(require('fs').readFileSync('/dev/stdin','utf8'));console.log(JSON.stringify(d.evidence||[]))" 2>/dev/null) - fi - OBS_DETAILS=$(echo "$OBS" | json_field "details" "") - - # Validate required fields - if [ -z "$OBS_ID" ] || [ -z "$OBS_TYPE" ] || [ -z "$OBS_PATTERN" ]; then - log "Skipping observation $i: empty required field (id='$OBS_ID' type='$OBS_TYPE')" - continue - fi - if [ "$OBS_TYPE" != "workflow" ] && [ "$OBS_TYPE" != "procedural" ]; then - log "Skipping observation $i: invalid type '$OBS_TYPE'" - continue - fi - case "$OBS_ID" in - obs_*) ;; - *) log "Skipping observation $i: invalid id format '$OBS_ID'"; continue ;; - esac - - # Check if observation already exists - EXISTING_LINE="" - if [ -f "$LEARNING_LOG" ]; then - EXISTING_LINE=$(grep -F "\"id\":\"$OBS_ID\"" "$LEARNING_LOG" 2>/dev/null | head -1) - fi - - if [ -n "$EXISTING_LINE" ]; then - # Update existing: increment count, update last_seen, merge evidence - OLD_COUNT=$(echo "$EXISTING_LINE" | json_field "observations" "0") - NEW_COUNT=$((OLD_COUNT + 1)) - FIRST_SEEN=$(echo "$EXISTING_LINE" | json_field "first_seen" "") - if [ "$_HAS_JQ" = "true" ]; then - OLD_EVIDENCE=$(echo "$EXISTING_LINE" | jq -c '.evidence' 2>/dev/null) - else - OLD_EVIDENCE=$(echo "$EXISTING_LINE" | node -e "const d=JSON.parse(require('fs').readFileSync('/dev/stdin','utf8'));console.log(JSON.stringify(d.evidence||[]))" 2>/dev/null) - fi - MERGED_EVIDENCE=$(echo "[$OLD_EVIDENCE, $OBS_EVIDENCE]" | json_merge_evidence) - - # Calculate confidence - if [ "$OBS_TYPE" = "workflow" ]; then - REQUIRED=3 - else - REQUIRED=2 - fi - CONF_RAW=$((NEW_COUNT * 100 / REQUIRED)) - if [ "$CONF_RAW" -gt 95 ]; then CONF_RAW=95; fi - CONF=$(echo "$CONF_RAW" | awk '{printf "%.2f", $1 / 100}') - - # Check temporal spread for workflows - STATUS=$(echo "$EXISTING_LINE" | json_field "status" "") - if [ "$OBS_TYPE" = "workflow" ] && [ "$STATUS" != "created" ]; then - FIRST_EPOCH=$(date -j -f "%Y-%m-%dT%H:%M:%SZ" "$FIRST_SEEN" +%s 2>/dev/null \ - || date -d "$FIRST_SEEN" +%s 2>/dev/null \ - || echo "0") - NOW_EPOCH=$(date +%s) - SPREAD=$((NOW_EPOCH - FIRST_EPOCH)) - if [ "$SPREAD" -lt 86400 ] && [ "$CONF_RAW" -ge 70 ]; then - STATUS="observing" - fi - fi - - # Determine status - ARTIFACT_PATH=$(echo "$EXISTING_LINE" | json_field "artifact_path" "") - if [ "$STATUS" != "created" ] && [ "$CONF_RAW" -ge 70 ]; then - if [ "$OBS_TYPE" = "workflow" ]; then - FIRST_EPOCH=$(date -j -f "%Y-%m-%dT%H:%M:%SZ" "$FIRST_SEEN" +%s 2>/dev/null \ - || date -d "$FIRST_SEEN" +%s 2>/dev/null \ - || echo "0") - NOW_EPOCH=$(date +%s) - SPREAD=$((NOW_EPOCH - FIRST_EPOCH)) - if [ "$SPREAD" -ge 86400 ]; then - STATUS="ready" - fi - else - STATUS="ready" - fi - fi - - # Build updated entry - UPDATED=$(json_obs_construct_full \ - --arg id "$OBS_ID" \ - --arg type "$OBS_TYPE" \ - --arg pattern "$OBS_PATTERN" \ - --argjson confidence "$CONF" \ - --argjson observations "$NEW_COUNT" \ - --arg first_seen "$FIRST_SEEN" \ - --arg last_seen "$NOW_ISO" \ - --arg status "$STATUS" \ - --argjson evidence "$MERGED_EVIDENCE" \ - --arg details "$OBS_DETAILS" \ - --arg artifact_path "$ARTIFACT_PATH") - - # Replace line in file - TEMP_LOG="$LEARNING_LOG.tmp" - grep -vF "\"id\":\"$OBS_ID\"" "$LEARNING_LOG" > "$TEMP_LOG" 2>/dev/null || true - echo "$UPDATED" >> "$TEMP_LOG" - mv "$TEMP_LOG" "$LEARNING_LOG" - - log "Updated observation $OBS_ID: count=$NEW_COUNT confidence=$CONF status=$STATUS" - else - # New observation - if [ "$OBS_TYPE" = "workflow" ]; then - CONF="0.33" - else - CONF="0.50" - fi - - NEW_ENTRY=$(json_obs_construct \ - --arg id "$OBS_ID" \ - --arg type "$OBS_TYPE" \ - --arg pattern "$OBS_PATTERN" \ - --argjson confidence "$CONF" \ - --argjson observations 1 \ - --arg first_seen "$NOW_ISO" \ - --arg last_seen "$NOW_ISO" \ - --arg status "observing" \ - --argjson evidence "$OBS_EVIDENCE" \ - --arg details "$OBS_DETAILS") - - echo "$NEW_ENTRY" >> "$LEARNING_LOG" - log "New observation $OBS_ID: type=$OBS_TYPE confidence=$CONF" - fi - done + local result + result=$(node "$_JSON_HELPER" process-observations "$RESPONSE_FILE" "$LEARNING_LOG" 2>> "$LOG_FILE") || return + [ "$DEBUG" = "true" ] && log "Observations: $result" } # --- Create Artifacts --- create_artifacts() { - ART_COUNT=$(echo "$RESPONSE" | json_array_length "artifacts") - - for i in $(seq 0 $((ART_COUNT - 1))); do - ART=$(echo "$RESPONSE" | json_array_item "artifacts" "$i") - ART_OBS_ID=$(echo "$ART" | json_field "observation_id" "") - ART_TYPE=$(echo "$ART" | json_field "type" "") - ART_NAME=$(echo "$ART" | json_field "name" "") - ART_DESC=$(echo "$ART" | json_field "description" "") - ART_CONTENT=$(echo "$ART" | json_field "content" "") - - # Sanitize ART_NAME — prevent path traversal (model-generated input) - ART_NAME=$(echo "$ART_NAME" | tr -d '/' | sed 's/\.\.//g') - if [ -z "$ART_NAME" ]; then - log "Skipping artifact with empty/invalid name" - continue - fi - - # Check the observation's status — only create if ready - if [ -f "$LEARNING_LOG" ]; then - OBS_STATUS=$(grep -F "\"id\":\"$ART_OBS_ID\"" "$LEARNING_LOG" 2>/dev/null | json_field "status" "") - if [ "$OBS_STATUS" != "ready" ]; then - log "Skipping artifact for $ART_OBS_ID (status: $OBS_STATUS, need: ready)" - continue - fi - fi - - if [ "$ART_TYPE" = "command" ]; then - ART_DIR="$CWD/.claude/commands/learned" - ART_PATH="$ART_DIR/$ART_NAME.md" - else - ART_DIR="$CWD/.claude/skills/learned-$ART_NAME" - ART_PATH="$ART_DIR/SKILL.md" - fi - - # Never overwrite existing files (user customization preserved) - if [ -f "$ART_PATH" ]; then - log "Artifact already exists at $ART_PATH — skipping" - continue - fi - - mkdir -p "$ART_DIR" - - # Precompute metadata (safe — our own data, not model-generated) - ART_DATE=$(date +%Y-%m-%d) - ART_CONF=$(grep -F "\"id\":\"$ART_OBS_ID\"" "$LEARNING_LOG" 2>/dev/null | json_field "confidence" "0") - ART_OBS_N=$(grep -F "\"id\":\"$ART_OBS_ID\"" "$LEARNING_LOG" 2>/dev/null | json_field "observations" "0") - - # Write artifact with learning marker - # Uses printf %s to safely write model-generated content (no shell expansion) - if [ "$ART_TYPE" = "command" ]; then - printf '%s\n' "---" \ - "description: \"$ART_DESC\"" \ - "# devflow-learning: auto-generated ($ART_DATE, confidence: $ART_CONF, obs: $ART_OBS_N)" \ - "---" \ - "" > "$ART_PATH" - printf '%s\n' "$ART_CONTENT" >> "$ART_PATH" - else - printf '%s\n' "---" \ - "name: learned-$ART_NAME" \ - "description: \"$ART_DESC\"" \ - "# devflow-learning: auto-generated ($ART_DATE, confidence: $ART_CONF, obs: $ART_OBS_N)" \ - "---" \ - "" > "$ART_PATH" - printf '%s\n' "$ART_CONTENT" >> "$ART_PATH" - fi - - # Update observation status to "created" and record artifact path - TEMP_LOG="$LEARNING_LOG.tmp" - while IFS= read -r line; do - if echo "$line" | grep -qF "\"id\":\"$ART_OBS_ID\""; then - echo "$line" | json_update_field "status" "created" | json_update_field "artifact_path" "$ART_PATH" - else - echo "$line" - fi - done < "$LEARNING_LOG" > "$TEMP_LOG" - mv "$TEMP_LOG" "$LEARNING_LOG" - - log "Created artifact: $ART_PATH" - done + local result + result=$(node "$_JSON_HELPER" create-artifacts "$RESPONSE_FILE" "$LEARNING_LOG" "$CWD" 2>> "$LOG_FILE") || return + [ "$DEBUG" = "true" ] && log "Artifacts: $result" } # --- Main --- @@ -601,14 +435,14 @@ create_artifacts() { # Wait for parent session to flush transcript sleep 3 -log "Starting learning analysis for session $SESSION_ID" +log "Starting learning analysis (mode: $MODE)" # Break stale locks from previous zombie processes break_stale_lock # Acquire lock if ! acquire_lock; then - log "Lock timeout after 90s — skipping for session $SESSION_ID" + log "Lock timeout after 90s — skipping" trap - EXIT exit 0 fi @@ -621,9 +455,9 @@ if ! check_daily_cap; then exit 0 fi -# Extract user messages +# Extract user messages (batch mode reads from .learning-batch-ids) USER_MESSAGES="" -if ! extract_user_messages; then +if ! extract_batch_messages; then log "No messages to analyze — skipping" exit 0 fi @@ -653,7 +487,10 @@ fi process_observations create_artifacts -increment_daily_counter -log "Learning analysis complete for session $SESSION_ID" +# Clean up response file +rm -f "$RESPONSE_FILE" + +# Note: daily counter already incremented by session-end-learning before spawning us +log "Learning analysis complete (batch mode)" exit 0 diff --git a/scripts/hooks/json-helper.cjs b/scripts/hooks/json-helper.cjs index 6565e83..e24cbfb 100755 --- a/scripts/hooks/json-helper.cjs +++ b/scripts/hooks/json-helper.cjs @@ -20,12 +20,15 @@ // slurp-cap Read JSONL, sort by field desc, output limit lines // array-length Get length of array at dotted path in stdin JSON // array-item Get item at index from array at path in stdin JSON -// obs-construct Build observation JSON from key=value pairs // session-output Build SessionStart output envelope // prompt-output Build UserPromptSubmit output envelope // backup-construct Build pre-compact backup JSON from --arg pairs // learning-created Extract created artifacts from learning log // learning-new Find new artifacts since epoch +// temporal-decay Apply temporal decay to learning log entries +// process-observations Merge model observations into learning log +// create-artifacts Create command/skill files from ready observations +// filter-observations [sort] [n] Filter valid observations, sort desc, limit 'use strict'; @@ -36,8 +39,10 @@ const op = process.argv[2]; const args = process.argv.slice(3); /** - * Resolve and validate a file path argument. Returns the resolved absolute path. - * Rejects paths containing '..' traversal sequences for defense-in-depth. + * Resolve a file path argument to an absolute path. + * Note: path.resolve() normalizes away '..' segments, so the includes check + * only catches the rare case of literal '..' in a directory name after resolution. + * Primary value is ensuring all file operations use absolute paths. */ function safePath(filePath) { const resolved = path.resolve(filePath); @@ -72,6 +77,47 @@ function parseJsonl(file) { }).filter(Boolean); } +// --- Learning system constants --- +const DECAY_FACTORS = [1.0, 0.90, 0.81, 0.73, 0.66, 0.59, 0.53]; +const CONFIDENCE_FLOOR = 0.10; +const DECAY_PERIOD_DAYS = 30; +const REQUIRED_OBSERVATIONS = 3; +const TEMPORAL_SPREAD_SECS = 86400; + +function learningLog(msg) { + const ts = new Date().toISOString().replace(/\.\d{3}Z$/, 'Z'); + process.stderr.write(`[${ts}] ${msg}\n`); +} + +function writeJsonlAtomic(file, entries) { + const tmp = file + '.tmp'; + const content = entries.length > 0 + ? entries.map(e => JSON.stringify(e)).join('\n') + '\n' + : ''; + fs.writeFileSync(tmp, content); + fs.renameSync(tmp, file); +} + +function calculateConfidence(count) { + const raw = Math.floor(count * 100 / REQUIRED_OBSERVATIONS); + return Math.min(raw, 95) / 100; +} + +function mergeEvidence(oldEvidence, newEvidence) { + const flat = [...(oldEvidence || []), ...(newEvidence || [])]; + const unique = [...new Set(flat)]; + return unique.slice(0, 10); +} + +/** Extract artifact display name from its file path. */ +function artifactName(obs) { + const parts = (obs.artifact_path || '').split('/'); + if (obs.type === 'workflow') { + return (parts.pop() || '').replace(/\.md$/, ''); + } + return parts.length >= 2 ? parts[parts.length - 2] : ''; +} + function parseArgs(argList) { const result = {}; const jsonArgs = {}; @@ -167,6 +213,10 @@ try { case 'extract-text-messages': { const input = JSON.parse(readStdin()); const content = input?.message?.content; + if (typeof content === 'string') { + console.log(content); + break; + } if (!Array.isArray(content)) { console.log(''); break; @@ -231,13 +281,6 @@ try { break; } - case 'obs-construct': { - // Build an observation JSON from --arg/--argjson pairs - const data = parseArgs(args); - console.log(JSON.stringify(data)); - break; - } - case 'session-output': { const ctx = args[0]; console.log(JSON.stringify({ @@ -283,51 +326,276 @@ try { const created = parsed.filter(o => o.status === 'created' && o.artifact_path); - const commands = created - .filter(o => o.type === 'workflow') - .slice(0, 5) - .map(o => { - const name = o.artifact_path.split('/').pop().replace(/\.md$/, ''); - const conf = (Math.floor(o.confidence * 10) / 10).toString(); - return { name, conf }; - }); - - const skills = created - .filter(o => o.type === 'procedural') - .slice(0, 5) - .map(o => { - const match = o.artifact_path.match(/learned-([^/]+)/); - const name = match ? match[1] : ''; - const conf = (Math.floor(o.confidence * 10) / 10).toString(); - return { name, conf }; - }); + const formatEntry = o => ({ + name: artifactName(o), + conf: (Math.floor(o.confidence * 10) / 10).toString(), + }); + + const commands = created.filter(o => o.type === 'workflow').slice(0, 5).map(formatEntry); + const skills = created.filter(o => o.type === 'procedural').slice(0, 5).map(formatEntry); console.log(JSON.stringify({ commands, skills })); break; } case 'learning-new': { - // Find new artifacts since epoch const file = args[0]; - // since_epoch argument unused in current implementation — always show created const parsed = parseJsonl(file); const created = parsed.filter(o => o.status === 'created' && o.last_seen); const messages = created.map(o => { - if (o.type === 'workflow') { - const name = o.artifact_path.split('/').pop().replace(/\.md$/, ''); - return `NEW: /learned/${name} command created from repeated workflow`; - } else { - const match = o.artifact_path.match(/learned-([^/]+)/); - const name = match ? match[1] : ''; - return `NEW: ${name} skill created from procedural knowledge`; - } + const name = artifactName(o); + return o.type === 'workflow' + ? `NEW: /self-learning/${name} command created from repeated workflow` + : `NEW: ${name} skill created from procedural knowledge`; }); console.log(messages.join('\n')); break; } + case 'temporal-decay': { + const file = safePath(args[0]); + if (!fs.existsSync(file)) { + console.log(JSON.stringify({ removed: 0, decayed: 0 })); + break; + } + const entries = parseJsonl(file); + const now = Date.now() / 1000; + let removed = 0; + let decayed = 0; + const results = []; + for (const entry of entries) { + if (entry.last_seen) { + const lastDate = new Date(entry.last_seen); + if (isNaN(lastDate.getTime())) { + learningLog(`Warning: invalid date in ${entry.id || 'unknown'}: ${entry.last_seen}`); + results.push(entry); + continue; + } + const lastEpoch = lastDate.getTime() / 1000; + const days = Math.floor((now - lastEpoch) / 86400); + const periods = Math.floor(days / DECAY_PERIOD_DAYS); + if (periods > 0) { + const factor = periods < DECAY_FACTORS.length + ? DECAY_FACTORS[periods] : DECAY_FACTORS[DECAY_FACTORS.length - 1]; + const newConf = Math.round(entry.confidence * factor * 100) / 100; + if (newConf < CONFIDENCE_FLOOR) { + removed++; + learningLog(`Removed ${entry.id || 'unknown'}: confidence ${newConf} below threshold`); + continue; + } + entry.confidence = newConf; + decayed++; + } + } + results.push(entry); + } + writeJsonlAtomic(file, results); + learningLog(`Temporal decay: removed=${removed}, decayed=${decayed}`); + console.log(JSON.stringify({ removed, decayed })); + break; + } + + case 'process-observations': { + const responseFile = safePath(args[0]); + const logFile = safePath(args[1]); + const response = JSON.parse(fs.readFileSync(responseFile, 'utf8')); + const observations = response.observations || []; + + let logEntries = []; + if (fs.existsSync(logFile)) { + logEntries = parseJsonl(logFile); + } + const logMap = new Map(logEntries.map(e => [e.id, e])); + const nowIso = new Date().toISOString().replace(/\.\d{3}Z$/, 'Z'); + let updated = 0, created = 0, skipped = 0; + + for (let i = 0; i < observations.length; i++) { + const obs = observations[i]; + if (!obs.id || !obs.type || !obs.pattern) { + learningLog(`Skipping observation ${i}: missing required field (id='${obs.id || ''}' type='${obs.type || ''}')`); + skipped++; + continue; + } + if (obs.type !== 'workflow' && obs.type !== 'procedural') { + learningLog(`Skipping observation ${i}: invalid type '${obs.type}'`); + skipped++; + continue; + } + if (!obs.id.startsWith('obs_')) { + learningLog(`Skipping observation ${i}: invalid id format '${obs.id}'`); + skipped++; + continue; + } + + const existing = logMap.get(obs.id); + if (existing) { + const newCount = (existing.observations || 0) + 1; + existing.observations = newCount; + existing.evidence = mergeEvidence(existing.evidence || [], obs.evidence || []); + existing.confidence = calculateConfidence(newCount); + existing.last_seen = nowIso; + if (obs.pattern) existing.pattern = obs.pattern; + if (obs.details) existing.details = obs.details; + + if (existing.status !== 'created') { + if (existing.confidence >= 0.70 && existing.first_seen) { + const firstDate = new Date(existing.first_seen); + if (!isNaN(firstDate.getTime())) { + const spread = Date.now() / 1000 - firstDate.getTime() / 1000; + existing.status = spread >= TEMPORAL_SPREAD_SECS ? 'ready' : 'observing'; + } + } + } + + learningLog(`Updated ${obs.id}: confidence ${existing.confidence}, status ${existing.status}`); + updated++; + } else { + const newEntry = { + id: obs.id, + type: obs.type, + pattern: obs.pattern, + confidence: 0.33, + observations: 1, + first_seen: nowIso, + last_seen: nowIso, + status: 'observing', + evidence: obs.evidence || [], + details: obs.details || '', + }; + logMap.set(obs.id, newEntry); + learningLog(`New observation ${obs.id}: type=${obs.type} confidence=0.33`); + created++; + } + } + + const logDir = path.dirname(logFile); + if (!fs.existsSync(logDir)) { + fs.mkdirSync(logDir, { recursive: true }); + } + writeJsonlAtomic(logFile, Array.from(logMap.values())); + console.log(JSON.stringify({ updated, created, skipped })); + break; + } + + case 'create-artifacts': { + const responseFile = safePath(args[0]); + const logFile = safePath(args[1]); + const baseDir = safePath(args[2]); + const response = JSON.parse(fs.readFileSync(responseFile, 'utf8')); + const artifacts = response.artifacts || []; + + if (artifacts.length === 0) { + console.log(JSON.stringify({ created: [], skipped: 0 })); + break; + } + + let logEntries = []; + if (fs.existsSync(logFile)) { + logEntries = parseJsonl(logFile); + } + const logMap = new Map(logEntries.map(e => [e.id, e])); + const createdPaths = []; + let skippedCount = 0; + const artDate = new Date().toISOString().slice(0, 10); + + for (const art of artifacts) { + let name = (art.name || '').toLowerCase().replace(/[^a-z0-9-]/g, '').slice(0, 50); + if (!name) { + learningLog('Skipping artifact with empty/invalid name'); + skippedCount++; + continue; + } + + const obs = logMap.get(art.observation_id); + if (!obs || obs.status !== 'ready') { + learningLog(`Skipping artifact for ${art.observation_id} (status: ${obs ? obs.status : 'not found'}, need: ready)`); + skippedCount++; + continue; + } + + let artDir, artPath; + if (art.type === 'command') { + artDir = path.join(baseDir, '.claude', 'commands', 'self-learning'); + artPath = path.join(artDir, `${name}.md`); + } else { + artDir = path.join(baseDir, '.claude', 'skills', name); + artPath = path.join(artDir, 'SKILL.md'); + } + + if (fs.existsSync(artPath)) { + learningLog(`Artifact already exists at ${artPath} — skipping`); + skippedCount++; + continue; + } + + const desc = (art.description || '').replace(/"/g, '\\"'); + const conf = obs.confidence || 0; + const obsN = obs.observations || 0; + + fs.mkdirSync(artDir, { recursive: true }); + + let content; + if (art.type === 'command') { + content = [ + '---', + `description: "${desc}"`, + `# devflow-learning: auto-generated (${artDate}, confidence: ${conf}, obs: ${obsN})`, + '---', + '', + art.content || '', + '', + ].join('\n'); + } else { + content = [ + '---', + `name: self-learning:${name}`, + `description: "${desc}"`, + 'user-invocable: false', + 'allowed-tools: Read, Grep, Glob', + `# devflow-learning: auto-generated (${artDate}, confidence: ${conf}, obs: ${obsN})`, + '---', + '', + art.content || '', + '', + ].join('\n'); + } + + fs.writeFileSync(artPath, content); + obs.status = 'created'; + obs.artifact_path = artPath; + learningLog(`Created artifact: ${artPath}`); + createdPaths.push(artPath); + } + + if (createdPaths.length > 0) { + writeJsonlAtomic(logFile, Array.from(logMap.values())); + } + + console.log(JSON.stringify({ created: createdPaths, skipped: skippedCount })); + break; + } + + case 'filter-observations': { + const file = args[0]; + const sortField = args[1] || 'confidence'; + const limit = parseInt(args[2]) || 30; + if (!fs.existsSync(safePath(file))) { + console.log('[]'); + break; + } + const entries = parseJsonl(file); + const valid = entries.filter(e => + e.id && e.id.startsWith('obs_') && + (e.type === 'workflow' || e.type === 'procedural') && + e.pattern + ); + valid.sort((a, b) => (b[sortField] || 0) - (a[sortField] || 0)); + console.log(JSON.stringify(valid.slice(0, limit))); + break; + } + default: process.stderr.write(`json-helper: unknown operation "${op}"\n`); process.exit(1); diff --git a/scripts/hooks/json-parse b/scripts/hooks/json-parse index 0d86ad6..6ca91c3 100755 --- a/scripts/hooks/json-parse +++ b/scripts/hooks/json-parse @@ -124,17 +124,6 @@ json_update_field_json() { # --- Slurp operations --- -# Slurp JSONL file, sort by field desc, output as JSON array. -# Usage: json_slurp_sort "file.jsonl" "confidence" 30 -json_slurp_sort() { - local file="$1" field="$2" limit="${3:-30}" - if [ "$_HAS_JQ" = "true" ]; then - jq -s "sort_by(.$field) | reverse | .[0:$limit]" "$file" 2>/dev/null - else - node "$_JSON_HELPER" slurp-sort "$file" "$field" "$limit" - fi -} - # Slurp JSONL file, sort by field desc, output as JSONL (one line per entry). # Usage: json_slurp_cap "file.jsonl" "confidence" 100 json_slurp_cap() { @@ -174,24 +163,15 @@ json_array_item() { json_extract_messages() { if [ "$_HAS_JQ" = "true" ]; then jq -r 'if .message.content then - [.message.content[] | select(.type == "text") | .text] | join("\n") + if (.message.content | type) == "string" then .message.content + else [.message.content[] | select(.type == "text") | .text] | join("\n") + end else "" end' 2>/dev/null else node "$_JSON_HELPER" extract-text-messages fi } -# --- Evidence merging --- - -# Merge two evidence arrays. Usage: echo '[[old], [new]]' | json_merge_evidence -json_merge_evidence() { - if [ "$_HAS_JQ" = "true" ]; then - jq -c 'flatten | unique | .[0:10]' 2>/dev/null - else - node "$_JSON_HELPER" merge-evidence - fi -} - # --- Hook output envelopes --- # Build SessionStart output. Usage: json_session_output "$CONTEXT" @@ -247,7 +227,7 @@ json_learning_created() { conf: (.confidence * 10 | floor / 10 | tostring) }] | .[0:5], skills: [.[] | select(.type == "procedural") | { - name: (.artifact_path | capture("learned-(?[^/]+)") | .n // ""), + name: (.artifact_path | split("/") | .[-2]), conf: (.confidence * 10 | floor / 10 | tostring) }] | .[0:5] } @@ -265,9 +245,9 @@ json_learning_new() { [.[] | select(.status == "created" and .last_seen != null)] | map( if .type == "workflow" then - "NEW: /learned/\(.artifact_path | split("/") | last | rtrimstr(".md")) command created from repeated workflow" + "NEW: /self-learning/\(.artifact_path | split("/") | last | rtrimstr(".md")) command created from repeated workflow" else - "NEW: \(.artifact_path | capture("learned-(?[^/]+)") | .n // "") skill created from procedural knowledge" + "NEW: \(.artifact_path | split("/") | .[-2]) skill created from procedural knowledge" end ) | join("\n") ' "$file" 2>/dev/null @@ -275,21 +255,3 @@ json_learning_new() { node "$_JSON_HELPER" learning-new "$file" "$since" fi } - -# Build observation JSON. Usage: json_obs_construct --arg id X --arg type Y ... -json_obs_construct() { - if [ "$_HAS_JQ" = "true" ]; then - jq -n -c "$@" '{id: $id, type: $type, pattern: $pattern, confidence: $confidence, observations: $observations, first_seen: $first_seen, last_seen: $last_seen, status: $status, evidence: $evidence, details: $details}' - else - node "$_JSON_HELPER" obs-construct "$@" - fi -} - -# Build observation JSON with artifact_path. Usage: json_obs_construct_full --arg id X ... -json_obs_construct_full() { - if [ "$_HAS_JQ" = "true" ]; then - jq -n -c "$@" '{id: $id, type: $type, pattern: $pattern, confidence: $confidence, observations: $observations, first_seen: $first_seen, last_seen: $last_seen, status: $status, evidence: $evidence, details: $details, artifact_path: $artifact_path}' - else - node "$_JSON_HELPER" obs-construct "$@" - fi -} diff --git a/scripts/hooks/session-end-learning b/scripts/hooks/session-end-learning new file mode 100755 index 0000000..ad2b610 --- /dev/null +++ b/scripts/hooks/session-end-learning @@ -0,0 +1,242 @@ +#!/bin/bash +# SessionEnd hook: learning system trigger with 3-session batching +# Replaces stop-update-learning (which ran on every Stop) +# +# Flow: +# 1. Guard clauses (skip if background process, no JSON tools, etc.) +# 2. Session depth check (skip if < 3 user turns) +# 3. Reinforce loaded self-learning artifacts (local grep, no LLM) +# 4. Batch counting: accumulate session IDs, trigger background learner at batch size +# 5. Daily cap enforcement +# +set -e + +# --- Guard clauses (break feedback loop from background claude sessions) --- +[ "${DEVFLOW_BG_LEARNER:-}" = "1" ] && exit 0 +[ "${DEVFLOW_BG_UPDATER:-}" = "1" ] && exit 0 + +# Source json-parse library (must be available before reading stdin JSON) +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +# shellcheck source=json-parse +source "$SCRIPT_DIR/json-parse" +if [ "$_JSON_AVAILABLE" = "false" ]; then exit 0; fi + +# Read hook input from stdin (Claude passes JSON with cwd, session_id, etc.) +INPUT=$(cat) + +CWD=$(echo "$INPUT" | json_field "cwd" "") +[ -z "$CWD" ] && exit 0 + +MEMORY_DIR="$CWD/.memory" +[ ! -d "$MEMORY_DIR" ] && exit 0 + +# Learning config +LEARNING_CONFIG="$MEMORY_DIR/learning.json" +[ ! -f "$LEARNING_CONFIG" ] && exit 0 + +# Resolve claude binary +CLAUDE_BIN=$(command -v claude 2>/dev/null || true) +[ -z "$CLAUDE_BIN" ] && exit 0 + +# --- Config --- +ENABLED=$(json_field "enabled" "false" < "$LEARNING_CONFIG") +[ "$ENABLED" != "true" ] && exit 0 + +DEBUG=$(json_field "debug" "false" < "$LEARNING_CONFIG") +MAX_DAILY=$(json_field "max_daily_runs" "5" < "$LEARNING_CONFIG") +BATCH_SIZE=$(json_field "batch_size" "3" < "$LEARNING_CONFIG") + +# Log path (shared helper — consistent slug with background-learning) +source "$SCRIPT_DIR/log-paths" +LOG_FILE="$(devflow_log_dir "$CWD")/.learning-update.log" + +log() { + echo "[$(date -u '+%Y-%m-%dT%H:%M:%SZ')] session-end-learning: $1" >> "$LOG_FILE" +} + +log "SessionEnd hook triggered" + +# --- Find transcript --- +# Encode CWD for Claude's project path +ENCODED_CWD=$(echo "$CWD" | sed 's|^/||' | tr '/' '-') +PROJECTS_DIR="$HOME/.claude/projects/-${ENCODED_CWD}" + +if [ ! -d "$PROJECTS_DIR" ]; then + log "No projects dir: $PROJECTS_DIR" + exit 0 +fi + +# Extract session ID from hook JSON (preferred), fall back to most recent transcript +SESSION_ID=$(echo "$INPUT" | json_field "session_id" "") +if [ -n "$SESSION_ID" ] && [ -f "$PROJECTS_DIR/${SESSION_ID}.jsonl" ]; then + TRANSCRIPT="$PROJECTS_DIR/${SESSION_ID}.jsonl" +else + TRANSCRIPT=$(ls -t "$PROJECTS_DIR"/*.jsonl 2>/dev/null | head -1 || true) + if [ -z "$TRANSCRIPT" ] || [ ! -f "$TRANSCRIPT" ]; then + log "No transcript found" + exit 0 + fi + SESSION_ID=$(basename "$TRANSCRIPT" .jsonl) +fi +log "Session: $SESSION_ID" + +# --- Session depth check (min 3 user turns) --- +USER_TURNS=$(grep -c '"type":"user"' "$TRANSCRIPT" 2>/dev/null || echo "0") +if [ "$USER_TURNS" -lt 3 ]; then + log "Shallow session ($USER_TURNS turns), skipping" + exit 0 +fi +log "Session depth: $USER_TURNS turns" + +# --- Reinforcement: update last_seen for loaded self-learning artifacts --- +reinforce_loaded_artifacts() { + local learning_log="$MEMORY_DIR/learning-log.jsonl" + [ ! -f "$learning_log" ] && return + + # Grep transcript for self-learning skill/command references + local loaded + loaded=$(grep -oE 'self-learning[:/][a-z0-9-]+' "$TRANSCRIPT" 2>/dev/null | sort -u || true) + [ -z "$loaded" ] && return + + local now_iso + now_iso=$(date -u '+%Y-%m-%dT%H:%M:%SZ') + + # Convert loaded slugs to a regex-safe pipe-delimited list for single-pass matching + local slugs_pattern + slugs_pattern=$(echo "$loaded" | sed 's|self-learning[:/]||' | paste -sd '|' -) + [ -z "$slugs_pattern" ] && return + + # Single-pass: use jq if available, otherwise node + local temp_log="${learning_log}.tmp" + if [ "$_HAS_JQ" = "true" ]; then + local updated + updated=$(jq -c --arg now "$now_iso" --arg slugs "$slugs_pattern" ' + def extract_slug: + .artifact_path as $p | + if ($p | contains("/commands/")) then ($p | split("/") | last | rtrimstr(".md")) + else ($p | split("/") | .[-2]) + end; + if .status == "created" and .artifact_path != null and .artifact_path != "" then + extract_slug as $slug | + if ($slugs | split("|") | any(. == $slug)) then .last_seen = $now | . + {"_reinforced": true} + else . + end + else . + end + ' "$learning_log" 2>/dev/null) + + if echo "$updated" | grep -qF '"_reinforced":true' 2>/dev/null; then + # Strip the internal marker and write back + echo "$updated" | jq -c 'del(._reinforced)' > "$temp_log" 2>/dev/null + mv "$temp_log" "$learning_log" + # Log which slugs were reinforced + echo "$updated" | jq -r 'select(._reinforced == true) | .artifact_path' 2>/dev/null | while IFS= read -r path; do + log "Reinforced: $(basename "$(dirname "$path")")" + done + else + rm -f "$temp_log" + fi + else + # Node fallback: process entire file at once + node -e " + const fs = require('fs'); + const lines = fs.readFileSync('/dev/stdin', 'utf8').trim().split('\n').filter(Boolean); + const slugs = process.argv[1].split('|'); + const now = process.argv[2]; + let updated = false; + const result = lines.map(line => { + try { + const obj = JSON.parse(line); + if (obj.status === 'created' && obj.artifact_path) { + const slug = obj.artifact_path.includes('/commands/') + ? obj.artifact_path.split('/').pop().replace(/\.md$/, '') + : obj.artifact_path.split('/').slice(-2, -1)[0]; + if (slugs.includes(slug)) { + obj.last_seen = now; + updated = true; + } + } + return JSON.stringify(obj); + } catch (e) { return line; } + }); + if (updated) { + fs.writeFileSync(process.argv[3], result.join('\n') + '\n'); + } + " "$slugs_pattern" "$now_iso" "$learning_log" < "$learning_log" + fi +} + +reinforce_loaded_artifacts + +# --- 3-session batching --- + +run_batch_check() { + SESSION_COUNT_FILE="$MEMORY_DIR/.learning-session-count" + BATCH_IDS_FILE="$MEMORY_DIR/.learning-batch-ids" + + # Adaptive batch size: 5 if >=15 observations, otherwise default + OBS_TOTAL=0 + if [ -f "$MEMORY_DIR/learning-log.jsonl" ]; then + OBS_TOTAL=$(wc -l < "$MEMORY_DIR/learning-log.jsonl" | tr -d ' ') + fi + if [ "$OBS_TOTAL" -ge 15 ]; then + BATCH_SIZE=5 + log "Adaptive batch size: 5 (${OBS_TOTAL} observations)" + fi + + # Validate session ID format before appending + if ! echo "$SESSION_ID" | grep -qE '^[a-zA-Z0-9_-]+$'; then + log "Invalid session ID: $SESSION_ID" + return + fi + + # Append this session ID (deduplicate) + if [ -f "$SESSION_COUNT_FILE" ] && grep -qF "$SESSION_ID" "$SESSION_COUNT_FILE" 2>/dev/null; then + log "Session already counted, skipping" + return + fi + + echo "$SESSION_ID" >> "$SESSION_COUNT_FILE" + CURRENT_COUNT=$(wc -l < "$SESSION_COUNT_FILE" | tr -d ' ') + log "Session count: $CURRENT_COUNT / $BATCH_SIZE" + + if [ "$CURRENT_COUNT" -lt "$BATCH_SIZE" ]; then + log "Batch not full yet ($CURRENT_COUNT < $BATCH_SIZE), waiting" + return + fi + + # --- Batch is full: prepare to spawn background learner --- + + # Daily cap check — tab-separated format: "DATE\tCOUNT" (matches background-learning) + RUNS_FILE="$MEMORY_DIR/.learning-runs-today" + TODAY=$(date '+%Y-%m-%d') + RUNS_TODAY=0 + if [ -f "$RUNS_FILE" ]; then + RUNS_DATE=$(cut -f1 "$RUNS_FILE") + if [ "$RUNS_DATE" = "$TODAY" ]; then + RUNS_TODAY=$(cut -f2 "$RUNS_FILE") + fi + fi + + if [ "$RUNS_TODAY" -ge "$MAX_DAILY" ]; then + log "Daily cap reached ($RUNS_TODAY >= $MAX_DAILY), skipping" + return + fi + + # Atomically hand off batch IDs to background-learning (mv is atomic on same filesystem) + mv "$SESSION_COUNT_FILE" "$BATCH_IDS_FILE" + + # Update daily run count (tab-separated: "DATE\tCOUNT") + printf '%s\t%d\n' "$TODAY" "$((RUNS_TODAY + 1))" > "$RUNS_FILE" + + log "Triggering batch learning (${CURRENT_COUNT} sessions)" + + # Spawn background learner + DEVFLOW_BG_LEARNER=1 nohup bash "$SCRIPT_DIR/background-learning" "$CWD" "--batch" "$CLAUDE_BIN" \ + >> "$LOG_FILE" 2>&1 & + disown +} + +run_batch_check + +exit 0 diff --git a/scripts/hooks/session-start-memory b/scripts/hooks/session-start-memory index ead0a74..75eaf97 100644 --- a/scripts/hooks/session-start-memory +++ b/scripts/hooks/session-start-memory @@ -143,13 +143,13 @@ if [ -f "$LEARNING_LOG" ]; then LEARNED_SKILLS="" if [ -n "$LEARNED_JSON" ]; then if [ "$_HAS_JQ" = "true" ]; then - LEARNED_COMMANDS=$(echo "$LEARNED_JSON" | jq -r '.commands | map("/learned/\(.name) (\(.conf))") | join(", ")' 2>/dev/null) + LEARNED_COMMANDS=$(echo "$LEARNED_JSON" | jq -r '.commands | map("/self-learning/\(.name) (\(.conf))") | join(", ")' 2>/dev/null) LEARNED_SKILLS=$(echo "$LEARNED_JSON" | jq -r '.skills | map("\(.name) (\(.conf))") | join(", ")' 2>/dev/null) else # Node fallback: parse the JSON and format LEARNED_COMMANDS=$(echo "$LEARNED_JSON" | node -e " const d=JSON.parse(require('fs').readFileSync('/dev/stdin','utf8')); - console.log(d.commands.map(c=>\"/learned/\"+c.name+\" (\"+c.conf+\")\").join(', ')); + console.log(d.commands.map(c=>\"/self-learning/\"+c.name+\" (\"+c.conf+\")\").join(', ')); " 2>/dev/null) LEARNED_SKILLS=$(echo "$LEARNED_JSON" | node -e " const d=JSON.parse(require('fs').readFileSync('/dev/stdin','utf8')); @@ -169,7 +169,7 @@ Commands: $LEARNED_COMMANDS" Skills: $LEARNED_SKILLS" fi LEARNED_SECTION="$LEARNED_SECTION -Edit or delete: .claude/commands/learned/ and .claude/skills/" +Edit or delete: .claude/commands/self-learning/ and .claude/skills/" # Check for new artifacts since last notification (single jq -s pass) LAST_NOTIFIED=0 diff --git a/scripts/hooks/stop-update-learning b/scripts/hooks/stop-update-learning index 9c14f38..77a0dce 100755 --- a/scripts/hooks/stop-update-learning +++ b/scripts/hooks/stop-update-learning @@ -1,98 +1,4 @@ #!/bin/bash - -# Self-Learning: Stop Hook -# Spawns a background process to analyze session patterns asynchronously. -# The session ends immediately — no visible effect in the TUI. -# On failure: does nothing (missing patterns are better than fake data). - -set -e - -# Break feedback loop: background learner's headless session triggers stop hook on exit. -# DEVFLOW_BG_LEARNER is set by background-learning before invoking claude. -if [ "${DEVFLOW_BG_LEARNER:-}" = "1" ]; then exit 0; fi -# Also guard against memory updater triggering learning -if [ "${DEVFLOW_BG_UPDATER:-}" = "1" ]; then exit 0; fi - -# Resolve script directory once (used for json-parse, ensure-memory-gitignore, and learner) -SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" - -# JSON parsing (jq with node fallback) — silently no-op if neither available -source "$SCRIPT_DIR/json-parse" -if [ "$_JSON_AVAILABLE" = "false" ]; then exit 0; fi - -INPUT=$(cat) - -# Resolve project directory — bail if missing -CWD=$(echo "$INPUT" | json_field "cwd" "") -if [ -z "$CWD" ]; then - exit 0 -fi - -# Auto-create .memory/ and ensure .gitignore entries (idempotent after first run) -source "$SCRIPT_DIR/ensure-memory-gitignore" "$CWD" || exit 0 - -# Logging -source "$SCRIPT_DIR/log-paths" -LOG_FILE="$(devflow_log_dir "$CWD")/.learning-update.log" -log() { echo "[$(date -u '+%Y-%m-%dT%H:%M:%SZ')] [stop-hook] $1" >> "$LOG_FILE"; } - -# Throttle: skip if triggered within configured throttle window -# Load throttle config -THROTTLE_MINUTES=5 -GLOBAL_CONFIG="$HOME/.devflow/learning.json" -PROJECT_CONFIG="$CWD/.memory/learning.json" -if [ -f "$PROJECT_CONFIG" ]; then - THROTTLE_MINUTES=$(json_field_file "$PROJECT_CONFIG" "throttle_minutes" "5") -elif [ -f "$GLOBAL_CONFIG" ]; then - THROTTLE_MINUTES=$(json_field_file "$GLOBAL_CONFIG" "throttle_minutes" "5") -fi -THROTTLE_SECONDS=$((THROTTLE_MINUTES * 60)) - -TRIGGER_MARKER="$CWD/.memory/.learning-last-trigger" -if [ -f "$TRIGGER_MARKER" ]; then - if stat --version &>/dev/null 2>&1; then - MARKER_MTIME=$(stat -c %Y "$TRIGGER_MARKER") - else - MARKER_MTIME=$(stat -f %m "$TRIGGER_MARKER") - fi - NOW=$(date +%s) - AGE=$(( NOW - MARKER_MTIME )) - if [ "$AGE" -lt "$THROTTLE_SECONDS" ]; then - log "Skipped: triggered ${AGE}s ago (throttle: ${THROTTLE_SECONDS}s)" - exit 0 - fi -fi - -# Resolve claude binary — if not found, skip (graceful degradation) -CLAUDE_BIN=$(command -v claude 2>/dev/null || true) -if [ -z "$CLAUDE_BIN" ]; then - log "Skipped: claude binary not found" - exit 0 -fi - -# Extract session ID from hook input -SESSION_ID=$(echo "$INPUT" | json_field "session_id" "") -if [ -z "$SESSION_ID" ]; then - log "Skipped: no session_id in hook input" - exit 0 -fi - -# Resolve the background learning script (same directory as this hook) -LEARNER="$SCRIPT_DIR/background-learning" -if [ ! -x "$LEARNER" ]; then - log "Skipped: learner not found/not executable at $LEARNER" - exit 0 -fi - -# Touch marker BEFORE spawning learner — prevents race with concurrent hooks -touch "$TRIGGER_MARKER" - -# Spawn background learner — detached, no effect on session exit -nohup "$LEARNER" "$CWD" "$SESSION_ID" "$CLAUDE_BIN" \ - /dev/null 2>&1 & -disown - -log "Spawned background learner: session=$SESSION_ID cwd=$CWD claude=$CLAUDE_BIN learner=$LEARNER" - -# Allow stop immediately (no JSON output = proceed) +# DEPRECATED: Learning moved from Stop -> SessionEnd hook. +# Run `devflow learn --disable && devflow learn --enable` to upgrade. exit 0 diff --git a/src/cli/commands/learn.ts b/src/cli/commands/learn.ts index 9aa6179..c269dbb 100644 --- a/src/cli/commands/learn.ts +++ b/src/cli/commands/learn.ts @@ -31,6 +31,8 @@ export interface LearningConfig { throttle_minutes: number; model: string; debug: boolean; + /** Number of observations processed per learning run. Default 3, adaptive 5 at 15+ observations. */ + batch_size: number; } /** @@ -51,24 +53,30 @@ export function isLearningObservation(obj: unknown): obj is LearningObservation && typeof o.details === 'string'; } -const LEARNING_HOOK_MARKER = 'stop-update-learning'; +const LEARNING_HOOK_MARKER = 'session-end-learning'; +const LEGACY_HOOK_MARKER = 'stop-update-learning'; /** - * Add the learning Stop hook to settings JSON. - * Idempotent — returns unchanged JSON if hook already exists. + * Add the learning SessionEnd hook to settings JSON. + * Idempotent — returns unchanged JSON if current hook already exists. + * Self-upgrading — removes legacy Stop hook before adding SessionEnd hook. */ export function addLearningHook(settingsJson: string, devflowDir: string): string { - const settings: Settings = JSON.parse(settingsJson); + const hookState = hasLearningHook(settingsJson); - if (hasLearningHook(settingsJson)) { + if (hookState === 'current') { return settingsJson; } + // Remove any legacy Stop hooks before adding the new SessionEnd hook + const cleanedJson = removeLearningHook(settingsJson); + const settings: Settings = JSON.parse(cleanedJson); + if (!settings.hooks) { settings.hooks = {}; } - const hookCommand = path.join(devflowDir, 'scripts', 'hooks', 'run-hook') + ' stop-update-learning'; + const hookCommand = path.join(devflowDir, 'scripts', 'hooks', 'run-hook') + ' session-end-learning'; const newEntry: HookMatcher = { hooks: [ @@ -80,40 +88,43 @@ export function addLearningHook(settingsJson: string, devflowDir: string): strin ], }; - if (!settings.hooks.Stop) { - settings.hooks.Stop = []; + if (!settings.hooks.SessionEnd) { + settings.hooks.SessionEnd = []; } - settings.hooks.Stop.push(newEntry); + settings.hooks.SessionEnd.push(newEntry); return JSON.stringify(settings, null, 2) + '\n'; } /** - * Remove the learning Stop hook from settings JSON. + * Remove the learning hook from settings JSON. + * Checks BOTH SessionEnd (new) and Stop (legacy cleanup). * Idempotent — returns unchanged JSON if hook not present. - * Preserves other Stop hooks. Cleans empty arrays/objects. + * Preserves other hooks. Cleans empty arrays/objects. */ export function removeLearningHook(settingsJson: string): string { const settings: Settings = JSON.parse(settingsJson); - - if (!settings.hooks?.Stop) { - return settingsJson; + let changed = false; + + function removeFromEvent(event: 'SessionEnd' | 'Stop', marker: string): void { + const matchers = settings.hooks?.[event]; + if (!matchers) return; + const before = matchers.length; + settings.hooks![event] = matchers.filter( + (m) => !m.hooks.some((h) => h.command.includes(marker)), + ); + if (settings.hooks![event]!.length < before) changed = true; + if (settings.hooks![event]!.length === 0) delete settings.hooks![event]; } - const before = settings.hooks.Stop.length; - settings.hooks.Stop = settings.hooks.Stop.filter( - (matcher) => !matcher.hooks.some((h) => h.command.includes(LEARNING_HOOK_MARKER)), - ); + removeFromEvent('SessionEnd', LEARNING_HOOK_MARKER); + removeFromEvent('Stop', LEGACY_HOOK_MARKER); - if (settings.hooks.Stop.length === before) { + if (!changed) { return settingsJson; } - if (settings.hooks.Stop.length === 0) { - delete settings.hooks.Stop; - } - if (settings.hooks && Object.keys(settings.hooks).length === 0) { delete settings.hooks; } @@ -123,17 +134,28 @@ export function removeLearningHook(settingsJson: string): string { /** * Check if the learning hook is registered in settings JSON. + * Returns 'current' for SessionEnd hook, 'legacy' for old Stop hook, or false if absent. */ -export function hasLearningHook(settingsJson: string): boolean { +export function hasLearningHook(settingsJson: string): 'current' | 'legacy' | false { const settings: Settings = JSON.parse(settingsJson); - if (!settings.hooks?.Stop) { - return false; + const hasSessionEnd = settings.hooks?.SessionEnd?.some((matcher) => + matcher.hooks.some((h) => h.command.includes(LEARNING_HOOK_MARKER)), + ) ?? false; + + if (hasSessionEnd) { + return 'current'; } - return settings.hooks.Stop.some((matcher) => - matcher.hooks.some((h) => h.command.includes(LEARNING_HOOK_MARKER)), - ); + const hasLegacyStop = settings.hooks?.Stop?.some((matcher) => + matcher.hooks.some((h) => h.command.includes(LEGACY_HOOK_MARKER)), + ) ?? false; + + if (hasLegacyStop) { + return 'legacy'; + } + + return false; } /** @@ -179,11 +201,16 @@ export function loadAndCountObservations(logContent: string): { /** * Format a human-readable status summary for learning state. + * hookState: 'current' (SessionEnd), 'legacy' (old Stop hook), or false (disabled). */ -export function formatLearningStatus(observations: LearningObservation[], hookEnabled: boolean): string { +export function formatLearningStatus(observations: LearningObservation[], hookState: 'current' | 'legacy' | false): string { const lines: string[] = []; - lines.push(`Self-learning: ${hookEnabled ? 'enabled' : 'disabled'}`); + if (hookState === 'legacy') { + lines.push('Self-learning: enabled (legacy — run `devflow learn --disable && devflow learn --enable` to upgrade)'); + } else { + lines.push(`Self-learning: ${hookState ? 'enabled' : 'disabled'}`); + } if (observations.length === 0) { lines.push('Observations: none'); @@ -208,7 +235,7 @@ export function formatLearningStatus(observations: LearningObservation[], hookEn * Skips fields with wrong types; swallows parse errors. */ // SYNC: Config loading duplicated in scripts/hooks/background-learning load_config() -// Synced fields: max_daily_runs, throttle_minutes, model, debug +// Synced fields: max_daily_runs, throttle_minutes, model, debug, batch_size export function applyConfigLayer(config: LearningConfig, json: string): LearningConfig { try { const raw = JSON.parse(json) as Record; @@ -217,6 +244,7 @@ export function applyConfigLayer(config: LearningConfig, json: string): Learning throttle_minutes: typeof raw.throttle_minutes === 'number' ? raw.throttle_minutes : config.throttle_minutes, model: typeof raw.model === 'string' ? raw.model : config.model, debug: typeof raw.debug === 'boolean' ? raw.debug : config.debug, + batch_size: typeof raw.batch_size === 'number' ? raw.batch_size : config.batch_size, }; } catch { return { ...config }; @@ -229,10 +257,11 @@ export function applyConfigLayer(config: LearningConfig, json: string): Learning */ export function loadLearningConfig(globalJson: string | null, projectJson: string | null): LearningConfig { let config: LearningConfig = { - max_daily_runs: 10, + max_daily_runs: 5, throttle_minutes: 5, model: 'sonnet', debug: false, + batch_size: 3, }; if (globalJson) config = applyConfigLayer(config, globalJson); @@ -241,6 +270,28 @@ export function loadLearningConfig(globalJson: string | null, projectJson: strin return config; } +/** + * Read and parse observations from the learning log file. + * Returns empty results if the file does not exist. + */ +async function readObservations(logPath: string): Promise<{ observations: LearningObservation[]; invalidCount: number }> { + try { + const logContent = await fs.readFile(logPath, 'utf-8'); + return loadAndCountObservations(logContent); + } catch { + return { observations: [], invalidCount: 0 }; + } +} + +/** + * Warn the user if invalid entries were found in the learning log. + */ +function warnIfInvalid(invalidCount: number): void { + if (invalidCount > 0) { + p.log.warn(`Note: ${invalidCount} invalid entry(ies) found. Run 'devflow learn --purge' to clean.`); + } +} + interface LearnOptions { enable?: boolean; disable?: boolean; @@ -253,7 +304,7 @@ interface LearnOptions { export const learnCommand = new Command('learn') .description('Enable or disable self-learning (workflow detection + auto-commands)') - .option('--enable', 'Register Stop hook for self-learning') + .option('--enable', 'Register SessionEnd hook for self-learning') .option('--disable', 'Remove self-learning hook') .option('--status', 'Show learning status and observation counts') .option('--list', 'Show all observations sorted by confidence') @@ -292,36 +343,24 @@ export const learnCommand = new Command('learn') settingsContent = '{}'; } + // Shared log path for --status, --list, --purge, --clear + const logPath = path.join(process.cwd(), '.memory', 'learning-log.jsonl'); + // --- --status --- if (options.status) { - const hookEnabled = hasLearningHook(settingsContent); - const cwd = process.cwd(); - const logPath = path.join(cwd, '.memory', 'learning-log.jsonl'); - - let observations: LearningObservation[] = []; - let invalidCount = 0; - try { - const logContent = await fs.readFile(logPath, 'utf-8'); - ({ observations, invalidCount } = loadAndCountObservations(logContent)); - } catch { - // No log file yet - } + const hookState = hasLearningHook(settingsContent); + const { observations, invalidCount } = await readObservations(logPath); - const status = formatLearningStatus(observations, hookEnabled); + const status = formatLearningStatus(observations, hookState); p.log.info(status); - if (invalidCount > 0) { - p.log.warn(`Note: ${invalidCount} invalid entry(ies) found. Run 'devflow learn --purge' to clean.`); - } + warnIfInvalid(invalidCount); return; } // --- --list --- if (options.list) { - const cwd = process.cwd(); - const logPath = path.join(cwd, '.memory', 'learning-log.jsonl'); - - let observations: LearningObservation[] = []; - let invalidCount = 0; + let observations: LearningObservation[]; + let invalidCount: number; try { const logContent = await fs.readFile(logPath, 'utf-8'); ({ observations, invalidCount } = loadAndCountObservations(logContent)); @@ -349,9 +388,7 @@ export const learnCommand = new Command('learn') `[${typeIcon}] ${color.cyan(obs.pattern)} (${conf}% | ${obs.observations}x | ${statusIcon})`, ); } - if (invalidCount > 0) { - p.log.warn(`Note: ${invalidCount} invalid entry(ies) found. Run 'devflow learn --purge' to clean.`); - } + warnIfInvalid(invalidCount); p.outro(color.dim(`${observations.length} observation(s) total`)); return; } @@ -362,8 +399,8 @@ export const learnCommand = new Command('learn') const maxRuns = await p.text({ message: 'Maximum background runs per day', - placeholder: '10', - defaultValue: '10', + placeholder: '5', + defaultValue: '5', validate: (v) => { const n = Number(v); if (isNaN(n) || n < 1 || n > 50) return 'Enter a number between 1 and 50'; @@ -412,6 +449,21 @@ export const learnCommand = new Command('learn') return; } + const batchSize = await p.text({ + message: 'Observations per learning run (adaptive: 5 at 15+ observations)', + placeholder: '3', + defaultValue: '3', + validate: (v) => { + const n = Number(v); + if (isNaN(n) || n < 1 || n > 20) return 'Enter a number between 1 and 20'; + return undefined; + }, + }); + if (p.isCancel(batchSize)) { + p.cancel('Configuration cancelled.'); + return; + } + const scope = await p.select({ message: 'Configuration scope', options: [ @@ -427,8 +479,9 @@ export const learnCommand = new Command('learn') const config: LearningConfig = { max_daily_runs: Number(maxRuns), throttle_minutes: Number(throttle), - model: String(model), - debug: !!debugMode, + model: model as string, + debug: debugMode, + batch_size: Number(batchSize), }; const configJson = JSON.stringify(config, null, 2) + '\n'; @@ -452,9 +505,6 @@ export const learnCommand = new Command('learn') // --- --purge --- if (options.purge) { - const cwd = process.cwd(); - const logPath = path.join(cwd, '.memory', 'learning-log.jsonl'); - let logContent: string; try { logContent = await fs.readFile(logPath, 'utf-8'); @@ -478,9 +528,6 @@ export const learnCommand = new Command('learn') // --- --clear --- if (options.clear) { - const cwd = process.cwd(); - const logPath = path.join(cwd, '.memory', 'learning-log.jsonl'); - try { await fs.access(logPath); } catch { @@ -509,10 +556,12 @@ export const learnCommand = new Command('learn') let devflowDir: string; try { const settings: Settings = JSON.parse(settingsContent); - // Try to extract devflowDir from existing hooks (e.g., Stop hook path) + // Try to extract devflowDir from existing hooks (SessionEnd first, Stop fallback) + const sessionEndHook = settings.hooks?.SessionEnd?.[0]?.hooks?.[0]?.command; const stopHook = settings.hooks?.Stop?.[0]?.hooks?.[0]?.command; - if (stopHook) { - const hookBinary = stopHook.split(' ')[0]; + const hookCommand = sessionEndHook || stopHook; + if (hookCommand) { + const hookBinary = hookCommand.split(' ')[0]; devflowDir = path.resolve(hookBinary, '..', '..', '..'); } else { devflowDir = getDevFlowDirectory(); @@ -522,13 +571,18 @@ export const learnCommand = new Command('learn') } if (options.enable) { + const priorState = hasLearningHook(settingsContent); const updated = addLearningHook(settingsContent, devflowDir); if (updated === settingsContent) { p.log.info('Self-learning already enabled'); return; } await fs.writeFile(settingsPath, updated, 'utf-8'); - p.log.success('Self-learning enabled — Stop hook registered'); + if (priorState === 'legacy') { + p.log.success('Self-learning upgraded — legacy Stop hook replaced with SessionEnd hook'); + } else { + p.log.success('Self-learning enabled — SessionEnd hook registered'); + } p.log.info(color.dim('Repeated workflows will be detected and turned into slash commands')); } diff --git a/tests/learn.test.ts b/tests/learn.test.ts index 2331aa2..bb01425 100644 --- a/tests/learn.test.ts +++ b/tests/learn.test.ts @@ -4,6 +4,7 @@ import { removeLearningHook, hasLearningHook, parseLearningLog, + loadAndCountObservations, formatLearningStatus, loadLearningConfig, isLearningObservation, @@ -16,9 +17,9 @@ describe('addLearningHook', () => { const result = addLearningHook('{}', '/home/user/.devflow'); const settings = JSON.parse(result); - expect(settings.hooks.Stop).toHaveLength(1); - expect(settings.hooks.Stop[0].hooks[0].command).toContain('stop-update-learning'); - expect(settings.hooks.Stop[0].hooks[0].timeout).toBe(10); + expect(settings.hooks.SessionEnd).toHaveLength(1); + expect(settings.hooks.SessionEnd[0].hooks[0].command).toContain('session-end-learning'); + expect(settings.hooks.SessionEnd[0].hooks[0].timeout).toBe(10); }); it('adds alongside existing hooks (Stop hooks from memory)', () => { @@ -30,9 +31,9 @@ describe('addLearningHook', () => { const result = addLearningHook(input, '/home/user/.devflow'); const settings = JSON.parse(result); - expect(settings.hooks.Stop).toHaveLength(2); - expect(settings.hooks.Stop[0].hooks[0].command).toBe('stop-update-memory'); - expect(settings.hooks.Stop[1].hooks[0].command).toContain('stop-update-learning'); + expect(settings.hooks.Stop).toHaveLength(1); + expect(settings.hooks.SessionEnd).toHaveLength(1); + expect(settings.hooks.SessionEnd[0].hooks[0].command).toContain('session-end-learning'); }); it('is idempotent — does not add duplicate', () => { @@ -45,10 +46,10 @@ describe('addLearningHook', () => { it('uses correct path via run-hook wrapper', () => { const result = addLearningHook('{}', '/custom/path/.devflow'); const settings = JSON.parse(result); - const command = settings.hooks.Stop[0].hooks[0].command; + const command = settings.hooks.SessionEnd[0].hooks[0].command; expect(command).toContain('/custom/path/.devflow/scripts/hooks/run-hook'); - expect(command).toContain('stop-update-learning'); + expect(command).toContain('session-end-learning'); }); it('preserves other settings', () => { @@ -61,26 +62,66 @@ describe('addLearningHook', () => { expect(settings.statusLine.command).toBe('statusline.sh'); expect(settings.env.SOME_VAR).toBe('1'); + expect(settings.hooks.SessionEnd).toHaveLength(1); + }); + + it('adds alongside existing SessionEnd hooks', () => { + const input = JSON.stringify({ + hooks: { + SessionEnd: [{ hooks: [{ type: 'command', command: 'other-session-end.sh' }] }], + UserPromptSubmit: [{ hooks: [{ type: 'command', command: 'ambient-prompt' }] }], + }, + }); + const result = addLearningHook(input, '/home/user/.devflow'); + const settings = JSON.parse(result); + + expect(settings.hooks.SessionEnd).toHaveLength(2); + expect(settings.hooks.UserPromptSubmit).toHaveLength(1); + }); + + it('self-upgrades legacy Stop hook to SessionEnd', () => { + const input = JSON.stringify({ + hooks: { + Stop: [ + { hooks: [{ type: 'command', command: 'stop-update-memory' }] }, + { hooks: [{ type: 'command', command: '/old/path/stop-update-learning' }] }, + ], + }, + }); + const result = addLearningHook(input, '/home/user/.devflow'); + const settings = JSON.parse(result); + + // Legacy Stop learning hook removed, memory hook preserved expect(settings.hooks.Stop).toHaveLength(1); + expect(settings.hooks.Stop[0].hooks[0].command).toBe('stop-update-memory'); + // New SessionEnd hook added + expect(settings.hooks.SessionEnd).toHaveLength(1); + expect(settings.hooks.SessionEnd[0].hooks[0].command).toContain('session-end-learning'); }); - it('adds alongside existing Stop hooks', () => { + it('self-upgrades legacy Stop hook and preserves other events', () => { const input = JSON.stringify({ hooks: { - Stop: [{ hooks: [{ type: 'command', command: 'other-stop.sh' }] }], + Stop: [ + { hooks: [{ type: 'command', command: '/old/path/stop-update-learning' }] }, + ], UserPromptSubmit: [{ hooks: [{ type: 'command', command: 'ambient-prompt' }] }], }, }); const result = addLearningHook(input, '/home/user/.devflow'); const settings = JSON.parse(result); - expect(settings.hooks.Stop).toHaveLength(2); + // Legacy Stop hook removed entirely (was the only Stop entry) + expect(settings.hooks.Stop).toBeUndefined(); + // New SessionEnd hook added + expect(settings.hooks.SessionEnd).toHaveLength(1); + // Other hooks preserved expect(settings.hooks.UserPromptSubmit).toHaveLength(1); }); }); describe('removeLearningHook', () => { - it('removes learning hook', () => { + it('removes learning hook from SessionEnd', () => { const withHook = addLearningHook('{}', '/home/user/.devflow'); const result = removeLearningHook(withHook); const settings = JSON.parse(result); @@ -88,27 +129,27 @@ describe('removeLearningHook', () => { expect(settings.hooks).toBeUndefined(); }); - it('preserves memory Stop hooks', () => { + it('preserves other SessionEnd hooks', () => { const input = JSON.stringify({ hooks: { - Stop: [ - { hooks: [{ type: 'command', command: 'stop-update-memory' }] }, - { hooks: [{ type: 'command', command: '/path/to/stop-update-learning' }] }, + SessionEnd: [ + { hooks: [{ type: 'command', command: 'other-session-end-hook' }] }, + { hooks: [{ type: 'command', command: '/path/to/session-end-learning' }] }, ], }, }); const result = removeLearningHook(input); const settings = JSON.parse(result); - expect(settings.hooks.Stop).toHaveLength(1); - expect(settings.hooks.Stop[0].hooks[0].command).toBe('stop-update-memory'); + expect(settings.hooks.SessionEnd).toHaveLength(1); + expect(settings.hooks.SessionEnd[0].hooks[0].command).toBe('other-session-end-hook'); }); it('cleans empty hooks object when last hook removed', () => { const input = JSON.stringify({ hooks: { - Stop: [ - { hooks: [{ type: 'command', command: '/path/to/stop-update-learning' }] }, + SessionEnd: [ + { hooks: [{ type: 'command', command: '/path/to/session-end-learning' }] }, ], }, }); @@ -122,8 +163,8 @@ describe('removeLearningHook', () => { const input = JSON.stringify({ hooks: { UserPromptSubmit: [{ hooks: [{ type: 'command', command: 'ambient-prompt' }] }], - Stop: [ - { hooks: [{ type: 'command', command: '/path/to/stop-update-learning' }] }, + SessionEnd: [ + { hooks: [{ type: 'command', command: '/path/to/session-end-learning' }] }, ], }, }); @@ -131,7 +172,7 @@ describe('removeLearningHook', () => { const settings = JSON.parse(result); expect(settings.hooks.UserPromptSubmit).toHaveLength(1); - expect(settings.hooks.Stop).toBeUndefined(); + expect(settings.hooks.SessionEnd).toBeUndefined(); }); it('is idempotent', () => { @@ -144,39 +185,108 @@ describe('removeLearningHook', () => { expect(result).toBe(input); }); + + it('cleans up legacy Stop entries', () => { + const input = JSON.stringify({ + hooks: { + Stop: [ + { hooks: [{ type: 'command', command: 'stop-update-memory' }] }, + { hooks: [{ type: 'command', command: '/path/to/stop-update-learning' }] }, + ], + }, + }); + const result = removeLearningHook(input); + const settings = JSON.parse(result); + expect(settings.hooks.Stop).toHaveLength(1); + expect(settings.hooks.Stop[0].hooks[0].command).toBe('stop-update-memory'); + }); + + it('cleans both SessionEnd and legacy Stop', () => { + const input = JSON.stringify({ + hooks: { + SessionEnd: [ + { hooks: [{ type: 'command', command: '/path/session-end-learning' }] }, + ], + Stop: [ + { hooks: [{ type: 'command', command: 'stop-update-memory' }] }, + { hooks: [{ type: 'command', command: '/old/path/stop-update-learning' }] }, + ], + }, + }); + const result = removeLearningHook(input); + const settings = JSON.parse(result); + expect(settings.hooks.SessionEnd).toBeUndefined(); + expect(settings.hooks.Stop).toHaveLength(1); + }); }); describe('hasLearningHook', () => { - it('returns true when present', () => { + it('returns current when present on SessionEnd', () => { const withHook = addLearningHook('{}', '/home/user/.devflow'); - expect(hasLearningHook(withHook)).toBe(true); + expect(hasLearningHook(withHook)).toBe('current'); }); it('returns false when absent', () => { expect(hasLearningHook('{}')).toBe(false); }); - it('returns false for non-learning Stop hooks', () => { + it('returns false for non-learning SessionEnd hooks', () => { const input = JSON.stringify({ hooks: { - Stop: [ - { hooks: [{ type: 'command', command: 'stop-update-memory' }] }, + SessionEnd: [ + { hooks: [{ type: 'command', command: 'some-other-hook' }] }, ], }, }); expect(hasLearningHook(input)).toBe(false); }); - it('returns true among other Stop hooks', () => { + it('returns current among other SessionEnd hooks', () => { + const input = JSON.stringify({ + hooks: { + SessionEnd: [ + { hooks: [{ type: 'command', command: 'some-other-hook' }] }, + { hooks: [{ type: 'command', command: '/path/to/session-end-learning' }] }, + ], + }, + }); + expect(hasLearningHook(input)).toBe('current'); + }); + + it('returns legacy for Stop hook with stop-update-learning', () => { const input = JSON.stringify({ hooks: { Stop: [ - { hooks: [{ type: 'command', command: 'stop-update-memory' }] }, { hooks: [{ type: 'command', command: '/path/to/stop-update-learning' }] }, ], }, }); - expect(hasLearningHook(input)).toBe(true); + expect(hasLearningHook(input)).toBe('legacy'); + }); + + it('returns current when both SessionEnd and legacy Stop present', () => { + const input = JSON.stringify({ + hooks: { + SessionEnd: [ + { hooks: [{ type: 'command', command: '/path/to/session-end-learning' }] }, + ], + Stop: [ + { hooks: [{ type: 'command', command: '/path/to/stop-update-learning' }] }, + ], + }, + }); + expect(hasLearningHook(input)).toBe('current'); + }); + + it('returns false for non-learning Stop hooks', () => { + const input = JSON.stringify({ + hooks: { + Stop: [ + { hooks: [{ type: 'command', command: 'stop-update-memory' }] }, + ], + }, + }); + expect(hasLearningHook(input)).toBe(false); }); }); @@ -218,10 +328,64 @@ describe('parseLearningLog', () => { }); }); +describe('loadAndCountObservations', () => { + it('counts mixed valid and invalid lines', () => { + const valid = JSON.stringify({ + id: 'obs_1', type: 'workflow', pattern: 'p1', + confidence: 0.5, observations: 1, first_seen: 't', + last_seen: 't', status: 'observing', evidence: [], details: 'd', + }); + const invalid = 'not json at all'; + const incomplete = JSON.stringify({ id: 'obs_2', type: 'workflow' }); + const log = [valid, invalid, incomplete].join('\n'); + const result = loadAndCountObservations(log); + expect(result.observations).toHaveLength(1); + expect(result.observations[0].id).toBe('obs_1'); + expect(result.invalidCount).toBe(2); + }); + + it('returns zero invalid count for all-valid lines', () => { + const lines = [ + JSON.stringify({ + id: 'obs_1', type: 'workflow', pattern: 'p1', + confidence: 0.5, observations: 1, first_seen: 't', + last_seen: 't', status: 'observing', evidence: [], details: 'd', + }), + JSON.stringify({ + id: 'obs_2', type: 'procedural', pattern: 'p2', + confidence: 0.8, observations: 2, first_seen: 't', + last_seen: 't', status: 'ready', evidence: ['e1'], details: 'd2', + }), + ].join('\n'); + const result = loadAndCountObservations(lines); + expect(result.observations).toHaveLength(2); + expect(result.invalidCount).toBe(0); + }); + + it('handles empty input', () => { + const result = loadAndCountObservations(''); + expect(result.observations).toHaveLength(0); + expect(result.invalidCount).toBe(0); + }); + + it('calculates invalidCount as rawLines minus valid observations', () => { + const valid = JSON.stringify({ + id: 'obs_1', type: 'workflow', pattern: 'p1', + confidence: 0.5, observations: 1, first_seen: 't', + last_seen: 't', status: 'observing', evidence: [], details: 'd', + }); + const log = [valid, 'bad1', 'bad2', 'bad3'].join('\n'); + const result = loadAndCountObservations(log); + expect(result.observations).toHaveLength(1); + expect(result.invalidCount).toBe(3); + }); +}); + describe('formatLearningStatus', () => { - it('shows enabled state', () => { - const result = formatLearningStatus([], true); + it('shows enabled state for current hook', () => { + const result = formatLearningStatus([], 'current'); expect(result).toContain('enabled'); + expect(result).not.toContain('legacy'); }); it('shows disabled state', () => { @@ -229,13 +393,19 @@ describe('formatLearningStatus', () => { expect(result).toContain('disabled'); }); + it('shows legacy upgrade message for legacy hook', () => { + const result = formatLearningStatus([], 'legacy'); + expect(result).toContain('legacy'); + expect(result).toContain('devflow learn --disable && devflow learn --enable'); + }); + it('shows observation counts', () => { const observations: LearningObservation[] = [ { id: 'obs_1', type: 'workflow', pattern: 'p1', confidence: 0.33, observations: 1, first_seen: 't', last_seen: 't', status: 'observing', evidence: [], details: 'd' }, { id: 'obs_2', type: 'procedural', pattern: 'p2', confidence: 0.50, observations: 1, first_seen: 't', last_seen: 't', status: 'observing', evidence: [], details: 'd' }, { id: 'obs_3', type: 'workflow', pattern: 'p3', confidence: 0.95, observations: 3, first_seen: 't', last_seen: 't', status: 'ready', evidence: [], details: 'd' }, ]; - const result = formatLearningStatus(observations, true); + const result = formatLearningStatus(observations, 'current'); expect(result).toContain('3 total'); expect(result).toContain('Workflows: 2'); expect(result).toContain('Procedural: 1'); @@ -246,13 +416,13 @@ describe('formatLearningStatus', () => { { id: 'obs_1', type: 'workflow', pattern: 'p1', confidence: 0.95, observations: 3, first_seen: 't', last_seen: 't', status: 'created', evidence: [], details: 'd', artifact_path: '/path' }, { id: 'obs_2', type: 'procedural', pattern: 'p2', confidence: 0.50, observations: 1, first_seen: 't', last_seen: 't', status: 'observing', evidence: [], details: 'd' }, ]; - const result = formatLearningStatus(observations, true); + const result = formatLearningStatus(observations, 'current'); expect(result).toContain('1 promoted'); expect(result).toContain('1 observing'); }); it('handles empty observations', () => { - const result = formatLearningStatus([], true); + const result = formatLearningStatus([], 'current'); expect(result).toContain('none'); }); }); @@ -260,10 +430,11 @@ describe('formatLearningStatus', () => { describe('loadLearningConfig', () => { it('returns defaults when no config files', () => { const config = loadLearningConfig(null, null); - expect(config.max_daily_runs).toBe(10); + expect(config.max_daily_runs).toBe(5); expect(config.throttle_minutes).toBe(5); expect(config.model).toBe('sonnet'); expect(config.debug).toBe(false); + expect(config.batch_size).toBe(3); }); it('loads global config', () => { @@ -272,6 +443,7 @@ describe('loadLearningConfig', () => { expect(config.max_daily_runs).toBe(20); expect(config.throttle_minutes).toBe(5); // default preserved expect(config.model).toBe('haiku'); + expect(config.batch_size).toBe(3); // default preserved }); it('project config overrides global', () => { @@ -285,10 +457,22 @@ describe('loadLearningConfig', () => { it('handles partial override (only some fields)', () => { const projectJson = JSON.stringify({ throttle_minutes: 15 }); const config = loadLearningConfig(null, projectJson); - expect(config.max_daily_runs).toBe(10); // default + expect(config.max_daily_runs).toBe(5); // default expect(config.throttle_minutes).toBe(15); // overridden expect(config.model).toBe('sonnet'); // default }); + + it('loads batch_size from config', () => { + const projectJson = JSON.stringify({ batch_size: 5 }); + const config = loadLearningConfig(null, projectJson); + expect(config.batch_size).toBe(5); + }); + + it('ignores non-numeric batch_size', () => { + const projectJson = JSON.stringify({ batch_size: 'large' }); + const config = loadLearningConfig(null, projectJson); + expect(config.batch_size).toBe(3); // default preserved + }); }); describe('isLearningObservation', () => { @@ -395,35 +579,47 @@ describe('parseLearningLog — type guard filtering', () => { describe('applyConfigLayer — immutability', () => { it('returns new object without mutating input', () => { - const original = { max_daily_runs: 10, throttle_minutes: 5, model: 'sonnet', debug: false }; + const original = { max_daily_runs: 10, throttle_minutes: 5, model: 'sonnet', debug: false, batch_size: 3 }; const result = applyConfigLayer(original, JSON.stringify({ max_daily_runs: 20 })); expect(result.max_daily_runs).toBe(20); expect(original.max_daily_runs).toBe(10); // not mutated }); it('returns copy on invalid JSON', () => { - const original = { max_daily_runs: 10, throttle_minutes: 5, model: 'sonnet', debug: false }; + const original = { max_daily_runs: 10, throttle_minutes: 5, model: 'sonnet', debug: false, batch_size: 3 }; const result = applyConfigLayer(original, 'not json'); expect(result).toEqual(original); expect(result).not.toBe(original); // different reference }); it('ignores wrong-typed fields', () => { - const original = { max_daily_runs: 10, throttle_minutes: 5, model: 'sonnet', debug: false }; + const original = { max_daily_runs: 10, throttle_minutes: 5, model: 'sonnet', debug: false, batch_size: 3 }; const result = applyConfigLayer(original, JSON.stringify({ max_daily_runs: 'lots', model: 42 })); expect(result.max_daily_runs).toBe(10); expect(result.model).toBe('sonnet'); }); it('applies debug field when boolean', () => { - const original = { max_daily_runs: 10, throttle_minutes: 5, model: 'sonnet', debug: false }; + const original = { max_daily_runs: 10, throttle_minutes: 5, model: 'sonnet', debug: false, batch_size: 3 }; const result = applyConfigLayer(original, JSON.stringify({ debug: true })); expect(result.debug).toBe(true); }); it('ignores debug field when non-boolean', () => { - const original = { max_daily_runs: 10, throttle_minutes: 5, model: 'sonnet', debug: false }; + const original = { max_daily_runs: 10, throttle_minutes: 5, model: 'sonnet', debug: false, batch_size: 3 }; const result = applyConfigLayer(original, JSON.stringify({ debug: 'yes' })); expect(result.debug).toBe(false); }); + + it('applies batch_size field when number', () => { + const original = { max_daily_runs: 10, throttle_minutes: 5, model: 'sonnet', debug: false, batch_size: 3 }; + const result = applyConfigLayer(original, JSON.stringify({ batch_size: 5 })); + expect(result.batch_size).toBe(5); + }); + + it('ignores batch_size field when non-number', () => { + const original = { max_daily_runs: 10, throttle_minutes: 5, model: 'sonnet', debug: false, batch_size: 3 }; + const result = applyConfigLayer(original, JSON.stringify({ batch_size: 'large' })); + expect(result.batch_size).toBe(3); + }); }); diff --git a/tests/shell-hooks.test.ts b/tests/shell-hooks.test.ts index 3bbf42b..5ec0479 100644 --- a/tests/shell-hooks.test.ts +++ b/tests/shell-hooks.test.ts @@ -10,6 +10,7 @@ const JSON_HELPER = path.join(HOOKS_DIR, 'json-helper.cjs'); const HOOK_SCRIPTS = [ 'background-learning', + 'session-end-learning', 'stop-update-learning', 'background-memory-update', 'stop-update-memory', @@ -35,31 +36,6 @@ describe('shell hook syntax checks', () => { }); describe('background-learning pure functions', () => { - it('decay_factor returns correct values for all periods', () => { - const expected: Record = { - '0': '100', '1': '90', '2': '81', - '3': '73', '4': '66', '5': '59', - '6': '53', '10': '53', '99': '53', - }; - - for (const [input, output] of Object.entries(expected)) { - const result = execSync( - `bash -c ' - decay_factor() { - case $1 in - 0) echo "100";; 1) echo "90";; 2) echo "81";; - 3) echo "73";; 4) echo "66";; 5) echo "59";; - *) echo "53";; - esac - } - decay_factor ${input} - '`, - { stdio: 'pipe' }, - ).toString().trim(); - expect(result).toBe(output); - } - }); - it('check_daily_cap respects counter file', () => { const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); const counterFile = path.join(tmpDir, '.learning-runs-today'); @@ -192,6 +168,19 @@ describe('json-helper.js operations', () => { expect(result).toBe('Hello world\nSecond message'); }); + it('extract-text-messages handles plain string content', () => { + const input = JSON.stringify({ + message: { + content: 'plain string message', + }, + }); + const result = execSync( + `echo '${input.replace(/'/g, "'\\''")}' | node "${JSON_HELPER}" extract-text-messages`, + { stdio: 'pipe' }, + ).toString().trim(); + expect(result).toBe('plain string message'); + }); + it('merge-evidence flattens, dedupes, and limits', () => { const input = JSON.stringify([['a', 'b', 'c'], ['b', 'c', 'd']]); const result = execSync( @@ -279,8 +268,8 @@ describe('json-helper.js operations', () => { try { fs.writeFileSync(file, [ - JSON.stringify({ id: 'obs_1', type: 'workflow', status: 'created', artifact_path: '/path/learned/deploy-flow.md', confidence: 0.95 }), - JSON.stringify({ id: 'obs_2', type: 'procedural', status: 'created', artifact_path: '/path/learned-debug-hooks/SKILL.md', confidence: 0.8 }), + JSON.stringify({ id: 'obs_1', type: 'workflow', status: 'created', artifact_path: '/.claude/commands/self-learning/deploy-flow.md', confidence: 0.95 }), + JSON.stringify({ id: 'obs_2', type: 'procedural', status: 'created', artifact_path: '/.claude/skills/debug-hooks/SKILL.md', confidence: 0.8 }), JSON.stringify({ id: 'obs_3', type: 'workflow', status: 'observing', confidence: 0.3 }), ].join('\n')); @@ -297,6 +286,811 @@ describe('json-helper.js operations', () => { fs.rmSync(tmpDir, { recursive: true, force: true }); } }); + + it('learning-new outputs new artifact notifications with self-learning prefix', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const file = path.join(tmpDir, 'learning.jsonl'); + + try { + fs.writeFileSync(file, [ + JSON.stringify({ id: 'obs_1', type: 'workflow', status: 'created', artifact_path: '/.claude/commands/self-learning/deploy-flow.md', confidence: 0.95, last_seen: '2026-03-22T00:00:00Z' }), + JSON.stringify({ id: 'obs_2', type: 'procedural', status: 'created', artifact_path: '/.claude/skills/debug-hooks/SKILL.md', confidence: 0.8, last_seen: '2026-03-22T00:00:00Z' }), + JSON.stringify({ id: 'obs_3', type: 'workflow', status: 'observing', confidence: 0.3, last_seen: '2026-03-22T00:00:00Z' }), + ].join('\n')); + + const result = execSync( + `node "${JSON_HELPER}" learning-new "${file}" 0`, + { stdio: 'pipe' }, + ).toString().trim(); + const lines = result.split('\n'); + expect(lines).toHaveLength(2); + expect(lines[0]).toContain('self-learning/deploy-flow'); + expect(lines[0]).toContain('command created'); + expect(lines[1]).toContain('debug-hooks'); + expect(lines[1]).toContain('skill created'); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); +}); + +describe('json-helper.cjs temporal-decay', () => { + it('applies decay to old entries', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const file = path.join(tmpDir, 'learning.jsonl'); + try { + const oldDate = new Date(Date.now() - 35 * 86400000).toISOString(); + fs.writeFileSync(file, JSON.stringify({ + id: 'obs_test1', confidence: 0.66, last_seen: oldDate, + }) + '\n'); + + const result = execSync( + `node "${JSON_HELPER}" temporal-decay "${file}"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ).toString().trim(); + const counts = JSON.parse(result); + expect(counts.decayed).toBe(1); + + const updated = fs.readFileSync(file, 'utf8').trim(); + const entry = JSON.parse(updated); + expect(entry.confidence).toBeLessThan(0.66); + expect(entry.confidence).toBeGreaterThan(0); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); + + it('removes entries below threshold', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const file = path.join(tmpDir, 'learning.jsonl'); + try { + const oldDate = new Date(Date.now() - 200 * 86400000).toISOString(); + fs.writeFileSync(file, JSON.stringify({ + id: 'obs_test1', confidence: 0.15, last_seen: oldDate, + }) + '\n'); + + const result = execSync( + `node "${JSON_HELPER}" temporal-decay "${file}"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ).toString().trim(); + const counts = JSON.parse(result); + expect(counts.removed).toBe(1); + + const content = fs.readFileSync(file, 'utf8').trim(); + expect(content).toBe(''); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); + + it('preserves fresh entries', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const file = path.join(tmpDir, 'learning.jsonl'); + try { + const recentDate = new Date().toISOString(); + fs.writeFileSync(file, JSON.stringify({ + id: 'obs_test1', confidence: 0.66, last_seen: recentDate, + }) + '\n'); + + const result = execSync( + `node "${JSON_HELPER}" temporal-decay "${file}"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ).toString().trim(); + const counts = JSON.parse(result); + expect(counts.decayed).toBe(0); + expect(counts.removed).toBe(0); + + const entry = JSON.parse(fs.readFileSync(file, 'utf8').trim()); + expect(entry.confidence).toBe(0.66); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); + + it('handles missing file gracefully', () => { + const result = execSync( + `node "${JSON_HELPER}" temporal-decay "/tmp/nonexistent-devflow-test-${Date.now()}.jsonl"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ).toString().trim(); + const counts = JSON.parse(result); + expect(counts.removed).toBe(0); + expect(counts.decayed).toBe(0); + }); + + it('handles empty file', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const file = path.join(tmpDir, 'learning.jsonl'); + try { + fs.writeFileSync(file, ''); + const result = execSync( + `node "${JSON_HELPER}" temporal-decay "${file}"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ).toString().trim(); + const counts = JSON.parse(result); + expect(counts.removed).toBe(0); + expect(counts.decayed).toBe(0); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); + + it('returns correct counts for mixed entries', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const file = path.join(tmpDir, 'learning.jsonl'); + try { + const old35 = new Date(Date.now() - 35 * 86400000).toISOString(); + const old200 = new Date(Date.now() - 200 * 86400000).toISOString(); + const recent = new Date().toISOString(); + fs.writeFileSync(file, [ + JSON.stringify({ id: 'obs_a', confidence: 0.66, last_seen: old35 }), + JSON.stringify({ id: 'obs_b', confidence: 0.15, last_seen: old200 }), + JSON.stringify({ id: 'obs_c', confidence: 0.95, last_seen: recent }), + ].join('\n') + '\n'); + + const result = execSync( + `node "${JSON_HELPER}" temporal-decay "${file}"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ).toString().trim(); + const counts = JSON.parse(result); + expect(counts.decayed).toBe(1); + expect(counts.removed).toBe(1); + + const lines = fs.readFileSync(file, 'utf8').trim().split('\n'); + expect(lines).toHaveLength(2); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); +}); + +describe('json-helper.cjs process-observations', () => { + it('creates new observations', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const responseFile = path.join(tmpDir, 'response.json'); + const logFile = path.join(tmpDir, 'learning.jsonl'); + try { + fs.writeFileSync(responseFile, JSON.stringify({ + observations: [{ + id: 'obs_abc123', type: 'workflow', pattern: 'test pattern', + evidence: ['evidence1'], details: 'test details', + }], + })); + + const result = execSync( + `node "${JSON_HELPER}" process-observations "${responseFile}" "${logFile}"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ).toString().trim(); + const counts = JSON.parse(result); + expect(counts.created).toBe(1); + expect(counts.updated).toBe(0); + + const entry = JSON.parse(fs.readFileSync(logFile, 'utf8').trim()); + expect(entry.id).toBe('obs_abc123'); + expect(entry.confidence).toBe(0.33); + expect(entry.status).toBe('observing'); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); + + it('updates existing observations', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const responseFile = path.join(tmpDir, 'response.json'); + const logFile = path.join(tmpDir, 'learning.jsonl'); + try { + fs.writeFileSync(logFile, JSON.stringify({ + id: 'obs_abc123', type: 'workflow', pattern: 'old pattern', + confidence: 0.33, observations: 1, + first_seen: '2026-03-20T00:00:00Z', last_seen: '2026-03-20T00:00:00Z', + status: 'observing', evidence: ['old evidence'], details: 'old', + }) + '\n'); + + fs.writeFileSync(responseFile, JSON.stringify({ + observations: [{ + id: 'obs_abc123', type: 'workflow', pattern: 'updated pattern', + evidence: ['new evidence'], details: 'updated', + }], + })); + + const result = execSync( + `node "${JSON_HELPER}" process-observations "${responseFile}" "${logFile}"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ).toString().trim(); + const counts = JSON.parse(result); + expect(counts.updated).toBe(1); + + const entry = JSON.parse(fs.readFileSync(logFile, 'utf8').trim()); + expect(entry.observations).toBe(2); + expect(entry.confidence).toBe(0.66); + expect(entry.evidence).toContain('old evidence'); + expect(entry.evidence).toContain('new evidence'); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); + + it('skips observations with missing fields', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const responseFile = path.join(tmpDir, 'response.json'); + const logFile = path.join(tmpDir, 'learning.jsonl'); + try { + fs.writeFileSync(responseFile, JSON.stringify({ + observations: [ + { id: 'obs_abc123', type: 'workflow' }, + ], + })); + + const result = execSync( + `node "${JSON_HELPER}" process-observations "${responseFile}" "${logFile}"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ).toString().trim(); + const counts = JSON.parse(result); + expect(counts.skipped).toBe(1); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); + + it('skips observations with invalid type', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const responseFile = path.join(tmpDir, 'response.json'); + const logFile = path.join(tmpDir, 'learning.jsonl'); + try { + fs.writeFileSync(responseFile, JSON.stringify({ + observations: [ + { id: 'obs_abc123', type: 'invalid', pattern: 'test' }, + ], + })); + + const result = execSync( + `node "${JSON_HELPER}" process-observations "${responseFile}" "${logFile}"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ).toString().trim(); + const counts = JSON.parse(result); + expect(counts.skipped).toBe(1); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); + + it('skips observations with invalid id format', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const responseFile = path.join(tmpDir, 'response.json'); + const logFile = path.join(tmpDir, 'learning.jsonl'); + try { + fs.writeFileSync(responseFile, JSON.stringify({ + observations: [ + { id: 'bad_id', type: 'workflow', pattern: 'test' }, + ], + })); + + const result = execSync( + `node "${JSON_HELPER}" process-observations "${responseFile}" "${logFile}"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ).toString().trim(); + const counts = JSON.parse(result); + expect(counts.skipped).toBe(1); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); + + it('calculates confidence correctly from count', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const responseFile = path.join(tmpDir, 'response.json'); + const logFile = path.join(tmpDir, 'learning.jsonl'); + try { + fs.writeFileSync(logFile, JSON.stringify({ + id: 'obs_abc123', type: 'workflow', pattern: 'test', + confidence: 0.66, observations: 2, + first_seen: '2026-03-20T00:00:00Z', last_seen: '2026-03-20T00:00:00Z', + status: 'observing', evidence: [], details: '', + }) + '\n'); + + fs.writeFileSync(responseFile, JSON.stringify({ + observations: [{ id: 'obs_abc123', type: 'workflow', pattern: 'test', evidence: [] }], + })); + + execSync( + `node "${JSON_HELPER}" process-observations "${responseFile}" "${logFile}"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ); + + const entry = JSON.parse(fs.readFileSync(logFile, 'utf8').trim()); + expect(entry.confidence).toBe(0.95); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); + + it('sets ready on temporal spread', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const responseFile = path.join(tmpDir, 'response.json'); + const logFile = path.join(tmpDir, 'learning.jsonl'); + try { + const twoDaysAgo = new Date(Date.now() - 2 * 86400000).toISOString(); + fs.writeFileSync(logFile, JSON.stringify({ + id: 'obs_abc123', type: 'workflow', pattern: 'test', + confidence: 0.66, observations: 2, + first_seen: twoDaysAgo, last_seen: twoDaysAgo, + status: 'observing', evidence: [], details: '', + }) + '\n'); + + fs.writeFileSync(responseFile, JSON.stringify({ + observations: [{ id: 'obs_abc123', type: 'workflow', pattern: 'test', evidence: [] }], + })); + + execSync( + `node "${JSON_HELPER}" process-observations "${responseFile}" "${logFile}"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ); + + const entry = JSON.parse(fs.readFileSync(logFile, 'utf8').trim()); + expect(entry.status).toBe('ready'); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); + + it('preserves created status', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const responseFile = path.join(tmpDir, 'response.json'); + const logFile = path.join(tmpDir, 'learning.jsonl'); + try { + fs.writeFileSync(logFile, JSON.stringify({ + id: 'obs_abc123', type: 'workflow', pattern: 'test', + confidence: 0.95, observations: 3, + first_seen: '2026-03-20T00:00:00Z', last_seen: '2026-03-22T00:00:00Z', + status: 'created', evidence: [], details: '', + artifact_path: '/some/path.md', + }) + '\n'); + + fs.writeFileSync(responseFile, JSON.stringify({ + observations: [{ id: 'obs_abc123', type: 'workflow', pattern: 'test', evidence: ['new'] }], + })); + + execSync( + `node "${JSON_HELPER}" process-observations "${responseFile}" "${logFile}"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ); + + const entry = JSON.parse(fs.readFileSync(logFile, 'utf8').trim()); + expect(entry.status).toBe('created'); + expect(entry.observations).toBe(4); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); + + it('handles missing log file by creating it', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const responseFile = path.join(tmpDir, 'response.json'); + const logFile = path.join(tmpDir, 'learning.jsonl'); + try { + fs.writeFileSync(responseFile, JSON.stringify({ + observations: [{ + id: 'obs_abc123', type: 'workflow', pattern: 'test', evidence: [], + }], + })); + + const result = execSync( + `node "${JSON_HELPER}" process-observations "${responseFile}" "${logFile}"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ).toString().trim(); + const counts = JSON.parse(result); + expect(counts.created).toBe(1); + expect(fs.existsSync(logFile)).toBe(true); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); + + it('returns correct counts for mixed operations', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const responseFile = path.join(tmpDir, 'response.json'); + const logFile = path.join(tmpDir, 'learning.jsonl'); + try { + fs.writeFileSync(logFile, JSON.stringify({ + id: 'obs_exist1', type: 'workflow', pattern: 'existing', + confidence: 0.33, observations: 1, + first_seen: '2026-03-20T00:00:00Z', last_seen: '2026-03-20T00:00:00Z', + status: 'observing', evidence: [], details: '', + }) + '\n'); + + fs.writeFileSync(responseFile, JSON.stringify({ + observations: [ + { id: 'obs_exist1', type: 'workflow', pattern: 'existing', evidence: [] }, + { id: 'obs_new001', type: 'procedural', pattern: 'new pattern', evidence: [] }, + { id: 'bad', type: 'workflow', pattern: 'test' }, + ], + })); + + const result = execSync( + `node "${JSON_HELPER}" process-observations "${responseFile}" "${logFile}"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ).toString().trim(); + const counts = JSON.parse(result); + expect(counts.updated).toBe(1); + expect(counts.created).toBe(1); + expect(counts.skipped).toBe(1); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); +}); + +describe('json-helper.cjs create-artifacts', () => { + it('creates command with correct frontmatter', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const responseFile = path.join(tmpDir, 'response.json'); + const logFile = path.join(tmpDir, 'learning.jsonl'); + try { + fs.writeFileSync(logFile, JSON.stringify({ + id: 'obs_abc123', type: 'workflow', pattern: 'deploy flow', + confidence: 0.95, observations: 3, status: 'ready', + first_seen: '2026-03-20T00:00:00Z', last_seen: '2026-03-22T00:00:00Z', + evidence: [], details: '', + }) + '\n'); + + fs.writeFileSync(responseFile, JSON.stringify({ + artifacts: [{ + observation_id: 'obs_abc123', type: 'command', + name: 'deploy-flow', description: 'Deploy workflow', + content: '# Deploy Flow\nDeploy the app.', + }], + })); + + const result = execSync( + `node "${JSON_HELPER}" create-artifacts "${responseFile}" "${logFile}" "${tmpDir}"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ).toString().trim(); + const counts = JSON.parse(result); + expect(counts.created).toHaveLength(1); + + const artPath = path.join(tmpDir, '.claude', 'commands', 'self-learning', 'deploy-flow.md'); + expect(fs.existsSync(artPath)).toBe(true); + const content = fs.readFileSync(artPath, 'utf8'); + expect(content).toContain('description: "Deploy workflow"'); + expect(content).toContain('devflow-learning: auto-generated'); + expect(content).toContain('# Deploy Flow'); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); + + it('creates skill with correct frontmatter', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const responseFile = path.join(tmpDir, 'response.json'); + const logFile = path.join(tmpDir, 'learning.jsonl'); + try { + fs.writeFileSync(logFile, JSON.stringify({ + id: 'obs_abc123', type: 'procedural', pattern: 'debug hooks', + confidence: 0.95, observations: 3, status: 'ready', + first_seen: '2026-03-20T00:00:00Z', last_seen: '2026-03-22T00:00:00Z', + evidence: [], details: '', + }) + '\n'); + + fs.writeFileSync(responseFile, JSON.stringify({ + artifacts: [{ + observation_id: 'obs_abc123', type: 'skill', + name: 'debug-hooks', description: 'Debug hook issues', + content: '# Debug Hooks\nHow to debug hooks.', + }], + })); + + const result = execSync( + `node "${JSON_HELPER}" create-artifacts "${responseFile}" "${logFile}" "${tmpDir}"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ).toString().trim(); + const counts = JSON.parse(result); + expect(counts.created).toHaveLength(1); + + const artPath = path.join(tmpDir, '.claude', 'skills', 'debug-hooks', 'SKILL.md'); + expect(fs.existsSync(artPath)).toBe(true); + const content = fs.readFileSync(artPath, 'utf8'); + expect(content).toContain('name: self-learning:debug-hooks'); + expect(content).toContain('user-invocable: false'); + expect(content).toContain('allowed-tools: Read, Grep, Glob'); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); + + it('skips non-ready observations', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const responseFile = path.join(tmpDir, 'response.json'); + const logFile = path.join(tmpDir, 'learning.jsonl'); + try { + fs.writeFileSync(logFile, JSON.stringify({ + id: 'obs_abc123', type: 'workflow', pattern: 'test', + confidence: 0.33, observations: 1, status: 'observing', + evidence: [], details: '', + }) + '\n'); + + fs.writeFileSync(responseFile, JSON.stringify({ + artifacts: [{ + observation_id: 'obs_abc123', type: 'command', + name: 'test-cmd', description: 'Test', content: 'Test', + }], + })); + + const result = execSync( + `node "${JSON_HELPER}" create-artifacts "${responseFile}" "${logFile}" "${tmpDir}"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ).toString().trim(); + const counts = JSON.parse(result); + expect(counts.skipped).toBe(1); + expect(counts.created).toHaveLength(0); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); + + it('skips empty or invalid names', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const responseFile = path.join(tmpDir, 'response.json'); + const logFile = path.join(tmpDir, 'learning.jsonl'); + try { + fs.writeFileSync(logFile, JSON.stringify({ + id: 'obs_abc123', type: 'workflow', pattern: 'test', + confidence: 0.95, observations: 3, status: 'ready', + evidence: [], details: '', + }) + '\n'); + + fs.writeFileSync(responseFile, JSON.stringify({ + artifacts: [{ + observation_id: 'obs_abc123', type: 'command', + name: '!!!', description: 'Test', content: 'Test', + }], + })); + + const result = execSync( + `node "${JSON_HELPER}" create-artifacts "${responseFile}" "${logFile}" "${tmpDir}"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ).toString().trim(); + const counts = JSON.parse(result); + expect(counts.skipped).toBe(1); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); + + it('never overwrites existing files', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const responseFile = path.join(tmpDir, 'response.json'); + const logFile = path.join(tmpDir, 'learning.jsonl'); + try { + fs.writeFileSync(logFile, JSON.stringify({ + id: 'obs_abc123', type: 'workflow', pattern: 'test', + confidence: 0.95, observations: 3, status: 'ready', + evidence: [], details: '', + }) + '\n'); + + const artDir = path.join(tmpDir, '.claude', 'commands', 'self-learning'); + fs.mkdirSync(artDir, { recursive: true }); + fs.writeFileSync(path.join(artDir, 'existing.md'), 'USER CONTENT'); + + fs.writeFileSync(responseFile, JSON.stringify({ + artifacts: [{ + observation_id: 'obs_abc123', type: 'command', + name: 'existing', description: 'Test', content: 'New content', + }], + })); + + const result = execSync( + `node "${JSON_HELPER}" create-artifacts "${responseFile}" "${logFile}" "${tmpDir}"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ).toString().trim(); + const counts = JSON.parse(result); + expect(counts.skipped).toBe(1); + + const content = fs.readFileSync(path.join(artDir, 'existing.md'), 'utf8'); + expect(content).toBe('USER CONTENT'); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); + + it('sanitizes artifact name', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const responseFile = path.join(tmpDir, 'response.json'); + const logFile = path.join(tmpDir, 'learning.jsonl'); + try { + fs.writeFileSync(logFile, JSON.stringify({ + id: 'obs_abc123', type: 'workflow', pattern: 'test', + confidence: 0.95, observations: 3, status: 'ready', + evidence: [], details: '', + }) + '\n'); + + fs.writeFileSync(responseFile, JSON.stringify({ + artifacts: [{ + observation_id: 'obs_abc123', type: 'command', + name: 'My Cool_Command!@#', description: 'Test', content: 'Test', + }], + })); + + const result = execSync( + `node "${JSON_HELPER}" create-artifacts "${responseFile}" "${logFile}" "${tmpDir}"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ).toString().trim(); + const counts = JSON.parse(result); + expect(counts.created).toHaveLength(1); + expect(counts.created[0]).toContain('mycoolcommand'); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); + + it('updates observation status in log', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const responseFile = path.join(tmpDir, 'response.json'); + const logFile = path.join(tmpDir, 'learning.jsonl'); + try { + fs.writeFileSync(logFile, JSON.stringify({ + id: 'obs_abc123', type: 'workflow', pattern: 'test', + confidence: 0.95, observations: 3, status: 'ready', + evidence: [], details: '', + }) + '\n'); + + fs.writeFileSync(responseFile, JSON.stringify({ + artifacts: [{ + observation_id: 'obs_abc123', type: 'command', + name: 'test-cmd', description: 'Test', content: 'Test', + }], + })); + + execSync( + `node "${JSON_HELPER}" create-artifacts "${responseFile}" "${logFile}" "${tmpDir}"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ); + + const entry = JSON.parse(fs.readFileSync(logFile, 'utf8').trim()); + expect(entry.status).toBe('created'); + expect(entry.artifact_path).toContain('test-cmd.md'); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); + + it('escapes description quotes', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const responseFile = path.join(tmpDir, 'response.json'); + const logFile = path.join(tmpDir, 'learning.jsonl'); + try { + fs.writeFileSync(logFile, JSON.stringify({ + id: 'obs_abc123', type: 'workflow', pattern: 'test', + confidence: 0.95, observations: 3, status: 'ready', + evidence: [], details: '', + }) + '\n'); + + fs.writeFileSync(responseFile, JSON.stringify({ + artifacts: [{ + observation_id: 'obs_abc123', type: 'command', + name: 'test-cmd', description: 'A "quoted" description', content: 'Test', + }], + })); + + execSync( + `node "${JSON_HELPER}" create-artifacts "${responseFile}" "${logFile}" "${tmpDir}"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ); + + const artPath = path.join(tmpDir, '.claude', 'commands', 'self-learning', 'test-cmd.md'); + const content = fs.readFileSync(artPath, 'utf8'); + expect(content).toContain('description: "A \\"quoted\\" description"'); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); + + it('returns created paths', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const responseFile = path.join(tmpDir, 'response.json'); + const logFile = path.join(tmpDir, 'learning.jsonl'); + try { + fs.writeFileSync(logFile, [ + JSON.stringify({ + id: 'obs_abc123', type: 'workflow', pattern: 'test1', + confidence: 0.95, observations: 3, status: 'ready', + evidence: [], details: '', + }), + JSON.stringify({ + id: 'obs_def456', type: 'procedural', pattern: 'test2', + confidence: 0.95, observations: 3, status: 'ready', + evidence: [], details: '', + }), + ].join('\n') + '\n'); + + fs.writeFileSync(responseFile, JSON.stringify({ + artifacts: [ + { observation_id: 'obs_abc123', type: 'command', name: 'cmd1', description: 'Cmd 1', content: 'C1' }, + { observation_id: 'obs_def456', type: 'skill', name: 'skill1', description: 'Skill 1', content: 'S1' }, + ], + })); + + const result = execSync( + `node "${JSON_HELPER}" create-artifacts "${responseFile}" "${logFile}" "${tmpDir}"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ).toString().trim(); + const counts = JSON.parse(result); + expect(counts.created).toHaveLength(2); + expect(counts.created[0]).toContain('cmd1.md'); + expect(counts.created[1]).toContain('SKILL.md'); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); +}); + +describe('json-helper.cjs filter-observations', () => { + it('returns valid entries as sorted array', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const file = path.join(tmpDir, 'learning.jsonl'); + try { + fs.writeFileSync(file, [ + JSON.stringify({ id: 'obs_a', type: 'workflow', pattern: 'p1', confidence: 0.3 }), + JSON.stringify({ id: 'obs_b', type: 'procedural', pattern: 'p2', confidence: 0.9 }), + JSON.stringify({ id: 'obs_c', type: 'workflow', pattern: 'p3', confidence: 0.5 }), + ].join('\n') + '\n'); + + const result = execSync( + `node "${JSON_HELPER}" filter-observations "${file}" confidence 2`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ).toString().trim(); + const parsed = JSON.parse(result); + expect(parsed).toHaveLength(2); + expect(parsed[0].id).toBe('obs_b'); + expect(parsed[1].id).toBe('obs_c'); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); + + it('filters out malformed entries', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'devflow-test-')); + const file = path.join(tmpDir, 'learning.jsonl'); + try { + fs.writeFileSync(file, [ + JSON.stringify({ id: 'obs_valid', type: 'workflow', pattern: 'valid', confidence: 0.5 }), + JSON.stringify({ id: 'bad_id', type: 'workflow', pattern: 'bad id' }), + JSON.stringify({ id: 'obs_notype', pattern: 'no type' }), + JSON.stringify({ id: 'obs_nopattern', type: 'workflow' }), + 'not json at all', + ].join('\n') + '\n'); + + const result = execSync( + `node "${JSON_HELPER}" filter-observations "${file}"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ).toString().trim(); + const parsed = JSON.parse(result); + expect(parsed).toHaveLength(1); + expect(parsed[0].id).toBe('obs_valid'); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }); + + it('returns empty array for missing file', () => { + const result = execSync( + `node "${JSON_HELPER}" filter-observations "/tmp/nonexistent-devflow-test-${Date.now()}.jsonl"`, + { stdio: ['pipe', 'pipe', 'pipe'] }, + ).toString().trim(); + expect(result).toBe('[]'); + }); +}); + +describe('session-end-learning structure', () => { + it('is included in bash -n syntax checks', () => { + expect(HOOK_SCRIPTS).toContain('session-end-learning'); + }); + + it('starts with bash shebang and sources json-parse', () => { + const scriptPath = path.join(HOOKS_DIR, 'session-end-learning'); + const content = fs.readFileSync(scriptPath, 'utf8'); + const lines = content.split('\n'); + expect(lines[0]).toBe('#!/bin/bash'); + expect(content).toContain('source "$SCRIPT_DIR/json-parse"'); + }); }); describe('json-parse wrapper', () => {