feat: learning system Wave 2 quality improvements

CHANGELOG.md

@@ -12,8 +12,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **Learning**: `devflow learn --purge` command to remove invalid entries from learning log
 - **Learning**: debug logging mode (`devflow learn --configure`) — logs to `~/.devflow/logs/`
 
+### Changed
+- **Learning**: Moved from Stop → SessionEnd hook with 3-session batching (adaptive: 5 at 15+ observations)
+- **Learning**: Raised procedural thresholds from 2 to 3 observations with 24h+ temporal spread for both types
+- **Learning**: Reduced default `max_daily_runs` from 10 to 5
+- **Learning**: Renamed artifact paths: `commands/learned/` → `commands/self-learning/`, `skills/learned-{name}/` → `skills/{name}/`
+- **Learning**: Skill artifacts now include `user-invocable: false`, Iron Law section, and `self-learning:` name prefix
+
 ### Fixed
 - **Learning**: reject observations with empty id/type/pattern fields (validation + auto-purge on migration)
+- **Learning**: Handle string-typed `.message.content` in transcript extraction (was only handling arrays)
+- **Learning**: Eliminate empty-array loop noise when Sonnet returns no observations
+- **Learning**: Race condition in batch file handoff (atomic `mv` replaces `cp`+`rm`)
+- **Learning**: `--enable` now auto-upgrades legacy Stop hook to SessionEnd
+- **Learning**: `--status` detects legacy hook and shows upgrade instructions
 
 ---
 

CLAUDE.md

@@ -40,7 +40,7 @@ Commands with Teams Variant ship as `{name}.md` (parallel subagents) and `{name}
 
 **Working Memory**: Three shell-script hooks (`scripts/hooks/`) provide automatic session continuity. Toggleable via `devflow memory --enable/--disable/--status` or `devflow init --memory/--no-memory`. Stop hook → reads last turn from session transcript (`~/.claude/projects/{encoded-cwd}/{session_id}.jsonl`), spawns background `claude -p --model haiku` to update `.memory/WORKING-MEMORY.md` with structured sections (`## Now`, `## Progress`, `## Decisions`, `## Modified Files`, `## Context`, `## Session Log`; throttled: skips if triggered <2min ago; concurrent sessions serialize via mkdir-based lock). SessionStart hook → injects previous memory + git state as `additionalContext` on `/clear`, startup, or compact (warns if >1h stale; injects pre-compact memory snapshot when compaction happened mid-session). PreCompact hook → saves git state + WORKING-MEMORY.md snapshot + bootstraps minimal WORKING-MEMORY.md if none exists. Zero-ceremony context preservation.
 
-**Self-Learning**: A Stop hook (`stop-update-learning`) spawns a background `claude -p --model sonnet` to detect repeated workflows and procedural knowledge from session transcripts. Observations accumulate in `.memory/learning-log.jsonl` with confidence scores, temporal decay, and daily run caps. When confidence thresholds are met (3 observations for workflows with 24h+ temporal spread, 2 for procedural), artifacts are auto-created as slash commands (`.claude/commands/learned/`) or skills (`.claude/skills/learned-*/`). Toggleable via `devflow learn --enable/--disable/--status` or `devflow init --learn/--no-learn`. Configurable model/throttle/caps/debug via `devflow learn --configure`. Debug logs stored at `~/.devflow/logs/{project-slug}/`. Use `devflow learn --purge` to remove invalid observations.
+**Self-Learning**: A SessionEnd hook (`session-end-learning`) accumulates session IDs and triggers a background `claude -p --model sonnet` every 3 sessions (5 at 15+ observations) to detect repeated workflows and procedural knowledge from batch transcripts. Observations accumulate in `.memory/learning-log.jsonl` with confidence scores, temporal decay, and daily run caps. When confidence thresholds are met (3 observations with 24h+ temporal spread for both workflow and procedural types), artifacts are auto-created as slash commands (`.claude/commands/self-learning/`) or skills (`.claude/skills/{slug}/`). Loaded artifacts are reinforced locally (no LLM) on each session end. Toggleable via `devflow learn --enable/--disable/--status` or `devflow init --learn/--no-learn`. Configurable model/throttle/caps/debug via `devflow learn --configure`. Debug logs stored at `~/.devflow/logs/{project-slug}/`. Use `devflow learn --purge` to remove invalid observations.
 
 ## Project Structure
 
@@ -51,7 +51,7 @@ devflow/
 ├── plugins/devflow-*/      # 17 plugins (8 core + 9 optional language/ecosystem)
 ├── docs/reference/         # Detailed reference documentation
 ├── scripts/                # Helper scripts (statusline, docs-helpers)
-│   └── hooks/              # Working Memory + ambient + learning hooks (stop, session-start, pre-compact, ambient-prompt, stop-update-learning, background-learning)
+│   └── hooks/              # Working Memory + ambient + learning hooks (stop, session-start, pre-compact, ambient-prompt, session-end-learning, stop-update-learning [deprecated], background-learning)
 ├── src/cli/                # TypeScript CLI (init, list, uninstall, ambient, learn)
 ├── .claude-plugin/         # Marketplace registry
 ├── .docs/                  # Project docs (reviews, design) — per-project
@@ -99,7 +99,8 @@ Working memory files live in a dedicated `.memory/` directory:
 ├── learning-log.jsonl        # Learning observations (JSONL, one entry per line)
 ├── learning.json             # Project-level learning config (max runs, throttle, model, debug)
 ├── .learning-runs-today      # Daily run counter (date + count)
-├── .learning-last-trigger    # Throttle marker (epoch timestamp)
+├── .learning-session-count   # Session IDs pending batch (one per line)
+├── .learning-batch-ids       # Session IDs for current batch run
 ├── .learning-notified-at     # New artifact notification marker (epoch timestamp)
 └── knowledge/
     ├── decisions.md           # Architectural decisions (ADR-NNN, append-only)

README.md

@@ -1,4 +1,4 @@
-# DevFlow: The Most Advanced Agentic Development Toolkit
+# DevFlow: The Most Advanced Agentic Development Toolkit for Production-Grade Code
 
 [![npm version](https://img.shields.io/npm/v/devflow-kit)](https://www.npmjs.com/package/devflow-kit)
 [![CI](https://github.com/dean0x/devflow/actions/workflows/ci.yml/badge.svg)](https://github.com/dean0x/devflow/actions/workflows/ci.yml)
@@ -36,6 +36,7 @@ Claude Code is powerful. DevFlow makes it extraordinary.
 - **Parallel debugging** with competing hypotheses investigated simultaneously
 - **35 quality skills** with 9 auto-activating core, 8 language/ecosystem, plus specialized review and orchestration skills
 - **Ambient mode** that classifies intent and loads proportional skill sets automatically
+- **Fully composable plugin system** where every feature is a plugin. Install only what you need. No bloat, no take-it-or-leave-it bundles.
 
 ## Quick Start
 
@@ -203,14 +204,14 @@ Working memory is **per-project** — scoped to each repo's `.memory/` directory
 
 DevFlow detects repeated workflows and procedural knowledge across your sessions and automatically creates slash commands and skills.
 
-A background agent runs on session stop (same as Working Memory) and analyzes your session transcript for patterns. When a pattern is observed enough times (3 for workflows with 24h+ temporal spread, 2 for procedural knowledge), it creates an artifact:
+A background agent runs on session end, batching every 3 sessions (5 at 15+ observations) to analyze transcripts for patterns. When a pattern is observed enough times (3 observations with 24h+ temporal spread for both types), it creates an artifact:
 
-- **Workflow patterns** become slash commands at `.claude/commands/learned/`
-- **Procedural patterns** become skills at `.claude/skills/learned-*/`
+- **Workflow patterns** become slash commands at `.claude/commands/self-learning/`
+- **Procedural patterns** become skills at `.claude/skills/{slug}/`
 
 | Command | Description |
 |---------|-------------|
-| `devflow learn --enable` | Register the learning Stop hook |
+| `devflow learn --enable` | Register the learning SessionEnd hook |
 | `devflow learn --disable` | Remove the learning hook |
 | `devflow learn --status` | Show learning status and observation counts |
 | `devflow learn --list` | Show all observations sorted by confidence |
@@ -235,7 +236,8 @@ DevFlow creates project documentation in `.docs/` and working memory in `.memory
 ├── learning-log.jsonl        # Learning observations (JSONL)
 ├── learning.json             # Project-level learning config
 ├── .learning-runs-today      # Daily run counter
-├── .learning-last-trigger    # Throttle marker
+├── .learning-session-count   # Session IDs pending batch (one per line)
+├── .learning-batch-ids       # Session IDs for current batch run
 ├── .learning-notified-at     # New artifact notification marker
 └── knowledge/
     ├── decisions.md           # Architectural decisions (ADR-NNN, append-only)
@@ -339,8 +341,8 @@ npx devflow-kit skills unshadow core-patterns
 
 | Tool | Role | What It Does |
 |------|------|-------------|
-| **[Skim](https://github.com/dean0x/skim)** | Context Optimization | Compresses code, test output, build output, and git output for optimal LLM reasoning |
-| **DevFlow** | Quality Orchestration | 18 parallel reviewers, working memory, self-learning, production-grade lifecycle workflows |
+| **[Skim](https://github.com/dean0x/skim)** | Context Optimization | Code-aware AST parsing across 12 languages, command rewriting, test/build/git output compression |
+| **DevFlow** | Quality Orchestration | 18 parallel reviewers, working memory, self-learning, composable plugin system |
 | **[Backbeat](https://github.com/dean0x/backbeat)** | Agent Orchestration | Orchestration at scale. Karpathy optimization loops, multi-agent pipelines, DAG dependencies, autoscaling |
 
 Skim optimizes what your AI sees. DevFlow enforces how it works. Backbeat scales everything across agents. No other stack covers all three.

docs/reference/file-organization.md

@@ -47,7 +47,8 @@ devflow/
 │       ├── session-start-memory     # SessionStart hook: injects memory + git state
 │       ├── pre-compact-memory       # PreCompact hook: saves git state backup
 │       ├── ambient-prompt           # UserPromptSubmit hook: ambient skill injection
-│       ├── stop-update-learning     # Stop hook: triggers background learning
+│       ├── session-end-learning      # SessionEnd hook: batched learning trigger
+│       ├── stop-update-learning     # Stop hook: deprecated stub (upgrade via devflow learn)
 │       ├── background-learning      # Background: pattern detection via Sonnet
 │       ├── json-helper.cjs           # Node.js jq-equivalent operations
 │       └── json-parse               # Shell wrapper: jq with node fallback
@@ -154,7 +155,7 @@ Included settings:
 
 Three hooks in `scripts/hooks/` provide automatic session continuity. Toggleable via `devflow memory --enable/--disable/--status` or `devflow init --memory/--no-memory`.
 
-A fourth hook (`stop-update-learning`) provides self-learning. Toggleable via `devflow learn --enable/--disable/--status` or `devflow init --learn/--no-learn`:
+A fourth hook (`session-end-learning`) provides self-learning. Toggleable via `devflow learn --enable/--disable/--status` or `devflow init --learn/--no-learn`:
 
 | Hook | Event | File | Purpose |
 |------|-------|------|---------|