docs: clarify 0.8 documentation direction

AGENTS.md

@@ -98,6 +98,30 @@ Do not change specification documents unless explicitly instructed.
 
 If implementation behavior does not match the specification, report the mismatch instead of modifying the specification.
 
+Documentation is not commentary.
+
+README examples, demos, integration examples, migration guides, CLI usage
+documentation, and explicitly requested documentation changes are part of the
+project contract.
+
+Treat documentation requirements in a task as acceptance criteria.
+
+Do not treat documentation as illustrative unless explicitly stated.
+Do not silently change documented behavior because implementation is easier.
+Do not update documentation merely to match unintended behavior.
+Do not weaken or remove user-facing tests to accommodate implementation.
+
+Documentation examples explicitly referenced by a task are part of the
+expected deliverable.
+
+If implementation, documentation, examples, tests, fixtures, and
+specifications disagree:
+
+1. Specifications and fixtures are authoritative.
+2. Report the mismatch.
+3. Request review before changing documented behavior.
+4. Do not resolve disagreements by silently changing docs.
+
 ## Documentation style
 
 For README, demo, integration, and package-listing docs, explain user-visible behavior before architecture.

CONTRIBUTING.md

@@ -35,6 +35,35 @@ pre-commit run --all-files
 - include tests if behavior changes
 - open an issue first for large design changes
 
+## Architectural Boundaries
+
+The flat policy model is intentional.
+
+Policies in core compiler state are modeled as independent flat assertions.
+They are not designed to carry relationship semantics such as:
+
+- policy interaction
+- policy grouping
+- policy inheritance
+- synonym handling
+- antonym handling
+- dependency modeling
+- hierarchy modeling
+- ontology-style reasoning
+
+Before proposing changes in those areas, understand that this boundary is part
+of the core architecture rather than a missing convenience layer.
+
+Relationship semantics are generally expected to live in:
+
+- drafting layers
+- orchestration layers
+- composition layers
+- domain-specific packages
+
+Changes to policy independence should be treated as architectural proposals,
+not routine feature requests.
+
 ## Documentation Style
 
 For README, demo, integration, and package-listing docs, explain user-visible behavior before architecture.

README.md

@@ -52,7 +52,7 @@ Like a compiler, it parses input, validates it, applies fixed rules, and
 produces a stable representation the host can use. It is not source-code
 compilation and not a reasoning model.
 
-## Does it work?
+## Results
 
 Yes, on the current scored demo set.
 
@@ -145,23 +145,51 @@ Your app decides whether to call the model based on the returned `Decision`.
 
 ## Quickstart
 
+Use Context Compiler in your host application first:
+
+```python
+from context_compiler import (
+    create_engine,
+    get_clarify_prompt,
+    is_clarify,
+    is_update,
+)
+
+engine = create_engine()
+
+user_input = "set premise current project uses uv"
+decision = engine.step(user_input)
+
+if is_clarify(decision):
+    show_to_user(get_clarify_prompt(decision))
+elif is_update(decision):
+    messages = build_messages(engine.state, user_input)
+    render(call_llm(messages))
+else:
+    render(call_llm(user_input))
+```
+
+This is the primary integration path: your app owns the model call, and the
+compiler owns deterministic state transitions.
+
+## Interactive Playground
+
+Use the REPL to explore behavior, learn the directive grammar, and debug or
+test host-side state handling.
+
 ```bash
 pip install context-compiler
 context-compiler
 context-compiler --with-preprocessor
-context-compiler --json < input.txt
 ```
 
 `context-compiler` launches the interactive REPL.
 
-`--with-preprocessor` enables the experimental preprocessor before each REPL turn
+`--with-preprocessor` enables the experimental preprocessor before each input
 (simple rule-based checks plus conservative validation). For near-miss inputs,
 the preprocessor does not rewrite the text. It passes the input to the engine,
 and the engine can return `clarify`.
 
-`--json` enables machine-readable NDJSON output for non-interactive usage
-(one complete JSON object per processed input line).
-
 Preload options keep saved rules separate from in-progress confirmation state:
 - `--initial-state-json` / `--initial-state-file` load saved state
   (via exported state JSON).
@@ -175,29 +203,27 @@ REPL commands (controller layer, not engine directives):
 
 Bare REPL input behavior remains unchanged.
 
-Or in code:
-```python
-from context_compiler import (
-    create_engine,
-    get_clarify_prompt,
-    is_clarify,
-    is_update,
-)
+## Machine-Readable CLI Usage
 
-engine = create_engine()
+Use `--json` when you want one complete JSON object per processed input line
+for non-interactive usage.
 
-user_input = "set premise current project uses uv"
-decision = engine.step(user_input)
-
-if is_clarify(decision):
-    show_to_user(get_clarify_prompt(decision))
-elif is_update(decision):
-    messages = build_messages(engine.state, user_input)
-    render(call_llm(messages))
-else:
-    render(call_llm(user_input))
+```bash
+context-compiler --json < input.txt
 ```
 
+`--json` enables machine-readable NDJSON output for non-interactive usage
+(one complete JSON object per processed input line).
+
+You can combine `--with-preprocessor` with `--json` when you want the same
+preprocessing path in non-interactive usage.
+
+Preload options keep saved rules separate from in-progress confirmation state:
+- `--initial-state-json` / `--initial-state-file` load saved state
+  (via exported state JSON).
+- `--initial-checkpoint-json` / `--initial-checkpoint-file` restore full
+  continuation checkpoint (saved state + pending confirmation state).
+
 ## Installation
 
 Requirements:

docs/DescriptionAndMilestones.md

@@ -26,6 +26,8 @@ The state engine is the source of truth and is model-independent.
 Model output is never interpreted to derive or modify state.
 All state transitions originate from explicit user directives.
 
+Context Compiler remains deterministic conversational state authority.
+
 Behavioral details are defined in `docs/DirectiveGrammarSpec.md`.
 
 ## Project Milestones
@@ -127,12 +129,109 @@ Make engine behavior inspectable and externally controllable without guessing.
 - No heuristic-heavy parsing
 - Preserve separation between engine, preprocessor, and host/controller layers
 
-### Post-0.7 Direction
+### 0.8 — Architectural Decomposition (approved direction)
+
+The approved 0.8 direction is to document and preserve the architectural
+decomposition that has emerged around the current engine and experimental
+preprocessor work. Behavior is expected to be preserved initially rather than
+redesigned.
+
+#### Authority Layer
+
+Context Compiler core remains the Authority Layer and continues to serve as
+deterministic conversational state authority.
+
+Responsibilities:
+
+- deterministic state transitions
+- contradiction handling
+- clarification handling
+- pending confirmation flows
+- checkpoints
+- preview/diff
+- authoritative state ownership
+
+The Authority Layer remains the only layer that applies directives and the only
+layer that mutates authoritative state.
+
+#### Acquisition Layer
+
+The current experimental preprocessor direction belongs to the Acquisition
+Layer.
+
+Planned 0.8 extraction target:
+`context-compiler-directive-drafter`
+
+Responsibilities:
+
+- directive-adjacent interpretation
+- conservative natural-language-to-directive drafting
+- candidate directive generation
+- possible future suggestion/proposal workflows outside the initial 0.8 extraction scope
+
+Important boundary:
+
+- drafts are non-authoritative
+- drafting proposes
+- authority decides
+- only core applies directives
+- only core mutates authoritative state
+
+The Directive Drafter may draft candidate directive strings.
+
+It must not:
+
+- mutate authoritative state
+- bypass `engine.step(...)`
+- edit `engine.state`
+- become a second authority
+
+0.8 direction:
+
+> Extraction of the experimental preprocessor into a separate package:
+>
+> `context-compiler-directive-drafter`
+
+The initial goal is extraction with preserved behavior, not API redesign.
+
+#### Composition Layer
+
+The Composition Layer is a future possibility only and is not planned for 0.8.
+
+Examples:
+
+- multiple authorities
+- profiles
+- domain-specific authorities
+- provenance
+- conflict resolution
+- effective-state computation
+
+### 0.9 Candidate Direction — Canonical Export Integrity / Hashing
+
+This is future planning only. No 0.9 implementation is defined here.
+
+Candidate goals:
+
+- canonical serialization
+- deterministic hashes of exported artifacts
+- Python/TypeScript verification
+- auditability
+- future signing compatibility
+
+Explicitly out of scope:
+
+- signing
+- key management
+- trust infrastructure
+- security guarantees from hashes alone
+- hashes embedded inside semantic engine state
+
+### Post-0.8 Direction
 
-- 0.8 candidate direction: model-assisted state suggestions (inspectable, previewable,
-  and never directly mutating authoritative state)
-- MCP adapter likely as a separate/later track after 0.8 direction is clearer
-- Optional 0.7.1 MCP-readiness helpers only if narrowly justified
+- 0.9 candidate direction: canonical export integrity and hashing
+- MCP adapter likely as a separate/later track after 0.8 extraction direction is clearer
+- Optional MCP-readiness helpers only if narrowly justified
 - Additional tooling built on auditability surfaces
 
 ### 1.0 Target

docs/architecture.md

@@ -9,15 +9,18 @@ Responsibilities:
 - recognize possible user state updates before core compilation
 - normalize candidate inputs conservatively
 - abstain when intent is uncertain
+- draft candidate directives without becoming a second authority
 
 Examples:
-- the optional heuristic/LLM preprocessor
+- the optional heuristic/LLM preprocessor, including a future extracted directive drafter package
 - host-side input shaping before `engine.step(...)`
 
 Out of scope:
 - authoritative state mutation
 - final conflict resolution
 - semantic classification as source of truth
+- bypassing `engine.step(...)`
+- editing `engine.state`
 
 ## Authority Layer
 
@@ -35,6 +38,7 @@ Out of scope:
 - prompt rendering
 - tool selection
 - moderation or policy classification
+- non-authoritative drafting or proposal UX
 
 ## Application Layer
 
@@ -84,3 +88,52 @@ Out of scope:
 The preprocessor belongs to the Acquisition Layer. It is optional,
 conservative, and never the source of truth. Context Compiler core belongs to
 the Authority Layer. Host applications own Application Layer behavior.
+
+Composition remains exploratory. It is a future possibility, not a planned 0.8
+core change.
+
+## Architectural Rationale: Flat Policy Independence
+
+Policies are intentionally modeled as independent flat assertions.
+
+The model intentionally does not include:
+
+- ordering
+- grouping
+- precedence
+- inheritance
+- synonym relationships
+- antonym relationships
+- dependencies
+- hierarchy
+- ontology-style reasoning
+- interaction semantics
+
+This is a deliberate architectural boundary, not an omitted convenience
+feature.
+
+Policy independence is a major contributor to:
+
+- determinism
+- portability
+- replay consistency
+- checkpoint stability
+- cross-language conformance
+
+Because policies are independent flat assertions, directive semantics stay
+simple, exported state stays portable, replay remains exact, and checkpoint
+continuation does not depend on hidden relational logic.
+
+Changing this model would be a major architectural change with consequences
+for:
+
+- directive semantics
+- fixtures
+- replay
+- checkpoints
+- language parity
+- future composition systems
+
+Relationship-heavy semantics may still be useful, but they generally belong in
+drafting, orchestration, composition, or domain-specific layers rather than in
+the core authority model.