Skip to content

docs: Pluggable predicate interface design doc #5

@dpup

Description

@dpup

Summary

Write a design doc for opt-in, non-deterministic predicates — the extension point for cases where CEL + regex aren't enough (PHI in unstructured text, intent classification, content moderation). This is a design doc, not implementation.

Context

The engine is deliberately deterministic, bounded-time, and stateless. rateCount() is the sole exception (local counters). But the PRD identifies real cases where operators would accept the latency/cost tradeoff:

  • PHI detection in unstructured clinical text — MRNs look like numbers, diagnoses need medical context, patient names are just names
  • Intent classification — "is this agent trying to exfiltrate data?" can't be answered with regex
  • Nuanced content moderation — beyond keyword matching

Design questions to address

  1. Interface shape — What does llmJudge(field, prompt) (or equivalent) look like as a CEL function? What does it return (bool, score, structured result)?
  2. Per-scope opt-in — How does a scope declare it uses async predicates? What's the config surface?
  3. Timeout and fallback — What happens when the LLM call times out? Fail-open or fail-closed per predicate?
  4. Audit semantics — How are non-deterministic evaluations logged? Should the audit entry flag that the decision involved an LLM call?
  5. Latency budget — The engine targets <10ms p99. An LLM call is 500ms-2s. How does this compose? Parallel eval? Async with callback?
  6. DLP middle ground — Google DLP / AWS Macie as an alternative to LLM calls. Same interface? Separate predicate?
  7. Provider abstraction — Should the predicate be model-agnostic or provider-specific?

Deliverable

A design doc in docs/plans/ covering the interface, opt-in model, failure semantics, and audit impact. No implementation.

References

  • PRD open question #8 (AI-assisted policy evaluation)
  • PRD open question feat: Local HTTP eval server for non-Go callers #6 (PHI detection feasibility)
  • VISION.md: "Evaluation is deterministic" — this extension must be explicitly opt-in and clearly marked

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions