Skip to content

feat(rag,kg): time-decay recency scoring + wire contradiction invalidation (Wave 4)#43

Closed
urmzd wants to merge 1 commit into
mainfrom
wave-4-temporal-decay
Closed

feat(rag,kg): time-decay recency scoring + wire contradiction invalidation (Wave 4)#43
urmzd wants to merge 1 commit into
mainfrom
wave-4-temporal-decay

Conversation

@urmzd

@urmzd urmzd commented May 31, 2026

Copy link
Copy Markdown
Owner

Wave 4 — Temporal differentiators

Bounded slice landed; green on go build ./..., go vet ./rag/... ./knowledge/..., and go test for every changed package.

What landed

(a) RAG time-decay / recency scoring

  • New types.WithRecency(halfLife time.Duration, weight float64) SearchOption.
  • After RRF fusion in rag/internal/pipeline, each hit's fused score is blended with an exponential recency factor exp(-ln2 * age / halfLife):
    score = (1-weight)*score + weight*score*recencyFactor.
  • Opt-in: a non-positive half-life is a no-op, so the default ranking is unchanged (backward compatible).
  • types.SearchHit gains a Timestamp time.Time field, populated from the document's UpdatedAt (fallback CreatedAt) by both rag/memstore and rag/pgstore during search. A zero timestamp means "unknown age" → no decay (factor 1.0).
  • Tests:
    • rag/memstore: TestSearchByEmbeddingPopulatesTimestamp (UpdatedAt preferred, CreatedAt fallback).
    • rag/internal/pipeline: TestPipelineRecencyTieBreak (two equally-relevant docs of different ages indexed in memstore; the recent doc outranks the older one only with WithRecency) and TestPipelineRecencyDisabledByDefault.

(b) KG contradiction invalidation

  • Wired the previously-callerless Store.InvalidateRelation into the engine ingestion path.
  • After creating a relation for an existing (source, target, type), invalidateSupersededRelations finds active prior relation(s) of the same type and calls InvalidateRelation(old, newValidAt), setting the old edge's InvalidAt.
  • Rule-based supersession only (same source+target+type supersedes); no LLM judge. The newly created relation and already-invalidated priors are left untouched.
  • Tests in knowledge/internal/engine:
    • TestIngestEpisode_SupersededRelationInvalidated (prior invalidated exactly once at the new relation's ValidAt; new relation not invalidated).
    • TestIngestEpisode_DifferentTypePriorNotInvalidated.
    • TestIngestEpisode_AlreadyInvalidatedPriorSkipped.

Deferred (TODO)

  • True transaction-time bitemporal versioning + WithAsOf(validTime, txTime) read path.
  • LLM-judged supersession (semantic contradiction detection beyond same source+target+type).
  • Recency blending in the KG RRF (SearchFacts) path.

Notes

  • gofmt -w . reports pre-existing formatting drift across many unrelated files on the foundation branch; this PR deliberately commits only the 8 files it changed (all gofmt-clean) to keep the diff focused.

Stacked on #38 (foundation) — merge after #38.

@urmzd urmzd force-pushed the wave-4-temporal-decay branch from 839e059 to 6e7dca3 Compare May 31, 2026 20:49
…ation

RAG: add WithRecency(halfLife, weight) SearchOption that blends an
exponential time-decay factor exp(-ln2*age/halfLife) into each hit's
fused score after RRF. Opt-in (non-positive half-life is a no-op).
SearchHit gains a Timestamp populated from the document UpdatedAt
(fallback CreatedAt) in memstore and pgstore.

KG: wire the previously-unused Store.InvalidateRelation into the engine
ingestion path. A new relation for an existing (source,target,type)
rule-based supersedes active prior relation(s) of the same type, setting
their InvalidAt to the new relation's ValidAt.
@urmzd

urmzd commented May 31, 2026

Copy link
Copy Markdown
Owner Author

Superseded by #44, which was squash-merged into main (53a6aff) and contains every change from this branch — a dry merge of this branch into main is a no-op. main is green (build/vet/golangci-lint, 39 packages, and 8/8 live validation on gpt-4o-mini). Closing as redundant.

@urmzd urmzd closed this May 31, 2026
@urmzd urmzd deleted the wave-4-temporal-decay branch May 31, 2026 21:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant