@omnifolio/tiered-cache

Two-tier caching with L1 in-memory + L2 Redis, stale-while-revalidate, circuit breaker state, request deduplication, and LRU eviction. Zero dependencies. Built for multi-instance deployments on Cloud Run, Kubernetes, etc.

Built by OmniFolio — Financial Intelligence Platform.

---

The Problem

When you auto-scale to N instances (Cloud Run, K8s, ECS), each instance has its own in-memory cache. Without a shared L2 layer:

• 20 instances × 30 API calls = 600 external calls/minute instead of 30
• API rate limits blown, bills explode, users get errors

The Solution

Request → L1 (in-memory, 0ms) → L2 (Redis, ~2ms) → Origin

First instance fetches from origin and writes to Redis. All other instances read from Redis — zero redundant external calls.

Graceful degradation: No Redis? Falls back to per-instance Map. Redis down? Falls back to per-instance Map. Zero crashes.

Install

npm install @omnifolio/tiered-cache

Quick Start

Shared Cache (L1 + L2)

import { configureL2, cacheGet, cacheSet } from '@omnifolio/tiered-cache';
import { Redis } from '@upstash/redis';

// Configure L2 (optional — works without it in dev)
const redis = new Redis({ url: '...', token: '...' });
configureL2({
  get: (key) => redis.get(key),
  set: (key, value, ttl) => redis.set(key, value, { ex: ttl }),
  del: (key) => redis.del(key).then(() => {}),
  ping: () => redis.ping().then(() => {}),
});

// Use it
const data = await cacheGet<MyData>('api:prices');
if (!data) {
  const fresh = await fetchPrices();
  await cacheSet('api:prices', fresh, 300); // 5min TTL
}

SWR Cache (Stale-While-Revalidate)

import { SWRCache, SWRPresets } from '@omnifolio/tiered-cache';

const cache = new SWRCache(500); // max 500 entries

// Returns stale data instantly, refreshes in background
const prices = await cache.swr(
  'prices:BTC',
  () => fetchBTCPrice(),
  SWRPresets.frequent, // 30s fresh, 5min max
);

// Subscribe to updates
const unsub = cache.subscribe('prices:BTC', (data) => {
  console.log('Price updated:', data);
});

// Optimistic mutation
cache.mutate('prices:BTC', (current) => ({
  ...current,
  price: newPrice,
}));

Circuit Breaker

import { getCircuitBreakerState, setCircuitBreakerState } from '@omnifolio/tiered-cache';

const state = await getCircuitBreakerState('coinbase-api');
if (state.open) {
  return cachedFallback();
}

try {
  const data = await fetchFromCoinbase();
  await setCircuitBreakerState('coinbase-api', {
    open: false, openedAt: 0, consecutiveErrors: 0,
  });
  return data;
} catch {
  await setCircuitBreakerState('coinbase-api', {
    open: state.consecutiveErrors >= 4,
    openedAt: Date.now(),
    consecutiveErrors: state.consecutiveErrors + 1,
  });
}

SWR Presets

Preset	TTL (fresh)	Max Age	Use Case
realtime	15s	1min	Live prices
frequent	30s	5min	Dashboards
standard	1min	10min	General data
slow	5min	1hr	Reference data
static	1hr	24hr	Config/metadata

API Reference

Shared Cache

Function	Description
configureL2(adapter, l1Ttl?)	Set up the Redis adapter
cacheGet<T>(key)	Get from L1 → L2
cacheSet<T>(key, value, ttlSeconds)	Write to L1 + L2
cacheDel(key)	Delete from L1 + L2
healthCheck()	Check L2 connectivity + latency
clearL1()	Clear L1 (for tests)

SWR Cache

Method	Description
swr(key, fetcher, options?)	Get with stale-while-revalidate
set(key, data, options?)	Direct set
get(key)	Direct get (no revalidation)
invalidate(key)	Remove a key
invalidatePattern(pattern)	Remove by glob pattern
mutate(key, mutator)	Optimistic update
subscribe(key, callback)	Listen for updates
getStats()	Cache statistics

Circuit Breaker

Function	Description
getCircuitBreakerState(provider)	Read state from shared cache
setCircuitBreakerState(provider, state)	Update state

License

MIT — see LICENSE.

---

Built with ❤️ by OmniFolio