Skip to content

jamals86/KalamDB

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1,162 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

KalamDB logo

Rust License CI Tests Release Docker Pulls

SDKs

TypeScript SDK Tests Dart SDK Tests TypeScript SDK Dart SDK

KalamDB is a SQL-first realtime state database for AI agents, chat products, and multi-tenant SaaS. It combines SQL execution, live subscriptions, pub/sub streams, and hot/cold storage in one Rust runtime.

Supported SDKs: TypeScript/JavaScript and Dart/Flutter.

Frontend clients can execute SQL directly against KalamDB. This is not only a backend database layer.

Frontend-First SQL Execution

To avoid confusion: KalamDB is designed for both frontend and backend SQL execution.

  • Frontend (web/mobile/desktop): run SQL for user-scoped reads/writes and realtime subscriptions.
  • Backend/workers: run automation, cross-tenant system jobs, and service workflows.
  • Same SQL model on both sides: SELECT, INSERT, UPDATE, DELETE, SUBSCRIBE TO, CONSUME, ACK.

This is what lets chat UIs and agent apps read/write state directly and receive live updates without adding separate polling or fanout infrastructure.

Why KalamDB

  • Frontend-direct SQL and realtime subscriptions, so web and mobile apps can read, write, and stay in sync without a separate sync layer.
  • Multi-tenant isolation by default, with the same SQL returning user-scoped data safely across frontend and backend clients.
  • Built-in authentication and authorization with Basic auth, JWT, OAuth 2.0 (Google Workspace, GitHub, Azure AD), and RBAC.
  • Unified application primitives: SQL tables, live queries, pub/sub topics, consumer groups, and file attachments in one runtime.
  • Hybrid storage engine tuned for product workloads: RocksDB for fast active writes, Parquet for compressed historical storage and analytics.
  • Portable object storage support across filesystem, S3, GCS, and Azure backends.
  • Vector embeddings and vector search workflows for semantic retrieval, agent memory, and AI-powered product features.
  • Distributed clustering with Multi-Raft replication and failover for resilient production deployments.
  • First-party tooling for operators and app teams: Admin UI, kalam CLI, and official TypeScript and Dart SDKs.

60-Second Quick Start (Docker)

Single node

KALAMDB_JWT_SECRET="$(openssl rand -base64 32)" \
curl -sSL https://raw.githubusercontent.com/jamals86/KalamDB/main/docker/run/single/docker-compose.yml | docker-compose -f - up -d

3-node cluster

KALAMDB_JWT_SECRET="$(openssl rand -base64 32)" \
curl -sSL https://raw.githubusercontent.com/jamals86/KalamDB/main/docker/run/cluster/docker-compose.yml | docker-compose -f - up -d

Local run

git clone https://github.com/jamals86/KalamDB.git
cd KalamDB/backend
cargo run --bin kalamdb-server

Browser/Frontend Client Example

import { createClient, Auth } from 'kalam-link';

const client = createClient({
  url: 'http://localhost:8080',
  authProvider: async () => Auth.jwt('<user-token>'),
});

const threadSql = `
  SELECT id, role, content, created_at
  FROM chat.messages
  WHERE thread_id = 'thread_42'
`;

await client.query(`
  INSERT INTO chat.messages (thread_id, role, content)
  VALUES ('thread_42', 'user', 'hello from frontend');
`);

const unsubscribe = await client.live(
  threadSql,
  (rows) => {
    // If `chat.messages` is a USER table, each signed-in user only sees
    // their own rows even though the SQL text is identical for everyone.
    console.log('live rows', rows);
  },
  {
    subscriptionOptions: { last_rows: 20 },
  },
);

// Later
await unsubscribe();
await client.disconnect();

AI Agent Example (Topic Subscription)

Subscribe an AI agent to a KalamDB topic and process each row with an LLM — fully managed retries, backpressure, and at-least-once delivery via runAgent.

import { createClient, Auth, runAgent } from 'kalam-link';

const client = createClient({
  url: 'http://localhost:8080',
  authProvider: async () => Auth.basic('root', 'kalamdb123'),
});

const abort = new AbortController();
process.on('SIGINT', () => abort.abort());

await runAgent<{ title: string; body: string }>({
  client,
  name: 'summarizer-agent',
  topic: 'blog.posts',       // KalamDB topic to consume
  groupId: 'summarizer-v1',  // consumer group — tracks per-agent offset
  start: 'earliest',
  batchSize: 20,
  timeoutSeconds: 30,
  stopSignal: abort.signal,

  // Called for every row; return value is written back as metadata
  onRow: async ({ row }) => {
    const summary = await myLlm.summarize(row.body);
    console.log(`[${row.title}] →`, summary);
  },

  onFailed: async ({ row, error }) => {
    console.error('failed row', row, error);
  },

  retry: {
    maxAttempts: 3,
    initialBackoffMs: 250,
    maxBackoffMs: 1500,
    multiplier: 2,
  },
  ackOnFailed: true,    // commit offset even on permanent failure
});

await client.disconnect();

See examples/summarizer-agent/ for a full working example with Gemini integration.

SQL Example

CREATE NAMESPACE IF NOT EXISTS chat;

CREATE TABLE chat.messages (
  id BIGINT PRIMARY KEY DEFAULT SNOWFLAKE_ID(),
  thread_id TEXT NOT NULL,
  role TEXT NOT NULL,
  content TEXT NOT NULL,
  attachment FILE,
  created_at TIMESTAMP DEFAULT NOW()
) WITH (TYPE = 'USER');

Architecture Snapshot

  • SQL and execution: kalamdb-sql + DataFusion/Arrow.
  • Hot path: kalamdb-store (RocksDB).
  • Cold path: kalamdb-filestore (Parquet/object storage).
  • Orchestration: kalamdb-core (DDL/DML, jobs, schema registry).
  • API surface: HTTP SQL API + WebSocket realtime + Admin UI + CLI.

Best-Fit Workloads

  • AI chat history and agent memory/state.
  • Tool-call logs and human-in-the-loop workflows.
  • Live dashboards, notifications, and collaborative feeds.
  • Multi-tenant SaaS that needs strong user-level isolation.

Docs and Links

KalamDB is under active development and evolving quickly.

About

KalamDB — a lightweight, real-time, storage-efficient SQL database. Designed for per-user data isolation, GDPR compliance, and scalable performance — ideal for the AI era.

Topics

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors