Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
485 changes: 27 additions & 458 deletions README.md

Large diffs are not rendered by default.

3 changes: 2 additions & 1 deletion apps/docs/pages/guides/_meta.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,6 @@
"dashboards": "Build a dashboard",
"versioning": "Version & diff notebooks",
"jaffle-shop": "Jaffle Shop walkthrough",
"migrate": "Migrate from Metabase / Looker / Hex"
"migrate": "Migrate from Metabase / Looker / Hex",
"faq": "FAQ"
}
69 changes: 69 additions & 0 deletions apps/docs/pages/guides/faq.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
---
title: FAQ
description: Frequently asked questions about DQL.
---

# FAQ

## What is DQL?

An open-source analytics language, CLI, and notebook. Every analytics answer
— SQL, chart config, owner, tests, parameters — lives in a single `.dql`
file in git. See [Concepts](/get-started/concepts).

## Do I need a cloud warehouse to try it?

No. DQL ships with DuckDB. Drop a CSV into `data/`, open `dql notebook`, and
query it immediately. No credentials, no configuration.

## Do I need dbt?

No. DQL works standalone. But if you already use dbt, `create-dql-app`
auto-detects a sibling dbt project and `dql sync dbt` imports your manifest,
metrics, and lineage. See [Import a dbt project](/guides/import-dbt).

## How is this different from a Jupyter notebook?

| | Jupyter | DQL notebook |
| ----------------- | -------------- | --------------------- |
| Primary language | Python | SQL |
| Runtime | Python kernel | DuckDB (or your warehouse) |
| Chart setup | matplotlib/Plotly code | declarative, auto-rendered |
| File format | `.ipynb` JSON | `.dqlnb` JSON |
| Git diff | noisy (outputs embedded) | clean (outputs sidecar) |

## How is this different from Metabase, Looker, or Hex?

Those are BI tools — queries live in a database or cloud service. DQL keeps
every artifact in git. You get code review, diff, branching, and blame on
your analytics. See [Migrate from Metabase / Looker / Hex](/guides/migrate).

## What's the difference between `custom` and `semantic` blocks?

- **`type = "custom"`** — the block's SQL executes directly against a connection
- **`type = "semantic"`** — the block references a semantic-layer metric; SQL is generated

Start with `custom` blocks if you're new.

## Can I use DQL from code, not just the CLI?

Yes. The useful entry points:

- `@duckcodeailabs/dql-core` — parser, AST, formatter, semantic analysis
- `@duckcodeailabs/dql-compiler` — IR lowering, codegen
- `@duckcodeailabs/dql-runtime` — browser runtime
- `@duckcodeailabs/dql-connectors` — database drivers

## Does `dql test` hit a real database?

Yes — assertions run against the configured connection. With no connection
configured, `dql test` falls back to structural checks only.

## What's **not** in the open-source repo?

Hosted workspaces, column-level lineage, scheduled runs, alerting, regulatory
governance packs, and AI/agentic block generation. See [Compatibility](/reference/compatibility).

## Where do I report bugs or request features?

[github.com/duckcode-ai/dql/issues](https://github.com/duckcode-ai/dql/issues).
4 changes: 3 additions & 1 deletion apps/docs/pages/reference/_meta.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,7 @@
"language": "DQL language",
"connectors": "Connectors",
"semantic-layer": "Semantic layer",
"file-formats": "File formats"
"file-formats": "File formats",
"project-layout": "Project layout",
"compatibility": "Compatibility"
}
51 changes: 51 additions & 0 deletions apps/docs/pages/reference/compatibility.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
---
title: Compatibility
description: Runtime, platform, and scope support matrix for OSS DQL.
---

# Compatibility

## Runtime

- **Node.js** — 18, 20, or 22 (active LTS)
- **Package managers** — npm, pnpm, yarn all work for consuming published packages
- **OS** — macOS, Linux, Windows (WSL recommended on Windows for the Tauri desktop build)

## Connectors

All 15 drivers ship in `@duckcodeailabs/dql-connectors`. For first-run local
exploration, start with `duckdb` or `file`. See [Connectors](/reference/connectors).

## Chart types

`bar`, `line`, `scatter`, `donut`, `pie`, `area`, `heatmap`, `kpi`, `table`,
`histogram`, `grouped-bar`, `stacked-bar`, `waterfall`, `funnel`, `gauge`,
`geo`, `combo`, `boxplot`, `forecast`, `stacked-area`.

## Editor support

- **VS Code** — [`dql.dql-language-support`](https://marketplace.visualstudio.com/items?itemName=dql.dql-language-support) extension (syntax, snippets, LSP)
- **Any LSP client** — `@duckcodeailabs/dql-lsp` speaks standard LSP

## What's in this repo

- DQL language (parser, formatter, AST)
- CLI (`init`, `notebook`, `compile`, `sync`, `diff`, `preview`, `serve`, `certify`, `test`, `fmt`, `doctor`)
- Notebook UI (SQL + DQL cells, block studio, lineage panel, git panel)
- Semantic layer (local YAML + dbt import)
- Lineage DAG (table / block / notebook granularity)
- 15 database connectors
- Plugin API (custom connectors, charts, governance rules)
- OpenLineage export

## What's **not** in this repo

Lives in the closed DuckCode Cloud product:

- Column-level lineage
- Hosted multi-user workspace (SSO, RBAC, audit log)
- Scheduled runs, alerting, orchestration
- Regulatory governance packs (SOX, HIPAA, GDPR)
- AI / agentic block generation
- Premium connectors (SAP, Oracle, mainframe)
- Federated multi-warehouse blocks
64 changes: 39 additions & 25 deletions apps/docs/pages/reference/file-formats.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: File formats
description: .dql, .dqlnb, .run.json, and cdql.yaml.
description: .dql, .dqlnb, .run.json, and dql.config.json.
---

# File formats
Expand Down Expand Up @@ -31,11 +31,21 @@ block revenue_by_segment {
- **Canonical whitespace** — `dql fmt` enforces it. CI should run `dql fmt --check .`
- **Forward compatibility** — unknown top-level keys are preserved verbatim

## `.dqlnb` — notebook bundle (legacy)
## `.dqlnb` — notebook

Multi-cell notebooks shipped a bundled `.dqlnb` in early versions. From
v0.11 onwards, notebooks are plain `.dql` files with multiple cells. The
`.dqlnb` parser remains for backward compatibility.
JSON document holding a sequence of cells (`markdown`, `sql`, `dql`, `chart`).
Written in a stable key-order so git diffs stay readable.

```json
{
"version": 1,
"metadata": { "name": "Welcome", "createdAt": "2026-04-17T00:00:00Z" },
"cells": [
{ "id": "c1", "type": "markdown", "source": "# Welcome" },
{ "id": "c2", "type": "sql", "source": "select 1" }
]
}
```

## `.run.json` — run snapshots

Expand All @@ -61,30 +71,34 @@ Sibling file next to a notebook holding the last execution's results:
- **Used for rehydration** — re-opening a notebook loads results without re-running queries
- **Save trigger** — debounced 600ms after a cell's execution count / row count changes

## `cdql.yaml` — project config

Lives at the project root.
## `dql.config.json` — project config

```yaml
project:
name: jaffle-analytics
version: 1
Lives at the project root. Written by `dql init` and `create-dql-app`.

connections:
default:
driver: duckdb
path: ./warehouse.duckdb
```json
{
"project": "jaffle-analytics",
"connections": {
"default": { "driver": "duckdb", "filepath": ":memory:" }
},
"semanticLayer": { "provider": "dql", "path": "semantic-layer" },
"dbt": {
"projectDir": "../dbt",
"manifestPath": "target/manifest.json"
},
"governance": {
"required_fields": ["domain", "owner", "description"]
}
}
```

dbt:
projectDir: ../jaffle-shop-dbt
include:
anchors: ["tag:core"]
maxDbtHops: 2
Fields:

governance:
required_fields: [domain, owner, description]
certification_gates: [row_count_gt_zero, unique_pk]
```
- **`project`** — human-readable project name
- **`connections`** — named database connections. `default` is used unless a block overrides it
- **`semanticLayer`** — `{ provider, path }`. Providers: `dql` (local YAML), `dbt`, `cubejs`, `snowflake`
- **`dbt`** — `{ projectDir, manifestPath }` for sibling dbt projects. `dql sync dbt` reads from here
- **`governance`** — required fields every block must declare to pass `dql certify`

## `dql-manifest.json` — build output

Expand Down
62 changes: 62 additions & 0 deletions apps/docs/pages/reference/project-layout.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
---
title: Project layout
description: Directory conventions for a DQL project.
---

# Project layout

A DQL project is a git repo with a few conventional directories. Everything
is plain files — no database, no lockfile, no hidden state.

```text
my-dql-project/
├─ dql.config.json # connection, semantic layer, dbt wiring
├─ package.json # npm scripts: notebook, compile, sync, doctor
├─ blocks/ # certified reusable .dql files
│ └─ revenue_by_segment.dql
├─ notebooks/ # .dqlnb interactive notebooks
│ └─ welcome.dqlnb
├─ semantic-layer/ # metrics, dimensions, hierarchies, cubes (YAML)
│ ├─ metrics/revenue.yaml
│ └─ dimensions/customer.yaml
├─ dashboards/ # composable dashboards (.dql dashboard blocks)
│ └─ overview.dql
├─ data/ # local CSV/Parquet — git-ignored by default
└─ .dql/ # manifest cache — git-ignored
```

## What each directory holds

- **`blocks/`** — one `.dql` file per block. Governance fields (`domain`,
`owner`) are required by default; the certification check runs on CI.
- **`notebooks/`** — interactive analysis. Saved results live beside the
notebook as `.run.json` (git-ignored).
- **`semantic-layer/`** — metrics and dimensions authored locally. When you
run `dql sync dbt`, entries imported from a sibling dbt project are merged
here too.
- **`dashboards/`** — composed views that reference blocks.
- **`data/`** — sample data for local exploration. Production projects usually
query a warehouse instead.

## Working with a sibling dbt project

The default layout assumes your dbt project lives alongside your DQL project:

```text
my-team/
├─ dbt/ # your dbt project (dbt_project.yml)
│ └─ target/manifest.json # produced by `dbt parse` or `dbt build`
└─ dql/ # your DQL project
└─ dql.config.json # dbt.projectDir: "../dbt"
```

`create-dql-app` and `dql init` both auto-detect the sibling and wire the
config for you. Then `dql sync dbt` imports the manifest on demand.

## What gets committed

**Commit:** `blocks/`, `notebooks/`, `semantic-layer/`, `dashboards/`,
`dql.config.json`, `package.json`.

**Don't commit:** `data/`, `.dql/`, `*.run.json`, `dql-manifest.json` (build
output). The default `.gitignore` from the scaffolder handles this.
28 changes: 0 additions & 28 deletions docs/01-start-here/README.md

This file was deleted.

Loading
Loading