Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
d9bf714
add MongoStorageIndex and MongoIndexKey contract types with Arktype v…
wmadden Apr 9, 2026
21469b3
implement marker/ledger storage and MongoControlDriverInstance
wmadden Apr 9, 2026
71bec0f
Add M2 spec and implementation plan for full index vocabulary, valida…
wmadden Apr 9, 2026
8f7cb72
feat(mongo-contract): add M2 index options (wildcardProjection, colla…
wmadden Apr 9, 2026
513e75b
feat(mongo-contract): add MongoStorageValidator and MongoStorageColle…
wmadden Apr 9, 2026
69b7368
feat(mongo-schema-ir): extend MongoSchemaIndex with M2 options
wmadden Apr 9, 2026
35f80ed
feat(mongo-schema-ir): add MongoSchemaValidator and MongoSchemaCollec…
wmadden Apr 9, 2026
392d090
feat(mongo-query-ast): add CreateCollectionCommand, DropCollectionCom…
wmadden Apr 9, 2026
aadc151
feat(mongo-schema-ir): add canonicalize() for key-order-independent s…
wmadden Apr 9, 2026
fb081e0
feat(adapter-mongo): extend contractToSchema for validators, options,…
wmadden Apr 9, 2026
e9f30a2
feat(adapter-mongo): extend serializer for new DDL commands and index…
wmadden Apr 9, 2026
4013cf8
feat(adapter-mongo): extend DDL formatter for new commands and index …
wmadden Apr 9, 2026
89b8418
feat(adapter-mongo): extend planner with canonical lookup keys and M2…
wmadden Apr 9, 2026
60616ba
feat(adapter-mongo): extend planner for validator diffing and collect…
wmadden Apr 9, 2026
223600e
feat(adapter-mongo): extend command executor for new DDL commands
wmadden Apr 9, 2026
a727bca
feat(mongo-contract-psl): add @@index/@@unique and @unique to Mongo P…
wmadden Apr 9, 2026
7d05d7d
feat(mongo-contract-psl): auto-derive $jsonSchema validator from mode…
wmadden Apr 10, 2026
3af1432
feat(mongo-contract-psl): wire $jsonSchema validator derivation into …
wmadden Apr 10, 2026
ee4d9ac
feat(adapter-mongo): E2E integration tests for full M2 vocabulary
wmadden Apr 10, 2026
7fefd4d
test(integration): E2E PSL authoring test for full M2 flow
wmadden Apr 10, 2026
df7353e
fix: reconcile MongoStorageIndex format after rebase onto origin/main
wmadden Apr 10, 2026
a7effdc
fix: implement validator classification matrix and add prechecks/post…
wmadden Apr 10, 2026
eaa2fbe
refactor: code cleanup — remove dead CLI formatter, rename options no…
wmadden Apr 10, 2026
c35aaea
test: add missing negative Arktype tests and language_override integr…
wmadden Apr 10, 2026
0a71c16
docs: update ADRs 187-189, schema-ir README, and contract-psl README …
wmadden Apr 10, 2026
d5383f9
test: add policy-gated validator and collection option integration te…
wmadden Apr 10, 2026
d2c0f97
fix: resolve TS errors in integration tests for CollectionInfo option…
wmadden Apr 10, 2026
89ed81b
test: improve adapter-mongo branch coverage above 90% threshold
wmadden Apr 10, 2026
b13db34
fix: remove weak postcheck from validator removal to prevent idempote…
wmadden Apr 10, 2026
d3c64ef
fix: address PR review findings for planner correctness
wmadden Apr 10, 2026
0aa36eb
fix: update integration test fixtures to new MongoStorageIndex keys f…
wmadden Apr 10, 2026
97380a3
fix: route StorageBase cast through unknown for exactOptionalProperty…
wmadden Apr 10, 2026
df8a587
fix: update contract fixtures for new MongoStorageIndex keys format a…
wmadden Apr 10, 2026
e63de91
fix: update emit-command test to new MongoStorageIndex keys format
wmadden Apr 10, 2026
023f753
use canonicalize() for immutable option comparison and fix clusteredI…
wmadden Apr 12, 2026
4326dc7
revert clusteredIndex fixture: schema intentionally omits key+unique
wmadden Apr 12, 2026
b109363
Add PSL index authoring surface design doc
wmadden Apr 12, 2026
b2f840e
Rewrite PSL index authoring design doc for clarity
wmadden Apr 12, 2026
3593d60
Add PSL language specification describing current parser
wmadden Apr 12, 2026
b72ba88
fix cli.emit-command test: update d.ts assertion for keys index format
wmadden Apr 12, 2026
c4e225a
Implement PSL index authoring surface
wmadden Apr 12, 2026
700617d
Add e2e tests for PSL index authoring surface on real MongoDB
wmadden Apr 12, 2026
68ce8aa
Establish PSL language model: grammar, binding, and index surface
wmadden Apr 12, 2026
cd707a3
Add TypeConstructor and TaggedLiteral value types to PSL design
wmadden Apr 12, 2026
d6e7a2f
Define filter as typed dictionary: field-ref keys with schema-validat…
wmadden Apr 12, 2026
33882f1
Fix typecheck: add null to parseCollation return type, fix getIndexes…
wmadden Apr 12, 2026
25d8b8e
Merge branch 'main' into tml-2231-m2-full-index-vocabulary-validators…
wmadden Apr 12, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ MongoDB has a small set of server-side objects that migrations manage. Each one
| `MongoSchemaCollectionOptions` | Capped, timeseries, collation, etc. | `{ capped: true, size: 1048576 }` |


Currently only `MongoSchemaCollection` and `MongoSchemaIndex` are implemented. Validators and collection options are defined in the visitor interface (so adding them later produces a compile error in all consumers) but not yet built.
All four node types are implemented. `MongoSchemaValidator` holds `$jsonSchema`, `validationLevel`, and `validationAction`. `MongoSchemaCollectionOptions` holds capped, timeseries, collation, clusteredIndex, and changeStreamPreAndPostImages.

## Decision

Expand All @@ -94,17 +94,19 @@ interface MongoSchemaIR {

An empty IR (for a new project with no prior contract) is `{ collections: {} }`.

A collection groups its indexes (and, in the future, its validator and options):
A collection groups its indexes, validator, and options:

```ts
class MongoSchemaCollection extends MongoSchemaNode {
readonly kind = 'collection' as const;
readonly name: string;
readonly indexes: ReadonlyArray<MongoSchemaIndex>;
readonly validator?: MongoSchemaValidator;
readonly options?: MongoSchemaCollectionOptions;
}
```

An index — the most important node — is defined by its keys and options:
An index is defined by its keys and options:

```ts
class MongoSchemaIndex extends MongoSchemaNode {
Expand All @@ -114,6 +116,11 @@ class MongoSchemaIndex extends MongoSchemaNode {
readonly sparse?: boolean;
readonly expireAfterSeconds?: number;
readonly partialFilterExpression?: Record<string, unknown>;
readonly wildcardProjection?: Record<string, 0 | 1>;
readonly collation?: Record<string, unknown>;
readonly weights?: Record<string, number>;
readonly default_language?: string;
readonly language_override?: string;
}
```

Expand All @@ -133,12 +140,12 @@ Each node extends `MongoSchemaNode` and implements `accept<R>(visitor: MongoSche
interface MongoSchemaVisitor<R> {
collection(node: MongoSchemaCollection): R;
index(node: MongoSchemaIndex): R;
validator(node: unknown): R;
collectionOptions(node: unknown): R;
validator(node: MongoSchemaValidator): R;
collectionOptions(node: MongoSchemaCollectionOptions): R;
}
```

Adding a new node type (e.g. `MongoSchemaValidator`) requires adding a method to this interface. Every existing visitor implementation gets a compile error until it handles the new case. This is the same exhaustiveness guarantee used by the DDL command visitors and filter expression visitors elsewhere in the codebase.
Adding a new node type requires adding a method to this interface. Every existing visitor implementation gets a compile error until it handles the new case. This is the same exhaustiveness guarantee used by the DDL command visitors and filter expression visitors elsewhere in the codebase.

### Structural identity for indexes

Expand Down Expand Up @@ -168,4 +175,4 @@ We considered a generic `DocumentSchemaIR` shared across all document databases

### Define only the nodes needed today

We considered defining nodes only for indexes and adding collection/validator/options nodes later. We chose to define the full visitor interface up front (with `unknown` parameter types for unimplemented nodes) so that future additions produce compile errors in existing code. The node classes themselves are added incrementally — only `MongoSchemaCollection` and `MongoSchemaIndex` exist today.
We considered defining nodes only for indexes and adding collection/validator/options nodes later. We chose to define the full visitor interface up front so that future additions produce compile errors in existing code. All four node types (`MongoSchemaCollection`, `MongoSchemaIndex`, `MongoSchemaValidator`, `MongoSchemaCollectionOptions`) are now implemented.
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ interface MongoMigrationStep {
}
```

The current command vocabulary is `CreateIndexCommand` and `DropIndexCommand`. Future additions (e.g. `CreateCollectionCommand`, `DropCollectionCommand`, `CollModCommand`) follow the same `MongoAstNode` pattern: frozen, `kind`-discriminated, `accept(visitor)` for dispatch. Adding a new command means one new class and one new case in the command executor — not a new operation type.
The command vocabulary is `CreateIndexCommand`, `DropIndexCommand`, `CreateCollectionCommand`, `DropCollectionCommand`, and `CollModCommand`. All follow the same `MongoAstNode` pattern: frozen, `kind`-discriminated, `accept(visitor)` for dispatch. Adding a new command means one new class and one new case in the command executor — not a new operation type.

### Checks

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,14 +35,21 @@ function buildIndexLookupKey(index: MongoSchemaIndex): string {
index.unique ? 'unique' : '',
index.sparse ? 'sparse' : '',
index.expireAfterSeconds != null ? `ttl:${index.expireAfterSeconds}` : '',
index.partialFilterExpression ? `pfe:${JSON.stringify(index.partialFilterExpression)}` : '',
index.partialFilterExpression ? `pfe:${canonicalize(index.partialFilterExpression)}` : '',
index.wildcardProjection ? `wp:${canonicalize(index.wildcardProjection)}` : '',
index.collation ? `col:${canonicalize(index.collation)}` : '',
index.weights ? `wt:${canonicalize(index.weights)}` : '',
index.default_language ? `dl:${index.default_language}` : '',
index.language_override ? `lo:${index.language_override}` : '',
]
.filter(Boolean)
.join(';');
return opts ? `${keys}|${opts}` : keys;
}
```

Object-valued options (`partialFilterExpression`, `wildcardProjection`, `collation`, `weights`) use `canonicalize()` — a key-order-independent serialization — so that `{ locale: 'en', strength: 2 }` and `{ strength: 2, locale: 'en' }` produce the same lookup key.

Two indexes that produce the same lookup key are the same index. For example:


Expand Down Expand Up @@ -72,6 +79,11 @@ Each component is included because it changes the index's behavior at the databa
- **`sparse`**. A sparse index omits documents missing the indexed field.
- **`expireAfterSeconds`**. A TTL index with a 24-hour expiry is different from one with a 7-day expiry.
- **`partialFilterExpression`**. A partial index scoped to `{ status: "active" }` is different from one scoped to `{ status: "archived" }`.
- **`wildcardProjection`**. A wildcard index on `{ name: 1, email: 1 }` differs from `{ name: 1 }`.
- **`collation`**. Per-index collation changes sort and comparison behavior.
- **`weights`**. Text index weights change relevance scoring.
- **`default_language`**. Changes how text indexes tokenize and stem words.
- **`language_override`**. Changes the per-document field used to determine language.

### What the lookup key excludes

Expand Down
4 changes: 2 additions & 2 deletions examples/mongo-demo/src/contract.json
Original file line number Diff line number Diff line change
Expand Up @@ -220,8 +220,8 @@
"users": {
"indexes": [
{
"fields": { "email": 1 },
"options": { "unique": true }
"keys": [{ "field": "email", "direction": 1 }],
"unique": true
}
]
}
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import type { MigrationPlanOperation } from '@prisma-next/framework-components/control';
import { extractMongoStatements } from './extract-mongo-statements';
import { extractSqlDdl } from './extract-sql-ddl';

export function extractOperationStatements(
Expand All @@ -9,8 +8,6 @@ export function extractOperationStatements(
switch (familyId) {
case 'sql':
return extractSqlDdl(operations);
case 'mongo':
return extractMongoStatements(operations);
default:
return undefined;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,58 +23,9 @@ describe('extractOperationStatements', () => {
expect(result).toContain('CREATE TABLE t (id int)');
});

it('delegates to Mongo extractor for mongo family', () => {
const ops: MigrationPlanOperation[] = [
{
id: 'op1',
label: 'test',
operationClass: 'additive',
execute: [
{
description: 'create index',
command: {
kind: 'createIndex',
collection: 'users',
keys: [{ field: 'email', direction: 1 }],
},
},
],
} as unknown as MigrationPlanOperation,
];
const result = extractOperationStatements('mongo', ops);
expect(result).toBeDefined();
expect(result).toHaveLength(1);
expect(result![0]).toContain('db.users.createIndex');
expect(result![0]).toContain('"email"');
});

it('extracts mongo dropIndex statement', () => {
const ops: MigrationPlanOperation[] = [
{
id: 'op1',
label: 'test',
operationClass: 'destructive',
execute: [
{
description: 'drop index',
command: {
kind: 'dropIndex',
collection: 'users',
name: 'email_1',
},
},
],
} as unknown as MigrationPlanOperation,
];
const result = extractOperationStatements('mongo', ops);
expect(result).toEqual(['db.users.dropIndex("email_1")']);
});

it('returns empty array for mongo family with no execute steps', () => {
const ops: MigrationPlanOperation[] = [
{ id: 'op1', label: 'test', operationClass: 'additive' } as unknown as MigrationPlanOperation,
];
it('returns undefined for mongo family', () => {
const ops: MigrationPlanOperation[] = [];
const result = extractOperationStatements('mongo', ops);
expect(result).toEqual([]);
expect(result).toBeUndefined();
});
});
Original file line number Diff line number Diff line change
Expand Up @@ -240,10 +240,65 @@ const IndexSchema = type({
'options?': IndexOptionsSchema,
});

const MongoIndexKeySchema = type({
'+': 'reject',
field: 'string',
direction: '1 | -1 | "text" | "2dsphere" | "2d" | "hashed"',
});

const MongoStorageIndexSchema = type({
'+': 'reject',
keys: MongoIndexKeySchema.array().atLeastLength(1),
'unique?': 'boolean',
'sparse?': 'boolean',
'expireAfterSeconds?': 'number',
'partialFilterExpression?': 'Record<string, unknown>',
'wildcardProjection?': 'Record<string, 0 | 1>',
'collation?': 'Record<string, unknown>',
'weights?': 'Record<string, number>',
'default_language?': 'string',
'language_override?': 'string',
});

const MongoStorageValidatorSchema = type({
'+': 'reject',
jsonSchema: 'Record<string, unknown>',
validationLevel: "'strict' | 'moderate'",
validationAction: "'error' | 'warn'",
});

const CappedOptionsSchema = type({
'+': 'reject',
size: 'number',
'max?': 'number',
});

const TimeseriesOptionsSchema = type({
'+': 'reject',
timeField: 'string',
'metaField?': 'string',
'granularity?': "'seconds' | 'minutes' | 'hours'",
});

const ClusteredIndexSchema = type({
'+': 'reject',
'name?': 'string',
});

const MongoCollectionOptionsSchema = type({
'+': 'reject',
'capped?': CappedOptionsSchema,
'timeseries?': TimeseriesOptionsSchema,
'collation?': 'Record<string, unknown>',
'changeStreamPreAndPostImages?': ChangeStreamPreAndPostImagesSchema,
'clusteredIndex?': ClusteredIndexSchema,
});

const StorageCollectionSchema = type({
'+': 'reject',
'indexes?': IndexSchema.array(),
'options?': CollectionOptionsSchema,
'indexes?': MongoStorageIndexSchema.array(),
'validator?': MongoStorageValidatorSchema,
'options?': MongoCollectionOptionsSchema,
});

export const MongoContractSchema = type({
Expand All @@ -269,3 +324,15 @@ export const MongoContractSchema = type({
'[string]': type({ '+': 'reject', fields: type({ '[string]': FieldSchema }) }),
}),
});

export {
CollationSchema,
CollectionOptionsSchema,
IndexFieldsSchema,
IndexOptionsSchema,
IndexSchema,
MongoIndexKeySchema,
MongoStorageIndexSchema,
NumberRecordSchema,
WildcardProjectionSchema,
};
Loading
Loading