spec(v1.0): Implement Strict Catalog JSON Schema Restrictions and Consolidation

[jacobsimionato]

Design Proposal: Restricting Catalog JSON Schemas for Alternative DSL Support and SDK Alignment

TLDR: In 1.0, let's document a number of restrictions on how Catalogs can be structured, to make it easier for our SDKs to manipulate them, expose them to models via different inference formats etc. E.g. no additional $defs especially. This also makes it easier to merge catalogs.

1. Rationale for the Change

1.1 The Challenge of Unconstrained JSON Schemas

Historically, the A2UI Protocol has permitted catalog files (catalog.json) to utilize the full, unconstrained expressive power of JSON Schema (Draft 2020-12). This includes deep nesting, arbitrary custom definitions in $defs, complex $refs, and complex condition types (e.g., dependencies, if/then/else).

While standard JSON Schema engines can validate JSON payloads against these unconstrained schemas, this open-ended design creates severe bottlenecks in two areas:

1. Alternative UI DSLs: We are actively exploring alternative DSLs that Large Language Models (LLMs) can write with lower token costs, faster inference speed, and higher syntactic accuracy (such as HTML-like XML or compact, programmatic DSLs like Text(text=$some/path)). Translating unconstrained JSON schemas into instructional prompts that describe these alternative DSL representations is incredibly difficult.
2. Client SDK Maintenance: We are struggling to maintain a canonical, structured representation of catalogs across different client SDKs (e.g., Python, Kotlin). In the Python SDK, we currently maintain two separate, complex pathways to define catalogs: programmatically via classes and dynamically from JSON. Restricting the schema layout allows us to map catalogs directly to structured models in all SDK programming languages without losing type-safety or correctness.

1.2 The Strategy

Our approach is divided into two phases:

• Phase 1 (Proposed for v1.0): Standardize and tightly restrict the structural patterns permitted within a catalog.json schema file.
• Phase 2 (Post-v1.0): Define a highly constrained, declarative catalog definition format (e.g., YAML or a dedicated DSL) that compiles down to standard JSON Schema solely for legacy client validation.

---

2. GitHub Source URL Links

For external reference outside of GitHub:

• A2UI Protocol Specification Docs
• Client Capabilities Schema (defines Catalog structure)
• Basic Catalog Implementation
• Common Types Schema

---

3. Phase 1: Strict Schema Restrictions (Proposed for v1.0)

To simplify catalog parsers, code-generators, and DSL translators, any compliant catalog.json file in v1.0 must adhere to the following strict rules:

3.1 Strict Top-Level vs. $defs Boundary

A catalog schema file must clearly partition its elements based on whether they are local schemas or referenced externally (e.g., by the envelope schemas server_to_client.json or client_to_server.json):

1. Top-Level Components & Functions:
   All component and function definitions must reside exclusively at the top level of the catalog under the components and functions keys, respectively. They must not be nested under $defs or any other sub-properties.
2. External References inside $defs:
   Any schema that is referred to from outside the catalog file (such as from the envelope schemas server_to_client.json or common_types.json) must reside within the $defs block. This strictly includes:
   • surfaceProperties: Referenced via catalog.json#/$defs/surfaceProperties.
   • anyComponent: Referenced via catalog.json#/$defs/anyComponent.
   • anyFunction: Referenced via catalog.json#/$defs/anyFunction.

3.2 Disallowing Custom $defs and Local Shared Schemas

To prevent arbitrary, deep structural branching, custom $defs or helper sub-schemas inside the catalog file are prohibited.

• Disallowing CatalogComponentCommon: The previous pattern of declaring a shared helper schema named CatalogComponentCommon (used to define properties like weight) under $defs is now disallowed.
• Inlining Requirements: Any properties that were previously inherited from CatalogComponentCommon (such as the relative flex layout weight property) must be inlined directly into the schemas of the individual components that support them (e.g., inlined directly under the properties of Text, Row, Column, etc., in the basic catalog).

3.3 Restricted $ref Targets

To keep reference pathways flat and easily traceable by simple AST parsers:

• Local $ref statements inside components/functions are only permitted to reference the top-level definitions within the same file (e.g., #/components/Text, #/functions/required).
• External $ref statements are strictly restricted to referencing the standardized common types defined in common_types.json (specifically https://a2ui.org/specification/v1_0/common_types.json#/$defs/...), limited to the following allowed schemas:
  • ComponentId
  • ChildList
  • DynamicString
  • DynamicNumber
  • DynamicBoolean
  • DynamicStringList
  • DynamicValue
  • CheckRule

3.4 Mandatory Leaf-Node Dynamic Wrappers (No Raw Primitives)

Every leaf node in a component's properties or a function's arguments schema must be defined as a dynamic type (DynamicString, DynamicNumber, DynamicBoolean, DynamicValue, etc.) rather than a literal primitive (e.g., raw string, number, boolean).

• The Dynamic Rule: Any field that can hold a static value must also be bindable to a path in the data model.
• DSL Translation Advantage: Forcing leaf nodes to be dynamic wrappers simplifies mapping to programmatic DSLs, ensuring that every value slot can seamlessly accept either a literal or a path reference without schema complexity.

3.5 Mitigating Prompt & Inference Verbosity

Disallowing custom helper $defs and forcing all leaf nodes to be dynamic wrappers naturally increases the raw verbosity of the catalog.json schema file. However, this is solved elegantly at the inference level:

• Shared Type Detection: The prompt generation engine or translator scans the catalog schema, identifies standard repetitive wrappers (like DynamicString), and deduplicates them in memory.
• Prompt Compression: Instead of passing the verbose JSON Schema representation to the LLM, the compiler condenses these patterns into simple, human-readable patterns in the prompt (e.g., "A property of type DynamicString accepts either a literal text or a JSON Pointer binding path like /user/name"). This keeps the active LLM context highly efficient.

---

4. Analysis of Catalog Conventions (The "Unwritten Rules")

Through an analysis of existing catalog implementations, several core structural conventions must be explicitly standardized to guarantee that automatic parsers can analyze and compile catalog files:

4.1 Component Discriminator Rule

Every component schema defined inside the components map must have a required property named component whose value is a constant (const) matching the key under which it is defined.

• Example: The component defined at components.Text must declare:

  "properties": {
    "component": {
      "const": "Text"
    }
  }

This enables O(1) route-dispatch matching via the discriminator block inside anyComponent (which designates "propertyName": "component").

4.2 Standard Component Structure

All components defined in the components object must use an allOf structure that combines:

1. An external reference to the baseline identity and accessibility attributes:
   {"$ref": "https://a2ui.org/specification/v1_0/common_types.json#/$defs/ComponentCommon"}
2. A local object schema defining the unique properties of that specific component (e.g., its children, variant, specific layouts).

4.3 Strict Function Interface Pattern

Every function schema defined inside the functions map must validate a wire-level FunctionCall object. This requires:

• A properties block with a call property containing a constant of the function's name (e.g., "call": { "const": "email" }).
• An optional args property representing arguments (or absent if the function accepts no arguments).
• Mandatory metadata fields outside the strict JSON validation properties to advertise interface details:
  • returnType: Must be a string enum indicating the return type (string, number, boolean, array, object, any, or void).
  • callableFrom: Must be a string enum indicating the execution boundary (clientOnly, remoteOnly, or clientOrRemote). If omitted, it defaults to clientOnly.

4.4 Unicode Variable Naming Rules

All component names (keys in components), function names (keys in functions), property names, and argument names must conform strictly to the Unicode Standard Annex #31 (UAX #31) naming rules as defined in the protocol specification. This ensures seamless code generation in Kotlin, Swift, TypeScript, and Python.

4.5 Strict Top-Level Schema Keys

To keep catalog schemas predictable and prevent custom extensions from polluting the global file space, a catalog.json file is restricted to the following root-level keys:

• $schema
• $id
• title
• description
• catalogId
• instructions
• components
• functions
• $defs

No other top-level keys are permitted.

---

5. Phase 2: Declarative Catalog Format (Post-v1.0)

In Phase 2, the team will introduce a dedicated, non-JSON Schema language or YAML-based declarative format to write catalogs. This will completely isolate developers from the complexities of writing raw JSON Schema files.

An SDK toolchain will automatically compile this clean, declarative representation into all required formats, keeping the ecosystem clean, type-safe, and highly adaptable to future LLM interaction paradigms.

---

6. Schema Consolidation: Consolidating the Catalog Schema

6.1 Current Discrepancy

There is currently a superficial schema-to-implementation mismatch between client_capabilities.json and actual catalogs like basic/catalog.json:

• In client_capabilities.json -> $defs -> Catalog -> properties:
  The surfaceProperties property is defined as a top-level property of the Catalog object.
• In the actual basic/catalog.json:
  surfaceProperties is placed under $defs (as $defs.surfaceProperties) because server_to_client.json references it as "catalog.json#/$defs/surfaceProperties".

Because of this mismatch, validating the actual catalog.json file directly against the Catalog schema in client_capabilities.json fails to properly validate the contents of surfaceProperties against its expected structure, since it resides in $defs rather than at the top level of the catalog instance.

6.2 Consolidation Recommendation

We recommend standardizing the layout by standardizing surfaceProperties under $defs inside the Catalog schema in client_capabilities.json. This aligns with our Phase 1 boundary guidelines, where elements referenced externally must reside under $defs within the catalog file.

[jacobsimionato]

Coding agent prompt:
You are an autonomous AI coding agent. Your task is to update the A2UI v1.0 specification, JSON schemas, documentation, and existing catalog files to enforce strict JSON Schema constraints for catalogs, and integrate all of these catalog restrictions directly into the main protocol documentation file.

Primary Files to Reference and Modify

• Client capabilities schema defining the Catalog format: @specification/v1_0/json/client_capabilities.json
• A2UI Protocol specification: @specification/v1_0/docs/a2ui_protocol.md
• Basic catalog implementation: @specification/v1_0/catalogs/basic/catalog.json
• Minimal catalog implementation: @specification/v1_0/catalogs/minimal/catalog.json
• Testing catalog and test suite directory: @specification/v1_0/test/
• Common types: @specification/v1_0/json/common_types.json

Implementation Instructions

1. Consolidate the Catalog Schema in client_capabilities.json

• Modify @specification/v1_0/json/client_capabilities.json to fix the surfaceProperties discrepancy.
• Under the Catalog definition schema inside client_capabilities.json:
  • Remove surfaceProperties from the top-level properties block.
  • In the schema's $defs of Catalog (or using properties constraint on the catalog's $defs), specify that $defs must contain surfaceProperties (type: object/schema).
  • Enforce that the catalog's $defs property only permits the following three keys: anyComponent, anyFunction, and surfaceProperties. All other keys in $defs should be forbidden to prevent arbitrary helpers.

2. Refactor the Basic Catalog (catalog.json)

• Edit @specification/v1_0/catalogs/basic/catalog.json:
  • Remove "CatalogComponentCommon" from the "$defs" block.
  • For every component in "components" (e.g., Text, Image, Icon, Video, AudioPlayer, Row, Column, List, Card, Tabs, Modal, Divider, Button, TextField, CheckBox, ChoicePicker, Slider, DateTimeInput), remove the {"$ref": "#/$defs/CatalogComponentCommon"} reference from its allOf array.
  • Inline the "weight" property definition directly into each component's unique properties block under "properties". The "weight" property schema is:

    "weight": {
      "type": "number",
      "description": "The relative weight of this component within a Row or Column. This is similar to the CSS 'flex-grow' property. Note: this may ONLY be set when the component is a direct descendant of a Row or Column."
    }

3. Refactor Minimal and Test Catalogs

• Update @specification/v1_0/catalogs/minimal/catalog.json to follow the same constraints (no custom definitions under $defs except the standard ones).
• Update @specification/v1_0/test/testing_catalog.json to inline weight and remove any prohibited helper schemas in $defs.

4. Document all Catalog Restrictions in a2ui_protocol.md

• Open and update @specification/v1_0/docs/a2ui_protocol.md.
• Locate "The component catalog" or component model section.
• Add a comprehensive, prominent section explaining the new strict catalog schema restrictions. You MUST document:
  1. Strict Top-Level vs. $defs Boundary: Component and function definitions must reside at the top level under components and functions, respectively. External references (e.g., surfaceProperties, anyComponent, anyFunction) must reside under $defs.
  2. No Custom $defs: Custom definitions or helpers (such as CatalogComponentCommon) are strictly prohibited in a catalog.json file. All helper fields (like weight) must be inlined.
  3. Restricted $ref Targets: $ref targets are restricted to local references inside the same file (e.g., #/components/...) or a strictly specified list of dynamic common types from common_types.json (ComponentId, ChildList, DynamicString, DynamicNumber, DynamicBoolean, DynamicStringList, DynamicValue, CheckRule).
  4. Mandatory Dynamic Wrappers: Raw primitive leaf-nodes are prohibited; all leaf properties or function arguments must utilize dynamic types.
  5. Inference-Level Prompt Compression: Explain that the increased verbosity of dynamic wrappers is mitigated by the prompt generator, which detects shared structures and compresses them into readable text instructions for LLMs.
  6. Existing Conventions (Unwritten Rules): Document the Component Discriminator Rule, Standard Component Structure, Strict Function Interface Pattern, Unicode Variable Naming Rules, and Strict Top-Level Schema Keys.

5. Verify and Test

• Run the python validation test suite: python3 specification/v1_0/test/run_tests.py in your workspace.
• Ensure all tests pass successfully, confirming that catalog.json validates correctly against the updated client_capabilities.json's Catalog schema.

[jacobsimionato]

Hey @gspencergoog and @jgindin , this occurred to me when reviewing our Python SDK that @nan-yu is building. WDYT? I sent #1629 to make the changes.