From e89c4ee96f8bb8968038fac43e4acbdcf73a154c Mon Sep 17 00:00:00 2001 From: James Broadhead Date: Wed, 27 May 2026 22:43:04 +0000 Subject: [PATCH 1/2] experimental: backfill metadata.version + parent + compatibility frontmatter MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds three frontmatter fields to 17 experimental SKILL.md files that stable skills already carry but the imported a-d-k snapshot does not: - `compatibility: Requires databricks CLI (>= v0.294.0)` - `metadata.version: "0.1.0"` (was the `0.0.1` script floor) - `parent: databricks-core` `experimental/databricks-vector-search` is intentionally skipped to avoid conflict with #87 (promotion to stable, which adds the same fields as part of the move). Manifest deltas: every experimental skill's `version` flips from `0.0.1` (scripts/skills.py fallback floor) to `0.1.0`. No other consumer-visible fields move — `compatibility` and `parent` are SKILL.md-only and not surfaced in manifest.json today. ## Test plan - `python3 scripts/skills.py generate` clean - `python3 scripts/skills.py validate` passes (`Everything is up to date.`) This PR was prepared by Claude. --- experimental/databricks-agent-bricks/SKILL.md | 4 +++ experimental/databricks-ai-functions/SKILL.md | 4 +++ .../databricks-aibi-dashboards/SKILL.md | 4 +++ experimental/databricks-apps-python/SKILL.md | 4 +++ experimental/databricks-dbsql/SKILL.md | 4 +++ experimental/databricks-docs/SKILL.md | 4 +++ .../databricks-execution-compute/SKILL.md | 4 +++ experimental/databricks-iceberg/SKILL.md | 4 +++ experimental/databricks-metric-views/SKILL.md | 4 +++ .../databricks-mlflow-evaluation/SKILL.md | 4 +++ experimental/databricks-python-sdk/SKILL.md | 4 +++ .../SKILL.md | 4 +++ .../databricks-synthetic-data-gen/SKILL.md | 4 +++ .../databricks-unity-catalog/SKILL.md | 4 +++ .../SKILL.md | 4 +++ .../databricks-zerobus-ingest/SKILL.md | 4 +++ .../spark-python-data-source/SKILL.md | 4 +++ manifest.json | 34 +++++++++---------- 18 files changed, 85 insertions(+), 17 deletions(-) diff --git a/experimental/databricks-agent-bricks/SKILL.md b/experimental/databricks-agent-bricks/SKILL.md index cb0bfb6..4147d0e 100644 --- a/experimental/databricks-agent-bricks/SKILL.md +++ b/experimental/databricks-agent-bricks/SKILL.md @@ -1,6 +1,10 @@ --- name: databricks-agent-bricks description: "Create Agent Bricks: Knowledge Assistants (KA) for document Q&A and Supervisor Agents for multi-agent orchestration (MAS)." +compatibility: Requires databricks CLI (>= v0.294.0) +metadata: + version: "0.1.0" +parent: databricks-core --- # Agent Bricks diff --git a/experimental/databricks-ai-functions/SKILL.md b/experimental/databricks-ai-functions/SKILL.md index ff40538..29fda1c 100644 --- a/experimental/databricks-ai-functions/SKILL.md +++ b/experimental/databricks-ai-functions/SKILL.md @@ -1,6 +1,10 @@ --- name: databricks-ai-functions description: "Use Databricks built-in AI Functions (ai_classify, ai_extract, ai_summarize, ai_mask, ai_translate, ai_fix_grammar, ai_gen, ai_analyze_sentiment, ai_similarity, ai_parse_document, ai_query, ai_forecast) to add AI capabilities directly to SQL and PySpark pipelines without managing model endpoints. Also covers document parsing and building custom RAG pipelines (parse → chunk → index → query)." +compatibility: Requires databricks CLI (>= v0.294.0) +metadata: + version: "0.1.0" +parent: databricks-core --- # Databricks AI Functions diff --git a/experimental/databricks-aibi-dashboards/SKILL.md b/experimental/databricks-aibi-dashboards/SKILL.md index 54defbb..0c4c623 100644 --- a/experimental/databricks-aibi-dashboards/SKILL.md +++ b/experimental/databricks-aibi-dashboards/SKILL.md @@ -1,6 +1,10 @@ --- name: databricks-aibi-dashboards description: "Create Databricks AI/BI dashboards. Must use when creating, updating, or deploying Lakeview dashboards as Databricks Dashboard have a unique json structure. CRITICAL: You MUST test ALL SQL queries via CLI BEFORE deploying. Follow guidelines strictly." +compatibility: Requires databricks CLI (>= v0.294.0) +metadata: + version: "0.1.0" +parent: databricks-core --- # AI/BI Dashboard Skill diff --git a/experimental/databricks-apps-python/SKILL.md b/experimental/databricks-apps-python/SKILL.md index 9b14627..601997c 100644 --- a/experimental/databricks-apps-python/SKILL.md +++ b/experimental/databricks-apps-python/SKILL.md @@ -1,6 +1,10 @@ --- name: databricks-apps-python description: "Builds Databricks applications. Prefers AppKit (TypeScript + React SDK) for new apps; falls back to Python frameworks (Dash, Streamlit, Gradio, Flask, FastAPI, Reflex) when Python is required. Handles OAuth authorization, app resources, SQL warehouse and Lakebase connectivity, model serving, foundation model APIs, and deployment. Use when building web apps, dashboards, ML demos, or REST APIs for Databricks, or when the user mentions AppKit, Streamlit, Dash, Gradio, Flask, FastAPI, Reflex, or Databricks app." +compatibility: Requires databricks CLI (>= v0.294.0) +metadata: + version: "0.1.0" +parent: databricks-core --- # Databricks Applications diff --git a/experimental/databricks-dbsql/SKILL.md b/experimental/databricks-dbsql/SKILL.md index b2d8ced..451cd58 100644 --- a/experimental/databricks-dbsql/SKILL.md +++ b/experimental/databricks-dbsql/SKILL.md @@ -12,6 +12,10 @@ description: >- "temp table", "temporary view", "pipe operator". SHOULD also invoke when the user asks about SQL best practices, data modeling patterns, or advanced SQL features on Databricks. +compatibility: Requires databricks CLI (>= v0.294.0) +metadata: + version: "0.1.0" +parent: databricks-core --- # Databricks SQL (DBSQL) - Advanced Features diff --git a/experimental/databricks-docs/SKILL.md b/experimental/databricks-docs/SKILL.md index 5c64f15..3bd1017 100644 --- a/experimental/databricks-docs/SKILL.md +++ b/experimental/databricks-docs/SKILL.md @@ -1,6 +1,10 @@ --- name: databricks-docs description: "Databricks documentation reference via llms.txt index. Use when other skills do not cover a topic, looking up unfamiliar Databricks features, or needing authoritative docs on APIs, configurations, or platform capabilities." +compatibility: Requires databricks CLI (>= v0.294.0) +metadata: + version: "0.1.0" +parent: databricks-core --- # Databricks Documentation Reference diff --git a/experimental/databricks-execution-compute/SKILL.md b/experimental/databricks-execution-compute/SKILL.md index e7c825c..a83f1be 100644 --- a/experimental/databricks-execution-compute/SKILL.md +++ b/experimental/databricks-execution-compute/SKILL.md @@ -9,6 +9,10 @@ description: >- "delete cluster", "terminate cluster", "create warehouse", "new warehouse", "resize warehouse", "delete warehouse", "node types", "runtime versions", "DBR versions", "spin up compute", "provision cluster". +compatibility: Requires databricks CLI (>= v0.294.0) +metadata: + version: "0.1.0" +parent: databricks-core --- # Databricks Execution & Compute diff --git a/experimental/databricks-iceberg/SKILL.md b/experimental/databricks-iceberg/SKILL.md index de05763..9b2e241 100644 --- a/experimental/databricks-iceberg/SKILL.md +++ b/experimental/databricks-iceberg/SKILL.md @@ -1,6 +1,10 @@ --- name: databricks-iceberg description: "Apache Iceberg tables on Databricks — Managed Iceberg tables, External Iceberg Reads (fka Uniform), Compatibility Mode, Iceberg REST Catalog (IRC), Iceberg v3, Snowflake interop, PyIceberg, OSS Spark, external engine access and credential vending. Use when creating Iceberg tables, enabling External Iceberg Reads (uniform) on Delta tables (including Streaming Tables and Materialized Views via compatibility mode), configuring external engines to read Databricks tables via Unity Catalog IRC, integrating with Snowflake catalog to read Foreign Iceberg tables" +compatibility: Requires databricks CLI (>= v0.294.0) +metadata: + version: "0.1.0" +parent: databricks-core --- # Apache Iceberg on Databricks diff --git a/experimental/databricks-metric-views/SKILL.md b/experimental/databricks-metric-views/SKILL.md index c2c396a..97d7d04 100644 --- a/experimental/databricks-metric-views/SKILL.md +++ b/experimental/databricks-metric-views/SKILL.md @@ -1,6 +1,10 @@ --- name: databricks-metric-views description: "Unity Catalog metric views: define, create, query, and manage governed business metrics in YAML. Use when building standardized KPIs, revenue metrics, order analytics, or any reusable business metrics that need consistent definitions across teams and tools." +compatibility: Requires databricks CLI (>= v0.294.0) +metadata: + version: "0.1.0" +parent: databricks-core --- # Unity Catalog Metric Views diff --git a/experimental/databricks-mlflow-evaluation/SKILL.md b/experimental/databricks-mlflow-evaluation/SKILL.md index 56f24ef..7248727 100644 --- a/experimental/databricks-mlflow-evaluation/SKILL.md +++ b/experimental/databricks-mlflow-evaluation/SKILL.md @@ -1,6 +1,10 @@ --- name: databricks-mlflow-evaluation description: "MLflow 3 GenAI agent evaluation. Use when writing mlflow.genai.evaluate() code, creating @scorer functions, using built-in scorers (Guidelines, Correctness, Safety, RetrievalGroundedness), building eval datasets from traces, setting up trace ingestion and production monitoring, aligning judges with MemAlign from domain expert feedback, or running optimize_prompts() with GEPA for automated prompt improvement." +compatibility: Requires databricks CLI (>= v0.294.0) +metadata: + version: "0.1.0" +parent: databricks-core --- # MLflow 3 GenAI Evaluation diff --git a/experimental/databricks-python-sdk/SKILL.md b/experimental/databricks-python-sdk/SKILL.md index d6cffc3..be53f53 100644 --- a/experimental/databricks-python-sdk/SKILL.md +++ b/experimental/databricks-python-sdk/SKILL.md @@ -1,6 +1,10 @@ --- name: databricks-python-sdk description: "Databricks development guidance including Python SDK, Databricks Connect, CLI, and REST API. Use when working with databricks-sdk, databricks-connect, or Databricks APIs." +compatibility: Requires databricks CLI (>= v0.294.0) +metadata: + version: "0.1.0" +parent: databricks-core --- # Databricks Development Guide diff --git a/experimental/databricks-spark-structured-streaming/SKILL.md b/experimental/databricks-spark-structured-streaming/SKILL.md index 001298e..8ead652 100644 --- a/experimental/databricks-spark-structured-streaming/SKILL.md +++ b/experimental/databricks-spark-structured-streaming/SKILL.md @@ -1,6 +1,10 @@ --- name: databricks-spark-structured-streaming description: "Comprehensive guide to Spark Structured Streaming for production workloads. Use when building streaming pipelines, working with Kafka ingestion, implementing Real-Time Mode (RTM), configuring triggers (processingTime, availableNow), handling stateful operations with watermarks, optimizing checkpoints, performing stream-stream or stream-static joins, writing to multiple sinks, or tuning streaming cost and performance." +compatibility: Requires databricks CLI (>= v0.294.0) +metadata: + version: "0.1.0" +parent: databricks-core --- # Spark Structured Streaming diff --git a/experimental/databricks-synthetic-data-gen/SKILL.md b/experimental/databricks-synthetic-data-gen/SKILL.md index 510f576..94a9ac9 100644 --- a/experimental/databricks-synthetic-data-gen/SKILL.md +++ b/experimental/databricks-synthetic-data-gen/SKILL.md @@ -1,6 +1,10 @@ --- name: databricks-synthetic-data-gen description: "Generate realistic synthetic data using Spark + Faker (strongly recommended). Supports serverless execution, multiple output formats (Parquet/JSON/CSV/Delta), and scales from thousands to millions of rows. For small datasets (<10K rows), can optionally generate locally and upload to volumes. Use when user mentions 'synthetic data', 'test data', 'generate data', 'demo dataset', 'Faker', or 'sample data'." +compatibility: Requires databricks CLI (>= v0.294.0) +metadata: + version: "0.1.0" +parent: databricks-core --- > Catalog and schema are **always user-supplied** — never default to any value. If the user hasn't provided them, ask. For any UC write, **always create the schema if it doesn't exist** before writing data. diff --git a/experimental/databricks-unity-catalog/SKILL.md b/experimental/databricks-unity-catalog/SKILL.md index bc2a9c8..f6fea7a 100644 --- a/experimental/databricks-unity-catalog/SKILL.md +++ b/experimental/databricks-unity-catalog/SKILL.md @@ -1,6 +1,10 @@ --- name: databricks-unity-catalog description: "Unity Catalog system tables and volumes. Use when querying system tables (audit, lineage, billing) or working with volume file operations (upload, download, list files in /Volumes/)." +compatibility: Requires databricks CLI (>= v0.294.0) +metadata: + version: "0.1.0" +parent: databricks-core --- # Unity Catalog diff --git a/experimental/databricks-unstructured-pdf-generation/SKILL.md b/experimental/databricks-unstructured-pdf-generation/SKILL.md index 1a1a636..f692e56 100644 --- a/experimental/databricks-unstructured-pdf-generation/SKILL.md +++ b/experimental/databricks-unstructured-pdf-generation/SKILL.md @@ -1,6 +1,10 @@ --- name: databricks-unstructured-pdf-generation description: "Generate PDF documents from HTML and upload to Unity Catalog volumes. Use for creating test PDFs, demo documents, reports, or evaluation datasets." +compatibility: Requires databricks CLI (>= v0.294.0) +metadata: + version: "0.1.0" +parent: databricks-core --- # PDF Generation from HTML diff --git a/experimental/databricks-zerobus-ingest/SKILL.md b/experimental/databricks-zerobus-ingest/SKILL.md index 160645b..f35ad18 100644 --- a/experimental/databricks-zerobus-ingest/SKILL.md +++ b/experimental/databricks-zerobus-ingest/SKILL.md @@ -1,6 +1,10 @@ --- name: databricks-zerobus-ingest description: "Build Zerobus Ingest clients for near real-time data ingestion into Databricks Delta tables via gRPC. Use when creating producers that write directly to Unity Catalog tables without a message bus, working with the Zerobus Ingest SDK in Python/Java/Go/TypeScript/Rust, generating Protobuf schemas from UC tables, or implementing stream-based ingestion with ACK handling and retry logic." +compatibility: Requires databricks CLI (>= v0.294.0) +metadata: + version: "0.1.0" +parent: databricks-core --- # Zerobus Ingest diff --git a/experimental/spark-python-data-source/SKILL.md b/experimental/spark-python-data-source/SKILL.md index 93cddc8..5f3ecb0 100644 --- a/experimental/spark-python-data-source/SKILL.md +++ b/experimental/spark-python-data-source/SKILL.md @@ -1,6 +1,10 @@ --- name: spark-python-data-source description: Build custom Python data sources for Apache Spark using the PySpark DataSource API — batch and streaming readers/writers for external systems. Use this skill whenever someone wants to connect Spark to an external system (database, API, message queue, custom protocol), build a Spark connector or plugin in Python, implement a DataSourceReader or DataSourceWriter, pull data from or push data to a system via Spark, or work with the PySpark DataSource API in any way. Even if they just say "read from X in Spark" or "write DataFrame to Y" and there's no native connector, this skill applies. +compatibility: Requires databricks CLI (>= v0.294.0) +metadata: + version: "0.1.0" +parent: databricks-core --- # spark-python-data-source diff --git a/manifest.json b/manifest.json index e6925dc..ec2c621 100644 --- a/manifest.json +++ b/manifest.json @@ -12,7 +12,7 @@ "references/2-supervisor-agents.md" ], "repo_dir": "experimental", - "version": "0.0.1" + "version": "0.1.0" }, "databricks-ai-functions": { "description": "Use Databricks built-in AI Functions (ai_classify, ai_extract, ai_summarize, ai_mask, ai_translate, ai_fix_grammar, ai_gen, ai_analyze_sentiment, ai_similarity, ai_parse_document, ai_query, ai_forecast) to add AI capabilities directly to SQL and PySpark pipelines without managing model endpoints. Also covers document parsing and building custom RAG pipelines (parse \u2192 chunk \u2192 index \u2192 query).", @@ -27,7 +27,7 @@ "references/4-document-processing-pipeline.md" ], "repo_dir": "experimental", - "version": "0.0.1" + "version": "0.1.0" }, "databricks-aibi-dashboards": { "description": "Create Databricks AI/BI dashboards. Must use when creating, updating, or deploying Lakeview dashboards as Databricks Dashboard have a unique json structure. CRITICAL: You MUST test ALL SQL queries via CLI BEFORE deploying. Follow guidelines strictly.", @@ -43,7 +43,7 @@ "references/5-troubleshooting.md" ], "repo_dir": "experimental", - "version": "0.0.1" + "version": "0.1.0" }, "databricks-apps": { "description": "Databricks Apps development and deployment (evaluates analytics vs synced tables data access)", @@ -90,7 +90,7 @@ "references/6-cli-approach.md" ], "repo_dir": "experimental", - "version": "0.0.1" + "version": "0.1.0" }, "databricks-core": { "description": "Core Databricks skill for CLI, auth, and data exploration", @@ -136,7 +136,7 @@ "references/sql-scripting.md" ], "repo_dir": "experimental", - "version": "0.0.1" + "version": "0.1.0" }, "databricks-docs": { "description": "Databricks documentation reference via llms.txt index. Use when other skills do not cover a topic, looking up unfamiliar Databricks features, or needing authoritative docs on APIs, configurations, or platform capabilities.", @@ -147,7 +147,7 @@ "assets/databricks.svg" ], "repo_dir": "experimental", - "version": "0.0.1" + "version": "0.1.0" }, "databricks-execution-compute": { "description": "Execute code and manage compute on Databricks. Use this skill when the user mentions: \"run code\", \"execute\", \"run on databricks\", \"serverless\", \"no cluster\", \"run python\", \"run scala\", \"run sql\", \"run R\", \"run file\", \"push and run\", \"notebook run\", \"batch script\", \"model training\", \"run script on cluster\", \"create cluster\", \"new cluster\", \"resize cluster\", \"modify cluster\", \"delete cluster\", \"terminate cluster\", \"create warehouse\", \"new warehouse\", \"resize warehouse\", \"delete warehouse\", \"node types\", \"runtime versions\", \"DBR versions\", \"spin up compute\", \"provision cluster\".", @@ -162,7 +162,7 @@ "scripts/compute.py" ], "repo_dir": "experimental", - "version": "0.0.1" + "version": "0.1.0" }, "databricks-iceberg": { "description": "Apache Iceberg tables on Databricks \u2014 Managed Iceberg tables, External Iceberg Reads (fka Uniform), Compatibility Mode, Iceberg REST Catalog (IRC), Iceberg v3, Snowflake interop, PyIceberg, OSS Spark, external engine access and credential vending. Use when creating Iceberg tables, enabling External Iceberg Reads (uniform) on Delta tables (including Streaming Tables and Materialized Views via compatibility mode), configuring external engines to read Databricks tables via Unity Catalog IRC, integrating with Snowflake catalog to read Foreign Iceberg tables", @@ -178,7 +178,7 @@ "references/5-external-engine-interop.md" ], "repo_dir": "experimental", - "version": "0.0.1" + "version": "0.1.0" }, "databricks-jobs": { "description": "Develop and deploy Lakeflow Jobs on Databricks via DABs, Python SDK, or the CLI \u2014 covers all task types, triggers, notifications, and worked examples", @@ -224,7 +224,7 @@ "references/yaml-reference.md" ], "repo_dir": "experimental", - "version": "0.0.1" + "version": "0.1.0" }, "databricks-mlflow-evaluation": { "description": "MLflow 3 GenAI agent evaluation. Use when writing mlflow.genai.evaluate() code, creating @scorer functions, using built-in scorers (Guidelines, Correctness, Safety, RetrievalGroundedness), building eval datasets from traces, setting up trace ingestion and production monitoring, aligning judges with MemAlign from domain expert feedback, or running optimize_prompts() with GEPA for automated prompt improvement.", @@ -246,7 +246,7 @@ "references/user-journeys.md" ], "repo_dir": "experimental", - "version": "0.0.1" + "version": "0.1.0" }, "databricks-model-serving": { "description": "Databricks Model Serving endpoint management", @@ -327,7 +327,7 @@ "references/doc-index.md" ], "repo_dir": "experimental", - "version": "0.0.1" + "version": "0.1.0" }, "databricks-serverless-migration": { "description": "Migrate Databricks workloads from classic compute to serverless compute, including compatibility checks and concrete fixes", @@ -363,7 +363,7 @@ "references/trigger-and-cost-optimization.md" ], "repo_dir": "experimental", - "version": "0.0.1" + "version": "0.1.0" }, "databricks-synthetic-data-gen": { "description": "Generate realistic synthetic data using Spark + Faker (strongly recommended). Supports serverless execution, multiple output formats (Parquet/JSON/CSV/Delta), and scales from thousands to millions of rows. For small datasets (<10K rows), can optionally generate locally and upload to volumes. Use when user mentions 'synthetic data', 'test data', 'generate data', 'demo dataset', 'Faker', or 'sample data'.", @@ -377,7 +377,7 @@ "scripts/generate_synthetic_data.py" ], "repo_dir": "experimental", - "version": "0.0.1" + "version": "0.1.0" }, "databricks-unity-catalog": { "description": "Unity Catalog system tables and volumes. Use when querying system tables (audit, lineage, billing) or working with volume file operations (upload, download, list files in /Volumes/).", @@ -391,7 +391,7 @@ "references/7-data-profiling.md" ], "repo_dir": "experimental", - "version": "0.0.1" + "version": "0.1.0" }, "databricks-unstructured-pdf-generation": { "description": "Generate PDF documents from HTML and upload to Unity Catalog volumes. Use for creating test PDFs, demo documents, reports, or evaluation datasets.", @@ -403,7 +403,7 @@ "scripts/pdf_generator.py" ], "repo_dir": "experimental", - "version": "0.0.1" + "version": "0.1.0" }, "databricks-vector-search": { "description": "Patterns for Databricks Vector Search: create endpoints and indexes, query with filters, manage embeddings. Use when building RAG applications, semantic search, or similarity matching. Covers both storage-optimized and standard endpoints.", @@ -434,7 +434,7 @@ "references/5-operations-and-limits.md" ], "repo_dir": "experimental", - "version": "0.0.1" + "version": "0.1.0" }, "spark-python-data-source": { "description": "Build custom Python data sources for Apache Spark using the PySpark DataSource API \u2014 batch and streaming readers/writers for external systems. Use this skill whenever someone wants to connect Spark to an external system (database, API, message queue, custom protocol), build a Spark connector or plugin in Python, implement a DataSourceReader or DataSourceWriter, pull data from or push data to a system via Spark, or work with the PySpark DataSource API in any way. Even if they just say \"read from X in Spark\" or \"write DataFrame to Y\" and there's no native connector, this skill applies.", @@ -453,7 +453,7 @@ "references/type-conversion.md" ], "repo_dir": "experimental", - "version": "0.0.1" + "version": "0.1.0" } }, "version": "2" From 624bc072beb62f57917bc8498a030d45c33b76ae Mon Sep 17 00:00:00 2001 From: James Broadhead Date: Thu, 28 May 2026 11:14:25 +0000 Subject: [PATCH 2/2] fix(experimental): bump CLI floor to v1.0.0, drop parent for non-core skills - compatibility floor bumped from v0.294.0 to v1.0.0 across all 17 experimental SKILL.md files. Several skills depend on the top-level 'databricks aitools install' surface promoted in cli#4917 (v1.0.0+); v0.294.0 understated the actual floor. - parent: databricks-core removed from spark-python-data-source (OSS Spark, not Databricks-specific) and databricks-zerobus-ingest (gRPC, different auth model). Over-couples the hierarchy otherwise. Co-authored-by: Isaac --- experimental/databricks-agent-bricks/SKILL.md | 2 +- experimental/databricks-ai-functions/SKILL.md | 2 +- experimental/databricks-aibi-dashboards/SKILL.md | 2 +- experimental/databricks-apps-python/SKILL.md | 2 +- experimental/databricks-dbsql/SKILL.md | 2 +- experimental/databricks-docs/SKILL.md | 2 +- experimental/databricks-execution-compute/SKILL.md | 2 +- experimental/databricks-iceberg/SKILL.md | 2 +- experimental/databricks-metric-views/SKILL.md | 2 +- experimental/databricks-mlflow-evaluation/SKILL.md | 2 +- experimental/databricks-python-sdk/SKILL.md | 2 +- experimental/databricks-spark-structured-streaming/SKILL.md | 2 +- experimental/databricks-synthetic-data-gen/SKILL.md | 2 +- experimental/databricks-unity-catalog/SKILL.md | 2 +- experimental/databricks-unstructured-pdf-generation/SKILL.md | 2 +- experimental/databricks-zerobus-ingest/SKILL.md | 3 +-- experimental/spark-python-data-source/SKILL.md | 3 +-- 17 files changed, 17 insertions(+), 19 deletions(-) diff --git a/experimental/databricks-agent-bricks/SKILL.md b/experimental/databricks-agent-bricks/SKILL.md index 4147d0e..d192cd2 100644 --- a/experimental/databricks-agent-bricks/SKILL.md +++ b/experimental/databricks-agent-bricks/SKILL.md @@ -1,7 +1,7 @@ --- name: databricks-agent-bricks description: "Create Agent Bricks: Knowledge Assistants (KA) for document Q&A and Supervisor Agents for multi-agent orchestration (MAS)." -compatibility: Requires databricks CLI (>= v0.294.0) +compatibility: Requires databricks CLI (>= v1.0.0) metadata: version: "0.1.0" parent: databricks-core diff --git a/experimental/databricks-ai-functions/SKILL.md b/experimental/databricks-ai-functions/SKILL.md index 29fda1c..fbfc3e3 100644 --- a/experimental/databricks-ai-functions/SKILL.md +++ b/experimental/databricks-ai-functions/SKILL.md @@ -1,7 +1,7 @@ --- name: databricks-ai-functions description: "Use Databricks built-in AI Functions (ai_classify, ai_extract, ai_summarize, ai_mask, ai_translate, ai_fix_grammar, ai_gen, ai_analyze_sentiment, ai_similarity, ai_parse_document, ai_query, ai_forecast) to add AI capabilities directly to SQL and PySpark pipelines without managing model endpoints. Also covers document parsing and building custom RAG pipelines (parse → chunk → index → query)." -compatibility: Requires databricks CLI (>= v0.294.0) +compatibility: Requires databricks CLI (>= v1.0.0) metadata: version: "0.1.0" parent: databricks-core diff --git a/experimental/databricks-aibi-dashboards/SKILL.md b/experimental/databricks-aibi-dashboards/SKILL.md index 0c4c623..9145891 100644 --- a/experimental/databricks-aibi-dashboards/SKILL.md +++ b/experimental/databricks-aibi-dashboards/SKILL.md @@ -1,7 +1,7 @@ --- name: databricks-aibi-dashboards description: "Create Databricks AI/BI dashboards. Must use when creating, updating, or deploying Lakeview dashboards as Databricks Dashboard have a unique json structure. CRITICAL: You MUST test ALL SQL queries via CLI BEFORE deploying. Follow guidelines strictly." -compatibility: Requires databricks CLI (>= v0.294.0) +compatibility: Requires databricks CLI (>= v1.0.0) metadata: version: "0.1.0" parent: databricks-core diff --git a/experimental/databricks-apps-python/SKILL.md b/experimental/databricks-apps-python/SKILL.md index 601997c..7070b30 100644 --- a/experimental/databricks-apps-python/SKILL.md +++ b/experimental/databricks-apps-python/SKILL.md @@ -1,7 +1,7 @@ --- name: databricks-apps-python description: "Builds Databricks applications. Prefers AppKit (TypeScript + React SDK) for new apps; falls back to Python frameworks (Dash, Streamlit, Gradio, Flask, FastAPI, Reflex) when Python is required. Handles OAuth authorization, app resources, SQL warehouse and Lakebase connectivity, model serving, foundation model APIs, and deployment. Use when building web apps, dashboards, ML demos, or REST APIs for Databricks, or when the user mentions AppKit, Streamlit, Dash, Gradio, Flask, FastAPI, Reflex, or Databricks app." -compatibility: Requires databricks CLI (>= v0.294.0) +compatibility: Requires databricks CLI (>= v1.0.0) metadata: version: "0.1.0" parent: databricks-core diff --git a/experimental/databricks-dbsql/SKILL.md b/experimental/databricks-dbsql/SKILL.md index 451cd58..f68a2c4 100644 --- a/experimental/databricks-dbsql/SKILL.md +++ b/experimental/databricks-dbsql/SKILL.md @@ -12,7 +12,7 @@ description: >- "temp table", "temporary view", "pipe operator". SHOULD also invoke when the user asks about SQL best practices, data modeling patterns, or advanced SQL features on Databricks. -compatibility: Requires databricks CLI (>= v0.294.0) +compatibility: Requires databricks CLI (>= v1.0.0) metadata: version: "0.1.0" parent: databricks-core diff --git a/experimental/databricks-docs/SKILL.md b/experimental/databricks-docs/SKILL.md index 3bd1017..6f7f5c5 100644 --- a/experimental/databricks-docs/SKILL.md +++ b/experimental/databricks-docs/SKILL.md @@ -1,7 +1,7 @@ --- name: databricks-docs description: "Databricks documentation reference via llms.txt index. Use when other skills do not cover a topic, looking up unfamiliar Databricks features, or needing authoritative docs on APIs, configurations, or platform capabilities." -compatibility: Requires databricks CLI (>= v0.294.0) +compatibility: Requires databricks CLI (>= v1.0.0) metadata: version: "0.1.0" parent: databricks-core diff --git a/experimental/databricks-execution-compute/SKILL.md b/experimental/databricks-execution-compute/SKILL.md index a83f1be..3fb3f80 100644 --- a/experimental/databricks-execution-compute/SKILL.md +++ b/experimental/databricks-execution-compute/SKILL.md @@ -9,7 +9,7 @@ description: >- "delete cluster", "terminate cluster", "create warehouse", "new warehouse", "resize warehouse", "delete warehouse", "node types", "runtime versions", "DBR versions", "spin up compute", "provision cluster". -compatibility: Requires databricks CLI (>= v0.294.0) +compatibility: Requires databricks CLI (>= v1.0.0) metadata: version: "0.1.0" parent: databricks-core diff --git a/experimental/databricks-iceberg/SKILL.md b/experimental/databricks-iceberg/SKILL.md index 9b2e241..eb442be 100644 --- a/experimental/databricks-iceberg/SKILL.md +++ b/experimental/databricks-iceberg/SKILL.md @@ -1,7 +1,7 @@ --- name: databricks-iceberg description: "Apache Iceberg tables on Databricks — Managed Iceberg tables, External Iceberg Reads (fka Uniform), Compatibility Mode, Iceberg REST Catalog (IRC), Iceberg v3, Snowflake interop, PyIceberg, OSS Spark, external engine access and credential vending. Use when creating Iceberg tables, enabling External Iceberg Reads (uniform) on Delta tables (including Streaming Tables and Materialized Views via compatibility mode), configuring external engines to read Databricks tables via Unity Catalog IRC, integrating with Snowflake catalog to read Foreign Iceberg tables" -compatibility: Requires databricks CLI (>= v0.294.0) +compatibility: Requires databricks CLI (>= v1.0.0) metadata: version: "0.1.0" parent: databricks-core diff --git a/experimental/databricks-metric-views/SKILL.md b/experimental/databricks-metric-views/SKILL.md index 97d7d04..6602910 100644 --- a/experimental/databricks-metric-views/SKILL.md +++ b/experimental/databricks-metric-views/SKILL.md @@ -1,7 +1,7 @@ --- name: databricks-metric-views description: "Unity Catalog metric views: define, create, query, and manage governed business metrics in YAML. Use when building standardized KPIs, revenue metrics, order analytics, or any reusable business metrics that need consistent definitions across teams and tools." -compatibility: Requires databricks CLI (>= v0.294.0) +compatibility: Requires databricks CLI (>= v1.0.0) metadata: version: "0.1.0" parent: databricks-core diff --git a/experimental/databricks-mlflow-evaluation/SKILL.md b/experimental/databricks-mlflow-evaluation/SKILL.md index 7248727..0b91285 100644 --- a/experimental/databricks-mlflow-evaluation/SKILL.md +++ b/experimental/databricks-mlflow-evaluation/SKILL.md @@ -1,7 +1,7 @@ --- name: databricks-mlflow-evaluation description: "MLflow 3 GenAI agent evaluation. Use when writing mlflow.genai.evaluate() code, creating @scorer functions, using built-in scorers (Guidelines, Correctness, Safety, RetrievalGroundedness), building eval datasets from traces, setting up trace ingestion and production monitoring, aligning judges with MemAlign from domain expert feedback, or running optimize_prompts() with GEPA for automated prompt improvement." -compatibility: Requires databricks CLI (>= v0.294.0) +compatibility: Requires databricks CLI (>= v1.0.0) metadata: version: "0.1.0" parent: databricks-core diff --git a/experimental/databricks-python-sdk/SKILL.md b/experimental/databricks-python-sdk/SKILL.md index be53f53..5f1d8cf 100644 --- a/experimental/databricks-python-sdk/SKILL.md +++ b/experimental/databricks-python-sdk/SKILL.md @@ -1,7 +1,7 @@ --- name: databricks-python-sdk description: "Databricks development guidance including Python SDK, Databricks Connect, CLI, and REST API. Use when working with databricks-sdk, databricks-connect, or Databricks APIs." -compatibility: Requires databricks CLI (>= v0.294.0) +compatibility: Requires databricks CLI (>= v1.0.0) metadata: version: "0.1.0" parent: databricks-core diff --git a/experimental/databricks-spark-structured-streaming/SKILL.md b/experimental/databricks-spark-structured-streaming/SKILL.md index 8ead652..a85e9f2 100644 --- a/experimental/databricks-spark-structured-streaming/SKILL.md +++ b/experimental/databricks-spark-structured-streaming/SKILL.md @@ -1,7 +1,7 @@ --- name: databricks-spark-structured-streaming description: "Comprehensive guide to Spark Structured Streaming for production workloads. Use when building streaming pipelines, working with Kafka ingestion, implementing Real-Time Mode (RTM), configuring triggers (processingTime, availableNow), handling stateful operations with watermarks, optimizing checkpoints, performing stream-stream or stream-static joins, writing to multiple sinks, or tuning streaming cost and performance." -compatibility: Requires databricks CLI (>= v0.294.0) +compatibility: Requires databricks CLI (>= v1.0.0) metadata: version: "0.1.0" parent: databricks-core diff --git a/experimental/databricks-synthetic-data-gen/SKILL.md b/experimental/databricks-synthetic-data-gen/SKILL.md index 94a9ac9..f534f60 100644 --- a/experimental/databricks-synthetic-data-gen/SKILL.md +++ b/experimental/databricks-synthetic-data-gen/SKILL.md @@ -1,7 +1,7 @@ --- name: databricks-synthetic-data-gen description: "Generate realistic synthetic data using Spark + Faker (strongly recommended). Supports serverless execution, multiple output formats (Parquet/JSON/CSV/Delta), and scales from thousands to millions of rows. For small datasets (<10K rows), can optionally generate locally and upload to volumes. Use when user mentions 'synthetic data', 'test data', 'generate data', 'demo dataset', 'Faker', or 'sample data'." -compatibility: Requires databricks CLI (>= v0.294.0) +compatibility: Requires databricks CLI (>= v1.0.0) metadata: version: "0.1.0" parent: databricks-core diff --git a/experimental/databricks-unity-catalog/SKILL.md b/experimental/databricks-unity-catalog/SKILL.md index f6fea7a..bacfed4 100644 --- a/experimental/databricks-unity-catalog/SKILL.md +++ b/experimental/databricks-unity-catalog/SKILL.md @@ -1,7 +1,7 @@ --- name: databricks-unity-catalog description: "Unity Catalog system tables and volumes. Use when querying system tables (audit, lineage, billing) or working with volume file operations (upload, download, list files in /Volumes/)." -compatibility: Requires databricks CLI (>= v0.294.0) +compatibility: Requires databricks CLI (>= v1.0.0) metadata: version: "0.1.0" parent: databricks-core diff --git a/experimental/databricks-unstructured-pdf-generation/SKILL.md b/experimental/databricks-unstructured-pdf-generation/SKILL.md index f692e56..191c65d 100644 --- a/experimental/databricks-unstructured-pdf-generation/SKILL.md +++ b/experimental/databricks-unstructured-pdf-generation/SKILL.md @@ -1,7 +1,7 @@ --- name: databricks-unstructured-pdf-generation description: "Generate PDF documents from HTML and upload to Unity Catalog volumes. Use for creating test PDFs, demo documents, reports, or evaluation datasets." -compatibility: Requires databricks CLI (>= v0.294.0) +compatibility: Requires databricks CLI (>= v1.0.0) metadata: version: "0.1.0" parent: databricks-core diff --git a/experimental/databricks-zerobus-ingest/SKILL.md b/experimental/databricks-zerobus-ingest/SKILL.md index f35ad18..694ef00 100644 --- a/experimental/databricks-zerobus-ingest/SKILL.md +++ b/experimental/databricks-zerobus-ingest/SKILL.md @@ -1,10 +1,9 @@ --- name: databricks-zerobus-ingest description: "Build Zerobus Ingest clients for near real-time data ingestion into Databricks Delta tables via gRPC. Use when creating producers that write directly to Unity Catalog tables without a message bus, working with the Zerobus Ingest SDK in Python/Java/Go/TypeScript/Rust, generating Protobuf schemas from UC tables, or implementing stream-based ingestion with ACK handling and retry logic." -compatibility: Requires databricks CLI (>= v0.294.0) +compatibility: Requires databricks CLI (>= v1.0.0) metadata: version: "0.1.0" -parent: databricks-core --- # Zerobus Ingest diff --git a/experimental/spark-python-data-source/SKILL.md b/experimental/spark-python-data-source/SKILL.md index 5f3ecb0..0e43bc2 100644 --- a/experimental/spark-python-data-source/SKILL.md +++ b/experimental/spark-python-data-source/SKILL.md @@ -1,10 +1,9 @@ --- name: spark-python-data-source description: Build custom Python data sources for Apache Spark using the PySpark DataSource API — batch and streaming readers/writers for external systems. Use this skill whenever someone wants to connect Spark to an external system (database, API, message queue, custom protocol), build a Spark connector or plugin in Python, implement a DataSourceReader or DataSourceWriter, pull data from or push data to a system via Spark, or work with the PySpark DataSource API in any way. Even if they just say "read from X in Spark" or "write DataFrame to Y" and there's no native connector, this skill applies. -compatibility: Requires databricks CLI (>= v0.294.0) +compatibility: Requires databricks CLI (>= v1.0.0) metadata: version: "0.1.0" -parent: databricks-core --- # spark-python-data-source