skills: split into databricks-model-serving (ops) + databricks-ml-training (experimental)

[QuentinAmbard]

Stacks on top of #84. Merge #84 first; this PR includes those commits at the base and adds the new skill on top.

Why this PR exists

PR #84 lands the model-serving content (endpoint create, query, update, traffic config, AI Gateway, Foundation Model API discovery) into databricks-model-serving. That's the right shape for a serving-ops skill, and it's what reviewers should expect a skill called "model serving" to contain.

The remaining a-d-k content — training a model with MLflow autolog, registering it to Unity Catalog, promoting versions via @prod aliases, custom PyFunc authoring, hand-rolled ResponsesAgent code — is a different lifecycle. It runs before an endpoint exists, often in a notebook submitted as a serverless job, and an agent asked to "train an XGBoost model and deploy it" needs both concerns surfaced cleanly rather than blended into one skill description.

This PR lands the dev-side content as a separate databricks-ml-training experimental skill, and weaves a few small but high-leverage serving-side fixes from the original a-d-k content into databricks-model-serving where they belong.

What this PR improves

A focused dev-side skill. New experimental/databricks-ml-training/ owns the training → register → consume narrative: MLflow autolog with Optuna for hyperparameter tuning, mlflow.set_registry_uri(\"databricks-uc\") + experiment-parent-folder pre-creation, alias-based promotion (@prod / @challenger), batch scoring via mlflow.pyfunc.spark_udf, custom PyFunc with the file-based "Models from Code" pattern, hand-rolled ResponsesAgent with LangGraph + UC Function + Vector Search tools, and the databricks jobs submit --no-wait train-and-deploy pattern.

Frontmatter triggers that actually triage. Each skill's description lists what it IS for and what it explicitly is NOT for, with cross-pointers (databricks-ml-training says "use databricks-model-serving for endpoint ops"; databricks-model-serving says "use databricks-ml-training for training and PyFunc authoring"). When the user says "train a model and deploy it," the orchestrator pulls both skills exactly once each.

Cross-skill links that resolve. Every databricks-model-serving → databricks-ml-training link and every reverse link uses the right relative path for the stable-skills/ ↔ experimental-experimental/ layout. No broken anchors, no stale paths to the old training-and-serving.md filename anywhere.

Five small but high-leverage gaps closed in databricks-model-serving. The original a-d-k port left a few non-obvious serving behaviors implicit. Each fix is woven into the existing section that already covers the topic — no new mega-sections, no duplication of MLflow boilerplate an LLM already knows from training data. The result: serving-side behavior an agent would otherwise have to guess at is now explicit and signposted.

Summary of changes

Area	What changed
New experimental skill	experimental/databricks-ml-training/ with SKILL.md + agents/ + assets/ + references/{custom-pyfunc.md, genai-agents.md}. Owns the full dev-side narrative (autolog + Optuna, UC registration, alias promotion, batch scoring, custom PyFunc, custom ResponsesAgent, train-and-deploy serverless job pattern).
Frontmatter scoping	Both skills' descriptions list scope + explicit NOT-for callouts pointing at their sibling. Triggers triage cleanly when the user mentions both training and deployment.
Cross-links	All cross-skill paths fixed for the stable ↔ experimental layout. Foundation Model API discovery moved into databricks-model-serving/SKILL.md inline (was previously linked into the relocated training file).
Serving gaps closed	Five small fixes in databricks-model-serving/SKILL.md: MLflow Deployments client gotchas (tags= top-level, served_model_name derivation), zero-downtime version-swap pattern (alias-repoint AND update_endpoint), two-state-field readiness rationale (state.ready lies during version-swap), classical-ML dataframe_records query example, Serving-UI "Owned by me" SP-filter troubleshooting row. Each merged into the existing section that already covered the topic.
DAS-only content preserved	PR #84's idempotency check before agent deploy, AppKit integration section, off-platform AI SDK v6 streaming, endpoint-structure ASCII diagram, OpenAPI schema section — all kept.
Versioning	databricks-model-serving bumped to 0.4.0 (description retightened, gaps closed). New databricks-ml-training at 0.1.0 under experimental/. Manifest regenerated; 27 skills total.

Reviewer aid

The split is on the natural seam — anything that runs before mlflow.deployments.get_deploy_client(...).create_endpoint(...) is dev-side and lives in databricks-ml-training, anything from create_endpoint onward is ops-side and lives in databricks-model-serving. The Python create_endpoint(...) / update_endpoint(...) call itself is canonically a serving operation and is documented there with the two non-obvious gotchas.

Validation: python3 scripts/skills.py validate passes; zero broken links across both touched skills.

This pull request and its description were written by Isaac.