Extension metadata capability

[wbreza]

Overview

This specification defines a new extension framework capability that enables azd to discover and cache comprehensive metadata about an extension's commands, configuration, and capabilities without requiring the extension to be running.

Problem Statement

Today, azd has limited visibility into extension capabilities:

• Only knows about the extension's entrypoint
• Only knows about special well-known commands (listen, mcp serve)
• Cannot discover the full command tree
• Cannot discover what configuration options an extension exposes
• Cannot provide rich IntelliSense, validation, or documentation for extension commands

This limits the ability to:

• Provide help and documentation for extension commands
• Validate command usage before execution
• Generate configuration schema validation
• Enable rich editor experiences (IntelliSense, autocomplete)
• Generate comprehensive CLI documentation
• Track extension command usage through telemetry with proper command structure knowledge

Goals

1. Command Discovery: Enable azd to discover the complete command tree of an extension
2. Command Metadata: Expose detailed metadata about each command (arguments, flags, parameters, help text)
3. Configuration Schema: Expose configuration options supported by the extension at global/project/service levels
4. Post-Installation Caching: Retrieve and cache metadata after extension installation
5. Portable Format: Use JSON-based format compatible with CLI frameworks (Cobra, Clap, etc.)
6. Telemetry Tracking: Enable azd to track extension command usage through telemetry by knowing the complete command structure, allowing for accurate usage analytics and insights into which extension commands are being used

Non-Goals

• Real-time metadata updates (cached metadata is sufficient)
• Runtime command validation (extensions are responsible for their own validation)
• Automatic command registration (extensions still use their own CLI framework)

Design

1. Extension Capability Declaration

Extensions that support metadata introspection must declare the metadata capability in their extension.yaml:

id: microsoft.azd.example
version: 1.0.0
capabilities:
  - custom-commands
  - metadata  # New capability

2. Metadata Command

Extensions supporting this capability must implement a hidden metadata command that outputs metadata in JSON format:

# azd calls this command to retrieve metadata
<extension-binary> metadata

Command Characteristics:

• Hidden from help output (not user-facing)
• Returns JSON to stdout
• Exit code 0 on success
• Must execute quickly (< 2 seconds)
• No side effects (read-only operation)

3. Metadata Schema

The metadata command returns a JSON object with the following structure:

interface ExtensionMetadata {
  // Schema version for forward compatibility
  schemaVersion: "1.0";
  
  // Basic extension info (should match extension.yaml)
  id: string;
  version: string;
  
  // Complete command tree
  commands: Command[];
  
  // Configuration schema
  configuration?: ConfigurationSchema;
}

interface Command {
  // Command path (e.g., ["demo", "context"] for "azd demo context")
  name: string[];
  
  // Short description (one line)
  short: string;
  
  // Long description (multi-line, markdown supported)
  long?: string;
  
  // Usage template (e.g., "azd demo context [flags]")
  usage?: string;
  
  // Example commands
  examples?: Example[];
  
  // Command arguments
  args?: Argument[];
  
  // Command flags
  flags?: Flag[];
  
  // Subcommands
  subcommands?: Command[];
  
  // Whether command is hidden from help
  hidden?: boolean;
  
  // Aliases for the command
  aliases?: string[];
  
  // Deprecation notice
  deprecated?: string;
}

interface Argument {
  // Argument name
  name: string;
  
  // Description
  description: string;
  
  // Whether argument is required
  required: boolean;
  
  // Whether argument accepts multiple values
  variadic?: boolean;
  
  // Valid values (for enum-like arguments)
  validValues?: string[];
}

interface Flag {
  // Flag name (without dashes, e.g., "verbose")
  name: string;
  
  // Short flag (single character, e.g., "v")
  shorthand?: string;
  
  // Description
  description: string;
  
  // Flag type
  type: "string" | "bool" | "int" | "stringArray" | "intArray";
  
  // Default value
  default?: any;
  
  // Whether flag is required
  required?: boolean;
  
  // Valid values (for enum-like flags)
  validValues?: string[];
  
  // Whether flag is hidden from help
  hidden?: boolean;
  
  // Deprecation notice
  deprecated?: string;
}

interface Example {
  // Example description
  description: string;
  
  // Example command
  command: string;
}

interface ConfigurationSchema {
  // Global-level configuration (user config)
  global?: ConfigSection;
  
  // Project-level configuration (azure.yaml)
  project?: ConfigSection;
  
  // Service-level configuration (azure.yaml services)
  service?: ConfigSection;
}

interface ConfigSection {
  // JSON Schema describing the configuration
  // This allows for nested objects, arrays, validation rules, etc.
  schema: object; // JSON Schema Draft 7
  
  // Example configuration
  example?: object;
  
  // Description of this configuration section
  description?: string;
}

4. Example Metadata Output

{
  "schemaVersion": "1.0",
  "id": "microsoft.azd.demo",
  "version": "0.1.0",
  "commands": [
    {
      "name": ["demo"],
      "short": "Demo extension commands",
      "long": "Provides demonstration commands for the azd extension framework.",
      "subcommands": [
        {
          "name": ["demo", "context"],
          "short": "Display current azd context",
          "long": "Shows the current project and environment context including project name, environment name, and configuration values.",
          "usage": "azd demo context [flags]",
          "examples": [
            {
              "description": "Show current context",
              "command": "azd demo context"
            },
            {
              "description": "Show context with verbose output",
              "command": "azd demo context --verbose"
            }
          ],
          "flags": [
            {
              "name": "verbose",
              "shorthand": "v",
              "description": "Show verbose output",
              "type": "bool",
              "default": false
            },
            {
              "name": "output",
              "shorthand": "o",
              "description": "Output format",
              "type": "string",
              "default": "table",
              "validValues": ["json", "table", "yaml"]
            }
          ]
        },
        {
          "name": ["demo", "deploy"],
          "short": "Deploy using custom target",
          "long": "Deploys the application using the demo custom service target provider.",
          "usage": "azd demo deploy <service-name> [flags]",
          "args": [
            {
              "name": "service-name",
              "description": "Name of the service to deploy",
              "required": true
            }
          ],
          "flags": [
            {
              "name": "target",
              "description": "Deployment target",
              "type": "string",
              "default": "vm",
              "validValues": ["vm", "container"]
            }
          ]
        }
      ]
    }
  ],
  "configuration": {
    "global": {
      "description": "Global configuration for the demo extension",
      "schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "type": "object",
        "properties": {
          "ext": {
            "type": "object",
            "properties": {
              "demo": {
                "type": "object",
                "properties": {
                  "defaultVerbose": {
                    "type": "boolean",
                    "description": "Default verbose mode for all demo commands",
                    "default": false
                  },
                  "customEndpoint": {
                    "type": "string",
                    "description": "Custom API endpoint for demo operations",
                    "format": "uri"
                  }
                }
              }
            }
          }
        }
      },
      "example": {
        "ext": {
          "demo": {
            "defaultVerbose": true,
            "customEndpoint": "https://api.example.com"
          }
        }
      }
    },
    "project": {
      "description": "Project-level configuration (azure.yaml)",
      "schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "type": "object",
        "properties": {
          "demo": {
            "type": "object",
            "properties": {
              "deploymentMode": {
                "type": "string",
                "enum": ["fast", "safe"],
                "description": "Deployment mode for demo services"
              }
            }
          }
        }
      },
      "example": {
        "demo": {
          "deploymentMode": "safe"
        }
      }
    },
    "service": {
      "description": "Service-level configuration (azure.yaml services)",
      "schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "type": "object",
        "properties": {
          "demo": {
            "type": "object",
            "properties": {
              "vmSize": {
                "type": "string",
                "description": "VM size for service deployment",
                "default": "Standard_B2s"
              },
              "enableMonitoring": {
                "type": "boolean",
                "description": "Enable monitoring for this service",
                "default": true
              }
            }
          }
        }
      },
      "example": {
        "demo": {
          "vmSize": "Standard_D4s_v3",
          "enableMonitoring": true
        }
      }
    }
  }
}

5. Implementation Flow

┌─────────────────────────────────────────────────────────────┐
│ Extension Installation                                       │
└─────────────────────────────────────────────────────────────┘
                           │
                           ▼
┌─────────────────────────────────────────────────────────────┐
│ azd checks extension.yaml for "metadata" capability         │
│                                                              │
└─────────────────────────────────────────────────────────────┘
                           │
                           ▼
                    [Supports metadata?]
                           │
                ┌──────────┴──────────┐
                │                     │
               Yes                   No
                │                     │
                ▼                     ▼
┌───────────────────────────┐  [Skip metadata collection]
│ Execute:                  │
│ <extension> metadata      │
└───────────────────────────┘
                │
                ▼
┌───────────────────────────┐
│ Parse JSON output         │
└───────────────────────────┘
                │
                ▼
┌───────────────────────────┐
│ Cache metadata in:        │
│ ~/.azd/extensions/        │
│   <ext-id>/               │
│     metadata.json         │
└───────────────────────────┘
                │
                ▼
┌───────────────────────────┐
│ Use cached metadata for:  │
│ - Help generation         │
│ - IntelliSense            │
│ - Validation              │
│ - Documentation           │
│ - Telemetry tracking      │
└───────────────────────────┘

6. Metadata Cache Location

Metadata is cached at:

~/.azd/extensions/<extension-id>/metadata.json

Cache Invalidation:

• Cleared when extension is uninstalled
• Refreshed when extension is upgraded
• Can be manually refreshed with azd extension refresh <extension-id>

References

• Extension Framework Documentation
• Extension Schema
• Cobra Command Structure
• JSON Schema Draft 7

[JeffreyCA]

What are your thoughts on a variation of this where the metadata.json gets bundled inside the extension artifact (.tar.gz/.zip), and as part of azd x pack azd could invoke the metadata command before generating the artifact? That way we wouldn't need the caching/refresh logic.

This could also make implementing a metadata subcommand optional for extension devs, since as long as a metadata.json exists, azd would just use that. Extension authors would be free to control how the metadata.json gets generated: it could be handwritten or generated by a script or metadata command.

Or maybe extension authors can modify their build.sh/build.ps1 to generate the metadata.json however they want, which azd x pack copies into the .tar.gz/.zip.

[wbreza]

What are your thoughts on a variation of this where the metadata.json gets bundled inside the extension artifact (.tar.gz/.zip), and as part of azd x pack azd could invoke the metadata command before generating the artifact? That way we wouldn't need the caching/refresh logic.

This could also make implementing a metadata subcommand optional for extension devs, since as long as a metadata.json exists, azd would just use that. Extension authors would be free to control how the metadata.json gets generated: it could be handwritten or generated by a script or metadata command.

Or maybe extension authors can modify their build.sh/build.ps1 to generate the metadata.json however they want, which azd x pack copies into the .tar.gz/.zip.

This is a great suggestion. I think integrating this into the azd x pack experience might be the way to go but still generated from within the extension itself.