From 3c3f9cd1b174a157498db53b32bda2516b428953 Mon Sep 17 00:00:00 2001 From: Patrick Date: Wed, 11 Feb 2026 15:47:50 +0700 Subject: [PATCH] feat: add source discovery guidance to skills Add a source discovery phase to the investigate and query skills so they identify the correct _sourceCategory before running queries. This prevents wasting search API quota on trial-and-error discovery. Update the reference skill with new CLI flags (list-collectors -q/-l, list-sources --collector/--name/--category/-l, discover-source-metadata -k/-l) and compound relative time examples. Update the discover skill to use filtering flags for scoped discovery. Closes #14, closes #15. --- skills/sumo-query/_reference.md | 133 ++++++++++++++++++++++--------- skills/sumo-query/discover.md | 24 +++++- skills/sumo-query/investigate.md | 54 +++++++++++-- skills/sumo-query/query.md | 32 +++++++- 4 files changed, 195 insertions(+), 48 deletions(-) diff --git a/skills/sumo-query/_reference.md b/skills/sumo-query/_reference.md index 320c83f..824c7b8 100644 --- a/skills/sumo-query/_reference.md +++ b/skills/sumo-query/_reference.md @@ -20,6 +20,7 @@ If the command fails, tell the user to install it: `gem install sumologic-query` The user asked about: **$ARGUMENTS** If `$ARGUMENTS` is empty, show a brief table of contents and ask which topic they want: + - `commands` — All CLI commands and flags - `query-syntax` — Sumo Logic query language reference - `time` — Time format and timezone reference @@ -48,16 +49,17 @@ sumo-query search -q 'error' -f -1h -t now -z America/New_York ``` ### Search Flags -| Flag | Long | Required | Description | -|------|------|----------|-------------| -| `-q` | `--query` | Yes | Sumo Logic query string | -| `-f` | `--from` | Yes | Start time | -| `-t` | `--to` | Yes | End time | -| `-l` | `--limit` | No | Max results to return | -| `-a` | `--aggregate` | No | Return aggregation records | -| `-z` | `--time-zone` | No | Timezone (default: UTC) | -| `-d` | `--debug` | No | Debug output | -| `-o` | `--output` | No | Write JSON to file | + +| Flag | Long | Required | Description | +| ---- | ------------- | -------- | -------------------------- | +| `-q` | `--query` | Yes | Sumo Logic query string | +| `-f` | `--from` | Yes | Start time | +| `-t` | `--to` | Yes | End time | +| `-l` | `--limit` | No | Max results to return | +| `-a` | `--aggregate` | No | Return aggregation records | +| `-z` | `--time-zone` | No | Timezone (default: UTC) | +| `-d` | `--debug` | No | Debug output | +| `-o` | `--output` | No | Write JSON to file | ## Monitors @@ -79,11 +81,12 @@ sumo-query get-monitor --monitor-id 00000000001A2B3C ``` ### list-monitors Flags -| Flag | Long | Required | Description | -|------|------|----------|-------------| -| `-s` | `--status` | No | Filter: Normal, Critical, Warning, MissingData, Disabled, AllTriggered | -| `-q` | `--query` | No | Search name/description | -| `-l` | `--limit` | No | Max results (default: 100) | + +| Flag | Long | Required | Description | +| ---- | ---------- | -------- | ---------------------------------------------------------------------- | +| `-s` | `--status` | No | Filter: Normal, Critical, Warning, MissingData, Disabled, AllTriggered | +| `-q` | `--query` | No | Search name/description | +| `-l` | `--limit` | No | Max results (default: 100) | ## Health Events @@ -98,18 +101,56 @@ sumo-query list-health-events -l 50 # List all collectors sumo-query list-collectors +# Filter collectors by name or category +sumo-query list-collectors -q "my-service" -l 20 + # List sources for a specific collector sumo-query list-sources --collector-id 123456789 # List all sources across all collectors sumo-query list-sources +# Filter sources by collector name, source name, or category +sumo-query list-sources --collector "my-service" --name "nginx" -l 20 +sumo-query list-sources --category "production" + # Discover dynamic source metadata from logs sumo-query discover-source-metadata sumo-query discover-source-metadata --filter '_sourceCategory=*ecs*' sumo-query discover-source-metadata -f -7d -t now + +# Discover with keyword filter (matches name or category) +sumo-query discover-source-metadata -k "nginx" -l 20 ``` +### list-collectors Flags + +| Flag | Long | Required | Description | +| ---- | --------- | -------- | --------------------------------------------- | +| `-q` | `--query` | No | Filter by name or category (case-insensitive) | +| `-l` | `--limit` | No | Max results to return | + +### list-sources Flags + +| Flag | Long | Required | Description | +| ---- | ---------------- | -------- | -------------------------------------------- | +| | `--collector-id` | No | Collector ID to list sources for | +| | `--collector` | No | Filter by collector name (case-insensitive) | +| `-n` | `--name` | No | Filter by source name (case-insensitive) | +| | `--category` | No | Filter by source category (case-insensitive) | +| `-l` | `--limit` | No | Max total sources to return | + +### discover-source-metadata Flags + +| Flag | Long | Required | Description | +| ---- | ------------- | -------- | ------------------------------------------------------- | +| `-f` | `--from` | No | Start time (default: -24h) | +| `-t` | `--to` | No | End time (default: now) | +| `-z` | `--time-zone` | No | Timezone (default: UTC) | +| | `--filter` | No | Sumo Logic query filter (e.g., `_sourceCategory=*ecs*`) | +| `-k` | `--keyword` | No | Filter results by keyword (matches name or category) | +| `-l` | `--limit` | No | Max sources to return | + ## Dashboards & Folders ```bash @@ -164,32 +205,39 @@ sumo-query get-lookup --lookup-id 00000000001A2B3C # Time Format Reference ## Relative Times -| Format | Meaning | -|--------|---------| -| `-15m` | 15 minutes ago | -| `-1h` | 1 hour ago | -| `-6h` | 6 hours ago | -| `-1d` | 1 day ago | -| `-7d` | 7 days ago | -| `-1w` | 1 week ago | -| `-1M` | 1 month ago | -| `now` | Current time | + +| Format | Meaning | +| -------- | --------------------- | +| `-15m` | 15 minutes ago | +| `-1h` | 1 hour ago | +| `-1h30m` | 1 hour 30 minutes ago | +| `-6h` | 6 hours ago | +| `-1d` | 1 day ago | +| `-2d3h` | 2 days 3 hours ago | +| `-7d` | 7 days ago | +| `-1w` | 1 week ago | +| `-1M` | 1 month ago | +| `now` | Current time | + +Compound expressions (e.g., `-1h30m`, `-2d3h15m`) are supported. ## Absolute Times -| Format | Example | -|--------|---------| -| ISO 8601 | `2025-01-15T14:00:00` | -| Unix timestamp | `1700000000` | + +| Format | Example | +| -------------- | --------------------- | +| ISO 8601 | `2025-01-15T14:00:00` | +| Unix timestamp | `1700000000` | ## Timezones -| Value | Description | -|-------|-------------| -| `UTC` | Default | -| `EST` | US Eastern | -| `AEST` | Australian Eastern | -| `America/New_York` | IANA US Eastern | + +| Value | Description | +| ------------------ | ----------------------- | +| `UTC` | Default | +| `EST` | US Eastern | +| `AEST` | Australian Eastern | +| `America/New_York` | IANA US Eastern | | `Australia/Sydney` | IANA Australian Eastern | -| `+05:30` | UTC offset format | +| `+05:30` | UTC offset format | --- @@ -213,6 +261,7 @@ _collector=my-collector ``` Combine with `AND`, `OR`, `NOT`: + ``` _sourceCategory=prod AND error _sourceCategory=prod NOT "health check" @@ -231,17 +280,20 @@ error NOT "expected error" ## Parse Operators ### parse (anchor-based) + ``` | parse "status=* method=* path=*" as status, method, path ``` ### parse regex + ``` | parse regex "(?\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})" | parse regex "status=(?\d+)" ``` ### json + ``` | json "response.status" as status | json "user.name" as username @@ -249,6 +301,7 @@ error NOT "expected error" ``` ### csv / keyvalue / xml + ``` | csv _raw extract 1 as ip, 2 as method, 3 as url | keyvalue infer "=" "," @@ -258,6 +311,7 @@ error NOT "expected error" ## Filter Operators ### where + ``` | where status >= 400 | where status = 200 @@ -269,9 +323,11 @@ error NOT "expected error" ``` ### Comparison operators + `=`, `!=`, `>`, `>=`, `<`, `<=`, `in`, `matches` ### String matching + ``` | where method matches "GET*" | where url matches "*api/v2*" @@ -281,6 +337,7 @@ error NOT "expected error" ## Aggregation Operators ### count + ``` | count # total count | count by _sourceCategory # count per category @@ -289,6 +346,7 @@ error NOT "expected error" ``` ### Statistical + ``` | avg(response_time) by endpoint | sum(bytes) by _sourceHost @@ -298,6 +356,7 @@ error NOT "expected error" ``` ### first / last + ``` | first(_raw) by _sourceHost | last(message) by user @@ -315,6 +374,7 @@ error NOT "expected error" ## Time Operations ### timeslice + ``` | timeslice 1m # group by 1-minute buckets | timeslice 5m # 5-minute buckets @@ -323,6 +383,7 @@ error NOT "expected error" ``` ### formatDate + ``` | formatDate(_messageTime, "yyyy-MM-dd HH:mm:ss") as timestamp ``` diff --git a/skills/sumo-query/discover.md b/skills/sumo-query/discover.md index 8af966b..975929e 100644 --- a/skills/sumo-query/discover.md +++ b/skills/sumo-query/discover.md @@ -37,6 +37,12 @@ Supported scopes: sumo-query list-collectors ``` +If the user mentions a specific service, filter by name: + +```bash +sumo-query list-collectors -q "" -l 20 +``` + For each collector found, list its sources: ```bash @@ -53,10 +59,24 @@ List all sources and discover dynamic metadata: sumo-query list-sources ``` +To narrow scope, filter by collector name, source name, or category: + +```bash +sumo-query list-sources --collector "" --name "" -l 30 +``` + +Discover dynamic sources from log data: + ```bash sumo-query discover-source-metadata -f -7d -t now ``` +Filter discovery results by keyword: + +```bash +sumo-query discover-source-metadata -k "" -l 20 +``` + ### Scope: dashboards Run in parallel: @@ -103,8 +123,8 @@ sumo-query list-folders --tree --depth 2 **Then:** ```bash -sumo-query list-sources -sumo-query discover-source-metadata -f -7d -t now +sumo-query list-sources -l 100 +sumo-query discover-source-metadata -f -7d -t now -l 50 ``` For the first 10 collectors, also fetch their sources individually. diff --git a/skills/sumo-query/investigate.md b/skills/sumo-query/investigate.md index 4d692a8..ecf8856 100644 --- a/skills/sumo-query/investigate.md +++ b/skills/sumo-query/investigate.md @@ -48,7 +48,49 @@ Scope: Time window: to ``` -### Phase 2: Check Monitors & Health +### Phase 2: Discover Log Sources + +Before running queries, identify the correct source categories for the service. +This avoids wasting search queries on trial-and-error source discovery. + +**If the source category is already known** (user provided it, or it's obvious), skip this phase. + +**If the source is unknown**, run a discovery query: + +```bash +sumo-query search \ + -q ' | count by _sourceCategory, _sourceName | sort by _count desc' \ + -f '-1h' -t 'now' -a -l 30 +``` + +Or use the discovery command with keyword filtering: + +```bash +sumo-query discover-source-metadata -k "" -l 20 +``` + +Or filter collectors/sources directly: + +```bash +sumo-query list-collectors -q "" +sumo-query list-sources --collector "" +``` + +**Evaluate the results:** +- Prefer sources with high message counts (active) +- Ignore sources with zero or very low counts (stale) +- Note the exact `_sourceCategory` values for use in subsequent queries + +Tell the user what sources were found: + +``` +Found sources: +- _sourceCategory=production/my-app (1.2M msgs) — Active +- _sourceCategory=production/my-app-legacy (0 msgs) — Stale, skipping +Using: _sourceCategory=production/my-app +``` + +### Phase 3: Check Monitors & Health Run these in parallel to get the current alert landscape: @@ -72,7 +114,7 @@ sumo-query list-monitors -q "" -l 20 **Analyze**: Are there active alerts related to the reported problem? Note any correlated monitors. -### Phase 3: Initial Log Search +### Phase 4: Initial Log Search Based on the problem description, construct a targeted search: @@ -85,14 +127,14 @@ sumo-query search \ ``` **Guidelines for query construction:** +- Use the `_sourceCategory` discovered in Phase 2 for precise scoping - Start broad, then narrow down - Use keywords from the problem description - Include status codes, error messages, exception names as appropriate -- If unsure about the source category, search broadly first **Analyze the results**: Look for patterns, error messages, stack traces, and timestamps. -### Phase 4: Aggregate & Drill Down +### Phase 5: Aggregate & Drill Down Based on Phase 3 findings, run aggregation queries to quantify the problem: @@ -119,7 +161,7 @@ sumo-query search \ Choose the aggregation queries that make sense for the specific problem. You may run multiple queries in parallel. -### Phase 5: Correlate +### Phase 6: Correlate Look for correlation across services or time: @@ -133,7 +175,7 @@ sumo-query search \ -f -t -a -l 100 ``` -### Phase 6: Present Incident Report +### Phase 7: Present Incident Report Format findings as a structured incident report: diff --git a/skills/sumo-query/query.md b/skills/sumo-query/query.md index e973f16..7cee5b7 100644 --- a/skills/sumo-query/query.md +++ b/skills/sumo-query/query.md @@ -43,7 +43,31 @@ Break down `$ARGUMENTS` into: - **Timezone**: If mentioned (default: UTC) - **Limit**: How many results (default: 50 for raw, 100 for aggregations) -### Step 2: Build Query +### Step 2: Discover Sources (if scope is unknown) + +If the user mentions a service or system but you don't know the exact `_sourceCategory`, discover it first: + +```bash +sumo-query discover-source-metadata -k "" -l 20 +``` + +Or run a broad aggregation: + +```bash +sumo-query search \ + -q ' | count by _sourceCategory, _sourceName | sort by _count desc' \ + -f '-1h' -t 'now' -a -l 30 +``` + +Use the discovered `_sourceCategory` in the query. Tell the user which source you found: + +``` +Found _sourceCategory=production/my-service (50K msgs/hr) — using this for the query. +``` + +**Skip this step** if the user already specified a `_sourceCategory` or the scope is clear. + +### Step 3: Build Query Construct the Sumo Logic query string following this syntax: @@ -94,7 +118,7 @@ _source= | limit 20 ``` -### Step 3: Show Query Plan +### Step 4: Show Query Plan Before executing, show the user: @@ -106,7 +130,7 @@ Mode: Limit: ``` -### Step 4: Execute +### Step 5: Execute Build the `sumo-query search` command: @@ -126,7 +150,7 @@ sumo-query search \ - Always include `-l` to limit output - Never use the `-i` flag (interactive mode is incompatible with this tool) -### Step 5: Present Results +### Step 6: Present Results Format the output for readability: - For **aggregation results**: Present as a formatted table or summary