Skip to content

feat: Implemented search annotations for runs #68

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
yuechao-qin wants to merge 2 commits into
base: master
Choose a base branch
from
+1,144 −53
Open

feat: Implemented search annotations for runs #68

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
bef4c47
Implemented search annotations for runs
yuechao-qin Jan 13, 2026
2e69b8f
Remove key from ValueFilter
yuechao-qin Jan 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 91 additions & 0 deletions cloud_pipelines_backend/api_router.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -269,6 +269,97 @@ def stream_container_log(
router.get("/api/pipeline_runs/", tags=["pipelineRuns"], **default_config)(
inject_session_dependency(list_pipeline_runs_func)
)
router.post(
"/api/pipeline_runs/search/",
Copy link

@morgan-wowk morgan-wowk Jan 19, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a piece of feedback on REST semantics mainly. Typically you wouldn't see a POST request for searching for resources, in other words getting resources. If it were a GraphQL API then things would be different.

Here's the suggestion:

Turn this into a GET endpoint:

GET /api/pipeline_runs

This is clear that you get retrieving a list of pipeline runs, which would be an unfiltered, paginated response by default. Then add the search capability after the foundation (unfiltered search) is established.

From loading the UI, I can see there is already a request being made:

curl 'http://localhost:8000/api/pipeline_runs?include_pipeline_names=true&include_execution_stats=true'

and upon asking cursor, I know that it has a pagination implementation already. We can add the filter options to this existing endpoint rather than creating a new endpoint. Which would set a good precedence for all future search capabilities on other resources.

Here is an example of how I've seen this implemented:

curl 'http://localhost:8000/api/pipeline_runs?filter=in(annotations.env:staging,prod)+eq(annotations.app:agenticsearch)

With other operators like gte (greater than or equal), lte, etc.

Accounting for complex AND / OR combinations is something that would require some thought but I wouldn't go there unless that's something we see people using. The above example is a basic OR and AND.

tags=["pipelineRuns"],
summary="Search Pipeline Runs",
description="""Search pipeline runs with annotation filters.

## What is an Annotation?

An annotation is a **key-value pair** attached to a pipeline run.
For example the following annotations (i.e. key = value):
- `environment` = `production`
- `team` = `backend`
- `priority` = `high`

## Filter Types

### KeyFilter - Search by annotation key
| Operator | Description |
|----------|-------------|
| `exists` | Key exists (any value) |
| `equals` | Key exactly matches string |
| `contains` | Key contains substring |
| `in_set` | Key matches one of multiple values |

### ValueFilter - Search by annotation value (across ALL annotations)
| Operator | Description |
|----------|-------------|
| `equals` | Value exactly matches string |
| `contains` | Value contains substring |
| `in_set` | Value matches one of multiple values |

### FilterGroup - Combine filters with logic
| Operator | Description |
|----------|-------------|
| `and` | ALL filters must match |
| `or` | ANY filter must match |

All filters support `negate: true` to invert the condition (e.g., NOT equals).

---

## Examples

### 1. Key equals a string
Find runs where annotation key equals "environment":
```json
{
"annotation_filters": {
"filters": [
{"operator": "equals", "key": "environment"}
]
}
}
```

### 2. Key contains substring AND value in set
Find runs where key contains "env" AND value is "prod" or "staging":
```json
{
"annotation_filters": {
"filters": [
{"operator": "contains", "key": "env"},
{"operator": "in_set", "values": ["prod", "staging"]}
],
"operator": "and"
}
}
```

### 3. Complex: (key contains OR value contains) AND key NOT contains
Find runs where (key contains "env" OR any value contains "prod") AND key NOT contains "deprecated":
```json
{
"annotation_filters": {
"filters": [
{
"filters": [
{"operator": "contains", "key": "env"},
{"operator": "contains", "value": "prod"}
],
"operator": "or"
},
{"operator": "contains", "value": "deprecated", "negate": true}
],
"operator": "and"
}
}
```
""",
**default_config,
)(inject_session_dependency(pipeline_run_service.search))
router.get("/api/pipeline_runs/{id}", tags=["pipelineRuns"], **default_config)(
inject_session_dependency(pipeline_run_service.get)
)
Expand Down
Loading