feat: Implemented search annotations for runs

[yuechao-qin]

TODO

• Discuss if indexing and migrations is needed for annotations (key, value).
• How do I confirm that the unit test will be part of CI/CD?
• Is EXISTS needed for searching for Keys? Is EQUALS sufficient?

Description

Closes #45

Implemented a new API to search annotations for runs.

Background

Annotations are key-value pairs. The following examples of annotations (e.g. key = value):

• env = production
• team = backend

This PR allows searches for key/value strings.

Features

• Created a new API (POST /api/pipeline_runs/search/) to search key/value in annotations.
• Keys searchable operations
  • EXISTS: If any key exists regardless of key string
  • CONTAINS: If key contains a substring
  • IN_SET: If key string matches set of strings
  • EQUALS: If key string equals string
• Values searchable operations
  • CONTAINS: If value contains a substring
  • IN_SET: If value matches set of strings
  • EQUALS: If value equals string
• Keys and Values search operations can be negated (i.e. NOT ).
• N searches can be grouped (AND, OR) together and recursive.
  • Example: (S1 and S2) or (S3 or S4)

Use Cases and Examples

1. Key equals a string

Find runs where annotation key equals "environment":

{
  "annotation_filters": {
    "filters": [
      {"operator": "equals", "key": "environment"}
    ]
  }
}

2. Key contains substring AND value in set

Find runs where key contains "env" AND value is "prod" or "staging":

{
  "annotation_filters": {
    "filters": [
      {"operator": "contains", "key": "env"},
      {"operator": "in_set", "values": ["prod", "staging"]}
    ],
    "operator": "and"
  }
}

3. Complex: (key contains OR value contains) AND key NOT contains

Find runs where (key contains "env" OR any value contains "prod") AND key NOT contains "deprecated":

{
  "annotation_filters": {
    "filters": [
      {
        "filters": [
          {"operator": "contains", "key": "env"},
          {"operator": "contains", "value": "prod"}
        ],
        "operator": "or"
      },
      {"operator": "contains", "value": "deprecated", "negate": true}
    ],
    "operator": "and"
  }
}

Test Plan

• Unit test
  • uv run pytest tests/test_pipeline_run_search.py -v
• Manual Testing
  • Watch demo videos below
  • Add annotations for testing (PUT /api/pipeline_runs/<ID>/annotations/<KEY>/)
  • Query annotations for ID (GET /api/pipeline_runs/<ID>/annotations/)
  • Test with new search (POST /api/pipeline_runs/search/)
• Test on Staging (same procedure as Manual Testing above)

Demo

part1.mov

part2.mov

[morgan-wowk]

This is a piece of feedback on REST semantics mainly. Typically you wouldn't see a POST request for searching for resources, in other words getting resources. If it were a GraphQL API then things would be different.

Here's the suggestion:

Turn this into a GET endpoint:

GET /api/pipeline_runs

This is clear that you get retrieving a list of pipeline runs, which would be an unfiltered, paginated response by default. Then add the search capability after the foundation (unfiltered search) is established.

From loading the UI, I can see there is already a request being made:

curl 'http://localhost:8000/api/pipeline_runs?include_pipeline_names=true&include_execution_stats=true'

and upon asking cursor, I know that it has a pagination implementation already. We can add the filter options to this existing endpoint rather than creating a new endpoint. Which would set a good precedence for all future search capabilities on other resources.

Here is an example of how I've seen this implemented:

curl 'http://localhost:8000/api/pipeline_runs?filter=in(annotations.env:staging,prod)+eq(annotations.app:agenticsearch)

With other operators like gte (greater than or equal), lte, etc.

Accounting for complex AND / OR combinations is something that would require some thought but I wouldn't go there unless that's something we see people using. The above example is a basic OR and AND.