Skip to content

Indexes 6: Changes version_filter field in index schema#1344

Open
dcookspi wants to merge 1 commit intoindex-5-repo-cmds-config-and-cli-flagsfrom
index-5-zplus-1-versionfilters-not-strings
Open

Indexes 6: Changes version_filter field in index schema#1344
dcookspi wants to merge 1 commit intoindex-5-repo-cmds-config-and-cli-flagsfrom
index-5-zplus-1-versionfilters-not-strings

Conversation

@dcookspi
Copy link
Copy Markdown
Collaborator

@dcookspi dcookspi commented Mar 28, 2026

This changes the version_filter field in the index schema to a flatbuffers list and table representation, instead of a string. This allows the version filter rules (VersionRange entries) to be extracted from the index without string parsing.

The resulting index is slightly larger at 122 MB (see Indexes PR 5: #1340). Index generation time is about the same. But the solve times improve for the larger solves (marked with *):

Requests        | Solution size | Num.    | Solve time  |  Indexed solve time |  Solve time now
                | (# packages)  | Retries | (seconds)   |  (seconds)          |  (seconds)  
------------------------------------------------------------------------------------------------
python          |        4      |    1    |     0.17    |   0.03              |  0.04
boost-python    |        8      |    1    |     0.31    |   0.05              |  0.07
python-torch    |       37      |    2    |     0.58    |   0.15              |  0.14
widget toolset  |       60      |    2    |     3.44    |   0.48              |  0.38
katana toolset  |      181      |   10    |    18.32    |   2.97              |  2.14
nuke toolset    |      280      |   12    |    24.32    |   4.55              |  1.70  (*)
houdini toolset |      211      |   19    |    37.24    |   6.58              |  3.78  (*)
maya toolset    |      403      |   20    |    59.50    |   9.52              |  5.30  (*)

This is the 6th of the chained PRs for adding indexes to spk solves:

  1. Indexes 1: Change Package and related traits to not return references to fields #1336
  2. Indexes 2: Add new_unchecked() constructors to spk schema objects #1337
  3. Indexes 3: Adds flatbuffers schema and SolverPackageSpec for indexes to spk #1338
  4. Indexes 4: Adds Indexes for SPK repositories #1339
  5. Indexes 5: Adds spk repo index subcommand for index generation and updates #1340
  6. this PR

@dcookspi dcookspi self-assigned this Mar 28, 2026
@dcookspi dcookspi added enhancement New feature or request SPI AOI Area of interest for SPI pr-chain This PR doesn't target the main branch, don't merge! labels Mar 28, 2026
@dcookspi dcookspi requested review from jrray and rydrman March 28, 2026 01:02
@codecov
Copy link
Copy Markdown

codecov bot commented Mar 28, 2026

Codecov Report

❌ Patch coverage is 38.04878% with 127 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
crates/spk-schema/src/fb_converter.rs 39.75% 100 Missing ⚠️
...-schema/crates/foundation/src/version_range/mod.rs 30.76% 27 Missing ⚠️

📢 Thoughts on this report? Let us know!

// LessThanOrEqualToRange, LessThanRange, LowestSpecifiedRange,
// NotEqualsVersion, SemverRange WildCardRange
version: Version;
// Used for: Compat,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this comma intentional?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I've taken it out.

let base = fb_rule
.version()
.map(|v| fb_version_to_version(v))
.expect("A CompatRange should have a version in an index");
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's hurting my brain that the type names mentioned in these expect strings are aligned with the version_range types, but they don't match the VersionRangeOperator enum arm names, and it makes this code look like there's "random" inconsistencies. But now I see they do match the VersionRange enum names.

How about changing the names used in these strings to match the enum names?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Todo:

  • Update the names in the messages to match the enum names

None
} else {
// A version filter is a bunch of version ranges, e.g.
// [kg/3.10
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I need to hunt down the PR where this [ came from? 😅

@dcookspi dcookspi force-pushed the index-5-repo-cmds-config-and-cli-flags branch 3 times, most recently from 39f3a57 to 50c68f1 Compare April 9, 2026 18:17
list/table representation instead of a string. The field no longer
needs to be parsed when extracted from the index.

Signed-off-by: David Gilligan-Cook <dcook@imageworks.com>
@dcookspi dcookspi force-pushed the index-5-zplus-1-versionfilters-not-strings branch from 6e12345 to c3fb021 Compare April 10, 2026 01:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request pr-chain This PR doesn't target the main branch, don't merge! SPI AOI Area of interest for SPI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants