Conversation
This is particularly important for the available_providers list, where we want to list all of the matching providers if there are multiple. For example, "gemini-flash" from vertex and "gemini/gemini-flash" would be collapsed into a single entry, but we want both providers in available_providers. This doesn't handle different pricing or parameters for the same model on different providers, but is at least more deterministic than previously. For now we always choose the provider that is alphabetically first.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 416af553ee
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".
| const resolvedRemote = resolveRemoteModels(remoteModels, argv.provider); | ||
|
|
||
| if (argv.provider) { | ||
| const lowerArgProvider = argv.provider.toLowerCase(); | ||
| const modelProvider = modelDetail.litellm_provider?.toLowerCase(); | ||
| const modelNameProviderPart = remoteModelName | ||
| .split("/")[0] | ||
| .toLowerCase(); | ||
|
|
||
| if ( | ||
| !modelProvider?.includes(lowerArgProvider) && | ||
| !modelNameProviderPart.includes(lowerArgProvider) && | ||
| !(modelProvider === lowerArgProvider) && | ||
| !(modelNameProviderPart === lowerArgProvider) | ||
| ) { | ||
| continue; | ||
| } | ||
| } | ||
| consideredRemoteModels[remoteModelName] = modelDetail; | ||
| } | ||
|
|
||
| const remoteModelNamesFiltered = new Set( | ||
| Object.keys(consideredRemoteModels), | ||
| ); | ||
|
|
||
| for (const modelName of remoteModelNamesFiltered) { | ||
| const translatedModelName = translateToBraintrust( | ||
| modelName, | ||
| consideredRemoteModels[modelName]?.litellm_provider, | ||
| ); | ||
| for (const [translatedName, { remoteModelName, remoteModel }] of resolvedRemote) { | ||
| consideredRemoteModels[remoteModelName] = remoteModel; |
There was a problem hiding this comment.
Iterating over resolveRemoteModels(...) here collapses collisions down to a single primary provider before consideredRemoteModels is built, so find-missing --summarize no longer reports accurate totals for secondary providers. In collision cases (the commit message’s gemini/vertex example), those provider entries are dropped from the summary entirely, which can hide missing coverage for that provider and mislead follow-up sync work.
Useful? React with 👍 / 👎.
1 participant
This is particularly important for the available_providers list, where we want
to list all of the matching providers if there are multiple. For example,
"gemini-flash" from vertex and "gemini/gemini-flash" would be collapsed into a
single entry, but we want both providers in available_providers.
This doesn't handle different pricing or parameters for the same model on
different providers, but is at least more deterministic than previously. For
now we always choose the provider that is alphabetically first.