Skip to content

fix(core): stop offering deprecated models in fallback lists - #3494

Merged
jackwener merged 2 commits into
apache:mainfrom
yihanzhu:fix/deprecated-fallback-models
Aug 22, 2026
Merged

fix(core): stop offering deprecated models in fallback lists#3494
jackwener merged 2 commits into
apache:mainfrom
yihanzhu:fix/deprecated-fallback-models

Conversation

@yihanzhu

Copy link
Copy Markdown
Contributor

Summary

fallbackModels is a recommendation, not an inventory: everything in it lands in the catalog as available and default-capable, and its head is what a new connection defaults to and what the connection test probes. toolCallingModelIds filters on tool-calling capability only, so six providers still offered 28 models their vendors have deprecated.

This applies the call-site lifecycle filter #3324 established for ZenMux to xiaomi, nvidia, mistral, togetherai and deepinfra, and drops the one deprecated id from openai's hand-written list. That empties PROVIDERS_WITH_DEPRECATED_FALLBACKS, so the recorded boundary goes away with it and the remaining assertion becomes the unconditional invariant — which is what its comment says it was built to do.

Removal is from the offer only. An id a user already chose still sends (authorizeConnectionModel never reads this list), and all six do kind: 'protocol' discovery, so the endpoint's own catalog returns on the first /models fetch. Every list head is unchanged, so no connection default or connection-test probe model moves.

Reasoning for the offer-policy question the issue poses, and the measurements behind the notes below, are in #3355 (comment).

Fixes #3355

Review focus

  • Two corrections to the issue body. The 28 is right as a registry count, but only 27 are user-visible: CURATED_CATALOG_FALLBACK_MODELS.openai shadows the registry list at model-catalog.ts:239, so gpt-4-turbo is in no picker today — removing it is hygiene for the non-catalog consumers (connection test, onboarding verify, CLI projection) and for the contract test, which reads the registry directly. And "discovery keeps only the fallback set" is the kind: 'fallback' branch; none of these six is on it, which is why the exclusion costs no reachability.
  • Two product judgments worth naming. mistral loses its whole devstral line from the pre-credential list (all eight deprecated ids are devstral plus open-mistral-nemo), and xiaomi is the largest proportional cut, 6 → 3. Both come straight back from live discovery.
  • The clause now sits at 13 call sites, so a new provider is wrong by default. Folding it into toolCallingModelIds would be a no-op today — of 46 snapshots, the only ones carrying a tool-capable deprecated id are the nine now filtered at their call site plus openai, google and the three xiaomi-token-plan entries, which all use hand-written lists that never reach the helper. Happy to roll that in here or leave it as a follow-up; the wrinkle is placement, since ollama-cloud's pre-filter is what makes a deprecated recommended id throw at module load rather than vanish.
  • Known limit, pre-existing: the invariant sweeps CATALOG_PROVIDER_TYPES, which is 55 of 59 registry entries. xai-oauth, github-copilot, claude-subscription and openai-codex sit outside it; I checked and none carries a deprecated id today.

Verification

  • Red state: with the registry change reverted, keeps deprecated snapshot models out of fallback lists fails naming all six providers and all 28 ids (togetherai 11, mistral 8, xiaomi 3, deepinfra 3, nvidia 2, openai 1)
  • @maka/core 588 pass; @maka/storage 895 pass; @maka/runtime 3066 pass with one environment failure (claude-subscription-usage cannot reach the OAuth token endpoint in my sandbox — fails identically on unmodified main)
  • Root lint, format:check and typecheck all pass
  • Post-fix list sizes, zero deprecated remaining in each, heads unchanged: togetherai 19, mistral 22, xiaomi 3, deepinfra 56, nvidia 59, openai 3
  • No protocol files touched, so RUNTIME_HOST_COMPATIBILITY_EPOCH stays put. model-pricing.generated.ts still carries rows for the removed ids and is deliberately untouched — it is generated and keyed independently.

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Claude Code (Opus 5) — implementation, tests and the supporting measurements, under my direction and review. The commit carries a Generated-by trailer.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — a connection created after this change sees a shorter pre-discovery model list; described under Summary above
  • No

yihanzhu and others added 2 commits August 22, 2026 08:30
`fallbackModels` is a recommendation, not an inventory: everything in it
lands in the catalog as available and default-capable, and its head is
what a new connection defaults to and what the connection test probes.
`toolCallingModelIds` filters on tool-calling capability only, so six
providers still offered 28 models their vendors have deprecated.

Apply the lifecycle filter the other eight derivations already use to
xiaomi, nvidia, mistral, togetherai and deepinfra, and drop the one
deprecated id from openai's hand-written list. Removal is from the offer
only: an id a user already chose still sends (`authorizeConnectionModel`
never reads this list), and all six do live protocol discovery, so the
endpoint's own catalog still comes back on the first fetch.

Every head is unchanged, so no connection default or probe model moves.
The recorded boundary in the contract test empties and goes away with
it, which is what it was built to do.

Closes apache#3355

Generated-by: Claude Code (Opus 5)
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The comment said the catalog marks whatever the list contains available
and default-capable, then singled out `openai` — but `openai` is one of
eight providers whose `CURATED_CATALOG_FALLBACK_MODELS` entry replaces
the registry list before the catalog is built, so for those eight this
list reaches CLI onboarding and the connection-test probe candidates
rather than the picker. It also said each derivation drops deprecated
ids at its call site; thirteen of twenty-seven do.

Generated-by: Claude Code (Opus 5)
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@jackwener jackwener left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the exact head against the current base. The fallback lifecycle policy and scope are sound, the follow-up only corrects the contract comment, and I found no blocking correctness issue.

@jackwener
jackwener merged commit 8b60ddf into apache:main Aug 22, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fallback model lists still offer deprecated models for six providers

2 participants