fix(core): let a Claude the generated table has not heard of still read images - #1949
Conversation
…ad images The models.dev snapshot is regenerated periodically, so a Claude released after the last regeneration is absent from it, not listed as text-only. Resolving absent to "no vision" drops the user's image attachment and turns an image tool result into a sentence, with nothing on screen to say why. Default the Anthropic opus/sonnet/haiku/fable families to vision when neither the connection nor the table has an opinion. An explicit value from either one still wins, in both directions.
…y hand
The default this branch adds only recognises the id shape Anthropic adopted at
Claude 4, where the family comes first. Before that the version came first and
the family sat behind it: `claude-3-5-sonnet-20241022`,
`claude-3-7-sonnet-20250219`, `claude-3-opus-20240229`. None of the three is in
the generated table, all three read images, and all three still resolved false
— so pinning one and attaching a screenshot reached `ai-sdk-backend.ts` and had
the image replaced with a notice. That is the case in the title, and it was the
one still broken.
`(?:[\d.]+-)*` now skips any leading version segments before the family name.
The generation that genuinely cannot read images is untouched: `claude-2.1`,
`claude-2.0` and `claude-instant-*` have no family segment at all, so they never
reach the alternation, and a test now holds that line rather than leaving it to
be re-derived from the regex.
`claude-subscription` joins the default. It is a first-class provider type that
fetches Anthropic's own models over the Anthropic protocol and stores the same
bare `{ id }` entries — its metadata table is literally the anthropic one — and
it is usually where a new Claude becomes usable first. The set stops there on
purpose: `anthropic-compatible`, `kimi-coding-plan` and `minimax-coding-plan`
speak the same wire format while serving somebody else's models, so a `claude-`
prefix there says nothing about what is behind it.
The tests were not holding the properties they named. A full production revert
left pass 9 / fail 1: only the first test moved, and "does not guess for other
providers" survived a complete revert of the change it claimed to test.
Dropping the provider guard — which would grant vision to `claude-opus-5` on
OpenRouter — passed the entire @maka/core suite. Both are covered now, along
with the pre-4 shape, the subscription path, the claude-2 floor, and both
directions of a connection overriding the default. Against the same full
revert the file is now pass 18 / fail 3, and each mutant lands on the test that
names it:
drop the provider guard → confines the default to the providers …
narrow the regex back → treats a Claude older than the snapshot …
drop 'claude-subscription' → extends the default to the subscription path
drop the stored-models branch → yields to what a connection reports …
Also puts the doc back where it belongs. The surviving block above the function
still said unknown "resolves to false, keeping the send path fail-closed",
which the change had made untrue, and inserting the new comment between it and
the function left `resolveModelVisionSupport` with no doc at all.
|
All four findings confirmed and fixed. a. The regex only recognised the id shape Anthropic adopted at Claude 4, where the family comes first. Before that the version came first and the family sat behind it, and b. Confirmed. c. Confirmed on both counts, and both are covered now. The second test survived a complete revert, and dropping the provider guard passed the whole @maka/core suite. Every mutant was run: revert, rebuild, observe red, restore, observe green. full production revert pass 18 / fail 3 (was pass 9 / fail 1) drop the provider guard pass 20 / fail 1 in the file, and fail 1 across the whole @maka/core narrow the regex back to the pass 19 / fail 2 drop 'claude-subscription' pass 20 / fail 1 drop the stored-models branch pass 19 / fail 2 d. Confirmed and fixed. The surviving block still said unknown "resolves to false, keeping the send path fail-closed", which the change had made untrue, and inserting the new comment between it and the function left Restored, rebuilt, green: model-metadata tests 21 pass 21 fail 0, whole @maka/core 709 pass 709 fail 0. Node 24.18.1, |
1 participant
resolveModelVisionSupport asks the connection first, then the generated models.dev table, and answers false when neither knows. That fail-closed rule was written for text-only models, but it also catches a model the table has simply not been regenerated for yet. On main today, lookupModelMetadata("anthropic", "claude-opus-5") returns {} and resolveModelVisionSupport returns false for it.
What the user sees when that happens: an image attached to the message is replaced by the non-vision fallback notice instead of being sent, and an image returned by a tool becomes the text "Image was read, but the selected model does not support image input." (both in packages/runtime/src/ai-sdk-backend.ts). Nothing on screen attributes this to a stale table, so it reads as the model lacking a capability it has, and it comes back every time a new Claude ships ahead of the snapshot.
The change defaults the Anthropic opus, sonnet, haiku and fable families to vision when neither the connection nor the table has an opinion. It is deliberately narrow: it is scoped to providerType "anthropic", it is only consulted after both explicit sources come back undefined, and an explicit value from either still wins in both directions. No other provider gains a default.
The asymmetry is the argument for it. Guessing yes and being wrong costs one turn and produces a provider error. Guessing no and being wrong silently discards the user's attachment.
How to tell it works: two tests in packages/core/src/tests/model-metadata.test.ts. The first pins that the generated table really returns {} for claude-opus-5 (so the test is exercising the unknown path, not a listed entry) and that vision now resolves true for it. The second holds the blast radius down: an unlisted OpenAI model still resolves false, and a connection that declares vision: false for claude-opus-5 still wins.