[brandonkelly]

Description

Adds AI image analysis to assets: alt text, titles, focal points, tagging, and image types, run through AI Connections, plus color names, text-color hints, and embedded metadata without AI. Builds on #20099: image tasks require vision and structured output.

on upload / file replace / indexing (new or changed file, or any image with the “Analyze images” option) / “Analyze images” asset action
  ImageAnalyzer::queue(asset)                 // only if some task has something to fill in, or a new image
                                              // is still missing colors or metadata (sample: true)
    AnalyzeImage job (low-priority queue, unique per asset)
      ImageAnalyzer::analyze(asset)
        sample colors / metadata the image is missing, from the same copy of the file
        group enabled, needed tasks by provider + model
        for each group: one structured-output request with a ≤1568px copy of the image
          each task adds its instructions + JSON schema properties, then applies its part
        save per-site changes (alt text, titles) and asset-wide ones (focal point, analysis)

 Core AI tasks (Settings → AI Connections)
+  craft:altText     Generate alt text   per site language; sites that share alt text get one
+  craft:title       Generate titles     replaces filename titles (“IMG 4759”), per site language
+  craft:focalPoint  Detect focal points sets missing focal points, invalidates transforms
+  craft:tagImage    Tag images          keywords + text in the image → searchable
+  craft:imageType   Classify images     photo / illustration / screenshot / diagram / logo /
                                         document / pattern → filterable

 Assets
+  $analysis (ImageAnalysis)             one `analysis` JSON column: colors, metadata, type,
                                         keywords, text
-  $colors                               moved to $analysis->colors (also in GraphQL)
+  analysis->colors->palette, labels()   main colors and their names (no AI; server + browser)
+  analysis->colors->weights             how much of the image each palette color covers (server + browser)
+  analysis->colors->luminance           P10/P90 lightness per grid region (server + browser)
+  analysis->colors->textColor(region)   white or black text over the image or an edge, + contrast
                                         (worked out from luminance when read, not stored)
+  analysis->metadata                    XMP/IPTC/EXIF title, description, alt text, creator,
                                         credit, copyright, keywords, date taken, camera, GPS
                                         latitude/longitude, city, state, and country, read before
                                         cleaning (server + browser); coordinates aren’t in GraphQL
                                         or sandboxed templates
+  hasDefaultTitle()                     whether the title is still the filename one
+  getImageType()                        the image's type, from analysis
+  assets.imageType                      generated, indexed column mirroring analysis.type
 Queries / conditions / GraphQL
+  AssetQuery::imageType()               'logo', ['screenshot', 'diagram'], ['not', 'logo'], ':empty:'
+  “Image Type” condition rule           index filters, field sources, etc.
+  assets(imageType: [...])              GraphQL argument
 Search keywords
+  keywords                              AI keywords + embedded keywords + embedded description
+  text                                  text in the image (OCR)
 Uploads
+  title from embedded metadata          when the file has one, or a replaced file’s while the title is still
                                         the filename one; alt text too (Asset::valuesFromEmbeddedMetadata(),
                                         shared with the queue job; replaces Asset’s own XMP/IPTC alt reading)
+  metadata read in the browser          image-metadata.ts, a port of ImageMetadataReader
~  completion request                    sends `analysis: {colors, metadata}`, like what’s stored (was
                                         `colors`); only those two parts are taken, AI results never are
~  Asset::$uploadColors                  replaced by $uploadAnalysis
+  remote staging disks                  images aren’t downloaded unless they’re cleaned or their dimensions
                                         can’t be read from the start of the file; without browser data,
                                         the AnalyzeImage job samples them
+  S3 → S3                               staged uploads are copied within S3 when both disks share
                                         credentials and an endpoint

 Asset Indexes utility / index-assets commands
+  “Analyze images” / --analyzeImages     also sample colors and queue AI tasks for unchanged images
                                         that are missing something (off by default)

 Edit page
+  Alternative Text field → “Generate alt text”   fills in a suggestion to review; nothing is saved
+  Info panel                            what was found, after a rule: image type, a color bar
                                         (sized by weight), embedded title/alt text, creator,
                                         credit, copyright, date taken, camera, and location
                                         (place name, Google Maps link, OSM-tile map)
~  “Location” (the asset’s folder)       now “Folder”, in the Info panel, index column, and card

 Asset indexes
+  columns + card attributes             Image Type, Colors, Creator, Credit, Copyright, Date Taken,
                                         Camera, Embedded Keywords

 MCP
+  assets.analyze                        queues images for the image analysis tasks that are set up
+  content-model.audit notes             missing-alt-text says whether “Generate alt text” is set up, and to
                                         use assets.analyze if so

 Elements
+  ElementInterface::getMetadataGroups() metadata groups, separated by a rule in the sidebar
~  ContentHtml::metadataHtml()           takes several groups, and lists of values (one <dd> each)

What was found in an image, in its Info panel:

The map is drawn from OpenStreetMap tiles rather than an embed, so it has no controls or text over it, and it’s credited below. Tiles are inverted for the dark theme. It’s hidden from assistive technology, since the place name and link say the same.

The new tasks in Settings → AI Connections:

Images’ colors are sampled from the file, without AI, and stored alongside what the AI tasks find, so templates read asset.analysis.colors, .metadata, .type, .keywords and .text, and GraphQL queries analysis { colors { … luminance textColor(region: "bottom") { color contrast } } metadata { … } type keywords text }. A new or changed file starts its analysis over with only its colors and metadata, so it gets tagged again.

textColor() compares white text with the lightest parts of a region (P90) and black with the darkest (P10), since those are hardest to read over; a contrast under 4.5 means the text needs a scrim. Metadata is read in beforeSave(), before uploads are cleaned, and the GPS flag is checked again after cleaning, so it says whether the stored file still has coordinates.

For example, to overlay a heading on the bottom edge of an entry’s hero image, falling back to white text over a scrim when the colors aren’t known yet (a new upload whose analysis hasn’t run, an image indexed before this change) or neither color is legible on its own:

{% set image = entry.heroImage.one() %}

{% if image %}
  {# null when the image hasn’t been analyzed, or its colors haven’t been sampled #}
  {% set text = image.analysis.colors.textColor('bottom') ?? null %}
  {# no data, or neither white nor black is legible on its own: white over a scrim #}
  {% set scrim = not text or text.contrast < 4.5 %}
  {% set color = scrim ? '#ffffff' : text.color %}

  <div class="hero">
    <img src="{{ image.getUrl({width: 1600}) }}" alt="{{ image.alt }}">
    <h1 class="hero__title{{ scrim ? ' hero__title--scrim' }}" style="--text: {{ color }}">
      {{ entry.title }}
    </h1>
  </div>
{% else %}
  <h1>{{ entry.title }}</h1>
{% endif %}

The same in Blade, where ?-> stops at a missing analysis or colors:

@php
    $image = $entry->heroImage->one();

    // null when the image hasn’t been analyzed, or its colors haven’t been sampled
    $text = $image?->analysis?->colors?->textColor('bottom');

    // no data, or neither white nor black is legible on its own: white over a scrim
    $scrim = $text === null || $text['contrast'] < 4.5;
    $color = $scrim ? '#ffffff' : $text['color'];
@endphp

@if ($image)
    <div class="hero">
        <img src="{{ $image->getUrl(['width' => 1600]) }}" alt="{{ $image->alt }}">
        <h1 @class(['hero__title', 'hero__title--scrim' => $scrim]) style="--text: {{ $color }}">
            {{ $entry->title }}
        </h1>
    </div>
@else
    <h1>{{ $entry->title }}</h1>
@endif

.hero { position: relative; }
.hero img { display: block; width: 100%; }
.hero__title { position: absolute; inset: auto 0 0; margin: 0; padding: 2rem; color: var(--text); }
/* 65% black keeps white text above 4.5:1, even over pure white */
.hero__title--scrim { padding-top: 4rem; background: linear-gradient(transparent, rgb(0 0 0 / 65%) 3rem); }

Use textColor('top'), 'left' or 'right' for text along another edge, or textColor() for text that could sit anywhere on the image.

Indexing only samples colors and queues analysis for new and changed files by default, so indexing a large existing volume doesn’t send every image to an AI provider, or download every remote image. Turning on “Analyze images” (or --analyzeImages=1) backfills unchanged images too. The option is stored on the indexing session, since the utility indexes over several requests.

Analysis only fills in what’s missing: titles are only replaced while they’re still the filename one. Tasks that change the same attribute in one request, like Tag images and Classify images both changing analysis, build on each other through ImageAnalysisChanges::get(). The bulk action never overwrites; only the alt text field action replaces a value, and it isn’t saved until the editor saves. The field action uses a generic craft:generate-field-value event, so other fields can reuse it.

Craft’s tasks are registered in AiTasks itself. Plugins can add their own image analysis tasks by extending ImageAnalysisTask and registering them from their service providers. Tasks that use the same provider and model share one request per image.

src/
├── Asset/Analysis/
│   ├── ImageAnalysisTask.php / Contracts/      # task contract + base (requires vision, structured output)
│   ├── ImageAnalyzer.php                       # grouping, image prep, analyze() / suggest(), saving
│   ├── ImageAnalysisAgent.php                  # combined prompt + schema
│   ├── Data/ImageAnalysis.php                  # colors + metadata + type + keywords + text, one JSON column
│   ├── Jobs/AnalyzeImage.php
│   └── Tasks/                                  # GenerateAltText, GenerateTitles, DetectFocalPoint, TagImage,
│                                               # ClassifyImage, TranslatedSite (per-site groups)
├── Asset/Enums/ImageType.php
├── Asset/Actions/AnalyzeImages.php
├── Database/Migrations/…_add_analysis_to_assets.php  # moves colors (and dev builds’ keywords/visibleText) into analysis
├── Database/Migrations/…_add_image_type_column_to_assets.php  # generated imageType column + index; schema 6.0.0.31
├── Element/Queries/Concerns/Asset/QueriesImageType.php, Asset/Conditions/ImageTypeConditionRule.php
├── Gql/Types/ImageAnalysis.php (+ generator)
├── Http/Controllers/Assets/GenerateAltTextController.php
├── Image/ColorNames.php                        # Oklch-based color names
├── Image/ColorGrid.php                         # + luminance() P10/P90 per region
├── Image/ImageMetadataReader.php, Data/ImageMetadata.php   # XMP → IPTC → EXIF
├── Filesystem/Data/UploadedFile.php            # isLocal(), imageSize(), S3 → S3 copies
└── Gql/Types/{TextColor,ImageMetadata}.php (+ generators)
resources/js/modules/uploader/
├── image-metadata.ts                           # browser port of ImageMetadataReader
└── image-completion-data.ts                    # `analysis` (colors + metadata), sent when the upload completes

Evidence

All model replies are faked through laravel/ai.

• tests/Feature/Asset/ImageAnalyzerTest.php:
  • one request for tasks that share a model, separate requests otherwise
  • what succeeded is saved when another group fails
  • only tasks that are available and needed run
  • images are queued on replace and by the bulk action
• tests/Feature/Asset/AssetIndexerTest.php: new and changed images are queued and sampled; unchanged ones only are with “Analyze images”, including images missing colors or sampled before palettes. Both fail on the code before this change.
• tests/Feature/Asset/Analysis/GenerateAltTextTest.php:
  • alt text per site language, and one value for sites that share it
  • only fills in missing alt text unless overwriting
  • the endpoint suggests without saving, and refuses users who can’t save the asset
  • the field action only appears when the task is set up
• tests/Feature/Asset/Analysis/GenerateTitlesTest.php: titles per site language, one for sites that share it, only filename titles replaced unless overwriting.
• tests/Feature/Asset/Analysis/ClassifyImageTest.php: types are stored, filterable by imageType() in every form, and in GraphQL (as a field and an argument); unknown types are ignored; tagging and classifying in one request keep both (fails without ImageAnalysisChanges::get()).
• tests/Unit/Image/ImageMetadataReaderTest.php: XMP, IPTC and EXIF (hand-built segments), precedence, camera names, dates, Windows-1252 text. AssetsTest: metadata is read before cleaning, fills alt text and new uploads’ titles, and hasLocation follows the stored file (fails without the post-cleaning check).
• Server/browser parity: ColorGridTest/ImagesTest and image-colors.test.ts pin both samplers to the same luminance, palette and dominant color for one fixture. The palette test fails on the old browser sampler, which seeded its k-means++ with mulberry32 rather than PHP’s Mt19937 Randomizer that Intervention uses; php-randomizer.ts now ports it, and php-randomizer.test.ts checks it against PHP’s own output. The ImagesTest case passes with GD and with Imagick. ImageColorsTest covers textColor(); the upload controller rejects out-of-range luminance (fails without the new rules).
• tests/Feature/Asset/Analysis/DetectFocalPointTest.php and TagImageTest.php:
  • values are clamped and cleaned up
  • transforms are invalidated when the focal point changes
  • keywords and text are searchable, in GraphQL, and tagging keeps the image’s colors
  • nothing is overwritten unless asked
• Colors:
  • tests/Unit/Image/ColorNamesTest.php covers reference colors.
  • ImageColorsTest, ImagesTest and AssetColorsTest cover the palette, labels and GraphQL.
  • ImagesTest: images without an alpha channel are sampled as opaque, which some ImageMagick builds otherwise get wrong.
• Image type filtering: AssetConditionRulesTest covers the condition rule; AddImageTypeColumnToAssetsTest checks the generated column follows analysis.type. Passes on SQLite, MySQL and Postgres.
• tests/Feature/Database/Migrations/AddAnalysisToAssetsTest.php: colors, keywords and visible text move into analysis and their columns are dropped; up-to-date installs are left alone. Passes on SQLite and MySQL.
• tests/Feature/Asset/AssetsTest.php: replacing a file forgets the old file’s keywords and text, and analysis can’t be set from a request.
• Browser metadata: image-metadata.browser.test.ts (Chromium, since HappyDOM lacks XML namespace lookups) covers XMP, IPTC, EXIF, precedence and Windows-1252 text. PHP’s reader returns the same values for the same five fixtures. Reading both colors and metadata from a 24-megapixel JPEG takes about 50 ms, almost all of it decoding for colors, and never blocks the page for more than 28 ms.
• Direct uploads: AssetUploadHandlerTest stages images on a remote disk. With browser data they aren’t downloaded or queued; without it, they aren’t downloaded but are queued (fails on the old code, which always downloaded them); cleaned ones are read before cleaning. UploadSessionControllerTest drops invalid browser metadata, and ignores AI results sent in analysis (type, keywords, text). S3UploaderTest copies between S3 disks with CopyObject, without GetObject/PutObject (fails without the copy). ImageAnalyzerTest queues new images for files already in place, and samples what they’re missing, filling in alt text and titles.
• resources/js/modules/ui/FieldNodeGenerateValue.test.ts: the generated value fills the field, the field is aria-busy while generating, and errors are reported.
• Mutation-checked: removing the upload/replace trigger, the indexing trigger, or the transform invalidation each fails its test.
• composer ci: Pint, Rector and PHPStan pass. Pest only fails HandleInertiaRequestsTest locally, which happens whenever the Vite dev server is running. vue-tsc, vp lint and the JS tests pass. Since the analysis change: the affected suites (1,227 tests), full PHPStan, Rector and Pint pass.

Since the browser metadata and direct upload changes, the affected suites (1,481 tests), the architecture tests, PHPStan, Pint, vue-tsc, vp lint and the uploader JS tests pass.

Info panel, columns, location, and color weights:

• EditAssetControllerTest: the analysis group follows a rule; the color bar’s segments, accessible name and title; embedded rows, leaving out a title that matches the asset’s; date taken in the time it was taken; the location row (place name without repeats, Google Maps link, tile map, OSM credit); table and card attributes; “Folder”.
• ImageMetadataReaderTest + image-metadata.browser.test.ts: GPS coordinates from XMP and EXIF (south/west negative), city/state/country from XMP and IPTC; XMP packets that end past the first 8 KB (fails on the old fread() read, which returned 8 KB). Both readers return the same values for IPTC’s 2024.1 reference image and a Nikon sample.
• ImagesTest + image-colors.test.ts: both samplers give the same palette weights for the shared fixture. AssetsTest: coordinates are kept after cleaning; embedded descriptions are searchable.
• ContentHtmlTest, AssetColorsTest: metadata groups and list values; weights, city/state/country in GraphQL, and no latitude.
• Embedded metadata fill and MCP: AssetsTest titles a replaced file from its embedded title while the title is still the filename one (fails on the old new-assets-only rule); MCP AssetsTest covers assets.analyze (queued, skipped, no tasks set up, unknown tasks); ContentModelTest covers the missing-alt-text note.
• 4,803 PHP tests across the affected areas, 80 JS tests, PHPStan, Pint, Stylelint and vue-tsc pass. Checked in Chrome in both themes; not checked with a screen reader or in forced colors mode.

Not verified: real providers, Imagick’s XMP fallback (no Imagick locally), and a browser pass of the edit page’s “Generate alt text” action (keyboard, screen reader, forced colors).

Related issues

• [6.x] AI task capability requirements #20099

🤖 Generated with Claude Code

[github-actions]

📚 Storybook previews

@craftcms/ui — open Storybook

No changed components detected in this Storybook.

resources/js — open Storybook

No changed components detected in this Storybook.

[github-actions]

This PR had conflicts with 6.x. Claude attempted an automatic resolution and pushed a merge commit. Please review the merge commit carefully before approving/merging.

Claude's response

I resolved both conflicts and committed the merge as d04e344. Neither conflict was ambiguous, so I added no CONFLICT-REVIEW comments. Both conflicts came from the same pair of changes: this branch added keywords and visibleText for the "Tag images" AI task, while 6.x replaced keptFile with trashedFileExists and added deletedWithVolume to the query.

• src/Asset/Elements/Asset.php: I kept this branch's keywords property (including its setter) and its visibleText property. I then used 6.x's version of the old keptFile property, which is now public bool $trashedFileExists = false with 6.x's docblock.
• src/Element/Queries/AssetQuery.php: The query now selects assets.keywords and assets.visibleText from this branch, plus assets.deletedWithVolume and assets.trashedFileExists from 6.x. I dropped the assets.keptFile select because 6.x's migration removes that column.

A search found no remaining keptFile references outside migrations. I didn't run the tests. php -l couldn't check the files either, because the runner has PHP 8.3, which can't parse the property hooks this code uses; the pre-merge version of Asset.php fails the same way.

[github-actions]

This PR had conflicts with 6.x. Claude attempted an automatic resolution and pushed a merge commit. Please review the merge commit carefully before approving/merging.

Claude's response

I resolved the merge conflict and committed the merge as 2fa43b0. I haven't pushed it.

The only conflict was in CHANGELOG.md, in the unreleased section. Both branches had added one line at the end of the same list. This branch added the entry about uploads staged on S3 being copied within S3 (#20103), and 6.x added the fix for withTransforms criteria being ignored for eager-loaded assets (#20117). I kept both lines, this branch's first and then 6.x's, and changed nothing else.

The other changed files from 6.x merged without conflicts. Nothing was ambiguous, so I added no CONFLICT-REVIEW comments. I didn't run any tests or checks.

Changed

• CHANGELOG.md: kept both entries and removed the conflict markers.
• Committed the merge with git commit --no-edit.

Found

• The [6.x] AI image analysis: alt text, titles, focal points, tagging, image types, metadata, and text colors #20103 entry reads as an improvement but sits in a list of "Fixed a bug…" entries. That placement came from this branch, so I left it as it was.

[github-actions]

This PR had conflicts with 6.x. Claude attempted an automatic resolution and pushed a merge commit. Please review the merge commit carefully before approving/merging.

Claude's response

I resolved the one conflict and committed the merge as 8e4445d. I haven't pushed it.

The conflict: it was in resources/js/modules/uploader/asset-upload-queue.ts, and only in the import block.

• This branch replaced the imageColorsCompletionData import with imageCompletionData, which sends image colors and embedded metadata to the server together (4eea57c).
• 6.x added an import for the ConflictResolution type from @/modules/assets/assetMover.

I kept both: this branch's imageCompletionData import and 6.x's ConflictResolution import. ConflictResolution is used in resolve(), and nothing else imports imageColorsCompletionData, so dropping that old import is safe. 6.x's newer color work in image-colors.ts, including BlurHashes (30d2d2a), still reaches uploads through readImageColors, which imageCompletionData calls. I didn't think any of this was ambiguous, so there are no CONFLICT-REVIEW comments.

Not checked: I didn't get a type-check or the uploader tests to run. npx refuses to run here because the repo requires pnpm. Running pnpm vitest run resources/js/modules/uploader and a type-check would confirm the merge.