Client Side Media iteration for WordPress 7.1

[adamsilverstein]

Overall Goal

Land client side media as a rich core feature in WordPress 7.1. This feature was originally planned for 7.0 but was punted to allow more time for more features to land.

Client-side media provides a progressive enhancement to the current server-based approach to media processing and delivers more robust and consistent media processing across WordPress installs, regardless of their underlying hosting stack. If the user's browser is not capable of processing media, the editor silently falls back to existing server-side processing.

Previous tracking issue: #74333 (WordPress 7.0 iteration)

What shipped during the 7.0 development cycle

In 7.0, the client media feature graduated from a Gutenberg experiment to a core Gutenberg feature.

Significant groundwork was completed during the 7.0 cycle, including:

• Core client-side image compression infrastructure using WebAssembly (WASM) and wasm-vips
• @wordpress/upload-media package with upload queue system
• @wordpress/vips package with fully inlined wasm-vips build
• @wordpress/web-worker package to offload processing in a worker thread
• SharedArrayBuffer / Cross-Origin Isolation support
• AVIF, WebP, and efficient JPEG output encoding
• Sub-size generation, EXIF metadata handling, auto-rotation, and large-image-size logic
• Graceful fallback to server-side processing for unsupported browsers or formats
• WebAssembly support detection and capability-based feature gating to avoid running on low powered devices

Goals for WordPress 7.1

Ship client-side media as a WordPress 7.1 core feature and deliver the following enhancements:

HEIC image support

iPhones capture photos in HEIC format by default. WordPress currently cannot process these on the server without additional libraries that most hosts don't install. Client-side media unlocks the ability to decode HEIC in the browser and transcode to a web-compatible format before upload.

• Media: Add HEIC canvas fallback for client-side processing #76732 — Add HEIC canvas fallback for client-side processing

Ultra HDR support

Ultra HDR (ISO 21496-1) embeds a gain map alongside an SDR base image, enabling HDR rendering on capable displays while remaining backward-compatible. Client-side processing must preserve gain map data through the resize and encode pipeline so sub-sizes retain HDR quality.

• Ensure UltraHDR images are handled properly #74874 — Ensure UltraHDR images are handled properly

Resilient uploading

Large or numerous uploads can fail due to network issues, timeouts, or server limits. Adding comprehensive error handling, retry logic, and clear progress feedback will make the upload experience significantly more robust.

• Add comprehensive error handling  #74366 — Add comprehensive error handling and retry logic
• Images: Implement upload progress UI #74363 — Implement upload progress UI in image overlay
• Track progress during media uploads #74362 — Track progress during media uploads

Performance

Client-side processing should be comperable in speed to server-side processing. Investigating concurrent uploads and optimizing the processing pipeline will reduce total upload time, especially for batch uploads.

• Investigate concurrent uploading of images #75257 — Investigate concurrent uploading of images
• Add performance metrics for client-side media processing #76789 — Add performance metrics for client-side media processing

Bug fixes & stability

• Client Side Media does not work in Playground #75941 — Client Side Media does not work in Playground
• Move image_editor_output_format data to the full sized image upload response #75784 — Move image_editor_output_format filtering to upload response
• Ensure REST API endpoint for side-loaded media covers all requirements #74360 — Ensure REST API endpoint for side-loaded media covers all requirements
• Investigate and fix crashes and console errors during client-side image upload processing #76706 — Investigate and fix crashes and console errors during client-side image upload processing

Testing & documentation

• Ensure complete e2e testing for client side media #74367 — Ensure complete e2e testing for client side media
• Add thorough documentation for Client Side Media #75111 — Add thorough documentation for Client Side Media

Related open PRs

• Revert client-side media processing plugin-only gate #76751 — Revert client-side media processing plugin-only gate
• Media: Add HEIC canvas fallback for client-side processing #76731 — Media: Add HEIC canvas fallback for client-side processing
• Fix image upload crashes and add media processing benchmarks #76707 — Fix image upload crashes and add media processing benchmarks
• Media: Add HEIC upload support via canonical plugin installer #76702 — Media: Add HEIC upload support via canonical plugin installer
• Media: Make client-side supported MIME types configurable #76549 — Media: Make client-side supported MIME types configurable
• WIP: Disable client-side media processing for non-Chromium browsers #76227 — Disable client-side media processing for non-Chromium browsers
• Build: Skip sourcemaps for WASM-inlined script module workers #75993 — Build: Reduce VIPS library size in build output
• Fill in E2E tests for client-side media processing #75949 — Fill in E2E tests for client-side media processing
• Docs: Add client-side media processing documentation #75895 — Docs: Add client-side media processing documentation
• Upload Media: Enable concurrent sideload uploads #75888 — Upload Media: Enable concurrent sideload uploads
• Update feature detection docblock for planned checks #75851 — Update feature detection docblock for planned checks
• Media: Move image output format filtering to upload response #75793 — Media: Move image output format filtering to upload response
• Image: Add upload progress indicator with cancel button and a11y announcements #75110 — Image: Add upload progress indicator with cancel button and a11y announcements
• Track progress during media uploads #75099 — Track progress during media uploads
• Add error handling improvements for upload-media #74917 — Add comprehensive error handling and retry logic for upload-media
• Add dimension validation to sideload endpoint #74903 — Add dimension validation to sideload endpoint
• Media: Add UltraHDR (ISO 21496-1) gain map support #74873 — Media: Add UltraHDR (ISO 21496-1) gain map support

Additional Features

New capabilities with dedicated tracking issues:

• Animated GIF to web video conversion #76942 — Client-side media: Animated GIF to video conversion (MP4/WebM)
• Add JPEG XL (JXL) support #76981 — Client Side Media: Add JPEG XL (JXL) support

7.1 or later

Capabilities that may land in 7.1 or remain on the longer-term roadmap:

• Video transcoding and compression
• Audio transcoding
• PDF poster image generation
• Image editing capabilities (crop, rotate, scale)
• BlurHash / dominant color placeholders
• Media Library integration and bulk optimization
• Per-size format optimization
• Saliency detection for thumbnail cropping