Explore compressing URL Metrics to work within the 64 KiB limit

[westonruter]

This is a follow-up to #1851.

There is a maximum possible limit of 64 kibibytes (65,536 bytes) which can be submitted via navigator.sendBeacon(). This limit is the sum of all data sent by all calls to navigator.sendBeacon(), so when sending a URL Metric the size of the JSON may be less than this limit.

When there are 100 images on a page, this results in a URL Metric JSON body of that is ~53,371 bytes or 83% of 64 KiB limit. When there are 200 images, then the URL Metric is ~106,024 bytes, or 166% of 64 KiB limit.

So as more elements are tracked we could quickly run over our budget.

We can reduce the precision in the numbers to reduce the size somewhat. When there are 100 images, rounding the numbers to the nearest whole number reduces the size from 83% down to 68% of the budget. Other strategies could be employed, like coming up with a compression algorithm to replace common JSON strings. But we can explore that later as we start monitoring how close we are getting to the maximum.

I did some quick prototyping with Gemini and after a few iterations it seemed to come up with something workable: https://g.co/gemini/share/38e21d505e0f

For a URL Metric with 100 elements, this saves ~17% on the JSON size: https://codepen.io/westonruter/pen/OPJbpap?editors=1011

I'm sure the algorithm could be further refined! The tricky part would be after sending the compressed JSON to the REST API, that we would have to uncompress the JSON before it is processed by the REST API. This could look like adding a rest_pre_dispatch action which runs extremely early (like PHP_INT_MIN) and then checks to see if the endpoint is for storing a URL Metric and if so, to do $request->get_body() and then decompress the JSON and put it back via $request->set_body().

We should explore how feasible these compression strategies are.

Note: The URL Metric storage endpoint currently enforces a 64 KiB limit as well. This logic should be moved to the rest_pre_dispatch handler instead, since the decompressed URL Metric could be larger than 64 KiB.

[b1ink0]

I think we could use the Compression Streams API to compress the JSON. Here is the browser support for Compression Streams API.

I created this codepen to test the compression quality. I used the same URL Metrics JSON from your codepen. Below is the result:

I also wrote a PHP code snippet to decompress the same metrics, which you can check here:
https://3v4l.org/HVVd4#v8.4.4

Here’s a quick proof of concept (POC) I created:
b1ink0@cc1c372

[westonruter]

@b1ink0 wow, great idea! I love it. Way better compression as well. I see the CompressionStream API is available to >93% of browsers too via caniuse.

The WP REST API in core should add support for such automatically decompressing such gzipped JSON bodies. When saving a post in the block editor, for example, it could be sped up by using gzipping the body.

[westonruter]

Unfortunately, I found a critical bug with the Compression Stream API where if you try to use it during pagehide it causes the function execution to be terminated: #1928