remotecache/s3: honor compression attributes

[ernetas]

--export-cache type=s3 silently ignores compression, compression-level and force-compression: the exporter returns a hardcoded compression.New(compression.Default) from Config(), so every layer is exported as gzip.

buildctl build ... \
  --export-cache type=s3,region=eu-west-1,bucket=my_bucket,compression=zstd,force-compression=true
# layers are still application/vnd.oci.image.layer.v1.tar+gzip

That value was never deliberate. e3f6d7b added Config() to the Exporter interface and wired compression.ParseAttributes into registry and local, leaving a placeholder returning the default in gha and inline; the s3 backend landed two months later in 09c5a7c and, having to satisfy the now-mandatory method, copied that placeholder rather than the registry implementation. Nothing in the s3 format is tied to gzip — Finalize already records the real media type of every layer and the importer reads it back, so blobs of any supported compression round-trip.

Changes

remotecache/s3: honor compression attributes — parse the attributes in ResolveCacheExporterFunc and return them from Config(), the same way registry and local do. Tests: a table test over the parsing (each valid value, and the error each invalid one produces), plus two integration tests that export, re-import, and assert the cache manifest records the requested media type and that the uploaded blobs really are in that format:

• testZstdS3CacheImportExport also exports llb.Image("busybox").Run(...) under a second manifest name. That build is what pins force-compression — the first only creates layers born in the requested compression, while this one has to convert the gzip base image layers.
• testUncompressedS3CacheImportExport covers the other end of the range, where the blob digest equals the diffID and the media type carries no compression suffix.

test: isolate mc config dir for minio servers — a prerequisite. mc keeps its aliases in a single folder defaulting to $HOME/.mc and NewMinioServer registers an alias there per server, so concurrent servers race while rewriting it: mc mb intermittently created the bucket on another test's server and the export then failed with NoSuchBucket. Each server now gets its own folder through MC_CONFIG_DIR, set in the environment rather than passed as --config-dir so that no invocation can miss it — mc treats <alias>/<bucket> of an unknown alias as a local path and silently succeeds.

README documents the three attributes, with a caveat that estargz layers are recorded as plain gzip in the s3 manifest and so cannot be lazily pulled from s3: LayerAnnotations has nowhere to keep their eStargz annotations.

Testing

$ go test ./cache/remotecache/s3/            # and with AWS_PROFILE=nonexistent
ok  	github.com/moby/buildkit/cache/remotecache/s3

$ TESTPKGS=./client TESTFLAGS="--run TestIntegration/Test.*S3CacheImportExport -v" ./hack/test integration
DONE 28 tests in 14.439s   # 9 workers, all pass, 0 skips

Both new integration tests fail without the Config() change, e.g.:

--- FAIL: TestIntegration/TestUncompressedS3CacheImportExport/worker=oci
    layer sha256:3a4bc23… has media type "application/vnd.oci.image.layer.v1.tar+gzip", expected uncompressed

Notes

• Behavior change: an invalid compression, compression-level or force-compression value on type=s3 now fails when the exporter is resolved — before any build work, and regardless of ignore-error — instead of being silently ignored. This matches registry and local.
• azblob and gha still carry the same placeholder; the gha one is noted in Support ZSTD compression in s3 remote cache #3071. Left out here to keep the change focused.

Fixes #3071

[ernetas]

@tonistiigi sorry, didn't see that this was already approved before pushing. Do you want me to drop minio test changes?

[crazy-max]

Can you check CI, seems there are some lint issues. Also can you rebase with master