[ci] Cache Android toolchain downloads + bump DownloadFile retries

[jonathanpeppers]

Context

PR builds frequently fail on dl.google.com DNS / TCP errors during the initial download of the Android toolchain (commandlinetools, build-tools, emulator, system images, m2repository, docs, source, etc.) on dnceng-public agents. Example from a recent build:

error MSB3923: Failed to download file
"https://dl.google.com/android/repository/source-36_r01.zip".
nodename nor servname provided, or not known (dl.google.com:443)

This costs the entire ~4 hour pipeline run for one transient DNS hiccup.

Changes

1. Cache $(AndroidToolchainCacheDirectory) between runs

New build-tools/automation/yaml-templates/cache-android-archives.yaml template, modeled exactly on the existing cache-gradle.yaml. Wired into all three of build-{linux,macos,windows}-steps.yaml right next to the existing Gradle cache step.

Cache safety. Every download under $HOME/android-archives is verified after download by the corresponding MSBuild target (androidsdk / openjdk / binutils / aapt2 / bundletool). Most targets check the downloaded file's SHA-256 against a hash hardcoded in Configuration.props; openjdk instead verifies against Microsoft's published .sha256sum.txt. In all cases a corrupt or stale archive is deleted, the build is failed, and the next run re-downloads it fresh — so cache hits cannot mask a version bump.

Cache key: Configuration.props + each download *.targets file. Any version or hash change invalidates the cache. The restoreKeys fallback (Agent.OS only) still recovers most bytes when only a subset of components changes.

Cache size. ~2 GB physical / ~10.7 GB raw per OS, comfortably within Azure Pipelines' per-key limit.

2. Bump DownloadFile retries

Retries="3" → Retries="5" and added RetryDelayMilliseconds="5000" on every DownloadFile invocation. This helps the first (cold-cache) and scheduled runs — which are responsible for warming the shared cache — better tolerate transient flakes on dl.google.com / aka.ms / github.com.

3. Delete corrupt bundletool.jar on SHA-256 mismatch

bundletool.targets previously errored on hash mismatch but left the corrupt jar in place. With the new pipeline cache, that bad jar could be persisted and repeatedly break future builds. Now matches the pattern used in androidsdk/binutils/aapt2/openjdk.

Verification — what we actually save

Measured against build #1456864 (first run with the cache, so all three OSes are a cache miss — they still downloaded everything; the cache save for the next build is what completed successfully):

Job	Cache step (miss)	What actually got downloaded	Time the cache will save on hit
Linux > Build	✅ 5s — miss	~2 GB SDK fresh (00:37:00 → 00:38:33)	~1.5 min per build
macOS > Build	✅ 54s — miss	Nothing — agent already had every archive on disk ("Did not download file … SkipUnchangedFiles=true")	~0 (agents persist $HOME)
Windows > Build & Smoke Test	✅ 1s — miss	Only JDK (~200 MB, a few seconds); SDK comes from agent image	~0

So the wall-clock savings are modest: ~1.5–2 min per Linux build on a cache hit, near-zero on macOS / Windows.

The bigger value of this PR is reliability, not throughput. dnceng-public Linux agents are ephemeral and download ~2 GB from dl.google.com on every build today. The previous motivating failure (build #1456555) lost a 4-hour run to a single DNS resolution error on one of those downloads. Cache hits avoid the request entirely; the bumped retries help the cold-cache and scheduled runs that have to repopulate the cache.

[Copilot]

Pull request overview

This PR improves CI reliability for dotnet/android by reducing sensitivity to transient network/DNS failures when downloading the Android toolchain, and by reusing previously downloaded archives across ephemeral agents.

Changes:

• Add an Azure Pipelines cache for $(AndroidToolchainCacheDirectory) (default ~/android-archives) and wire it into Linux/macOS/Windows build step templates.
• Increase DownloadFile retries from 3 → 5 and add a 5s retry delay across toolchain download targets (Android SDK packages, aapt2/build-tools, OpenJDK, binutils, bundletool).
• Introduce a cache key strategy based on Configuration.props and the download targets so cache invalidates on version/hash changes.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/openjdk/openjdk.targets	Increase download retries and add retry delay for OpenJDK archive + hash file downloads.
src/bundletool/bundletool.targets	Increase download retries and add retry delay for bundletool jar download.
src/binutils/binutils.targets	Increase download retries and add retry delay for binutils archive download.
src/androidsdk/androidsdk.targets	Increase download retries and add retry delay for Android SDK package downloads.
src/aapt2/aapt2.targets	Increase download retries and add retry delay for build-tools (aapt2) downloads.
build-tools/automation/yaml-templates/cache-android-archives.yaml	New template to cache Android toolchain archives directory between pipeline runs.
build-tools/automation/yaml-templates/build-windows-steps.yaml	Wire android-archives cache template into Windows build steps.
build-tools/automation/yaml-templates/build-macos-steps.yaml	Wire android-archives cache template into macOS build steps.
build-tools/automation/yaml-templates/build-linux-steps.yaml	Wire android-archives cache template into Linux build steps.