Skip to content

Conversation

@danlapid
Copy link
Collaborator

@danlapid danlapid commented Dec 30, 2025

  • Enable LLD linker for macOS (35x faster linking)
  • Apply ci-limit-storage to all CI configs (smaller debug info, Some jobs were running out of disk space because they lacked this.)
  • Exclude type generation from //... builds (saves ~65s)
  • Increase CI parallelism from 8 to 32 jobs

@github-actions
Copy link

github-actions bot commented Dec 30, 2025

The generated output of @cloudflare/workers-types matches the snapshot in types/generated-snapshot 🎉

@codspeed-hq
Copy link

codspeed-hq bot commented Dec 30, 2025

CodSpeed Performance Report

Merging #5796 will not alter performance

Comparing dlapid/ci-improvements (3e9c1da) with main (413a412)

Summary

✅ 140 untouched
⏩ 38 skipped1

Footnotes

  1. 38 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@danlapid danlapid force-pushed the dlapid/ci-improvements branch 6 times, most recently from 8c500dc to c6f96ef Compare December 31, 2025 09:12
@danlapid danlapid changed the title [WIP] Try to make CI faster Improve CI build performance, especially on macOS Dec 31, 2025
@danlapid danlapid marked this pull request as ready for review December 31, 2025 09:12
@danlapid danlapid requested review from a team as code owners December 31, 2025 09:12
- Enable LLD linker for macOS (35x faster linking)
- Apply ci-limit-storage to all CI configs (smaller debug info, Some jobs were running out of disk space because they lacked this.)
- Exclude type generation from //... builds (saves ~65s)
- Increase CI parallelism from 8 to 32 jobs
@danlapid danlapid force-pushed the dlapid/ci-improvements branch from c6f96ef to 3e9c1da Compare December 31, 2025 10:05
@danlapid danlapid merged commit 79a7661 into main Dec 31, 2025
21 checks passed
@danlapid danlapid deleted the dlapid/ci-improvements branch December 31, 2025 12:55
@fhanau
Copy link
Contributor

fhanau commented Dec 31, 2025

I think several changes in this PR will be really useful. That being said, it should have been given some time for review:

  • I compile workerd on macOS daily, I can't possibly imagine lld resulting in "35x faster linking" in any configuration. Locally, the speedup is significantly smaller/non-existent under default settings: The default ld64 linker links workerd in 1.66s user time/0.536s CPU time, lld links in 0.97s user time/1.025s CPU time. Note that lld is slower since it is less parallel in this particular case, in terms of user time which matters more when linking several tests lld is faster but less than 2x so. It is possible that I/O bottlenecks in CI affect ld64 more than lld, but since both need to write roughly the same amount of data to disk this seems unlikely to be a big factor. We're currently using the previous Xcode (and accordingly lld) release on CI, but ld64 is unlikely to have become much faster in that release either.
  • Based on the limited speedup from local testing, I was expecting the time to download llvm and lld to be higher than the time saved on linking, that's why lld wasn't being used by default. I'll look at the bazel profiles to see how much the speedup actually is.
  • ci-limit-storage's -g0 will be a no-op for non-debug CI jobs since we do not enable debug info by default. For debug CI jobs, we already set the flag, so this won't actually affect debug info.
  • ci-limit-storage disables building benchmarks – I think that doesn't actually take effect for ci-linux and ci-linux-arm since we overwrite the filter set later to enable container tests, so we'll still have some coverage of building benchmarks. It might also not be taking effect in other CI jobs since we only modify build_tag_filters and not test_tag_filters, so if we run bazel test benchmarks should still be built since bazel is instructed to test them... will have to think more about that.
  • With the types change, we still end up building src/workerd/tools/param-names.json as part of //types/..., which can also be a bottleneck – I'll see what target is responsible for that Edit: This is an inherent limitation since js_run_binary does not propagate the "manual" tag to all of the targets it defines, so we depend on //src/workerd/tools:param_extractor anyway.
  • Let's see if macOS CI is more stable/not freezing anymore with this – in that case we should update to macOS 26 again

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants