Skip to content

Reland "Compute GUIDs once and store in metadata" (#184065) #201849

Merged
mtrofin merged 1 commit into
mainfrom
users/mtrofin/06-05-reland_184065
Jul 7, 2026
Merged

Reland "Compute GUIDs once and store in metadata" (#184065) #201849
mtrofin merged 1 commit into
mainfrom
users/mtrofin/06-05-reland_184065

Conversation

@mtrofin

@mtrofin mtrofin commented Jun 5, 2026

Copy link
Copy Markdown
Member

This reverts #201194, thus relanding @orodley's PR #184065 (and #200323):

This allows us to keep GUIDs consistent across compilation phases which may change the name or linkage type.
See https://discourse.llvm.org/t/rfc-keep-globalvalue-guids-stable/84801

The CFI issues that triggered the original revert are fixed by #201370, together with the addressing of the TODOs in LowerTypeTests.cpp left in the latter. The graphite diff between this change's V1 and V2 shows what's been added:

Currently, we reassign GUIDs when CFI promotes internal linkage symbols, which is counter to the goal of the RFC. This is addressed in PR #203171. The reason for this split fix can be explained on compiler-rt/test/cfi/icall/wrong-signature-mixed-lto.c. Here, a module with the exact same source path is compiled twice, under different conditional compilation, to produce 2 objects. Each object defines an internal linkage symbol with the same name (this is install_trap_loop_detection from compiler-rt/test/cfi/trap_loop_signal_handler.inc which is -include-d by both - see how %clang_cfi is defined). The ThinLTO GUID of this symbol will be the same. Its name won't be - because CFI promotes it and renames it using a hash that is based on the IR Module content (rather than the source path). During thinlink, LTO::addThinLTOwill mark each of the 2 exported symbols as prevailing in their corresponding modules. But that is done by associating their GUID to the module. So whichever comes last wins. The other symbol will be marked available externally and its body DCEd later in backend. But each module will refer to its copy of install_trap_loop_detection, and so we end up with a linker error.

As mentioned, the fix is in PR #203171, and this relanding PR just maintains the existing ThinLTO behavior by rewriting the GUIDs. Since we haven't yet leveraged the GUID mechanics for e.g. simplifying PGO, this aspect of this change is essentially NFC.

Co-authored-by: Owen Rodley orodley@google.com

mtrofin commented Jun 5, 2026

Copy link
Copy Markdown
Member Author

@mtrofin mtrofin force-pushed the users/mtrofin/06-05-reland_184065 branch 2 times, most recently from ad94f68 to a1a789e Compare June 5, 2026 15:21
@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown

🐧 Linux x64 Test Results

  • 208570 tests passed
  • 6790 tests skipped

✅ The build succeeded and all tests passed.

@mtrofin mtrofin force-pushed the users/mtrofin/06-05-reland_184065 branch 4 times, most recently from 8cd8eb5 to f053482 Compare June 9, 2026 02:08
@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown

✅ With the latest revision this PR passed the C/C++ code formatter.

@mtrofin mtrofin force-pushed the users/mtrofin/06-05-reland_184065 branch from f053482 to 9926393 Compare June 9, 2026 02:10
@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown

🪟 Windows x64 Test Results

  • 139734 tests passed
  • 4885 tests skipped

✅ The build succeeded and all tests passed.

@mtrofin mtrofin force-pushed the users/mtrofin/06-05-reland_184065 branch 2 times, most recently from 13d3ad0 to f8a1ec9 Compare June 11, 2026 04:58
@mtrofin mtrofin force-pushed the users/mtrofin/06-05-reland_184065 branch 2 times, most recently from 854a6f9 to 6dc97cf Compare June 16, 2026 03:17
@github-actions

github-actions Bot commented Jun 16, 2026

Copy link
Copy Markdown

✅ With the latest revision this PR passed the LLVM ABI annotation checker.

@mtrofin mtrofin force-pushed the users/mtrofin/06-05-reland_184065 branch 2 times, most recently from 6f02138 to c7ff207 Compare June 17, 2026 14:54
@mtrofin mtrofin changed the base branch from main to users/mtrofin/06-17-_coro_handle_aliases June 17, 2026 18:14
@mtrofin mtrofin force-pushed the users/mtrofin/06-05-reland_184065 branch from c7ff207 to 525046e Compare June 17, 2026 18:15
@mtrofin mtrofin marked this pull request as ready for review June 17, 2026 18:18
@llvmorg-github-actions llvmorg-github-actions Bot added lld backend:X86 clang:codegen IR generation bugs: mangling, exceptions, etc. llvm:codegen lld:ELF PGO Profile Guided Optimizations LTO Link time optimization (regular/full LTO or ThinLTO) labels Jun 17, 2026
@mtrofin mtrofin force-pushed the users/mtrofin/06-05-reland_184065 branch from fa7961a to 1817d11 Compare June 18, 2026 04:14

mtrofin commented Jun 18, 2026

Copy link
Copy Markdown
Member Author

@pektezol - please see my notes about removing the test introduced in #194383. Does it make sense?

@mtrofin mtrofin force-pushed the users/mtrofin/06-05-reland_184065 branch from 1817d11 to 2829ab3 Compare June 18, 2026 05:19
@pektezol

Copy link
Copy Markdown
Contributor

@pektezol - please see my notes about removing the test introduced in #194383. Does it make sense?

@mtrofin LGTM

@nikic

nikic commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Does this also address the compile-time regressions from the previous implementation?

}

if (!ShouldCloneDefinition(&I)) {
CopyMD();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this line create a blowup of unnecessary copies?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch - fixed. Thanks!

Base automatically changed from users/mtrofin/06-17-_coro_handle_aliases to main June 18, 2026 14:34

mtrofin commented Jun 18, 2026

Copy link
Copy Markdown
Member Author

I looked more into it but didn't update the other thread.

IIRC the remaining surprise was the clang build regressing by 0.15%, because for the single-TU compilations, the regression was attributable to modules with lots of global values but not so much IR, where the bitcode serialization of the extra GUID table would start showing up.

I'm actually not able to repro the clang build - I figured in the meantime that the "instructions" link in the clang-build part of the report gives me a per-module compilation change, and I found a module (tools/clang/lib/CodeGen/CMakeFiles/obj.clangCodeGen.dir/ModuleLinker.cpp.o) for which we regress like 30% (!) - but when I repro locally and perf stat -e instructions -r 10 <clang> I see maybe a 0.1% change.

What's the cmake you use for building the host clang, and what's the cmake for the build target?

@mtrofin mtrofin force-pushed the users/mtrofin/06-05-reland_184065 branch 2 times, most recently from a6fdde1 to 4da2c32 Compare June 23, 2026 01:23

mtrofin commented Jun 23, 2026

Copy link
Copy Markdown
Member Author

Turns out the clang build regression was in large due to some #includes in the change that weren't necessary. The 30% regression when compiling ModuleLinker goes away or reappears when either old or new compiler (i.e. pre/post this change) compiles the old or the new source.

Fixed and re-ran the compile time tracker, the largest clang-build regression is now 1%.

@mtrofin mtrofin force-pushed the users/mtrofin/06-05-reland_184065 branch from 4da2c32 to 4bf5379 Compare June 23, 2026 01:28
@boomanaiden154

Copy link
Copy Markdown
Contributor

The stage1-ReleaseThinLTO results still don't look great and it seems like the geomean for the clang build is about the same?

@mtrofin mtrofin force-pushed the users/mtrofin/06-05-reland_184065 branch from 4bf5379 to 1cbfe8b Compare June 23, 2026 02:06

mtrofin commented Jun 23, 2026

Copy link
Copy Markdown
Member Author

The stage1-ReleaseThinLTO results still don't look great and it seems like the geomean for the clang build is about the same?

Note that some compile time performance effect in thinlto builds should be expected: we serialize/deserialize more. I think the main concern was "is there something else".

re. stage1 - yes, see my earlier response ("for the single-TU compilations, the regression was attributable to modules with lots of global values but not so much IR, where the bitcode serialization of the extra GUID table would start showing up.")

Re. clang-build, there's little change in the overall build timing (I don't think it's a geomean) because the individual compilation outliers like ModuleLinker were few and their negative contribution likely lost, before, in the overall parallel compilation part of the build. The rest is very likely for the same reason (more serialization) seen in the benchmarks builds, as at most we see compilation regressions within the same relative ranges for individual modules.

@nikic

nikic commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

To clarify, the regression I was concerned about is the one on bin/clang-23, i.e. the one during thin linking, not the one on individual files (pre-link).

mtrofin commented Jun 23, 2026

Copy link
Copy Markdown
Member Author

Ah, ok. About that one: perf stat -e instructions -r 10 actually shows ~6% improvement with the new change; when I look at cyclesI see a 1% improvement. Wallclock average showed the same. But, the wallclock measurements are actually pretty noisy - as much as 1.31% over the 10 reps.

Looking at the profile itself, (perf record -e instructions):

Before:

+   13.23%    13.23%  ld.lld          lld                   [.] llvm::MD5::body(llvm::ArrayRef<unsigne
+   12.83%    12.82%  ld.lld          lld                   [.] llvm::SimpleBitstreamCursor::Read(unsi
+   10.19%    10.19%  ld.lld          lld                   [.] llvm::SimpleBitstreamCursor::ReadVBR64
+    6.54%     0.00%  ld.lld          [unknown]             [.] 0000000000000000
+    4.64%     4.64%  ld.lld          lld                   [.] llvm::BitstreamCursor::readRecord(unsi
+    2.11%     2.11%  ld.lld          lld                   [.] llvm::xxh3_64bits(unsigned char const*

After:

+   15.02%    15.02%  ld.lld          lld                   [.] llvm::SimpleBitstreamCursor::Read(unsi
+   10.64%    10.64%  ld.lld          lld                   [.] llvm::SimpleBitstreamCursor::ReadVBR64
+    8.56%     0.00%  ld.lld          [unknown]             [.] 0000000000000000
+    6.48%     6.48%  ld.lld          lld                   [.] llvm::BitstreamCursor::readRecord(unsi
+    3.19%     3.19%  ld.lld          lld                   [.] llvm::MD5::body(llvm::ArrayRef<unsigne
+    2.96%     2.96%  ld.lld          lld                   [.] llvm::xxh3_64bits(unsigned char const*

This lines up with the "we serialize more" (and so we deserialize during linking).

Not sure what to make of the perf stat results difference. Perhaps cold runs -> cold caches -> patch is deserialization heavy -> deserialization (of the GUIDs) costs more than MD5 hashing, but then (hot caches) "the tables flip".

@teresajohnson teresajohnson left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mtrofin mtrofin force-pushed the users/mtrofin/06-05-reland_184065 branch from 1cbfe8b to 55df97c Compare June 30, 2026 01:29
@mtrofin mtrofin force-pushed the users/mtrofin/06-05-reland_184065 branch from 55df97c to e432443 Compare July 6, 2026 14:51
@mtrofin mtrofin merged commit 39dcb0f into main Jul 7, 2026
11 checks passed
@mtrofin mtrofin deleted the users/mtrofin/06-05-reland_184065 branch July 7, 2026 23:04
@llvm-ci

llvm-ci commented Jul 7, 2026

Copy link
Copy Markdown

LLVM Buildbot has detected a new failure on builder libc-riscv32-qemu-yocto-fullbuild-dbg running on rv32gc-qemu-system while building clang,lld,llvm at step 3 "checkout".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/196/builds/20771

Here is the relevant piece of the build log for the reference
Step 3 (checkout) failure: update (failure)
git version 2.45.2
error: RPC failed; HTTP 429 curl 22 The requested URL returned error: 429
fatal: expected flush after ref listing
error: RPC failed; HTTP 429 curl 22 The requested URL returned error: 429
fatal: expected flush after ref listing

@llvm-ci

llvm-ci commented Jul 7, 2026

Copy link
Copy Markdown

LLVM Buildbot has detected a new failure on builder amdgpu-hip-blender running on AMD-bb-w-04 while building clang,lld,llvm at step 3 "annotate".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/232/builds/3124

Here is the relevant piece of the build log for the reference
Step 3 (annotate) failure: '../llvm-zorg/zorg/buildbot/builders/annotated/hip-build.sh --jobs=' (failure)
...
+ . ../llvm-zorg/zorg/buildbot/builders/annotated/buildbot-helper.sh
+ set -eu
+ halt_on_failure
+ echo @@@HALT_ON_FAILURE@@@
@@@HALT_ON_FAILURE@@@
+ setup_env
+ build_step 'Setting up the buildbot'
+ echo '@@@BUILD_STEP Setting up the buildbot@@@'
+ BUILDBOT_ROOT=/home/botworker/bbot/hip-blender
@@@BUILD_STEP Setting up the buildbot@@@
++ whoami
+ BUILDBOT_SLAVENAME=botworker
+ AMDGPU_ARCHS='gfx908;gfx90a;gfx1030;gfx1100'
+ EXTERNAL_DIR=/opt/botworker/llvm/External
+ BUILD_DIR=/home/botworker/bbot/hip-blender/botworker/amdgpu-hip-blender
+ DESTDIR=/home/botworker/bbot/hip-blender/botworker/amdgpu-hip-blender/install
+ LLVM_ROOT=/home/botworker/bbot/hip-blender/llvm-project
+ LLVM_REVISION=39dcb0ff91b312bb269334168d5e32be78b60417
+ LLVM_BUILD_DIR=/home/botworker/bbot/hip-blender/botworker/amdgpu-hip-blender/llvm
+ TESTSUITE_ROOT=/home/botworker/bbot/hip-blender/llvm-test-suite
+ TEST_BUILD_DIR=/home/botworker/bbot/hip-blender/botworker/amdgpu-hip-blender/test-suite-build
+ NINJAOPT=
+ echo BUILDBOT_ROOT=/home/botworker/bbot/hip-blender
+ echo BUILDBOT_SLAVENAME=botworker
+ echo 'AMDGPU_ARCHS=gfx908;gfx90a;gfx1030;gfx1100'
+ echo LLVM_ROOT=/home/botworker/bbot/hip-blender/llvm-project
BUILDBOT_ROOT=/home/botworker/bbot/hip-blender
BUILDBOT_SLAVENAME=botworker
AMDGPU_ARCHS=gfx908;gfx90a;gfx1030;gfx1100
LLVM_ROOT=/home/botworker/bbot/hip-blender/llvm-project
TESTSUITE_ROOT=/home/botworker/bbot/hip-blender/llvm-test-suite
EXTERNAL_DIR=/opt/botworker/llvm/External
BUILD_DIR=/home/botworker/bbot/hip-blender/botworker/amdgpu-hip-blender
DESTDIR=/home/botworker/bbot/hip-blender/botworker/amdgpu-hip-blender/install
+ echo TESTSUITE_ROOT=/home/botworker/bbot/hip-blender/llvm-test-suite
+ echo EXTERNAL_DIR=/opt/botworker/llvm/External
+ echo BUILD_DIR=/home/botworker/bbot/hip-blender/botworker/amdgpu-hip-blender
+ echo DESTDIR=/home/botworker/bbot/hip-blender/botworker/amdgpu-hip-blender/install
+ echo LLVM_BUILD_DIR=/home/botworker/bbot/hip-blender/botworker/amdgpu-hip-blender/llvm
+ echo TEST_BUILD_DIR=/home/botworker/bbot/hip-blender/botworker/amdgpu-hip-blender/test-suite-build
LLVM_BUILD_DIR=/home/botworker/bbot/hip-blender/botworker/amdgpu-hip-blender/llvm
TEST_BUILD_DIR=/home/botworker/bbot/hip-blender/botworker/amdgpu-hip-blender/test-suite-build
+ update_llvm
+ '[' '!' -d /home/botworker/bbot/hip-blender/llvm-project ']'
+ build_step 'Updating llvm-project repo'
+ echo '@@@BUILD_STEP Updating llvm-project repo@@@'
@@@BUILD_STEP Updating llvm-project repo@@@
+ git -C /home/botworker/bbot/hip-blender/llvm-project fetch --prune origin
error: RPC failed; HTTP 429 curl 22 The requested URL returned error: 429
fatal: expected flush after ref listing
Step 5 (Updating llvm-project repo) failure: Updating llvm-project repo (failure)
@@@BUILD_STEP Updating llvm-project repo@@@
+ git -C /home/botworker/bbot/hip-blender/llvm-project fetch --prune origin
error: RPC failed; HTTP 429 curl 22 The requested URL returned error: 429
fatal: expected flush after ref listing
program finished with exit code 128
elapsedTime=7.489016

@llvm-ci

llvm-ci commented Jul 7, 2026

Copy link
Copy Markdown

LLVM Buildbot has detected a new failure on builder llvm-clang-x86_64-gcc-ubuntu running on sie-linux-worker3 while building clang,lld,llvm at step 6 "test-build-unified-tree-check-all".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/174/builds/37887

Here is the relevant piece of the build log for the reference
Step 6 (test-build-unified-tree-check-all) failure: test (failure)
******************** TEST 'libFuzzer-x86_64-default-Linux :: symbolize-deadlock.test' FAILED ********************
Exit Code: 1

Command Output (stdout):
--
# RUN: at line 1
/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/./bin/clang   -m64  --driver-mode=g++ -O2 -gline-tables-only -fsanitize=address,fuzzer -I/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/compiler-rt/lib/fuzzer  /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/compiler-rt/test/fuzzer/SymbolizeDeadlock.cpp -o /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/runtimes/runtimes-bins/compiler-rt/test/fuzzer/X86_64DefaultLinuxConfig/Output/symbolize-deadlock.test.tmp
# executed command: /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/./bin/clang -m64 --driver-mode=g++ -O2 -gline-tables-only -fsanitize=address,fuzzer -I/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/compiler-rt/lib/fuzzer /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/compiler-rt/test/fuzzer/SymbolizeDeadlock.cpp -o /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/runtimes/runtimes-bins/compiler-rt/test/fuzzer/X86_64DefaultLinuxConfig/Output/symbolize-deadlock.test.tmp
# note: command had no output on stdout or stderr
# RUN: at line 2
not  /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/runtimes/runtimes-bins/compiler-rt/test/fuzzer/X86_64DefaultLinuxConfig/Output/symbolize-deadlock.test.tmp -rss_limit_mb=20 2>&1
# executed command: not /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/runtimes/runtimes-bins/compiler-rt/test/fuzzer/X86_64DefaultLinuxConfig/Output/symbolize-deadlock.test.tmp -rss_limit_mb=20
# .---command stdout------------
# | INFO: Running with entropic power schedule (0xFF, 100).
# | INFO: Seed: 817533373
# | INFO: Loaded 1 modules   (10 inline 8-bit counters): 10 [0x558156653e48, 0x558156653e52), 
# | INFO: Loaded 1 PC tables (10 PCs): 10 [0x558156653e58,0x558156653ef8), 
# | INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
# | INFO: A corpus is not provided, starting from an empty corpus
# | #2	INITED cov: 2 ft: 2 corp: 1/1b exec/s: 0 rss: 101Mb
# | #4	NEW    cov: 3 ft: 3 corp: 2/4b lim: 4 exec/s: 0 rss: 101Mb L: 3/3 MS: 2 CrossOver-CrossOver-
# | #572	NEW    cov: 4 ft: 4 corp: 3/12b lim: 8 exec/s: 0 rss: 101Mb L: 8/8 MS: 3 InsertRepeatedBytes-ChangeBinInt-ShuffleBytes-
# | #599	REDUCE cov: 4 ft: 4 corp: 3/9b lim: 8 exec/s: 0 rss: 101Mb L: 5/5 MS: 2 CopyPart-CrossOver-
# | #745	REDUCE cov: 4 ft: 4 corp: 3/7b lim: 8 exec/s: 0 rss: 101Mb L: 3/3 MS: 1 EraseBytes-
# | #37395	REDUCE cov: 5 ft: 5 corp: 4/11b lim: 365 exec/s: 0 rss: 101Mb L: 4/4 MS: 5 EraseBytes-ChangeBit-CopyPart-ShuffleBytes-CopyPart-
# | #37666	REDUCE cov: 5 ft: 5 corp: 4/10b lim: 365 exec/s: 0 rss: 101Mb L: 3/3 MS: 1 EraseBytes-
# | 	NEW_FUNC[1/1]: 0x558156606550 in aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
# | #50692	REDUCE cov: 9 ft: 9 corp: 5/14b lim: 493 exec/s: 0 rss: 101Mb L: 4/4 MS: 1 InsertByte-
# | ==2638296== ERROR: libFuzzer: out-of-memory (used: 101Mb; limit: 20Mb)
# |    To change the out-of-memory limit use -rss_limit_mb=<N>
# | 
# | Live Heap Allocations: 24465951 bytes in 74 chunks; quarantined: 1195178 bytes in 52444 chunks; 17561 other chunks; total chunks: 70079; showing top 95% (at most 8 unique contexts)
# | 21233984 byte(s) (86%) in 1 allocation(s)
# |     #0 0x55815660515d in operator new(unsigned long) /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/compiler-rt/lib/asan/asan_new_delete.cpp:109:35
# |     #1 0x5581564e821a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:822:18
# |     #2 0x5581565161c2 in main /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
# |     #3 0x7fa082b0e082 in __libc_start_main /build/glibc-B3wQXB/glibc-2.31/csu/../csu/libc-start.c:308:16
# | 
# | 3148000 byte(s) (12%) in 1 allocation(s)
# |     #0 0x55815660515d in operator new(unsigned long) /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/compiler-rt/lib/asan/asan_new_delete.cpp:109:35
# |     #1 0x5581564e81f4 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:821:14
# |     #2 0x5581565161c2 in main /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
# |     #3 0x7fa082b0e082 in __libc_start_main /build/glibc-B3wQXB/glibc-2.31/csu/../csu/libc-start.c:308:16
# | 
# | MS: 2 ShuffleBytes-ChangeByte-
# `-----------------------------
# error: command failed with exit status: 1

--

...

gandhi56 pushed a commit that referenced this pull request Jul 9, 2026
This reverts #201194, thus relanding @orodley's PR #184065 (and
#200323):

> This allows us to keep GUIDs consistent across compilation phases
which may change the name or linkage type.
> See
https://discourse.llvm.org/t/rfc-keep-globalvalue-guids-stable/84801

The CFI issues that triggered the original revert are fixed by #201370,
together with the addressing of the TODOs in `LowerTypeTests.cpp` left
in the latter. The [graphite
diff](https://app.graphite.com/github/pr/llvm/llvm-project/201849/Reland-%23184065)
between this change's V1 and V2 shows what's been added:

- the `TODO`s from #201370 are done
- in LowerTypeTests.cpp, passing `!guid` when creating a new declaration
and when converting a definition to a declaration.
- `llvm/test/Transforms/LowerTypeTests/export-icall.ll` tests also the
above def->decl conversion
- removed
`test/Analysis/CtxProfAnalysis/flatten-prethinlink-requires-guid-metadata.ll`
introduced in #194383 (this was between the revert and this PR), as now
the general expectation is that GUID assignment happens appropriately
and all passes use `getGUID`, so there's no reason for `CtxProfAnalysis`
to do something different.

Currently, we reassign GUIDs when CFI promotes internal linkage symbols,
which is counter to the goal of the RFC. This is addressed in PR
#203171. The reason for this split fix can be explained on
`compiler-rt/test/cfi/icall/wrong-signature-mixed-lto.c`. Here, a module
with the exact same source path is compiled twice, under different
conditional compilation, to produce 2 objects. Each object defines an
internal linkage symbol with the same name (this is
`install_trap_loop_detection` from
`compiler-rt/test/cfi/trap_loop_signal_handler.inc` which is
`-include`\-d by both - see how `%clang_cfi` is defined). The ThinLTO
GUID of this symbol will be the same. Its name won't be - because CFI
promotes it and renames it using a hash that is based on the IR Module
content (rather than the source path). During thinlink,
`LTO::addThinLTO`will mark each of the 2 exported symbols as prevailing
in their corresponding modules. But that is done by associating their
GUID to the module. So whichever comes last wins. The other symbol will
be marked available externally and its body DCEd later in backend. But
each module will refer to its copy of `install_trap_loop_detection`, and
so we end up with a linker error.

As mentioned, the fix is in PR #203171, and this relanding PR just
maintains the existing ThinLTO behavior by rewriting the GUIDs. Since we
haven't yet leveraged the GUID mechanics for e.g. simplifying PGO, this
aspect of this change is essentially NFC.

Co-authored-by: Owen Rodley <orodley@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend:X86 clang:codegen IR generation bugs: mangling, exceptions, etc. lld:ELF lld llvm:analysis Includes value tracking, cost tables and constant folding llvm:codegen llvm:ir llvm:transforms LTO Link time optimization (regular/full LTO or ThinLTO) PGO Profile Guided Optimizations

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants