[IR] Only allow lifetime.start/end on allocas #149310
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Author
|
@efriedma-quic @fhahn @preames @dtcxzyw I'd like some early feedback on this before I sink time into updating more tests. Are there any concerns with limiting lifetime markers to allocas only? |
Member
|
Generally, it makes sense to me. I also thought about this last year: dtcxzyw/llvm-tools@c5ce24a Unfortunately, the code reminds me that lifetime markers are also used on BTW, does the order of lifetime markers matter? I mean the start-end sequence should look like a balanced bracket sequence. See also https://github.com/dtcxzyw/llvm-project/pull/2/files#diff-afb2b04eff951cb67d214bc6dfa62a087a72591451817dba7929b48c5fe635f9. |
Contributor
Author
Hm, I've not encountered this on any clang tests or in llvm-test-suite. I wonder whether this is an older issue that has since been fixed?
I don't think it matters right now as long as the interference graph is preserved. We'd want this property though for a more intrusive redesign of lifetime intrinsics, along the lines of what is discussed in #45725. |
Contributor
Author
Looking a bit closer, it looks like even though this is in the |
nikic
force-pushed
the
lifetime-alloca-only-2
branch
from
0fa2a96 to
955b133
Compare
dtcxzyw
reviewed
Jul 18, 2025
Member
dtcxzyw
left a comment
dtcxzyw
left a comment
There was a problem hiding this comment.
dead code:
llvm-project/llvm/lib/Transforms/Utils/MemoryTaggingSupport.cpp
Lines 237 to 239 in 7d040d4
Contributor
Author
Removed in 0121314. |
llvmbot
added
backend:AArch64
backend:X86
llvm:codegen
llvm:globalisel
mlir:llvm
mlir
llvm:instcombine
Member
|
@llvm/pr-subscribers-llvm-ir @llvm/pr-subscribers-backend-aarch64 Author: Nikita Popov (nikic) Changeslifetime.start and lifetime.end are primarily intended for use on allocas, to enable stack coloring and other liveness optimizations. This is necessary because all (static) allocas are hoisted into the entry block, so lifetime markers are necessary to convey the actual lifetimes. However, lifetime.start and lifetime.end are currently allowed to be used on non-alloca pointers. We don't actually do this in practice, but just the mere fact that this is possible breaks the core purpose of the lifetime markers, which is stack coloring of allocas. Stack coloring can only work correctly if all lifetime markers for an alloca are analyzable.
I don't think there is any way to have coherent semantics for lifetime markers on allocas, while also permitting them on arbitrary pointer values. This PR restricts lifetimes to operate on allocas only. As a followup, I will also drop the size argument, which is superfluous if we always operate on an alloca. (This change also renders various code handling lifetime markers on non-alloca dead. I plan to clean up that kind of code after dropping the size argument as well.) In practice, I've only found a few places that currently produce lifetimes on non-allocas:
Patch is 157.93 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/149310.diff 63 Files Affected:
diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst
index 371f356c80b0a..aacdf914ba61d 100644
--- a/llvm/docs/LangRef.rst
+++ b/llvm/docs/LangRef.rst
@@ -26634,19 +26634,14 @@ Arguments:
The first argument is a constant integer representing the size of the
object, or -1 if it is variable sized. The second argument is a pointer
-to the object.
+to an ``alloca`` instruction.
Semantics:
""""""""""
-If ``ptr`` is a stack-allocated object and it points to the first byte of
-the object, the object is initially marked as dead.
-``ptr`` is conservatively considered as a non-stack-allocated object if
-the stack coloring algorithm that is used in the optimization pipeline cannot
-conclude that ``ptr`` is a stack-allocated object.
-
-After '``llvm.lifetime.start``', the stack object that ``ptr`` points is marked
-as alive and has an uninitialized value.
+The stack-allocated object that ``ptr`` points to is initially marked as dead.
+After '``llvm.lifetime.start``', the stack object is marked as alive and has an
+uninitialized value.
The stack object is marked as dead when either
:ref:`llvm.lifetime.end <int_lifeend>` to the alloca is executed or the
function returns.
@@ -26656,11 +26651,6 @@ After :ref:`llvm.lifetime.end <int_lifeend>` is called,
The second '``llvm.lifetime.start``' call marks the object as alive, but it
does not change the address of the object.
-If ``ptr`` is a non-stack-allocated object, it does not point to the first
-byte of the object or it is a stack object that is already alive, it simply
-fills all bytes of the object with ``poison``.
-
-
.. _int_lifeend:
'``llvm.lifetime.end``' Intrinsic
@@ -26684,24 +26674,16 @@ Arguments:
The first argument is a constant integer representing the size of the
object, or -1 if it is variable sized. The second argument is a pointer
-to the object.
+to an ``alloca`` instruction.
Semantics:
""""""""""
-If ``ptr`` is a stack-allocated object and it points to the first byte of the
-object, the object is dead.
-``ptr`` is conservatively considered as a non-stack-allocated object if
-the stack coloring algorithm that is used in the optimization pipeline cannot
-conclude that ``ptr`` is a stack-allocated object.
+The stack-allocated object that ``ptr`` points to becomes dead after the call
+to this intrinsic.
Calling ``llvm.lifetime.end`` on an already dead alloca is no-op.
-If ``ptr`` is a non-stack-allocated object or it does not point to the first
-byte of the object, it is equivalent to simply filling all bytes of the object
-with ``poison``.
-
-
'``llvm.invariant.start``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
diff --git a/llvm/lib/Bitcode/Reader/BitcodeReader.cpp b/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
index 66ecc69c9874d..82c94b689aca6 100644
--- a/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
+++ b/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
@@ -7116,9 +7116,11 @@ Error BitcodeReader::materializeModule() {
if (CallInst *CI = dyn_cast<CallInst>(U))
UpgradeIntrinsicCall(CI, I.second);
}
- if (!I.first->use_empty())
- I.first->replaceAllUsesWith(I.second);
- I.first->eraseFromParent();
+ if (I.first != I.second) {
+ if (!I.first->use_empty())
+ I.first->replaceAllUsesWith(I.second);
+ I.first->eraseFromParent();
+ }
}
UpgradedIntrinsics.clear();
diff --git a/llvm/lib/CodeGen/SafeStack.cpp b/llvm/lib/CodeGen/SafeStack.cpp
index 996207034d076..908ed96172615 100644
--- a/llvm/lib/CodeGen/SafeStack.cpp
+++ b/llvm/lib/CodeGen/SafeStack.cpp
@@ -614,6 +614,13 @@ Value *SafeStack::moveStaticAllocasToUnsafeStack(
Use &U = *AI->use_begin();
Instruction *User = cast<Instruction>(U.getUser());
+ // Drop lifetime markers now that this is no longer an alloca.
+ // SafeStack has already performed its own stack coloring.
+ if (User->isLifetimeStartOrEnd()) {
+ User->eraseFromParent();
+ continue;
+ }
+
Instruction *InsertBefore;
if (auto *PHI = dyn_cast<PHINode>(User))
InsertBefore = PHI->getIncomingBlock(U)->getTerminator();
diff --git a/llvm/lib/IR/AutoUpgrade.cpp b/llvm/lib/IR/AutoUpgrade.cpp
index 86285a03c66bb..28ed1e520ce52 100644
--- a/llvm/lib/IR/AutoUpgrade.cpp
+++ b/llvm/lib/IR/AutoUpgrade.cpp
@@ -1310,6 +1310,18 @@ static bool upgradeIntrinsicFunction1(Function *F, Function *&NewFn,
return true;
}
break;
+ case 'l':
+ if (Name.starts_with("lifetime.start") ||
+ Name.starts_with("lifetime.end")) {
+ // Unless remangling is required, do not upgrade the function declaration,
+ // but do upgrade the calls.
+ if (auto Result = llvm::Intrinsic::remangleIntrinsicFunction(F))
+ NewFn = *Result;
+ else
+ NewFn = F;
+ return true;
+ }
+ break;
case 'm': {
// Updating the memory intrinsics (memcpy/memmove/memset) that have an
// alignment parameter to embedding the alignment as an attribute of
@@ -1629,7 +1641,6 @@ bool llvm::UpgradeIntrinsicFunction(Function *F, Function *&NewFn,
NewFn = nullptr;
bool Upgraded =
upgradeIntrinsicFunction1(F, NewFn, CanUpgradeDebugIntrinsicsToRecords);
- assert(F != NewFn && "Intrinsic function upgraded to the same function");
// Upgrade intrinsic attributes. This does not change the function.
if (NewFn)
@@ -4570,6 +4581,9 @@ void llvm::UpgradeIntrinsicCall(CallBase *CI, Function *NewFn) {
}
const auto &DefaultCase = [&]() -> void {
+ if (F == NewFn)
+ return;
+
if (CI->getFunctionType() == NewFn->getFunctionType()) {
// Handle generic mangling change.
assert(
@@ -5109,6 +5123,31 @@ void llvm::UpgradeIntrinsicCall(CallBase *CI, Function *NewFn) {
MTI->setSourceAlignment(Align->getMaybeAlignValue());
break;
}
+
+ case Intrinsic::lifetime_start:
+ case Intrinsic::lifetime_end: {
+ Value *Size = CI->getArgOperand(0);
+ Value *Ptr = CI->getArgOperand(1);
+ if (isa<AllocaInst>(Ptr)) {
+ DefaultCase();
+ return;
+ }
+
+ // Try to strip pointer casts, such that the lifetime works on an alloca.
+ Ptr = Ptr->stripPointerCasts();
+ if (isa<AllocaInst>(Ptr)) {
+ // Don't use NewFn, as we might have looked through an addrspacecast.
+ if (NewFn->getIntrinsicID() == Intrinsic::lifetime_start)
+ NewCall = Builder.CreateLifetimeStart(Ptr, cast<ConstantInt>(Size));
+ else
+ NewCall = Builder.CreateLifetimeEnd(Ptr, cast<ConstantInt>(Size));
+ break;
+ }
+
+ // Otherwise remove the lifetime marker.
+ CI->eraseFromParent();
+ return;
+ }
}
assert(NewCall && "Should have either set this variable or returned through "
"the default case");
@@ -5131,7 +5170,8 @@ void llvm::UpgradeCallsToIntrinsic(Function *F) {
UpgradeIntrinsicCall(CB, NewFn);
// Remove old function, no longer used, from the module.
- F->eraseFromParent();
+ if (F != NewFn)
+ F->eraseFromParent();
}
}
diff --git a/llvm/lib/IR/Verifier.cpp b/llvm/lib/IR/Verifier.cpp
index 8c8ed3c5e47ba..196a52cc07120 100644
--- a/llvm/lib/IR/Verifier.cpp
+++ b/llvm/lib/IR/Verifier.cpp
@@ -6679,6 +6679,11 @@ void Verifier::visitIntrinsicCall(Intrinsic::ID ID, CallBase &Call) {
"llvm.threadlocal.address operand isThreadLocal() must be true");
break;
}
+ case Intrinsic::lifetime_start:
+ case Intrinsic::lifetime_end:
+ Check(isa<AllocaInst>(Call.getArgOperand(1)),
+ "llvm.lifetime.start/end can only be used on alloca", &Call);
+ break;
};
// Verify that there aren't any unmediated control transfers between funclets.
diff --git a/llvm/lib/Target/SPIRV/SPIRVPrepareFunctions.cpp b/llvm/lib/Target/SPIRV/SPIRVPrepareFunctions.cpp
index 2bffbf73b574a..6766bd866cd2e 100644
--- a/llvm/lib/Target/SPIRV/SPIRVPrepareFunctions.cpp
+++ b/llvm/lib/Target/SPIRV/SPIRVPrepareFunctions.cpp
@@ -380,7 +380,7 @@ bool SPIRVPrepareFunctions::substituteIntrinsicCalls(Function *F) {
bool Changed = false;
const SPIRVSubtarget &STI = TM.getSubtarget<SPIRVSubtarget>(*F);
for (BasicBlock &BB : *F) {
- for (Instruction &I : BB) {
+ for (Instruction &I : make_early_inc_range(BB)) {
auto Call = dyn_cast<CallInst>(&I);
if (!Call)
continue;
@@ -408,12 +408,16 @@ bool SPIRVPrepareFunctions::substituteIntrinsicCalls(Function *F) {
if (!STI.isShader()) {
Changed |= toSpvOverloadedIntrinsic(
II, Intrinsic::SPVIntrinsics::spv_lifetime_start, {1});
+ } else {
+ II->eraseFromParent();
}
break;
case Intrinsic::lifetime_end:
if (!STI.isShader()) {
Changed |= toSpvOverloadedIntrinsic(
II, Intrinsic::SPVIntrinsics::spv_lifetime_end, {1});
+ } else {
+ II->eraseFromParent();
}
break;
case Intrinsic::ptr_annotation:
diff --git a/llvm/lib/Transforms/Coroutines/CoroEarly.cpp b/llvm/lib/Transforms/Coroutines/CoroEarly.cpp
index e279fec18bdbc..6561b1cd4ade1 100644
--- a/llvm/lib/Transforms/Coroutines/CoroEarly.cpp
+++ b/llvm/lib/Transforms/Coroutines/CoroEarly.cpp
@@ -170,6 +170,12 @@ void Lowerer::hidePromiseAlloca(CoroIdInst *CoroId, CoroBeginInst *CoroBegin) {
auto *PI = Builder.CreateIntrinsic(
Builder.getPtrTy(), Intrinsic::coro_promise, Arg, {}, "promise.addr");
PI->setCannotDuplicate();
+ // Remove lifetime markers, as these are only allowed on allocas.
+ for (User *U : make_early_inc_range(PA->users())) {
+ auto *I = cast<Instruction>(U);
+ if (I->isLifetimeStartOrEnd())
+ I->eraseFromParent();
+ }
PA->replaceUsesWithIf(PI, [CoroId](Use &U) {
bool IsBitcast = U == U.getUser()->stripPointerCasts();
bool IsCoroId = U.getUser() == CoroId;
diff --git a/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp b/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp
index 5957940add577..fbaa651641566 100644
--- a/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp
+++ b/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp
@@ -3637,6 +3637,7 @@ void FunctionStackPoisoner::processStaticAllocas() {
"Variable descriptions relative to ASan stack base will be dropped");
// Replace Alloca instructions with base+offset.
+ SmallVector<Value *> NewAllocaPtrs;
for (const auto &Desc : SVD) {
AllocaInst *AI = Desc.AI;
replaceDbgDeclare(AI, LocalStackBaseAllocaPtr, DIB, DIExprFlags,
@@ -3645,6 +3646,7 @@ void FunctionStackPoisoner::processStaticAllocas() {
IRB.CreateAdd(LocalStackBase, ConstantInt::get(IntptrTy, Desc.Offset)),
AI->getType());
AI->replaceAllUsesWith(NewAllocaPtr);
+ NewAllocaPtrs.push_back(NewAllocaPtr);
}
// The left-most redzone has enough space for at least 4 pointers.
@@ -3694,6 +3696,15 @@ void FunctionStackPoisoner::processStaticAllocas() {
}
}
+ // Remove lifetime markers now that these are no longer allocas.
+ for (Value *NewAllocaPtr : NewAllocaPtrs) {
+ for (User *U : make_early_inc_range(NewAllocaPtr->users())) {
+ auto *I = cast<Instruction>(U);
+ if (I->isLifetimeStartOrEnd())
+ I->eraseFromParent();
+ }
+ }
+
SmallVector<uint8_t, 64> ShadowClean(ShadowAfterScope.size(), 0);
SmallVector<uint8_t, 64> ShadowAfterReturn;
@@ -3829,6 +3840,13 @@ void FunctionStackPoisoner::handleDynamicAllocaCall(AllocaInst *AI) {
Value *NewAddressPtr = IRB.CreateIntToPtr(NewAddress, AI->getType());
+ // Remove lifetime markers now that this is no longer an alloca.
+ for (User *U : make_early_inc_range(AI->users())) {
+ auto *I = cast<Instruction>(U);
+ if (I->isLifetimeStartOrEnd())
+ I->eraseFromParent();
+ }
+
// Replace all uses of AddessReturnedByAlloca with NewAddressPtr.
AI->replaceAllUsesWith(NewAddressPtr);
diff --git a/llvm/lib/Transforms/Scalar/InferAddressSpaces.cpp b/llvm/lib/Transforms/Scalar/InferAddressSpaces.cpp
index 66836ef05d5db..85ee824b67121 100644
--- a/llvm/lib/Transforms/Scalar/InferAddressSpaces.cpp
+++ b/llvm/lib/Transforms/Scalar/InferAddressSpaces.cpp
@@ -430,6 +430,8 @@ bool InferAddressSpacesImpl::rewriteIntrinsicOperands(IntrinsicInst *II,
}
case Intrinsic::lifetime_start:
case Intrinsic::lifetime_end: {
+ // Always force lifetime markers to work directly on the alloca.
+ NewV = NewV->stripPointerCasts();
Function *NewDecl = Intrinsic::getOrInsertDeclaration(
M, II->getIntrinsicID(), {NewV->getType()});
II->setArgOperand(1, NewV);
diff --git a/llvm/test/Analysis/BasicAA/modref.ll b/llvm/test/Analysis/BasicAA/modref.ll
index 0619f8e615b80..1aab28f3f1871 100644
--- a/llvm/test/Analysis/BasicAA/modref.ll
+++ b/llvm/test/Analysis/BasicAA/modref.ll
@@ -67,27 +67,33 @@ define i8 @test2a(ptr %P) {
ret i8 %A
}
-define void @test3(ptr %P, i8 %X) {
+define void @test3(i8 %X) {
; CHECK-LABEL: @test3(
-; CHECK-NEXT: [[P2:%.*]] = getelementptr i8, ptr [[P:%.*]], i32 2
+; CHECK-NEXT: [[P:%.*]] = alloca i64, align 8
+; CHECK-NEXT: [[P2:%.*]] = getelementptr i8, ptr [[P]], i32 2
; CHECK-NEXT: call void @llvm.lifetime.end.p0(i64 1, ptr [[P]])
; CHECK-NEXT: store i8 2, ptr [[P2]], align 1
+; CHECK-NEXT: call void @external(ptr [[P]])
; CHECK-NEXT: ret void
;
+ %P = alloca i64
%Y = add i8 %X, 1 ;; Dead, because the only use (the store) is dead.
%P2 = getelementptr i8, ptr %P, i32 2
store i8 %Y, ptr %P2 ;; Not read by lifetime.end, should be removed.
call void @llvm.lifetime.end.p0(i64 1, ptr %P)
store i8 2, ptr %P2
+ call void @external(ptr %P)
ret void
}
-define void @test3a(ptr %P, i8 %X) {
+define void @test3a(i8 %X) {
; CHECK-LABEL: @test3a(
-; CHECK-NEXT: call void @llvm.lifetime.end.p0(i64 10, ptr [[P:%.*]])
+; CHECK-NEXT: [[P:%.*]] = alloca i64, align 8
+; CHECK-NEXT: call void @llvm.lifetime.end.p0(i64 10, ptr [[P]])
; CHECK-NEXT: ret void
;
+ %P = alloca i64
%Y = add i8 %X, 1 ;; Dead, because the only use (the store) is dead.
%P2 = getelementptr i8, ptr %P, i32 2
diff --git a/llvm/test/Analysis/CallGraph/ignore-assumelike-calls.ll b/llvm/test/Analysis/CallGraph/ignore-assumelike-calls.ll
index 658d73804c174..1c9d20193869e 100644
--- a/llvm/test/Analysis/CallGraph/ignore-assumelike-calls.ll
+++ b/llvm/test/Analysis/CallGraph/ignore-assumelike-calls.ll
@@ -10,7 +10,7 @@
; CHECK-EMPTY:
; CHECK-NEXT: Call graph node for function: 'bitcast_only'<<{{.*}}>> #uses=0
; CHECK-EMPTY:
-; CHECK-NEXT: Call graph node for function: 'llvm.lifetime.start.p0'<<{{.*}}>> #uses=3
+; CHECK-NEXT: Call graph node for function: 'llvm.lifetime.start.p0'<<{{.*}}>> #uses=2
; CHECK-EMPTY:
; CHECK-NEXT: Call graph node for function: 'llvm.memset.p0.i64'<<{{.*}}>> #uses=2
; CHECK-EMPTY:
@@ -25,18 +25,11 @@
; CHECK-NEXT: Call graph node for function: 'used_by_lifetime'<<{{.*}}>> #uses=0
; CHECK-NEXT: CS<{{.*}}> calls function 'llvm.lifetime.start.p0'
; CHECK-EMPTY:
-; CHECK-NEXT: Call graph node for function: 'used_by_lifetime_cast'<<{{.*}}>> #uses=0
-; CHECK-NEXT: CS<{{.*}}> calls function 'llvm.lifetime.start.p0'
-; CHECK-EMPTY:
define internal void @used_by_lifetime() {
entry:
- call void @llvm.lifetime.start.p0(i64 4, ptr @used_by_lifetime)
- ret void
-}
-
-define internal void @used_by_lifetime_cast() addrspace(1) {
- call void @llvm.lifetime.start.p0(i64 4, ptr addrspacecast (ptr addrspace(1) @used_by_lifetime_cast to ptr))
+ %a = alloca i8
+ call void @llvm.lifetime.start.p0(i64 4, ptr %a)
ret void
}
diff --git a/llvm/test/Analysis/CostModel/X86/free-intrinsics.ll b/llvm/test/Analysis/CostModel/X86/free-intrinsics.ll
index a8c5c43c3a9f8..3a54428bd8291 100644
--- a/llvm/test/Analysis/CostModel/X86/free-intrinsics.ll
+++ b/llvm/test/Analysis/CostModel/X86/free-intrinsics.ll
@@ -4,6 +4,7 @@
define i32 @trivially_free() {
; CHECK-SIZE-LABEL: 'trivially_free'
+; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %alloca = alloca i8, align 1
; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %a0 = call i32 @llvm.annotation.i32.p0(i32 undef, ptr undef, ptr undef, i32 undef)
; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.assume(i1 undef)
; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.experimental.noalias.scope.decl(metadata !3)
@@ -13,14 +14,15 @@ define i32 @trivially_free() {
; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %a2 = call ptr @llvm.launder.invariant.group.p0(ptr undef)
; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %a3 = call ptr @llvm.strip.invariant.group.p0(ptr undef)
; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %a4 = call i1 @llvm.is.constant.i32(i32 undef)
-; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.lifetime.start.p0(i64 1, ptr undef)
-; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.lifetime.end.p0(i64 1, ptr undef)
+; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.lifetime.start.p0(i64 1, ptr %alloca)
+; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.lifetime.end.p0(i64 1, ptr %alloca)
; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %a5 = call i64 @llvm.objectsize.i64.p0(ptr undef, i1 true, i1 true, i1 true)
; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %a6 = call ptr @llvm.ptr.annotation.p0.p0(ptr undef, ptr undef, ptr undef, i32 undef, ptr undef)
; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.var.annotation.p0.p0(ptr undef, ptr undef, ptr undef, i32 undef, ptr undef)
; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 1 for instruction: ret i32 undef
;
; CHECK-THROUGHPUT-LABEL: 'trivially_free'
+; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %alloca = alloca i8, align 1
; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %a0 = call i32 @llvm.annotation.i32.p0(i32 undef, ptr undef, ptr undef, i32 undef)
; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.assume(i1 undef)
; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.experimental.noalias.scope.decl(metadata !3)
@@ -30,13 +32,14 @@ define i32 @trivially_free() {
; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %a2 = call ptr @llvm.launder.invariant.group.p0(ptr undef)
; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %a3 = call ptr @llvm.strip.invariant.group.p0(ptr undef)
; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %a4 = call i1 @llvm.is.constant.i32(i32 undef)
-; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.lifetime.start.p0(i64 1, ptr undef)
-; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.lifetime.end.p0(i64 1, ptr undef)
+; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.lifetime.start.p0(i64 1, ptr %alloca)
+; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.lifetime.end.p0(i64 1, ptr %alloca)
; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %a5 = call i64 @llvm.objectsize.i64.p0(ptr undef, i1 true, i1 true, i1 true)
; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %a6 = call ptr @llvm.ptr.annotation.p0.p0(ptr undef, ptr undef, ptr undef, i32 undef, ptr undef)
; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.var.annotation.p0.p0(ptr undef, ptr undef, ptr undef, i32 undef, ptr undef)
; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret i32 undef
;
+ %alloca = alloca i8
%a0 = call i32 @llvm.annotation.i32(i32 undef, ptr undef, ptr undef, i32 undef)
call void @llvm.assume(i1 undef)
call void @llvm.experimental.noalias.scope.decl(metadata !...
[truncated]
|
efriedma-quic
approved these changes
Jul 18, 2025
Collaborator
efriedma-quic
left a comment
efriedma-quic
left a comment
There was a problem hiding this comment.
LGTM
There's possibly some underlying need for "lifetimes" with different rules. Like, if you want to implement malloc. But allocas are special because the backend handles allocation/deallocation, so I think it makes sense to have dedicated intrinsics specifically for alloca.
mahesh-attarde
pushed a commit
to mahesh-attarde/llvm-project
that referenced
this pull request
Jul 28, 2025
…149838) After llvm#149310 we are guaranteed that the argument is an alloca, so we don't need to look at underlying objects (which was not a correct thing to do anyway). This also drops the offset argument for lifetime nodes in SDAG. The offset is fixed to zero now. (Peculiarly, while SDAG pretended to have an offset, it just gets silently dropped during selection.)
mahesh-attarde
pushed a commit
to mahesh-attarde/llvm-project
that referenced
this pull request
Jul 28, 2025
…lvm#149994) After llvm#149310 the pointer argument of lifetime.start/lifetime.end is guaranteed to be an alloca, so we don't need to go through findAllocaForValue() anymore, and don't have to have special handling for the case where it fails.
mahesh-attarde
pushed a commit
to mahesh-attarde/llvm-project
that referenced
this pull request
Jul 28, 2025
…#150032) When materializing a function, we'd upgrade all calls to all upgraded intrinsics. However, this would operate on all calls to the intrinsic (including previously materialized ones), which leads to quadratic complexity. Instead, only upgrade the calls that are in the materialized function. This fixes a compile-time regression introduced by llvm#149310.
mahesh-attarde
pushed a commit
to mahesh-attarde/llvm-project
that referenced
this pull request
Jul 28, 2025
After llvm#149310 this is guaranteed to be an alloca.
nikic
added a commit
to nikic/llvm-project
that referenced
this pull request
Jul 28, 2025
After llvm#149310 lifetime intrinsics require an alloca argument, an invariant that this pass can break. I've fixed this in two ways: * First, move static allocas into the entry block. Currently, the way the pass splits the entry block makes all allocas dynamic, which I assume was not actually intended. This will avoid unnecessary SSA reconstruction for allocas as well, and thus avoid the problem. * If this fails (for dynamic allocas) drop all lifetime intrinsics if any one of them would require a rewrite during SSA reconstruction. Fixes llvm#150498.
nikic
added a commit
that referenced
this pull request
Jul 29, 2025
After #149310 lifetime intrinsics require an alloca argument, an invariant that this pass can break. I've fixed this in two ways: * First, move static allocas into the entry block. Currently, the way the pass splits the entry block makes all allocas dynamic, which I assume was not actually intended. This will avoid unnecessary SSA reconstruction for allocas as well, and thus avoid the problem. * If this fails (for dynamic allocas) drop all lifetime intrinsics if any one of them would require a rewrite during SSA reconstruction. Fixes #150498.
Member
|
our use is a bit more complex than that, but for the time being I added a special vendored intrinsic (llvm.enzyme.lifetime_start/end), which we can use until upstream adds an intrinsic with the restored semantics: https://github.com/EnzymeAD/Enzyme/pull/2402/files |
nikic
added a commit
to nikic/llvm-project
that referenced
this pull request
Aug 4, 2025
This slightly relaxes the invariant established in llvm#149310, by also allowing the lifetime argument to be poison. This is to support the typical pattern of RAUWing with poison when removing an instruction. It's worth noting that this does not require any conservative assumptions, lifetimes with poison arguments can simply be skipped. Fixes llvm#151119.
nikic
added a commit
that referenced
this pull request
Aug 4, 2025
This slightly relaxes the invariant established in #149310, by also allowing the lifetime argument to be poison. This is to support the typical pattern of RAUWing with poison when removing an instruction. It's worth noting that this does not require any conservative assumptions, lifetimes with poison arguments can simply be skipped. Fixes #151119.
nikic
added a commit
to nikic/llvm-project
that referenced
this pull request
Aug 8, 2025
Now that llvm#149310 has restricted lifetime intrinsics to only work on allocas, we can also drop the explicit size argument. Instead, the size is implied by the alloca. This removes the ability to only mark a prefix of an alloca alive/dead. We never used that capability, so we should remove the need to handle that possibility everywhere (many key places, including stack coloring, did not actually respect this).
nikic
added a commit
that referenced
this pull request
Aug 8, 2025
Now that #149310 has restricted lifetime intrinsics to only work on allocas, we can also drop the explicit size argument. Instead, the size is implied by the alloca. This removes the ability to only mark a prefix of an alloca alive/dead. We never used that capability, so we should remove the need to handle that possibility everywhere (though many key places, including stack coloring, did not actually respect this).
macdice
added a commit
to macdice/postgres
that referenced
this pull request
Nov 18, 2025
The semantics of the lifetime.end intrinsic changed. It must now point to an alloca instruction. We used it to mark the lifetime of fcinfo_data members inside ExprEvalStep allocated with palloc(). It doesn't sound like this loses any really optimization. We currently only use LLVMBuildAlloca() in deform functions for the stack variable "v_offp". In that case lifetime doesn't need to be marked explicitly, because the memory is reclaimed at the ret instruction that returns from the function. llvm/llvm-project#149310 https://llvm.org/docs/LangRef.html#llvm-lifetime-end-intrinsic https://llvm.org/docs/LangRef.html#i-alloca
macdice
added a commit
to macdice/postgres
that referenced
this pull request
Nov 18, 2025
The lifetime.end intrinsic can now only be used for stack memory allocated with alloca. We were using it to tell the optimizer that we are no longer interested in the arguments and null flag in a FunctionCallInfo struct, so it could avoid actually storing them if it managed to inline the function and keep everything in registers. It can't figure that out by itself because it's part of the ExecEvalStep struct and we scribble on it directly rather than copying it to stack memory allocated with alloca. Instead, generate stores of the special poison value (undef would work too). This is a no-op, but tells the compiler that we are not interested in the value and won't load it again until new values are stored there for a later call. XXX An alternative might be to populate a new FunctionCallInfo on the stack, and then call lifetime.end to free it after the call. That'd be a bigger change. XXX Verify inlined IR Deform functions use LLVMBuildAlloca() for a stack variable, but that memory is reclaimed by the ret instruction. llvm/llvm-project#149310 https://llvm.org/docs/LangRef.html#llvm-lifetime-end-intrinsic https://llvm.org/docs/LangRef.html#i-alloca
macdice
added a commit
to macdice/postgres
that referenced
this pull request
Nov 18, 2025
The lifetime.end intrinsic can now only be used for stack memory allocated with alloca. We were using it to tell the optimizer that we are no longer interested in the arguments and null flag in a FunctionCallInfo struct, so it could avoid actually storing them if it managed to inline the function and keep everything in registers. It can't figure that out by itself because it's part of the ExecEvalStep struct and we scribble on it directly rather than building a new one on the stack. Instead, store the special poison value (undef would work too). This is a no-op, but tells the optimizer that we are not interested in the values. XXX Verify inlined results! Deform functions use LLVMBuildAlloca() for a stack variable, but that memory is reclaimed implicitly by the ret instruction. llvm/llvm-project#149310 https://llvm.org/docs/LangRef.html#llvm-lifetime-end-intrinsic https://llvm.org/docs/LangRef.html#i-alloca
macdice
added a commit
to macdice/postgres
that referenced
this pull request
Nov 25, 2025
The lifetime.end intrinsic can now only be used for stack memory allocated with alloca. We were using it to tell the optimizer that we are no longer interested in the arguments and null flag in a FunctionCallInfo struct, so it could avoid actually storing them if it managed to inline the function and keep everything in registers. It can't figure that out by itself because it's part of the ExecEvalStep struct and we scribble on it directly rather than building a new one on the stack. Instead, store the special poison value (undef would work too). This is a no-op, but tells the optimizer that we are not interested in the values. XXX Verify inlined results! Deform functions use LLVMBuildAlloca() for a stack variable, but that memory is reclaimed implicitly by the ret instruction. llvm/llvm-project#149310 https://llvm.org/docs/LangRef.html#llvm-lifetime-end-intrinsic https://llvm.org/docs/LangRef.html#i-alloca
macdice
added a commit
to macdice/postgres
that referenced
this pull request
Nov 27, 2025
The lifetime.end intrinsic can now only be used for stack memory allocated with alloca. We were using it to tell the optimizer that we are no longer interested in the arguments and null flag in a FunctionCallInfo struct, so it could avoid actually storing them if it managed to inline the function and keep everything in registers. It can't figure that out by itself because it's part of the ExecEvalStep struct and we scribble on it directly rather than building a new one on the stack. Instead, store the special poison value (undef would work too). This generates no actual code, but tells the optimizer that we are not interested in the values. Deform functions use LLVMBuildAlloca() for a stack variable, but that memory is reclaimed implicitly by the ret instruction. llvm/llvm-project#149310 https://llvm.org/docs/LangRef.html#llvm-lifetime-end-intrinsic https://llvm.org/docs/LangRef.html#i-alloca
macdice
added a commit
to macdice/postgres
that referenced
this pull request
Nov 28, 2025
The lifetime.end intrinsic can now only be used for stack memory allocated with alloca. We were using it to tell the optimizer that we are no longer interested in the arguments and null flag in a FunctionCallInfo struct, so it could avoid actually storing them if it managed to inline the function and keep everything in registers. It can't figure that out by itself because it's part of the ExecEvalStep struct and we scribble on it directly rather than building a new one on the stack. Instead, store the special poison value (undef would work too). This generates no actual code, but tells the optimizer that we are not interested in the values. Deform functions use LLVMBuildAlloca() for a stack variable, but that memory is reclaimed implicitly by the ret instruction. llvm/llvm-project#149310 https://llvm.org/docs/LangRef.html#llvm-lifetime-end-intrinsic https://llvm.org/docs/LangRef.html#i-alloca
macdice
added a commit
to macdice/postgres
that referenced
this pull request
Nov 28, 2025
The lifetime.end intrinsic can now only be used for stack memory allocated with alloca. We were using it to tell the optimizer that we are no longer interested in the arguments and null flag in a FunctionCallInfo struct, so it could avoid actually storing them if it managed to inline the function and keep everything in registers. It can't figure that out by itself because it's part of the ExecEvalStep struct and we scribble on it directly rather than building a new one on the stack. Instead, store the special poison value (undef would work too). This generates no actual code, but tells the optimizer that we are not interested in the values. Deform functions use LLVMBuildAlloca() for a stack variable, but that memory is reclaimed implicitly by the ret instruction. llvm/llvm-project#149310 https://llvm.org/docs/LangRef.html#llvm-lifetime-end-intrinsic https://llvm.org/docs/LangRef.html#i-alloca
macdice
added a commit
to macdice/postgres
that referenced
this pull request
Nov 28, 2025
The lifetime.end intrinsic can now only be used for stack memory allocated with alloca. We were using it to tell the optimizer that we are no longer interested in the arguments and null flag in a FunctionCallInfo struct, so it could avoid actually storing them if it managed to inline the function and keep everything in registers. It can't figure that out by itself because it's part of the ExecEvalStep struct and we scribble on it directly rather than building a new one on the stack. Instead, store the special poison value (undef would work too). This generates no actual code, but tells the optimizer that we are not interested in the values. Deform functions use LLVMBuildAlloca() for a stack variable, but that memory is reclaimed implicitly by the ret instruction. llvm/llvm-project#149310 https://llvm.org/docs/LangRef.html#llvm-lifetime-end-intrinsic https://llvm.org/docs/LangRef.html#i-alloca
macdice
added a commit
to macdice/postgres
that referenced
this pull request
Nov 28, 2025
The lifetime.end intrinsic can now only be used for stack memory allocated with alloca. We were using it to tell the optimizer that we are no longer interested in the arguments and null flag in a FunctionCallInfo struct, so it could avoid actually storing them if it managed to inline the function and keep everything in registers. It can't figure that out by itself because it's part of the ExecEvalStep struct and we scribble on it directly rather than building a new one on the stack. Instead, store the special poison value (undef would work too). This generates no actual code, but tells the optimizer that we are not interested in the values. Deform functions use LLVMBuildAlloca() for a stack variable, but that memory is reclaimed implicitly by the ret instruction. llvm/llvm-project#149310 https://llvm.org/docs/LangRef.html#llvm-lifetime-end-intrinsic https://llvm.org/docs/LangRef.html#i-alloca
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
lifetime.start and lifetime.end are primarily intended for use on allocas, to enable stack coloring and other liveness optimizations. This is necessary because all (static) allocas are hoisted into the entry block, so lifetime markers are the only way to convey the actual lifetimes.
However, lifetime.start and lifetime.end are currently allowed to be used on non-alloca pointers. We don't actually do this in practice, but just the mere fact that this is possible breaks the core purpose of the lifetime markers, which is stack coloring of allocas. Stack coloring can only work correctly if all lifetime markers for an alloca are analyzable.
I don't think there is any way to have coherent semantics for lifetime markers on allocas, while also permitting them on arbitrary pointer values.
This PR restricts lifetimes to operate on allocas only. As a followup, I will also drop the size argument, which is superfluous if we always operate on an alloca. (This change also renders various code handling lifetime markers on non-alloca dead. I plan to clean up that kind of code after dropping the size argument as well.)
In practice, I've only found a few places that currently produce lifetimes on non-allocas: