Skip to content

Instrument native threads#308

Merged
zhengyu123 merged 18 commits intomainfrom
zgu/inst_native_thread
Dec 15, 2025
Merged

Instrument native threads#308
zhengyu123 merged 18 commits intomainfrom
zgu/inst_native_thread

Conversation

@zhengyu123
Copy link
Contributor

@zhengyu123 zhengyu123 commented Dec 9, 2025 •
edited by atlassian bot
Loading

What does this PR do?:
Instrument/profile native threads, the threads that are created/started outside of JVM, on hotspot/non-musl based JVM.

Motivation:
Enhance Java profiler to profiler native threads.

Additional Notes:
The feature is now only enabled for hotspot based JVM running on non-musl Linux platform. The reasons:

  • A crash seen on aarch64/musl /JDK11. It might not be related to this change, but it is hard to confirm. Disable the feature for musl based Linux for now.
  • J9 has issues to walk native only thread stack in release build, it shows only
    .no_java_frame
    while debug build shows correct stack.

How to test the change?:

  • Regular tests
  • JDK tier1 tests with profiler agent. Although, there are failures, but match java-profiler main line.
    There are failures that are expected:
    • HeapMonitor uses agent, which conflicts with profiler agent
    • Compiler frame is not compatible with agent.

For Datadog employees:

  • If this PR touches code that signs or publishes builds or packages, or handles
    credentials of any kind, I've requested a review from @DataDog/security-design-and-guidance.
  • This PR doesn't touch any of that.
  • JIRA: PROF-11577

Unsure? Have a question? Request a review!

@zhengyu123 zhengyu123 marked this pull request as ready for review December 11, 2025 16:21
jbachorik
jbachorik reviewed Dec 11, 2025
View reviewed changes
jbachorik
jbachorik requested changes Dec 11, 2025
View reviewed changes
Copy link
Collaborator

@jbachorik jbachorik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having the libraries patching tied to ctimer (CPU profiling) does not sound right.
We have a bunch of other engines that can be used either for CPU or wallclock profiling and their functionality would be very inconsistent.

I think the library patching should go to a more generic place - and also be called from a more generic place, like eg. profiler.cpp

@zhengyu123
Copy link
Contributor Author

zhengyu123 commented Dec 12, 2025 •
edited
Loading

Having the libraries patching tied to ctimer (CPU profiling) does not sound right. We have a bunch of other engines that can be used either for CPU or wallclock profiling and their functionality would be very inconsistent.

I think the library patching should go to a more generic place - and also be called from a more generic place, like eg. profiler.cpp

Agree. But I want to limit the changes in this PR and address this issue in separate PR.

@jbachorik
Copy link
Collaborator

jbachorik commented Dec 12, 2025

Having the libraries patching tied to ctimer (CPU profiling) does not sound right. We have a bunch of other engines that can be used either for CPU or wallclock profiling and their functionality would be very inconsistent.

I think the library patching should go to a more generic place - and also be called from a more generic place, like eg. profiler.cpp

Agree. But I want to limit the changes in this PR and address this issue in separate PR.

Ok. But create a followup ticket for that, plz. And let's move to the proper placement asap - we really don't want to get this partial task accidentally released.

@jbachorik
Copy link
Collaborator

jbachorik commented Dec 12, 2025

How will this work with dynamically loaded libraries? Am I reading the code right that we will not patch those?

@DataDog DataDog deleted a comment from zhengyu123 Dec 12, 2025
jbachorik
jbachorik approved these changes Dec 12, 2025
View reviewed changes
Copy link
Collaborator

@jbachorik jbachorik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved with the two followup tickets

@pr-commenter
Copy link

pr-commenter bot commented Dec 12, 2025 •
edited
Loading

Benchmarks [x86_64 wall]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.4 1.35.0-zgu_inst_native_thread-SNAPSHOT
See matching parameters
Baseline Candidate
alloc off off
cpu off off
iterations 5 5
java "11.0.28" "11.0.28"
memleak off off
modes wall wall
wall on on

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 15 metrics, 23 unstable metrics.

@pr-commenter
Copy link

pr-commenter bot commented Dec 12, 2025 •
edited
Loading

Benchmarks [x86_64 cpu]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.4 1.35.0-zgu_inst_native_thread-SNAPSHOT
See matching parameters
Baseline Candidate
alloc off off
cpu on on
iterations 5 5
java "11.0.28" "11.0.28"
memleak off off
modes cpu cpu
wall off off

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 14 metrics, 24 unstable metrics.

@pr-commenter
Copy link

pr-commenter bot commented Dec 12, 2025 •
edited
Loading

Benchmarks [x86_64 memleak]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.4 1.35.0-zgu_inst_native_thread-SNAPSHOT
See matching parameters
Baseline Candidate
alloc off off
cpu off off
iterations 5 5
java "11.0.28" "11.0.28"
memleak on on
modes memleak memleak
wall off off

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 16 metrics, 22 unstable metrics.

@pr-commenter
Copy link

pr-commenter bot commented Dec 12, 2025 •
edited
Loading

Benchmarks [x86_64 cpu,wall]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.4 1.35.0-zgu_inst_native_thread-SNAPSHOT
See matching parameters
Baseline Candidate
alloc off off
cpu on on
iterations 5 5
java "11.0.28" "11.0.28"
memleak off off
modes cpu,wall cpu,wall
wall on on

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 15 metrics, 23 unstable metrics.

@pr-commenter
Copy link

pr-commenter bot commented Dec 12, 2025 •
edited
Loading

Benchmarks [x86_64 alloc]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.4 1.35.0-zgu_inst_native_thread-SNAPSHOT
See matching parameters
Baseline Candidate
alloc on on
cpu off off
iterations 5 5
java "11.0.28" "11.0.28"
memleak off off
modes alloc alloc
wall off off

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 15 metrics, 23 unstable metrics.

@pr-commenter
Copy link

pr-commenter bot commented Dec 12, 2025 •
edited
Loading

Benchmarks [aarch64 wall]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.4 1.35.0-zgu_inst_native_thread-SNAPSHOT
See matching parameters
Baseline Candidate
alloc off off
cpu off off
iterations 5 5
java "11.0.28" "11.0.28"
memleak off off
modes wall wall
wall on on

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 18 metrics, 20 unstable metrics.

@pr-commenter
Copy link

pr-commenter bot commented Dec 12, 2025 •
edited
Loading

Benchmarks [aarch64 cpu]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.4 1.35.0-zgu_inst_native_thread-SNAPSHOT
See matching parameters
Baseline Candidate
alloc off off
cpu on on
iterations 5 5
java "11.0.28" "11.0.28"
memleak off off
modes cpu cpu
wall off off

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 19 metrics, 19 unstable metrics.

@pr-commenter
Copy link

pr-commenter bot commented Dec 12, 2025 •
edited
Loading

Benchmarks [x86_64 cpu,wall,alloc,memleak]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.4 1.35.0-zgu_inst_native_thread-SNAPSHOT
See matching parameters
Baseline Candidate
alloc on on
cpu on on
iterations 5 5
java "11.0.28" "11.0.28"
memleak on on
modes cpu,wall,alloc,memleak cpu,wall,alloc,memleak
wall on on

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 14 metrics, 24 unstable metrics.

@pr-commenter
Copy link

pr-commenter bot commented Dec 12, 2025 •
edited
Loading

Benchmarks [x86_64 memleak,alloc]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.4 1.35.0-zgu_inst_native_thread-SNAPSHOT
See matching parameters
Baseline Candidate
alloc on on
cpu off off
iterations 5 5
java "11.0.28" "11.0.28"
memleak on on
modes memleak,alloc memleak,alloc
wall off off

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 14 metrics, 24 unstable metrics.

@pr-commenter
Copy link

pr-commenter bot commented Dec 12, 2025 •
edited
Loading

Benchmarks [aarch64 memleak,alloc]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.4 1.35.0-zgu_inst_native_thread-SNAPSHOT
See matching parameters
Baseline Candidate
alloc on on
cpu off off
iterations 5 5
java "11.0.28" "11.0.28"
memleak on on
modes memleak,alloc memleak,alloc
wall off off

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 18 metrics, 20 unstable metrics.

@pr-commenter
Copy link

pr-commenter bot commented Dec 12, 2025 •
edited
Loading

Benchmarks [aarch64 alloc]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.4 1.35.0-zgu_inst_native_thread-SNAPSHOT
See matching parameters
Baseline Candidate
alloc on on
cpu off off
iterations 5 5
java "11.0.28" "11.0.28"
memleak off off
modes alloc alloc
wall off off

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 16 metrics, 22 unstable metrics.

@pr-commenter
Copy link

pr-commenter bot commented Dec 12, 2025 •
edited
Loading

Benchmarks [aarch64 cpu,wall]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.4 1.35.0-zgu_inst_native_thread-SNAPSHOT
See matching parameters
Baseline Candidate
alloc off off
cpu on on
iterations 5 5
java "11.0.28" "11.0.28"
memleak off off
modes cpu,wall cpu,wall
wall on on

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 18 metrics, 20 unstable metrics.

@pr-commenter
Copy link

pr-commenter bot commented Dec 12, 2025 •
edited
Loading

Benchmarks [aarch64 memleak]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.4 1.35.0-zgu_inst_native_thread-SNAPSHOT
See matching parameters
Baseline Candidate
alloc off off
cpu off off
iterations 5 5
java "11.0.28" "11.0.28"
memleak on on
modes memleak memleak
wall off off

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 17 metrics, 21 unstable metrics.

@pr-commenter
Copy link

pr-commenter bot commented Dec 12, 2025 •
edited
Loading

Benchmarks [aarch64 cpu,wall,alloc,memleak]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.4 1.35.0-zgu_inst_native_thread-SNAPSHOT
See matching parameters
Baseline Candidate
alloc on on
cpu on on
iterations 5 5
java "11.0.28" "11.0.28"
memleak on on
modes cpu,wall,alloc,memleak cpu,wall,alloc,memleak
wall on on

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 16 metrics, 22 unstable metrics.

@zhengyu123 zhengyu123 merged commit a598383 into main Dec 15, 2025
96 checks passed
@zhengyu123 zhengyu123 deleted the zgu/inst_native_thread branch December 15, 2025 15:31
@github-actions github-actions bot added this to the 1.35.0 milestone Dec 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jbachorik jbachorik jbachorik approved these changes

@r1viollet r1viollet Awaiting requested review from r1viollet

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

1.35.0

Development

Successfully merging this pull request may close these issues.

2 participants

@zhengyu123 @jbachorik