Collect logs in arm64 macrobenchmark tests#8057

NachoEchevarria · 2026-01-14T10:35:26Z

Summary of changes

The macrobenchmark yml has been modified to collect and artifact tracer logs in arm64 tests.

Reason for change

We were not collecting tracer logs in macrobenchmark arm64 tests.

Implementation details

Test coverage

Other details

dd-trace-dotnet-ci-bot · 2026-01-14T11:19:40Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing This PR (8057) and master.

✅ No regressions detected - check the details below

Full Metrics Comparison

FakeDbCommand

Metric	Master (Mean ± 95% CI)	Current (Mean ± 95% CI)	Change	Status
.NET Framework 4.8 - Baseline
duration	68.15 ± (68.19 - 68.43) ms	68.15 ± (68.22 - 68.43) ms	-0.0%	✅
.NET Framework 4.8 - Bailout
duration	72.20 ± (72.14 - 72.36) ms	72.19 ± (72.14 - 72.38) ms	-0.0%	✅
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration	997.92 ± (1004.52 - 1015.19) ms	1007.80 ± (1011.40 - 1021.20) ms	+1.0%	✅⬆️
.NET Core 3.1 - Baseline
process.internal_duration_ms	21.79 ± (21.76 - 21.82) ms	21.94 ± (21.92 - 21.96) ms	+0.7%	✅⬆️
process.time_to_main_ms	78.44 ± (78.26 - 78.61) ms	78.48 ± (78.30 - 78.67) ms	+0.1%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.87 ± (10.87 - 10.88) MB	10.88 ± (10.88 - 10.89) MB	+0.1%	✅⬆️
runtime.dotnet.threads.count	12 ± (12 - 12)	12 ± (12 - 12)	+0.0%	✅
.NET Core 3.1 - Bailout
process.internal_duration_ms	21.75 ± (21.73 - 21.77) ms	21.91 ± (21.89 - 21.93) ms	+0.8%	✅⬆️
process.time_to_main_ms	79.52 ± (79.42 - 79.62) ms	79.41 ± (79.31 - 79.51) ms	-0.1%	✅
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.92 ± (10.91 - 10.92) MB	10.93 ± (10.93 - 10.94) MB	+0.1%	✅⬆️
runtime.dotnet.threads.count	13 ± (13 - 13)	13 ± (13 - 13)	+0.0%	✅
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms	249.95 ± (247.00 - 252.90) ms	255.81 ± (253.75 - 257.87) ms	+2.3%	✅⬆️
process.time_to_main_ms	467.98 ± (467.31 - 468.64) ms	467.54 ± (467.12 - 467.96) ms	-0.1%	✅
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	48.29 ± (48.27 - 48.32) MB	48.38 ± (48.36 - 48.40) MB	+0.2%	✅⬆️
runtime.dotnet.threads.count	28 ± (28 - 28)	28 ± (28 - 28)	+0.3%	✅⬆️
.NET 6 - Baseline
process.internal_duration_ms	20.73 ± (20.70 - 20.76) ms	20.66 ± (20.64 - 20.69) ms	-0.3%	✅
process.time_to_main_ms	68.22 ± (68.09 - 68.36) ms	68.01 ± (67.92 - 68.11) ms	-0.3%	✅
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.60 ± (10.60 - 10.60) MB	10.62 ± (10.61 - 10.62) MB	+0.1%	✅⬆️
runtime.dotnet.threads.count	10 ± (10 - 10)	10 ± (10 - 10)	+0.0%	✅
.NET 6 - Bailout
process.internal_duration_ms	20.59 ± (20.57 - 20.61) ms	20.62 ± (20.60 - 20.65) ms	+0.2%	✅⬆️
process.time_to_main_ms	68.73 ± (68.67 - 68.79) ms	68.90 ± (68.85 - 68.95) ms	+0.2%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.65 ± (10.65 - 10.66) MB	10.71 ± (10.70 - 10.72) MB	+0.6%	✅⬆️
runtime.dotnet.threads.count	11 ± (11 - 11)	11 ± (11 - 11)	+0.0%	✅
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms	250.07 ± (249.18 - 250.97) ms	249.38 ± (248.55 - 250.22) ms	-0.3%	✅
process.time_to_main_ms	445.48 ± (444.98 - 445.98) ms	445.06 ± (444.48 - 445.64) ms	-0.1%	✅
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	49.17 ± (49.14 - 49.20) MB	49.17 ± (49.13 - 49.20) MB	-0.0%	✅
runtime.dotnet.threads.count	28 ± (28 - 28)	28 ± (28 - 28)	+0.0%	✅⬆️
.NET 8 - Baseline
process.internal_duration_ms	18.77 ± (18.74 - 18.80) ms	18.76 ± (18.74 - 18.79) ms	-0.0%	✅
process.time_to_main_ms	66.88 ± (66.77 - 66.98) ms	67.34 ± (67.22 - 67.46) ms	+0.7%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	7.64 ± (7.63 - 7.64) MB	7.66 ± (7.65 - 7.67) MB	+0.3%	✅⬆️
runtime.dotnet.threads.count	10 ± (10 - 10)	10 ± (10 - 10)	+0.0%	✅
.NET 8 - Bailout
process.internal_duration_ms	18.72 ± (18.70 - 18.75) ms	18.86 ± (18.83 - 18.88) ms	+0.7%	✅⬆️
process.time_to_main_ms	67.96 ± (67.89 - 68.04) ms	68.29 ± (68.23 - 68.34) ms	+0.5%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	7.71 ± (7.70 - 7.72) MB	7.72 ± (7.72 - 7.73) MB	+0.2%	✅⬆️
runtime.dotnet.threads.count	11 ± (11 - 11)	11 ± (11 - 11)	+0.0%	✅
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms	176.49 ± (175.58 - 177.41) ms	175.99 ± (175.20 - 176.79) ms	-0.3%	✅
process.time_to_main_ms	427.37 ± (426.65 - 428.10) ms	428.39 ± (427.72 - 429.05) ms	+0.2%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	36.54 ± (36.51 - 36.57) MB	36.56 ± (36.52 - 36.59) MB	+0.0%	✅⬆️
runtime.dotnet.threads.count	27 ± (26 - 27)	27 ± (27 - 27)	+0.5%	✅⬆️

HttpMessageHandler

Metric	Master (Mean ± 95% CI)	Current (Mean ± 95% CI)	Change	Status
.NET Framework 4.8 - Baseline
duration	193.57 ± (193.54 - 194.25) ms	192.76 ± (192.81 - 193.71) ms	-0.4%	✅
.NET Framework 4.8 - Bailout
duration	196.68 ± (196.64 - 197.19) ms	196.96 ± (196.54 - 197.25) ms	+0.1%	✅⬆️
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration	1117.89 ± (1121.39 - 1130.46) ms	1126.30 ± (1129.26 - 1138.29) ms	+0.8%	✅⬆️
.NET Core 3.1 - Baseline
process.internal_duration_ms	187.32 ± (186.96 - 187.68) ms	187.79 ± (187.46 - 188.12) ms	+0.3%	✅⬆️
process.time_to_main_ms	80.78 ± (80.57 - 81.00) ms	80.91 ± (80.70 - 81.12) ms	+0.2%	✅⬆️
runtime.dotnet.exceptions.count	3 ± (3 - 3)	3 ± (3 - 3)	+0.0%	✅
runtime.dotnet.mem.committed	16.10 ± (16.07 - 16.12) MB	16.16 ± (16.13 - 16.19) MB	+0.4%	✅⬆️
runtime.dotnet.threads.count	20 ± (20 - 20)	20 ± (19 - 20)	-0.8%	✅
.NET Core 3.1 - Bailout
process.internal_duration_ms	187.07 ± (186.66 - 187.47) ms	186.92 ± (186.61 - 187.24) ms	-0.1%	✅
process.time_to_main_ms	82.15 ± (81.96 - 82.35) ms	82.02 ± (81.89 - 82.15) ms	-0.2%	✅
runtime.dotnet.exceptions.count	3 ± (3 - 3)	3 ± (3 - 3)	+0.0%	✅
runtime.dotnet.mem.committed	16.23 ± (16.20 - 16.27) MB	16.17 ± (16.14 - 16.20) MB	-0.4%	✅
runtime.dotnet.threads.count	21 ± (21 - 21)	21 ± (20 - 21)	-0.7%	✅
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms	428.93 ± (425.97 - 431.89) ms	425.20 ± (421.85 - 428.55) ms	-0.9%	✅
process.time_to_main_ms	471.07 ± (470.42 - 471.72) ms	472.42 ± (471.86 - 472.98) ms	+0.3%	✅⬆️
runtime.dotnet.exceptions.count	3 ± (3 - 3)	3 ± (3 - 3)	+0.0%	✅
runtime.dotnet.mem.committed	58.79 ± (58.67 - 58.91) MB	58.77 ± (58.65 - 58.89) MB	-0.0%	✅
runtime.dotnet.threads.count	29 ± (29 - 29)	29 ± (29 - 30)	+0.1%	✅⬆️
.NET 6 - Baseline
process.internal_duration_ms	191.56 ± (191.15 - 191.97) ms	191.24 ± (190.86 - 191.62) ms	-0.2%	✅
process.time_to_main_ms	70.14 ± (69.94 - 70.34) ms	69.65 ± (69.47 - 69.84) ms	-0.7%	✅
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	15.93 ± (15.77 - 16.09) MB	16.17 ± (16.04 - 16.31) MB	+1.5%	✅⬆️
runtime.dotnet.threads.count	18 ± (18 - 18)	19 ± (19 - 19)	+3.2%	✅⬆️
.NET 6 - Bailout
process.internal_duration_ms	190.69 ± (190.41 - 190.97) ms	190.88 ± (190.58 - 191.17) ms	+0.1%	✅⬆️
process.time_to_main_ms	71.21 ± (71.11 - 71.32) ms	70.67 ± (70.54 - 70.79) ms	-0.8%	✅
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	15.88 ± (15.72 - 16.05) MB	15.98 ± (15.82 - 16.14) MB	+0.6%	✅⬆️
runtime.dotnet.threads.count	19 ± (19 - 19)	19 ± (19 - 19)	+0.9%	✅⬆️
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms	439.44 ± (436.05 - 442.83) ms	452.20 ± (449.83 - 454.57) ms	+2.9%	✅⬆️
process.time_to_main_ms	449.67 ± (449.17 - 450.17) ms	449.36 ± (448.84 - 449.89) ms	-0.1%	✅
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	59.01 ± (58.87 - 59.14) MB	58.77 ± (58.66 - 58.89) MB	-0.4%	✅
runtime.dotnet.threads.count	29 ± (29 - 29)	29 ± (29 - 29)	-0.1%	✅
.NET 8 - Baseline
process.internal_duration_ms	188.71 ± (188.42 - 188.99) ms	190.09 ± (189.68 - 190.50) ms	+0.7%	✅⬆️
process.time_to_main_ms	69.25 ± (69.07 - 69.43) ms	69.64 ± (69.46 - 69.81) ms	+0.6%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	11.76 ± (11.73 - 11.79) MB	11.75 ± (11.73 - 11.78) MB	-0.1%	✅
runtime.dotnet.threads.count	18 ± (18 - 18)	18 ± (18 - 18)	-0.1%	✅
.NET 8 - Bailout
process.internal_duration_ms	188.57 ± (188.31 - 188.83) ms	188.91 ± (188.66 - 189.16) ms	+0.2%	✅⬆️
process.time_to_main_ms	70.26 ± (70.17 - 70.36) ms	70.34 ± (70.24 - 70.44) ms	+0.1%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	11.79 ± (11.76 - 11.82) MB	11.84 ± (11.80 - 11.88) MB	+0.4%	✅⬆️
runtime.dotnet.threads.count	19 ± (19 - 19)	19 ± (19 - 19)	-0.1%	✅
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms	364.03 ± (362.32 - 365.74) ms	367.67 ± (366.00 - 369.33) ms	+1.0%	✅⬆️
process.time_to_main_ms	432.10 ± (431.42 - 432.79) ms	434.33 ± (433.60 - 435.07) ms	+0.5%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	48.15 ± (48.12 - 48.18) MB	48.14 ± (48.09 - 48.19) MB	-0.0%	✅
runtime.dotnet.threads.count	29 ± (29 - 29)	29 ± (29 - 29)	+0.2%	✅⬆️

Comparison explanation

Execution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

Duration charts

FakeDbCommand (.NET Framework 4.8)

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8057) - mean (68ms)  : 67, 70
    master - mean (68ms)  : 67, 70

    section Bailout
    This PR (8057) - mean (72ms)  : 71, 73
    master - mean (72ms)  : 71, 73

    section CallTarget+Inlining+NGEN
    This PR (8057) - mean (1,016ms)  : 945, 1088
    master - mean (1,010ms)  : 934, 1086

FakeDbCommand (.NET Core 3.1)

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8057) - mean (106ms)  : 102, 109
    master - mean (105ms)  : 103, 108

    section Bailout
    This PR (8057) - mean (106ms)  : 105, 108
    master - mean (106ms)  : 105, 107

    section CallTarget+Inlining+NGEN
    This PR (8057) - mean (749ms)  : 720, 778
    master - mean (743ms)  : 693, 794

FakeDbCommand (.NET 6)

gantt
    title Execution time (ms) FakeDbCommand (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8057) - mean (93ms)  : 91, 96
    master - mean (94ms)  : 91, 96

    section Bailout
    This PR (8057) - mean (94ms)  : 93, 95
    master - mean (94ms)  : 93, 95

    section CallTarget+Inlining+NGEN
    This PR (8057) - mean (718ms)  : 697, 738
    master - mean (721ms)  : 701, 741

FakeDbCommand (.NET 8)

gantt
    title Execution time (ms) FakeDbCommand (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8057) - mean (92ms)  : 90, 95
    master - mean (92ms)  : 89, 94

    section Bailout
    This PR (8057) - mean (93ms)  : 92, 94
    master - mean (93ms)  : 91, 94

    section CallTarget+Inlining+NGEN
    This PR (8057) - mean (632ms)  : 618, 646
    master - mean (633ms)  : 615, 651

HttpMessageHandler (.NET Framework 4.8)

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8057) - mean (193ms)  : 188, 198
    master - mean (194ms)  : 190, 197

    section Bailout
    This PR (8057) - mean (197ms)  : 194, 200
    master - mean (197ms)  : 194, 200

    section CallTarget+Inlining+NGEN
    This PR (8057) - mean (1,134ms)  : 1069, 1199
    master - mean (1,126ms)  : 1061, 1190

HttpMessageHandler (.NET Core 3.1)

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8057) - mean (277ms)  : 272, 281
    master - mean (277ms)  : 272, 281

    section Bailout
    This PR (8057) - mean (277ms)  : 272, 282
    master - mean (277ms)  : 272, 283

    section CallTarget+Inlining+NGEN
    This PR (8057) - mean (927ms)  : 875, 978
    master - mean (927ms)  : 873, 981

HttpMessageHandler (.NET 6)

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8057) - mean (269ms)  : 263, 274
    master - mean (270ms)  : 265, 275

    section Bailout
    This PR (8057) - mean (269ms)  : 265, 273
    master - mean (270ms)  : 266, 274

    section CallTarget+Inlining+NGEN
    This PR (8057) - mean (929ms)  : 885, 973
    master - mean (914ms)  : 852, 976

HttpMessageHandler (.NET 8)

gantt
    title Execution time (ms) HttpMessageHandler (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8057) - mean (270ms)  : 263, 277
    master - mean (268ms)  : 263, 273

    section Bailout
    This PR (8057) - mean (269ms)  : 265, 273
    master - mean (268ms)  : 265, 272

    section CallTarget+Inlining+NGEN
    This PR (8057) - mean (831ms)  : 809, 853
    master - mean (827ms)  : 805, 848

andrewlock

Thanks!

Collect logs in artifacts. Macrobenchmarks. [LANGPLAT-667]

1dc3c95

NachoEchevarria marked this pull request as ready for review January 14, 2026 12:00

NachoEchevarria requested a review from a team as a code owner January 14, 2026 12:00

andrewlock approved these changes Jan 14, 2026

View reviewed changes

NachoEchevarria merged commit 618beb2 into master Jan 14, 2026
187 checks passed

NachoEchevarria deleted the nacho/arm64LogsInMacroBenchmarkTests branch January 14, 2026 13:10

github-actions bot added this to the vNext-v3 milestone Jan 14, 2026

NachoEchevarria added area:tests unit tests, integration tests area:benchmarks Benchmarks, throughput tests, Crank, Bombardier, etc labels Jan 26, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Collect logs in arm64 macrobenchmark tests#8057