[https://nvbugs/6162860][fix] Set free_gpu_memory_fraction=0.6 only when torch_compile=True for test_bfloat16_

[tensorrt-cicd]

Summary

• Root cause: KvCacheConfig.free_gpu_memory_fraction=0.75 leaves no headroom for piecewise CUDA-graph workspaces and lazy cublasLt handle allocation when torch_compile + DP + MTP + ep4 are all enabled.
• Fix: Set free_gpu_memory_fraction=0.6 only when torch_compile=True for test_bfloat16_4gpus; keep 0.75 otherwise. Remove the matching nvbugs/6162860 waiver. Verified test passes (gsm8k 64.63, threshold 61.237) on a fresh GPU node.
• Automated fix generated by repair-bot

Test plan

• Verify fix on the same GPU type as the original failure
• Check for regressions in related tests

Links

• Bug: https://nvbugs/6162860

Summary by CodeRabbit

• Tests
  • Improved test reliability by dynamically adjusting memory configuration based on compilation settings.
  • Removed waived test entry to re-enable specific test validation.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 287b5977-556e-47b7-a994-c30694172b12

📥 Commits

Reviewing files that changed from the base of the PR and between 8cc5b61 and af5e3cc.

📒 Files selected for processing (2)

• tests/integration/defs/accuracy/test_llm_api_pytorch.py
• tests/integration/test_lists/waives.txt

💤 Files with no reviewable changes (1)

• tests/integration/test_lists/waives.txt

---

📝 Walkthrough

Walkthrough

This PR adjusts KV cache memory allocation in the test_bfloat16_4gpus test to account for torch_compile mode, making the free_gpu_memory_fraction dynamic (0.6 when torch_compile is enabled, otherwise 0.75), and removes a now-unnecessary test waiver entry that was skipping this configuration.

Changes

test_bfloat16_4gpus memory configuration and waiver cleanup

Layer / File(s)	Summary
Fix KV cache memory and remove waiver tests/integration/defs/accuracy/test_llm_api_pytorch.py, tests/integration/test_lists/waives.txt	test_bfloat16_4gpus now computes free_gpu_memory_fraction based on whether torch_compile is enabled (0.6 if enabled, 0.75 otherwise) and passes it into KvCacheConfig. The waived entry for this test with ep4 configuration and torch_compile=True is removed from the waivers list.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Suggested reviewers

• xinhe-nv
• jieli-matrix
• StanleySun639

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly describes the main change: adjusting free_gpu_memory_fraction based on torch_compile flag for a specific test.
Description check	✅ Passed	The description includes root cause, fix explanation, verification results, and test coverage, aligning well with the template requirements despite not using all template sections.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[liji-nv]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #49386 [ run ] triggered by Bot. Commit: 50bbfad Link to invocation

[tensorrt-cicd]

PR_Github #49386 [ run ] completed with state SUCCESS. Commit: 50bbfad
/LLM/main/L0_MergeRequest_PR pipeline #39038 completed with status: 'SUCCESS'

CI Report

Link to invocation

[liji-nv]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #50496 [ run ] triggered by Bot. Commit: 7391428 Link to invocation

[tensorrt-cicd]

PR_Github #50496 [ run ] completed with state SUCCESS. Commit: 7391428
/LLM/main/L0_MergeRequest_PR pipeline #40004 completed with status: 'SUCCESS'

CI Report

Link to invocation