Skip to content

[None][fix] Enforce NCCL >= 2.28 at CMake configure time#13108

Merged
eopXD merged 1 commit into
NVIDIA:mainfrom
eopXD:user/yuehtingc/enforce-nccl-version-check
Apr 22, 2026
Merged

[None][fix] Enforce NCCL >= 2.28 at CMake configure time#13108
eopXD merged 1 commit into
NVIDIA:mainfrom
eopXD:user/yuehtingc/enforce-nccl-version-check

Conversation

@eopXD

@eopXD eopXD commented Apr 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Bumps the CMake find_package(NCCL ...) minimum version from 2 to 2.28
  • allreduceOp.cpp uses NCCLWindowAllocator and createNCCLWindowTensor unconditionally, but these symbols are guarded by #if NCCL_VERSION_CODE >= NCCL_VERSION(2, 28, 0) in ncclUtils.h
  • Without this fix, building with NCCL < 2.28 fails deep into compilation with cryptic undeclared symbol errors. With this fix, the build fails immediately at CMake configure time with a clear version mismatch message

Test plan

  • Build with NCCL >= 2.28 succeeds (no change in behavior)

Summary by CodeRabbit

  • Chores
    • Updated minimum NCCL dependency version requirement to 2.28 for multi-device support builds.

@eopXD
eopXD requested a review from a team as a code owner April 16, 2026 06:16
@eopXD
eopXD force-pushed the user/yuehtingc/enforce-nccl-version-check branch from ce5e87f to cce974b Compare April 16, 2026 06:18
@coderabbitai

coderabbitai Bot commented Apr 16, 2026

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

The minimum required version of NCCL dependency was updated from version 2 to version 2.28 in the CMake build configuration for multi-device support builds. No control flow or functional changes were introduced.

Changes

Cohort / File(s) Summary
NCCL Dependency Version Constraint
cpp/CMakeLists.txt
Updated NCCL minimum version requirement from 2 to 2.28 when ENABLE_MULTI_DEVICE is enabled.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically describes the main change: enforcing NCCL >= 2.28 at CMake configure time, which matches the primary modification in CMakeLists.txt.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description check ✅ Passed The PR description clearly explains the issue, rationale, and test plan following the repository template structure with Summary and Test plan sections.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@eopXD

eopXD commented Apr 16, 2026

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@bobboli

bobboli commented Apr 16, 2026

Copy link
Copy Markdown
Collaborator

Hi @eopXD , could the version be further pinned to >= 2.29 ? There are some device APIs added.
TRTLLM image has also bumped to 2.29.2 #11227

@eopXD
eopXD force-pushed the user/yuehtingc/enforce-nccl-version-check branch from cce974b to 926ddb6 Compare April 16, 2026 07:38
@eopXD

eopXD commented Apr 16, 2026

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #43714 [ run ] triggered by Bot. Commit: 926ddb6 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #43714 [ run ] completed with state FAILURE. Commit: 926ddb6
/LLM/main/L0_MergeRequest_PR pipeline #34197 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

allreduceOp.cpp uses NCCLWindowAllocator and createNCCLWindowTensor
unconditionally, but these are guarded by #if NCCL_VERSION_CODE >=
NCCL_VERSION(2, 29, 0) in ncclUtils.h. With NCCL < 2.29, the build
fails deep into compilation with undeclared symbol errors. Bump the
CMake find_package minimum version so the build fails immediately at
configure time with a clear version mismatch message instead.

Signed-off-by: Yueh-Ting Chen <yueh.ting.chen@gmail.com>
@eopXD
eopXD force-pushed the user/yuehtingc/enforce-nccl-version-check branch from 926ddb6 to 8c41ccb Compare April 20, 2026 05:13
@eopXD

eopXD commented Apr 20, 2026

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #44308 [ run ] triggered by Bot. Commit: 8c41ccb Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #44308 [ run ] completed with state FAILURE. Commit: 8c41ccb
/LLM/main/L0_MergeRequest_PR pipeline #34729 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

@eopXD

eopXD commented Apr 20, 2026

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #44329 [ run ] triggered by Bot. Commit: 8c41ccb Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #44329 [ run ] completed with state FAILURE. Commit: 8c41ccb
/LLM/main/L0_MergeRequest_PR pipeline #34747 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

@eopXD

eopXD commented Apr 20, 2026

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #44454 [ run ] triggered by Bot. Commit: 8c41ccb Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #44454 [ run ] completed with state SUCCESS. Commit: 8c41ccb
/LLM/main/L0_MergeRequest_PR pipeline #34858 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

@eopXD

eopXD commented Apr 21, 2026

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #44597 [ run ] triggered by Bot. Commit: 8c41ccb Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #44597 [ run ] completed with state SUCCESS. Commit: 8c41ccb
/LLM/main/L0_MergeRequest_PR pipeline #34983 completed with status: 'SUCCESS'

CI Report

Link to invocation

@eopXD
eopXD enabled auto-merge (squash) April 22, 2026 06:00
@eopXD
eopXD merged commit 342adb3 into NVIDIA:main Apr 22, 2026
5 checks passed
ziyixiong-nv pushed a commit to ziyixiong-nv/TensorRT-LLM that referenced this pull request Apr 24, 2026
Signed-off-by: Yueh-Ting Chen <yueh.ting.chen@gmail.com>
@eopXD
eopXD deleted the user/yuehtingc/enforce-nccl-version-check branch May 7, 2026 06:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants