[None] [feat] Add PDL support for moeAlltoAllKernels#10591
[None] [feat] Add PDL support for moeAlltoAllKernels#10591kaiyux merged 3 commits intoNVIDIA:mainfrom
Conversation
kaiyux
force-pushed
the
user/kaiyu/a2a_pdl
branch
from
3e2bea7 to
36edead
Compare
📝 WalkthroughWalkthroughRefactors kernel launches in MoE all-to-all communication kernels to use a Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes 🚥 Pre-merge checks | ✅ 1 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (1 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
cpp/tensorrt_llm/kernels/communicationKernels/moeAlltoAllKernels.cu (1)
986-1027: RemovecudaTriggerProgrammaticLaunchCompletion()from early-return paths (lines 988, 999, 1012).The kernel calls
cudaTriggerProgrammaticLaunchCompletion()in multiple early-return paths where only some threads exit while others continue to the actual work at line 1023. This differs frommoeA2ADispatchKernel, which signals completion only after all work is done.Early-exiting threads should not signal grid completion before other threads in the same CTA finish copying data. Move
cudaTriggerProgrammaticLaunchCompletion()to a single location at the true end of the kernel (after line 1023), or validate conditions at the host level before kernel launch to ensure all threads follow the same path.
🤖 Fix all issues with AI agents
In @cpp/tensorrt_llm/kernels/communicationKernels/moeAlltoAllKernels.cu:
- Around line 553-554: The call to launchWithPdlWhenEnabled invoking
moeA2APrepareDispatchKernel is missing the ep_size argument; update the
invocation to pass params.ep_size as the third kernel parameter so the kernel
receives (send_counters, local_token_counter, params.ep_size, flag_val) as
expected by moeA2APrepareDispatchKernel, ensuring the argument order matches the
kernel signature and keeping params.stream and other launch parameters
unchanged.
🧹 Nitpick comments (1)
cpp/tensorrt_llm/kernels/communicationKernels/moeAlltoAllKernels.cu (1)
1039-1041: LGTM, but note inconsistency withmoeA2APrepareCombineKernel.This kernel follows the same pattern as
moeA2ADispatchKernel: early returns do not callcudaTriggerProgrammaticLaunchCompletion(), only the main exit path does. This is the safer pattern as it ensures completion is signaled only after all meaningful work is done.Consider making
moeA2APrepareCombineKernelconsistent with this approach—removing the completion calls from early returns (lines 988, 999, 1012) and keeping only the final one (line 1026).Also applies to: 1118-1121
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
cpp/tensorrt_llm/kernels/communicationKernels/moeAlltoAllKernels.cu
🧰 Additional context used
📓 Path-based instructions (4)
**/*.{cpp,cc,cxx,h,hpp,hxx,cu,cuh}
📄 CodeRabbit inference engine (CODING_GUIDELINES.md)
**/*.{cpp,cc,cxx,h,hpp,hxx,cu,cuh}: Closing braces of namespaces should have a comment saying the namespace it closes (e.g.,} // namespace foo)
Preferconstorconstexprvariables over#defineswhenever possible
A variable that is not modified after its initialization should be declared asconst
For naming of constants in C++, use uppercase snakecase with prefix 'k' (e.g.,kDIGIT_NUM)
Except for0,nullptr,true, andfalse, all other literals should only be used for variable initialization and not in comparisons or expressions
Use Allman indentation style for brace notation in C++ code
Put the semicolon for an emptyfororwhileloop in a new line
The statement forming the body of aswitch,while,do..while, orforstatement must be a compound statement (use brace-delimited statements)
Ifandelsestatements should always be followed by brace-delimited statements, even if empty or a single statement
C++ filenames should use camelCase with first letter lowercase (e.g.,thisIsAFilename.cpp)
All types (including class names) in C++ should use PascalCase with uppercase first letter (e.g.,FooBarClass)
Local variables, methods, and namespaces in C++ should use camelCase with first letter lowercase (e.g.,localFooBar)
Non-magic-number global variables that are non-static and not defined in anonymous namespace should use camelCase prefixed with 'g' (e.g.,gDontUseGlobalFoos)
Non-magic-number global variables that are static or defined in an anonymous namespace should use camelCase prefixed with 's' (e.g.,sMutableStaticGlobal)
Locally visible static variables should use camelCase with 's' as the first letter (e.g.,static std::once_flag sFlag;)
Public, private, and protected class member variables should use camelCase prefixed with 'm' (e.g.,mNbFooValues)
Do not use Hungarian notation in C++ except for 'apps hungarian' (e.g., 'nb' to indicate count:mNbLayers)
If a constructor parameter name conflicts with a public me...
Files:
cpp/tensorrt_llm/kernels/communicationKernels/moeAlltoAllKernels.cu
**/*.{cpp,cc,cxx,cu}
📄 CodeRabbit inference engine (CODING_GUIDELINES.md)
**/*.{cpp,cc,cxx,cu}: Use smart pointers for allocating objects on the heap in C++
Preferunique_ptrfor single resource ownership andshared_ptrfor shared resource ownership in C++. Useweak_ptronly in exceptional cases
In C++ function calls where parameters are not obvious, use inline C comments to document the parameter (e.g.,doSomeOperation(/* checkForErrors = */ false);)
Use the least forceful cast necessary in C++, or no cast if possible
Casting a pointer tovoid*in C++ should be implicit (except if removingconst)
Casting in C++ should not remove anyconstorvolatilequalification from the type of a pointer or reference
Do not use C-style casts (other than void casts) and functional notation casts (other than explicit constructor calls) in C++
Casting fromvoid*toT*in C++ should be done withstatic_cast, notreinterpret_cast
Usereinterpret_castin C++ as a last resort, whereconst_castandstatic_castwon't work
Avoiddynamic_castin C++
Do not use assignment operator in C++ subexpressions (e.g.,x = y = zorif (x = y))
When practical, a C++switchstatement controlled by anenumshould have a case for each enum value and not have a default clause
C++ switch statements should be well structured as structured multi-way branches, not as 'glorified gotos'
In C++ switch statements, prohibit fall-through except from one case label to another. Each case clause must be terminated with a break or throw
Do not end a C++ case clause with return; use break or throw instead
If a C++ switch clause is a compound statement, put the break inside the braces
Do not use C library functions in C++ whenever possible. Use C++ alternatives like brace initialization orstd::fill_n()instead ofmemset()
Files:
cpp/tensorrt_llm/kernels/communicationKernels/moeAlltoAllKernels.cu
**/*.{h,hpp,hxx,cpp,cc,cxx,cu,cuh}
📄 CodeRabbit inference engine (CODING_GUIDELINES.md)
All C++ class templates, function templates, class template member functions, and class template static members must be instantiated at least once
Files:
cpp/tensorrt_llm/kernels/communicationKernels/moeAlltoAllKernels.cu
**/*.{cpp,cc,cxx,h,hpp,hxx,cu,cuh,py}
📄 CodeRabbit inference engine (CODING_GUIDELINES.md)
All TensorRT-LLM source files (.cpp, .h, .cu, .py, and other source files) should contain an NVIDIA copyright header with the year of latest meaningful modification
Files:
cpp/tensorrt_llm/kernels/communicationKernels/moeAlltoAllKernels.cu
🧠 Learnings (11)
📓 Common learnings
Learnt from: djns99
Repo: NVIDIA/TensorRT-LLM PR: 6915
File: cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_kernels.cu:4010-4012
Timestamp: 2025-08-14T23:23:27.449Z
Learning: For MOE (Mixture of Experts) code reviews in TensorRT-LLM, avoid repeatedly suggesting finalize fusion validation checks and safety assertions. The user djns99 has indicated these suggestions are repetitive and unwanted across multiple MOE-related changes.
Learnt from: nv-lschneider
Repo: NVIDIA/TensorRT-LLM PR: 7910
File: cpp/tensorrt_llm/kernels/nccl_device/config.cu:42-49
Timestamp: 2025-09-23T14:58:05.372Z
Learning: In TensorRT-LLM NCCL device kernels (cpp/tensorrt_llm/kernels/nccl_device/), the token partitioning intentionally uses ceil-like distribution (same token_per_rank for all ranks) to ensure all ranks launch the same number of blocks. This is required for optimal NCCL device API barrier performance, even though it may launch extra blocks for non-existent tokens on later ranks. Runtime bounds checking in the kernel (blockID validation) handles the overshoot cases.
📚 Learning: 2025-08-19T03:35:20.866Z
Learnt from: djns99
Repo: NVIDIA/TensorRT-LLM PR: 6915
File: cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_kernels.cu:4616-4626
Timestamp: 2025-08-19T03:35:20.866Z
Learning: In the MOE profiler TMA workspace preparation (cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_kernels.cu), the overlapping of TMA WS regions for NONE and FINALIZE variants is deliberate design to save memory space, as confirmed by djns99. The comment "reuse the same pointers to save space" reflects this intentional behavior.
Applied to files:
cpp/tensorrt_llm/kernels/communicationKernels/moeAlltoAllKernels.cu
📚 Learning: 2025-09-23T15:01:00.070Z
Learnt from: nv-lschneider
Repo: NVIDIA/TensorRT-LLM PR: 7910
File: cpp/tensorrt_llm/kernels/nccl_device/config.cu:15-17
Timestamp: 2025-09-23T15:01:00.070Z
Learning: In TensorRT-LLM NCCL device kernels, the <sstream> header is not needed as an explicit include in config.cu because it's provided transitively through other headers. Local compilation testing confirms this works without the explicit include.
Applied to files:
cpp/tensorrt_llm/kernels/communicationKernels/moeAlltoAllKernels.cu
📚 Learning: 2025-08-21T02:39:12.009Z
Learnt from: djns99
Repo: NVIDIA/TensorRT-LLM PR: 7104
File: cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_kernels.cu:1475-1480
Timestamp: 2025-08-21T02:39:12.009Z
Learning: The min latency mode functionality in TensorRT-LLM MOE kernels (cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_kernels.cu) is deprecated and no longer being maintained/updated, as confirmed by djns99. Bug reports and optimization suggestions for the computeStridesTmaWarpSpecializedLowLatencyKernel and related min latency code paths should be deprioritized.
Applied to files:
cpp/tensorrt_llm/kernels/communicationKernels/moeAlltoAllKernels.cu
📚 Learning: 2025-09-02T13:42:44.885Z
Learnt from: pcastonguay
Repo: NVIDIA/TensorRT-LLM PR: 7455
File: tensorrt_llm/_torch/pyexecutor/py_executor.py:1852-1860
Timestamp: 2025-09-02T13:42:44.885Z
Learning: In MPI communication within TensorRT-LLM pipeline parallelism, different communication types (tokens, logits, termination sync) must use disjoint tag namespaces to avoid message routing collisions when using the same source/destination patterns.
Applied to files:
cpp/tensorrt_llm/kernels/communicationKernels/moeAlltoAllKernels.cu
📚 Learning: 2025-09-23T15:01:00.070Z
Learnt from: nv-lschneider
Repo: NVIDIA/TensorRT-LLM PR: 7910
File: cpp/tensorrt_llm/kernels/nccl_device/config.cu:15-17
Timestamp: 2025-09-23T15:01:00.070Z
Learning: In TensorRT-LLM NCCL device kernels (cpp/tensorrt_llm/kernels/nccl_device/config.cu), std::ostringstream is used but <sstream> doesn't need to be explicitly included because it's provided transitively through other headers like tensorrt_llm/common/cudaUtils.h or config.h. Local compilation testing confirms this works without the explicit include.
Applied to files:
cpp/tensorrt_llm/kernels/communicationKernels/moeAlltoAllKernels.cu
📚 Learning: 2025-09-23T14:58:05.372Z
Learnt from: nv-lschneider
Repo: NVIDIA/TensorRT-LLM PR: 7910
File: cpp/tensorrt_llm/kernels/nccl_device/config.cu:42-49
Timestamp: 2025-09-23T14:58:05.372Z
Learning: In TensorRT-LLM NCCL device kernels (cpp/tensorrt_llm/kernels/nccl_device/), the token partitioning intentionally uses ceil-like distribution (same token_per_rank for all ranks) to ensure all ranks launch the same number of blocks. This is required for optimal NCCL device API barrier performance, even though it may launch extra blocks for non-existent tokens on later ranks. Runtime bounds checking in the kernel (blockID validation) handles the overshoot cases.
Applied to files:
cpp/tensorrt_llm/kernels/communicationKernels/moeAlltoAllKernels.cu
📚 Learning: 2025-08-14T23:23:27.449Z
Learnt from: djns99
Repo: NVIDIA/TensorRT-LLM PR: 6915
File: cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_kernels.cu:4010-4012
Timestamp: 2025-08-14T23:23:27.449Z
Learning: For MOE (Mixture of Experts) code reviews in TensorRT-LLM, avoid repeatedly suggesting finalize fusion validation checks and safety assertions. The user djns99 has indicated these suggestions are repetitive and unwanted across multiple MOE-related changes.
Applied to files:
cpp/tensorrt_llm/kernels/communicationKernels/moeAlltoAllKernels.cu
📚 Learning: 2025-09-23T15:12:38.312Z
Learnt from: nv-lschneider
Repo: NVIDIA/TensorRT-LLM PR: 7910
File: cpp/tensorrt_llm/thop/allreduceOp.cpp:352-446
Timestamp: 2025-09-23T15:12:38.312Z
Learning: In TensorRT-LLM NCCL device allreduce implementation (cpp/tensorrt_llm/thop/allreduceOp.cpp), the goto pattern in runNCCLAllReduceDeviceFusion is intentionally used for future extensibility, allowing multiple switch cases to fallback to the default handler. While not aesthetically ideal, this pattern supports adding more fusion cases later that can reuse the same fallback logic.
Applied to files:
cpp/tensorrt_llm/kernels/communicationKernels/moeAlltoAllKernels.cu
📚 Learning: 2025-08-20T07:43:36.447Z
Learnt from: ChristinaZ
Repo: NVIDIA/TensorRT-LLM PR: 7068
File: cpp/tensorrt_llm/kernels/moeTopKFuncs.cuh:169-172
Timestamp: 2025-08-20T07:43:36.447Z
Learning: In TensorRT-LLM MOE kernels, when processing up to 128 experts across 32 threads, each thread handles at most 4 experts (N < 5 constraint), where N represents candidates per thread rather than total system capacity.
Applied to files:
cpp/tensorrt_llm/kernels/communicationKernels/moeAlltoAllKernels.cu
📚 Learning: 2025-08-14T15:36:37.610Z
Learnt from: MatthiasKohl
Repo: NVIDIA/TensorRT-LLM PR: 6904
File: cpp/tensorrt_llm/kernels/mlaKernels.cu:436-439
Timestamp: 2025-08-14T15:36:37.610Z
Learning: CUDA kernels prioritize performance and should avoid runtime bounds checking or conditional operations that cause branching/warp divergence. Input validation should be done at the host level before kernel launch, not per-thread in the kernel.
Applied to files:
cpp/tensorrt_llm/kernels/communicationKernels/moeAlltoAllKernels.cu
🧬 Code graph analysis (1)
cpp/tensorrt_llm/kernels/communicationKernels/moeAlltoAllKernels.cu (1)
cpp/tensorrt_llm/common/envUtils.h (1)
launchWithPdlWhenEnabled(64-81)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Pre-commit Check
🔇 Additional comments (6)
cpp/tensorrt_llm/kernels/communicationKernels/moeAlltoAllKernels.cu (6)
32-33: LGTM!Clean using declaration to bring the PDL launch wrapper into scope.
339-359: LGTM!Correct PDL pattern:
cudaGridDependencySynchronize()at entry andcudaTriggerProgrammaticLaunchCompletion()at exit, properly guarded for Hopper+ architecture.
377-379: LGTM!Correct PDL pattern with
cudaGridDependencySynchronize()at entry andcudaTriggerProgrammaticLaunchCompletion()at the main exit path. The early returns for threads without work correctly do not signal completion—only the threads participating in synchronization do.Also applies to: 545-548
617-643: LGTM!Clean refactoring to use
launchWithPdlWhenEnabledfor bothBlockPolicyandWarpPolicycode paths. Kernel arguments are correctly passed.
1143-1152: LGTM!Clean refactoring using policy selection and
launchWithPdlWhenEnabled. Kernel arguments are correctly passed.
1205-1217: LGTM!Correctly uses nested macros for template specialization and
launchWithPdlWhenEnabledfor the launch. Arguments match the kernel signature.
cpp/tensorrt_llm/kernels/communicationKernels/moeAlltoAllKernels.cu
Outdated
Show resolved
Hide resolved
|
/bot run --add-multi-gpu-test |
|
PR_Github #31768 [ run ] triggered by Bot. Commit: |
|
PR_Github #31768 [ run ] completed with state
|
cpp/tensorrt_llm/kernels/communicationKernels/moeAlltoAllKernels.cu
Outdated
Show resolved
Hide resolved
kaiyux
force-pushed
the
user/kaiyu/a2a_pdl
branch
from
c22aa02 to
e65f940
Compare
cpp/tensorrt_llm/kernels/communicationKernels/moeAlltoAllKernels.cu
Outdated
Show resolved
Hide resolved
|
/bot run --add-multi-gpu-test |
|
PR_Github #32270 [ run ] triggered by Bot. Commit: |
|
PR_Github #32270 [ run ] completed with state
|
|
/bot run --add-multi-gpu-test --disable-fail-fast |
|
PR_Github #32416 [ run ] triggered by Bot. Commit: |
|
PR_Github #32416 [ run ] completed with state |
|
/bot run --add-multi-gpu-test --disable-fail-fast |
|
PR_Github #32458 [ run ] triggered by Bot. Commit: |
|
PR_Github #32458 [ run ] completed with state
|
|
/bot run --add-multi-gpu-test --disable-fail-fast |
|
PR_Github #32489 [ run ] triggered by Bot. Commit: |
|
PR_Github #32489 [ run ] completed with state
|
kaiyux
force-pushed
the
user/kaiyu/a2a_pdl
branch
from
a828bdc to
019f4ef
Compare
Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com> Use launchWithPdlWhenEnabled Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com> Fix Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com> More aggressive PDL Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com> Update Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com> Update Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com> Fix typo Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com> Move Sync Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com> fix Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
kaiyux
force-pushed
the
user/kaiyu/a2a_pdl
branch
from
019f4ef to
10b92a5
Compare
|
/bot run --add-multi-gpu-test --disable-fail-fast |
|
PR_Github #33808 [ run ] triggered by Bot. Commit: |
|
PR_Github #33808 [ run ] completed with state
|
|
/bot run --add-multi-gpu-test --disable-fail-fast |
|
PR_Github #33869 [ run ] triggered by Bot. Commit: |
|
PR_Github #33869 [ run ] completed with state
|
Signed-off-by: Zhenhuan Chen <zhenhuanc@nvidia.com>
|
/bot run --add-multi-gpu-test --disable-fail-fast |
|
PR_Github #33982 [ run ] triggered by Bot. Commit: |
|
PR_Github #33982 [ run ] completed with state
|
|
/bot run --add-multi-gpu-test --disable-fail-fast |
|
PR_Github #34031 [ run ] triggered by Bot. Commit: |
|
PR_Github #34031 [ run ] completed with state
|
|
/bot run --add-multi-gpu-test --disable-fail-fast |
|
PR_Github #34381 [ run ] triggered by Bot. Commit: |
|
PR_Github #34381 [ run ] completed with state |
4 participants
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.
Description
Dispatch
Before:
After:
Combine
Before:
After:
Test Coverage
PR Checklist
Please review the following before submitting your PR:
PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.
GitHub Bot Help
/bot [-h] ['run', 'kill', 'skip', 'reuse-pipeline'] ...Provide a user friendly way for developers to interact with a Jenkins server.
Run
/bot [-h|--help]to print this help message.See details below for each supported subcommand.
Details
run [--reuse-test (optional)pipeline-id --disable-fail-fast --skip-test --stage-list "A10-PyTorch-1, xxx" --gpu-type "A30, H100_PCIe" --test-backend "pytorch, cpp" --add-multi-gpu-test --only-multi-gpu-test --disable-multi-gpu-test --post-merge --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" --detailed-log --debug(experimental)]Launch build/test pipelines. All previously running jobs will be killed.
--reuse-test (optional)pipeline-id(OPTIONAL) : Allow the new pipeline to reuse build artifacts and skip successful test stages from a specified pipeline or the last pipeline if no pipeline-id is indicated. If the Git commit ID has changed, this option will be always ignored. The DEFAULT behavior of the bot is to reuse build artifacts and successful test results from the last pipeline.--disable-reuse-test(OPTIONAL) : Explicitly prevent the pipeline from reusing build artifacts and skipping successful test stages from a previous pipeline. Ensure that all builds and tests are run regardless of previous successes.--disable-fail-fast(OPTIONAL) : Disable fail fast on build/tests/infra failures.--skip-test(OPTIONAL) : Skip all test stages, but still run build stages, package stages and sanity check stages. Note: Does NOT update GitHub check status.--stage-list "A10-PyTorch-1, xxx"(OPTIONAL) : Only run the specified test stages. Examples: "A10-PyTorch-1, xxx". Note: Does NOT update GitHub check status.--gpu-type "A30, H100_PCIe"(OPTIONAL) : Only run the test stages on the specified GPU types. Examples: "A30, H100_PCIe". Note: Does NOT update GitHub check status.--test-backend "pytorch, cpp"(OPTIONAL) : Skip test stages which don't match the specified backends. Only support [pytorch, cpp, tensorrt, triton]. Examples: "pytorch, cpp" (does not run test stages with tensorrt or triton backend). Note: Does NOT update GitHub pipeline status.--only-multi-gpu-test(OPTIONAL) : Only run the multi-GPU tests. Note: Does NOT update GitHub check status.--disable-multi-gpu-test(OPTIONAL) : Disable the multi-GPU tests. Note: Does NOT update GitHub check status.--add-multi-gpu-test(OPTIONAL) : Force run the multi-GPU tests in addition to running L0 pre-merge pipeline.--post-merge(OPTIONAL) : Run the L0 post-merge pipeline instead of the ordinary L0 pre-merge pipeline.--extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx"(OPTIONAL) : Run the ordinary L0 pre-merge pipeline and specified test stages. Examples: --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx".--detailed-log(OPTIONAL) : Enable flushing out all logs to the Jenkins console. This will significantly increase the log volume and may slow down the job.--debug(OPTIONAL) : Experimental feature. Enable access to the CI container for debugging purpose. Note: Specify exactly one stage in thestage-listparameter to access the appropriate container environment. Note: Does NOT update GitHub check status.For guidance on mapping tests to stage names, see
docs/source/reference/ci-overview.mdand the
scripts/test_to_stage_mapping.pyhelper.kill
killKill all running builds associated with pull request.
skip
skip --comment COMMENTSkip testing for latest commit on pull request.
--comment "Reason for skipping build/test"is required. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.reuse-pipeline
reuse-pipelineReuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.