[None][feat] Add encoder CUDA graph support to llm.encode()

[tingyangk]

Summary by CodeRabbit

Release Notes

• New Features

  • Added CUDA graph acceleration support for encoder workloads, enabling faster inference through graph caching and replay.
  • Added return_raw_logits parameter to encoder API for direct logits output.
  • Extended CUDA graph configuration with encoder-specific sizing options.

• Improvements

  • Enhanced encoder execution stability and performance optimization.
  • Added stricter validation for encoder and multimodal encoder configurations.

• Tests

  • Expanded encoder testing to include CUDA graph equivalence checks against eager execution.

Description

The PyTorch backend encoder forward path llm.encode() currently runs in pure eager mode. Prefill-only encoder workloads (BERT classification, decoder-as-encoder) are dominated by launch overhead at small batch sizes or short sequence lengths.

This PR introduces encoder CUDA graph infrastructure to the encode-only path. The main advantage is that, unlike piecewise CUDA graphs, it captures the entire model, including the attention layers, into one CUDA graph per bucket.

It adds a dedicated EncoderCUDAGraphRunner alongside the decoder runner in cuda_graph_runner.py, wires graph capture/replay into PyTorchModelEngine.warmup_encoder() and the encode forward step, and surfaces the configuration through the existing CudaGraphConfig on TorchLlmArgs.

Key design points:

• Composite key. Encoder graphs use a 3-tuple (padded_batch_size, padded_num_tokens, padded_max_seq_len) because two batches with the same num_tokens can still differ in max(seq_lens) for context-only batches, and FMHA launch parameters depend on the latter. A new max_context_q_len_override field — plumbed through to the thop.attention op — locks the FMHA kernel launch parameters across replays even when the actual batch has shorter sequences.

• Stable attention metadata. TrtllmAttentionMetadata gains bind_encoder_cuda_graph_seq_lens() and prepare_encoder_cuda_graph_replay(). When CC (confidential computing) is off, pinned host buffers are used: per-replay seq_lens updates and model inputs are CPU memcpys into pinned host buffers, and the host-to-device copies are captured in the graph to minimize CPU work in the replay path. When CC is on, pinned host buffers are not preferred, so the replay path updates the persistent GPU buffers directly before graph replay.

• Attention backend constraints. Only TRTLLM attention metadata is eligible. Non-TRTLLM backends, including FlashInfer, fall back to eager because their planner/wrapper state is per batch and is not replay-safe.

• Input constraints. Encoder CUDA graphs currently support text-token inputs only. Additional encoder inputs such as inputs_embeds or other model-specific tensors remain supported by the eager llm.encode() path, but are not captured/replayed by EncoderCUDAGraphRunner yet.

New API surface

CudaGraphConfig (tensorrt_llm/llmapi/llm_args.py) gains four encoder-only fields. They mirror the existing batch_sizes / max_batch_size pair:

Field	Type	Default	Meaning
num_tokens	Optional[List[int]]	None	Total token counts (Σ per-request lengths) to capture graphs for.
max_num_token	NonNegativeInt	0	If num_tokens is provided, must equal max(num_tokens); otherwise the list is auto-generated from this.
seq_lens	Optional[List[int]]	None	Per-request max sequence lengths to capture graphs for.
max_seq_len	NonNegativeInt	0	Same auto-derivation rule as max_num_token, for seq_lens.

Leaving all four at their defaults keeps encoder CUDA graphs disabled, even when batch_sizes is set — graphs only activate when the user explicitly opts in to encoder bucketing.

Usage

Minimal example — BERT classification with encoder CUDA graphs enabled:

from tensorrt_llm import LLM
from tensorrt_llm.llmapi import EncodeCudaGraphConfig

cgc = EncodeCudaGraphConfig(
    batch_sizes=[1, 4],          # batch buckets, same as the decoder path
    num_tokens=[16, 64],         # encode-only: total tokens per batch
    seq_lens=[8, 32],            # encode-only: per-request max seq lens
    enable_padding=True,         # round inputs up to the nearest bucket
)

llm = LLM(
    model="bert-base-uncased-yelp-polarity",
    encode_only=True,
    cuda_graph_config=cgc,
)

outputs = llm.encode([
    "Hello, my name is",
    "The capital of France is",
])
for out in outputs:
    print(out.logits)

llm.shutdown()

The grid here captures graphs for the feasible graph buckets of the cross-product {1, 4} × {16, 64} × {8, 32}; at inference time enable_padding=True rounds each incoming batch up to the nearest supported bucket. Decoder-only models can also use the encode-only path with encode_only=True and the same config.

Performance

BERT 110M benchmark

• GPU: L4
• Batch size: 1
• ISL: ~10
• Column definition:
  • baseline_generate: runs with old llm.generate() path
  • baseline_encode: runs with new llm.encode() path in eager mode
  • cuda-graph: runs with new llm.encode() path + encoder CUDA graph enabled
  • torch-compile+cuda-graph: runs with new llm.encode() path + TorchCompileConfig & encoder CUDA graph enabled
• Row definition:
  • speedup generate: uses original llm.generate() latency as baseline; shows the speedup from moving to llm.encode().
  • speedup encode: uses eager llm.encode() latency as baseline; shows the speedup from encoder CUDA graphs.

Metric	baseline_generate	baseline_encode	cuda-graph	torch-compile+cuda-graph
p50	6.744 ms	4.337 ms	1.292 ms	1.202 ms
p99	7.323 ms	4.455 ms	1.486 ms	1.309 ms
mean	6.789 ms	4.340 ms	1.314 ms	1.218 ms
throughput	147.29 b/sec	230.42 b/sec	760.84 b/sec	821.13 b/sec
speedup generate	1.00x	1.56x	5.17x	5.57x
speedup encode	0.64x	1.00x	3.30x	3.56x

Qwen3-0.6B benchmark: PyTorch llm.encode() variants vs legacy TRT path

• GPU: L4
• Batch size: 1
• TRT engine configs (built 2 engines):
  • max_num_tokens=768 for shorter (ISL <= 512) sequence lengths
  • max_num_tokens=16384 for longer (ISL > 512) sequence lengths
• Column definition:
  • latency delta vs TRT: uses the best latency from eager, cuda-graph, or torch-compile+cuda-graph and compares it with TRT; negative means the PyTorch variant is faster than TRT.

ISL	TRT (ms)	eager (ms)	cuda-graph (ms)	torch-compile+cuda-graph (ms)	latency delta vs TRT
16	7.65	16.39	5.98	5.97	-21.9%
64	7.65	16.60	6.39	6.43	-16.5%
128	8.25	16.60	7.01	6.99	-15.3%
512	14.70	16.89	14.76	14.65	-0.4%
1024	28.20	27.93	29.35	29.15	-0.9%
2048	62.11	61.19	61.59	61.49	-1.5%
4096	154.56	142.04	143.57	142.85	-8.1%
8192	411.93	375.75	378.39	376.89	-8.8%
12288	767.49	696.46	699.78	698.69	-9.3%
16000	1172.01	1064.07	1065.81	1067.61	-9.2%

Test Coverage

Integration tests

Add new tests to tests/integration/defs/accuracy/test_llm_api_pytorch_encode.py:

• TestEncoderEncode (encoder-only architectures):

  • test_encoder_encode_cuda_graph_matches_eager_logits[bert-yelp] — Tight numerical bound (rtol=atol=1e-3) between graph replay and eager output on the same model, to catch subtle launch-parameter or metadata drift that HF-tolerance wouldn't flag.

• TestDecoderEncode (decoder-as-encoder):

  • test_decoder_encode_cuda_graph_matches_eager_logits[tinyllama-1.1b] — Tight graph-vs-eager bound for the single-prefill decoder path.

Unit tests

Add new tests to tests/unittest/llmapi/test_llm_encode.py:

• CUDA graph path: test_encode_cuda_graph_and_return_raw_logits — exercises capture + replay end-to-end and verifies return_raw_logits output wrapping on the graph path.
• Backend guard: test_trtllm_gen_is_supported_rejects_none_kv_cache_manager — asserts the trtllm_gen backend refuses to be selected without a kv_cache_manager.

tests/unittest/llmapi/test_llm_args.py adds coverage for the new CudaGraphConfig fields (auto-derivation of max_num_token / max_seq_len, mismatch error paths, sorted-list normalization).

CI wiring

The new integration tests are added to:

• tests/integration/test_lists/test-db/l0_a100.yml
• tests/integration/test_lists/qa/llm_function_core.txt
• tests/integration/test_lists/qa/llm_function_l20.txt
• tests/integration/test_lists/qa/llm_function_rtx6k.txt

CC: @schetlur-nv @nvrohanv @amukkara @symphonylyh

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[tingyangk]

/bot run

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR enables CUDA graph capture and replay for encoder-only LLM.encode() execution. It introduces a max_context_q_len_override parameter through the C++ attention kernel, extends attention backends to support encoder-only CUDA graphs with new metadata preparation utilities, adds encoder-specific configuration to CudaGraphConfig, implements an EncoderCUDAGraphRunner for graph bucketing and replay, enhances the model engine with encoder warmup and capture workflows, and extends tests to validate eager vs. graph logits equivalence.

Changes

Encoder-only CUDA Graph Capture and Replay

Layer / File(s)	Summary
C++ attention kernel parameter plumbing cpp/tensorrt_llm/nanobind/thop/bindings.cpp, cpp/tensorrt_llm/thop/attentionOp.cpp, cpp/tensorrt_llm/thop/attentionOp.h	max_context_q_len_override parameter added to C++ attention signatures (nanobind binding, RunnerBase::run, Runner<T>::run, and attention() wrapper) and plumbed through kernel execution to stabilize launch parameters.
Attention backend encoder CUDA graph support tensorrt_llm/_torch/attention_backend/flashinfer.py, tensorrt_llm/_torch/attention_backend/interface.py, tensorrt_llm/_torch/attention_backend/trtllm.py, tensorrt_llm/_torch/attention_backend/trtllm_gen.py	encode_only flag added to create_cuda_graph_metadata() across all backends. TrtllmAttentionMetadata gains max_context_q_len_override field and three new encoder preparation methods. trtllm_gen now requires non-None kv_cache_manager for support (encode-only paths fall back to thop.attention()).
Configuration for encoder CUDA graphs tensorrt_llm/llmapi/llm_args.py	CudaGraphConfig extended with encoder-only sizing fields (num_tokens, max_num_token, seq_lens, max_seq_len) and bucket generation helpers. TorchLlmArgs validators enforce encode_only mutual exclusion with mm_encoder_only and piecewise CUDA graphs.
LLM.encode() API and raw logits support tensorrt_llm/llmapi/llm.py	Added return_raw_logits parameter to encode() signature. Refactored input tokenization to track per-sequence lengths. Added validation blocking incompatible model_kwargs when encoder CUDA graphs enabled. Logits can now return raw tensor directly or packaged as EncoderOutput.
Encoder executor warmup initialization tensorrt_llm/_torch/pyexecutor/encoder_executor.py	EncoderExecutor.__init__ calls model_engine.warmup_encoder() during setup.
Encoder CUDA graph runner implementation tensorrt_llm/_torch/pyexecutor/cuda_graph_runner.py	New EncoderCUDAGraphRunner and EncoderCUDAGraphRunnerConfig manage encoder graph bucketing by (batch_size, num_tokens, seq_len), capture with warmup runs, maintain pinned/pageable seq_lens storage, and replay with automatic input/position_ids updates.
Model engine encoder execution, warmup, and graph capture tensorrt_llm/_torch/pyexecutor/model_engine.py	Added encoder CUDA graph bucket filtering (_filter_cuda_graph_num_tokens, _filter_cuda_graph_seq_lens), encode-only detection, encoder attention metadata caching, warmup pipelines (general, autotuner, graph capture), and refactored encoder_forward to route between eager and graph replay/capture based on runner state.
Multimodal encoder validation tensorrt_llm/llmapi/mm_encoder.py	MultimodalEncoder._validate_mm_args_for_torch_backend now rejects encode_only=True, directing users to use mm_encoder_only only.
Integration and unit test coverage tests/integration/defs/accuracy/test_llm_api_pytorch_encode.py, tests/integration/test_lists/qa/*.txt, tests/integration/test_lists/test-db/l0_a100.yml, tests/unittest/llmapi/test_llm_args.py, tests/unittest/llmapi/test_llm_encode.py	Extended encoder tests with CUDA graph parametrization, added eager vs. graph logits equivalence checks with tighter tolerances, renamed and extended decoder tests, updated QA and pre-merge test lists with new CUDA-graph-specific test variants, and added unit tests for encoder configuration validation and contract coverage.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Suggested reviewers

• schetlur-nv
• taylor-yb-lee
• venkywonka
• mikeiovine
• nvchenghaoz

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 67.61% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main addition: encoder CUDA graph support for llm.encode(), directly matching the PR's primary objective.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description check	✅ Passed	PR description is comprehensive, well-structured, and clearly explains the feature, design decisions, API surface, usage, performance benchmarks, and test coverage.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 7

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tensorrt_llm/_torch/pyexecutor/cuda_graph_runner.py`:
- Around line 885-915: The capture currently only refreshes input_ids,
position_ids and attn_metadata.seq_lens before replay, leaving other per-call
dynamic kwargs in capture_inputs (e.g., token_type_ids, inputs_embeds,
attention_mask) frozen; update the capture/replay logic in the CUDA graph
creation around capture_inputs / sliced_static_tensors /
sliced_static_tensors_cpu and forward_fn so that all dynamic per-call inputs are
refreshed before graph replay: either detect extra dynamic kwargs and fall back
to eager execution, or allocate pinned/static GPU buffers for each dynamic field
present and perform non_blocking.copy_ into capture_inputs for those fields
(similar to input_ids/position_ids handling, and also update attn_md fields),
and ensure the same change is applied to the corresponding replay code paths
later (the section around lines 924-984 mentioned in the review). Ensure checks
respect self._capture_h2d_copy and that graph_metadata still records
attn_metadata correctly.
- Around line 916-921: The code assigns self.graphs[key] before checking for
nested-tensor outputs, which can leave a half-registered graph if
self._contains_nested_tensor(output) raises; move the assignment of
self.graphs[key] so it occurs after the nested-tensor check (or ensure you
remove self.graphs[key] on error) and only set self.graph_outputs[key] =
make_weak_ref(output) after the check passes; adjust the logic in the method
that builds/stores graphs (the block that references self.graphs[key],
self._contains_nested_tensor, and self.graph_outputs[key]) so an exception won’t
leave an inconsistent entry.

In `@tensorrt_llm/_torch/pyexecutor/model_engine.py`:
- Around line 4243-4246: The returned graph outputs other than "logits" are
still backed by static replay buffers and must be cloned before returning;
update the return construction (where outputs = dict(graph_outputs) and logits
is sliced/cloned) to: for each key,value in graph_outputs (including non-logit
tensors) slice to batch_size if applicable and call .clone()/.detach() so no
tensor shares the underlying static buffer; specifically modify the code around
outputs = dict(graph_outputs) / outputs["logits"] = ... to iterate graph_outputs
items, clone/detach each tensor (and slice by batch_size when per-batch) and
place into outputs so future encode() calls cannot overwrite returned tensors.

In `@tensorrt_llm/llmapi/llm_args.py`:
- Around line 127-151: Make the encoder bucket list fields reject empty lists
and non-positive values: change num_tokens: Optional[List[int]] to
Optional[List[PositiveInt]] with Field(min_length=1, default=None,
description=...), change seq_lens similarly to Optional[List[PositiveInt]] with
Field(min_length=1, default=None,...), and change max_num_token and max_seq_len
from NonNegativeInt to PositiveInt (and update their default/validation so 0 is
not allowed). Apply the same adjustments to the analogous fields referenced
later (lines ~184-220) so list types require PositiveInt elements and lists have
min_length=1, and ensure validation logic that compares num_tokens/seq_lens to
max_num_token/max_seq_len remains consistent.

In `@tensorrt_llm/llmapi/llm.py`:
- Around line 757-759: The encode() signature and docstring are incorrect for
return_raw_logits: when return_raw_logits=True it returns a torch.Tensor, not
EncoderOutput(s); add explicit typing overloads using typing.overload above
encode()—one overload for return_raw_logits: Literal[True] -> torch.Tensor and
one for return_raw_logits: Literal[False] (or omitted) -> Union[EncoderOutput,
List[EncoderOutput]]—then update the implementation signature to return
Union[EncoderOutput, List[EncoderOutput], torch.Tensor] and revise the encode()
docstring to document the return_raw_logits behavior and the tensor return path;
apply the same overload+docstring change to the other similar encode variants
referenced (the other overload blocks for encode).
- Around line 851-855: The current guard in LLM.encode that raises
NotImplementedError when engine.encoder_cuda_graph_runner.enabled blocks all
non-reserved model_kwargs (filtered_kwargs) including documented supported
inputs like token_type_ids and inputs_embeds; change the check to whitelist
these allowed encoder inputs instead of rejecting everything: in the LLM.encode
code path that references filtered_kwargs and
engine.encoder_cuda_graph_runner.enabled, build an allowed_encoder_kwargs set
(e.g., {"token_type_ids", "inputs_embeds"}), subtract that from filtered_kwargs,
and only raise NotImplementedError if any unsupported keys remain (include the
remaining sorted keys in the error message as before).

In `@tests/integration/defs/accuracy/test_llm_api_pytorch_encode.py`:
- Around line 173-176: When running the CUDA-graph variants created with
LLM(model_path, encode_only=True, ..., cuda_graph_config=cgc), add an explicit
assertion that CUDA-graph capture/replay is actually active instead of relying
on logits differences: after calling outs = llm.encode(PROMPTS) (and before
comparing logits) assert that cgc is None or llm.is_cuda_graph_active() (or
llm.backend == "TRTLLM" if your code uses backend names), and if
is_cuda_graph_active() does not exist add a small accessor on LLM (e.g.,
is_cuda_graph_active or cuda_graph_active property) that returns whether
cuda_graph_config was activated so the test fails if execution silently fell
back to eager mode; apply the same pattern to the other occurrences around the
encode calls (the blocks at the other line ranges noted).

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 7dddcd35-5603-4689-a484-b0b2bf1c7c1e

📥 Commits

Reviewing files that changed from the base of the PR and between b892451 and dbb7997.

📒 Files selected for processing (20)

• cpp/tensorrt_llm/nanobind/thop/bindings.cpp
• cpp/tensorrt_llm/thop/attentionOp.cpp
• cpp/tensorrt_llm/thop/attentionOp.h
• tensorrt_llm/_torch/attention_backend/flashinfer.py
• tensorrt_llm/_torch/attention_backend/interface.py
• tensorrt_llm/_torch/attention_backend/trtllm.py
• tensorrt_llm/_torch/attention_backend/trtllm_gen.py
• tensorrt_llm/_torch/pyexecutor/cuda_graph_runner.py
• tensorrt_llm/_torch/pyexecutor/encoder_executor.py
• tensorrt_llm/_torch/pyexecutor/model_engine.py
• tensorrt_llm/llmapi/llm.py
• tensorrt_llm/llmapi/llm_args.py
• tensorrt_llm/llmapi/mm_encoder.py
• tests/integration/defs/accuracy/test_llm_api_pytorch_encode.py
• tests/integration/test_lists/qa/llm_function_core.txt
• tests/integration/test_lists/qa/llm_function_l20.txt
• tests/integration/test_lists/qa/llm_function_rtx6k.txt
• tests/integration/test_lists/test-db/l0_a100.yml
• tests/unittest/llmapi/test_llm_args.py
• tests/unittest/llmapi/test_llm_encode.py

[tensorrt-cicd]

PR_Github #49268 [ run ] triggered by Bot. Commit: dbb7997 Link to invocation

[tingyangk]

/bot kill

[tensorrt-cicd]

PR_Github #49271 [ kill ] triggered by Bot. Commit: dbb7997 Link to invocation

[tensorrt-cicd]

PR_Github #49268 [ run ] completed with state ABORTED. Commit: dbb7997

Link to invocation

[tensorrt-cicd]

PR_Github #49271 [ kill ] completed with state SUCCESS. Commit: dbb7997
Successfully killed previous jobs for commit dbb7997

Link to invocation

[tingyangk]

/bot run

[tensorrt-cicd]

PR_Github #49272 [ run ] triggered by Bot. Commit: adc3500 Link to invocation

[tingyangk]

/bot kill

[tingyangk]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #49278 [ kill ] triggered by Bot. Commit: adc3500 Link to invocation

[tensorrt-cicd]

PR_Github #49279 [ run ] triggered by Bot. Commit: adc3500 Link to invocation

[tensorrt-cicd]

PR_Github #49278 [ kill ] completed with state ABORTED. Commit: adc3500

Link to invocation

[tingyangk]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #49754 [ run ] triggered by Bot. Commit: 97d9495 Link to invocation

[tensorrt-cicd]

PR_Github #49754 [ run ] completed with state SUCCESS. Commit: 97d9495
/LLM/main/L0_MergeRequest_PR pipeline #39355 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[tingyangk]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #51421 [ run ] triggered by Bot. Commit: e1fc709 Link to invocation

[tensorrt-cicd]

PR_Github #51421 [ run ] completed with state SUCCESS. Commit: e1fc709
/LLM/main/L0_MergeRequest_PR pipeline #40830 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[tingyangk]

/bot run

[tensorrt-cicd]

PR_Github #51521 [ run ] triggered by Bot. Commit: e1fc709 Link to invocation

[tensorrt-cicd]

PR_Github #51521 [ run ] completed with state FAILURE. Commit: e1fc709
/LLM/main/L0_MergeRequest_PR pipeline #40920 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[tingyangk]

/bot run

[2ez4bz]

Approving changes to tensorrt_llm/llmapi/mm_encoder.py.

[tingyangk]

/bot run

[tensorrt-cicd]

PR_Github #51687 [ run ] triggered by Bot. Commit: e1fc709 Link to invocation

[tensorrt-cicd]

PR_Github #51687 [ run ] completed with state SUCCESS. Commit: e1fc709
/LLM/main/L0_MergeRequest_PR pipeline #41067 completed with status: 'SUCCESS'

CI Report

Link to invocation