Skip to content

[None][feat] DSv4: enable GVR Heuristic Top-K for compress_ratio=4#14219

Merged
lfr-0531 merged 3 commits into
NVIDIA:feat/deepseek_v4from
longcheng-nv:feat/deepseek_v4
May 19, 2026
Merged

[None][feat] DSv4: enable GVR Heuristic Top-K for compress_ratio=4#14219
lfr-0531 merged 3 commits into
NVIDIA:feat/deepseek_v4from
longcheng-nv:feat/deepseek_v4

Conversation

@longcheng-nv

@longcheng-nv longcheng-nv commented May 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Widens the canUseHeuristic gate in cpp/tensorrt_llm/kernels/indexerTopK.cu from compressRatio == 1 to compressRatio ∈ {1, 4}, enabling GVR Heuristic Top-K on the DSv4 indexer (which operates in compressed-token-index space).
  • Threads compressRatio through launchHeuristicTopKDecode (3 dtype overloads). Kernel forces preIdxOffset = 0 when cr != 1 (compressed entries are appended at the end, so prev-step indices in [0, c_prev-1] remain valid in [0, c_curr-1]), and divides seqLens by compressRatio internally to derive per-row compressed N.
  • Adds pass-through for 6 DeepSeekSparseAttentionConfig sibling fields in tensorrt_llm/_torch/model_config.py:681-731 so user-set enable_heuristic_topk: true (and friends) survive the V4 sparse_attention_config rebuild. Previously the V4 rebuild silently dropped these back to subclass defaults — a user setting enable_heuristic_topk: true in --extra_llm_api_options had no effect on V4.
  • Adds 108-case parametrized correctness test test_indexer_topk_decode_dist_v4_cr4 in tests/unittest/_torch/thop/parallel/test_indexer_topk.py (cr=4 × MTP {1,2,3} × K {512,1024,2048} × {fp32,bf16,fp16} × batch_size {1,64} × num_tokens {65536,131072}) plus apply_mtp_structure_compressed / generate_pre_idx_v4 helpers.

See the commit message for full kernel-isolated perf evidence and NCU SOL/occupancy analysis.

Perf evidence (8× B300 SXM6 AC)

Indexer kernel-isolated microbench at production shape (numColumns=16642, sr=0.6 matching generate_pre_idx_v4):

Variant K dtype R/H mean over nR ∈ {4,8,12,16}
Flash 512 bf16 1.55-1.67×
Flash 512 fp32 2.01-2.08×
Pro 1024 bf16 1.40-1.57×
Pro 1024 fp32 2.02-2.17×

End-to-end trtllm-bench (Flash ISL=65536 BS=4 MTP=3, autotuner-off greedy, clean conditions): +6.4% req/s / −4.8% TPOT / −5.4% TTFT vs Radix.

Note on TTFT. GVR Heuristic Top-K is decode-only at the kernel level — invokeIndexerTopKDecode has the canUseHeuristic gate (cpp/tensorrt_llm/kernels/indexerTopK.cu:910,1045); invokeIndexerTopKPrefill (indexerTopK.cu:1126-1146) is unchanged and never takes the GVR path. The TTFT improvement is an in-flight batching artifact: faster GVR decode steps interleaved with new-request prefill chunks shorten per-iteration wall, which the scheduler attributes to the new request's TTFT. This is consistent with TTFT (−5.4%) ≈ TPOT (−4.8%) in §6, whereas a true prefill-kernel speedup would push TTFT beyond TPOT.

E2E accuracy validation (8× B300)

Canonical accuracy judge: 4 independent GVR-ON GSM8K runs (strict-match, multi-token output)

Config GVR-ON Baseline Δ
Flash GSM8K, autotuner-OFF 96.66 ±0.49 96.85 −0.23
Flash GSM8K, autotuner-ON 96.51 ±0.51 96.85 −0.38
Pro GSM8K (pro_safe, graph-off + MTP off) 96.74 ±0.49 96.66 ±0.49 +0.08
Pro GSM8K (graph-on + reuse-off + MTP=3, K=1024 path) 96.44 ±0.51 96.66 ±0.49 −0.22

All Δ ≤ 0.5 pp (within 1× stderr). Pro K=1024 + CUDA Graph capture + sparse-MLA path validated with no OOB regression.

Brittle-benchmark coverage (informational)

Config GVR-ON Baseline Δ Note
Flash MMLU 5-shot (14 042) 78.12 78.19 −0.07 ✓ within noise
Flash GPQA-Diamond (198) 73.74 ±3.13 76.77 ±3.01 −3.03 within stderr (198-sample task; statistically tied)
Pro MMLU 5-shot (14 042) 65.50 67.03 −1.53 see 2×2 isolation below — not a GVR regression

Pro MMLU 5-shot drift is NOT caused by GVR. The published baseline (67.03) was measured with autotuner-ON; the GVR-ON matrix flipped TWO variables simultaneously (GVR-ON + autotuner-OFF). A 2×2 control with GVR-OFF + autotuner-OFF yielded 62.94 (Δ −4.09 pp), proving autotuner-OFF alone causes the dominant drift; the GVR-ON delta vs the autotuner-OFF baseline is actually +2.56 pp (recovery). MMLU 5-shot chat-mode forces 1-token outputs (14 017/14 042 Pro outputs are a single A/B/C/D letter), making it pp-level sensitive to floating-point reduction order changes from any config flip — unsuitable for sparse-attention feature regression detection. GSM8K strict-match (rows above) is the canonical judge.

How to enable GVR — V3.2 vs V4

The user-facing flag (enable_heuristic_topk: true) is identical in both, but the dispatch chain differs in two ways under the hood:

Aspect V3.2 (existing) V4 (this PR)
algorithm value (Pydantic discriminator at llm_args.py:318, 410) dsaDeepSeekSparseAttentionConfig deepseek_v4DeepSeekV4SparseAttentionConfig (subclass)
enable_heuristic_topk field origin inherited (llm_args.py:350-358) inherited (same field)
index_topk default 2048 512 (Flash) / 1024 (Pro)
compressRatio in indexer 1 (raw token-index space) 4 (compressed; numColumns = ISL / 4)
Kernel gate canUseHeuristic (indexerTopK.cu:910, 1045) compressRatio == 1 compressRatio ∈ {1, 4} (PR change)
model_config.py pass-through of user-set enable_heuristic_topk direct field copy in V3.2 rebuild branch (model_config.py:743-779) helper update_sparse_attention_indexer_config(...) (model_config.py:681-731) — added by this PR; previously V4 rebuild silently dropped the user override back to subclass default False
Prefill indexer (invokeIndexerTopKPrefill, indexerTopK.cu:1126-1146) always Radix always Radix (unchanged)

YAML — V3.2 (reference, unchanged)

sparse_attention_config:
    algorithm: dsa
    enable_heuristic_topk: true
# Optional cleanliness flags (recommended for measurement):
enable_autotuner: false      # removes tactic-interaction noise

YAML — V4 (new; requires this PR)

sparse_attention_config:
    algorithm: deepseek_v4
    enable_heuristic_topk: true
# Optional cleanliness flags (recommended for measurement):
enable_autotuner: false
speculative_config:
    decoding_type: MTP
    num_nextn_predict_layers: 3   # V4 production default

Without this PR, the V4 YAML above silently has no effect (V4 rebuild path drops the user override back to subclass False); with this PR, V4 honors it the same way V3.2 always has.

Verification (both models)

Run with TRTLLM_SCHEMEX_DEBUG=1 set in the environment and grep stderr / log for [Scheme X]:

[Scheme X] numRows=16 numColumns=16384 kBsWave=426 kBsL2=3645 kBsLarge=426 ... topK=512  ... -> Heuristic path   # V4 Flash
[Scheme X] numRows=4  numColumns=65536 kBsWave=426 kBsL2=440  kBsLarge=426 ... topK=2048 ... -> Heuristic path   # V3.2

If the dispatcher prints -> Radix path for a workload you expect to use GVR, one of the gate conditions is unmet (typical causes: index_topk ∉ {512, 1024, 2048}, numColumns < kSeqSmall = 16384, or numRows ≥ kBsLarge). The gate falls back to Radix safely — never wrong output, just no GVR speedup.


V3.2 backward compatibility

  • Kernel cr=1 path is character-for-character unchanged: preIdxOffset = (rowIdx % next_n) + 1 and N = actual_kv_len (since cr==1).
  • Torch op binding has int compress_ratio=1 as default in both C++ m.def (cpp/tensorrt_llm/thop/IndexerTopKOp.cpp:199) and Python fake op (tensorrt_llm/_torch/custom_ops/cpp_custom_ops.py:212).
  • Existing V3.2 unit test test_indexer_topk_decode_dist (no cr parameter) auto-uses cr=1 → no test modification needed; cr=1 production callers in dsa.py:102 similarly unchanged.
  • git merge-base origin/feat/deepseek_v4 HEAD == 37c2e05ecf — HEAD is a strict descendant of upstream (no rebase / sibling conflict).

Test plan

  • Unit test: pytest tests/unittest/_torch/thop/parallel/test_indexer_topk.py::test_indexer_topk_decode_dist_v4_cr4 — 108 cases, @skip_pre_blackwell
  • V3.2 backward compat: existing test_indexer_topk_decode_dist still passes (cr=1 default)
  • E2E accuracy: 4 GVR-ON GSM8K runs (Flash autotuner-OFF/-ON, Pro safe/graph-on) all Δ ≤ 0.5 pp from baseline; full defense table above
  • Sparse-MLA OOB regression check: Pro K=1024 + CUDA Graph capture + reuse-off + MTP=3 passes (validates the K=1024 + graph path under GVR)
  • /bot run --disable-fail-fast — triggered 2026-05-17T13:51:24Z (initial run failed on stale clang-format + ruff-format; re-triggered after auto-fix amend)
    Follow-up (separate PR): add test_heuristic_topk to TestDeepSeekV4Flash in tests/integration/defs/accuracy/test_llm_api_pytorch.py, mirroring V3.2's heuristic_topk_mtp1 parametrization at line 3344,3348; register in tests/integration/test_lists/test-db/l0_dgx_b300_ds.yml.

🤖 Generated with Claude Code

@longcheng-nv
longcheng-nv requested review from a team as code owners May 17, 2026 13:50
@longcheng-nv
longcheng-nv requested review from HuiGao-NV, liji-nv and schetlur-nv and removed request for a team May 17, 2026 13:50
@longcheng-nv

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

Widens the canUseHeuristic gate in indexerTopK.cu to allow
compressRatio == 4 in addition to 1, wires
sparse_attention_config.enable_heuristic_topk through model_config.py,
and adds a parameterized cr=4 unit test
(test_indexer_topk_decode_dist_v4_cr4).

Refactor:
  - heuristicTopKDecode.{cu,h}: take compressRatio as argument; force
    preIdxOffset = 0 when compressRatio != 1 (V4 indexer operates in
    compressed-token-index space where new entries are appended at the
    end, so prev-step indices in [0, c_prev-1] remain valid as-is in
    [0, c_curr-1] — no +1/+2/+3 shift needed).
  - indexerTopK.cu: bool compressRatioOk = (compressRatio == 1 ||
    compressRatio == 4).
  - model_config.py: pass-through sparse_attention_config fields
    (use_cute_dsl_topk, q_split_threshold, indexer_rope_interleave,
    enable_heuristic_topk, indexer_k_dtype, …) so user-set values
    survive the V4 sparse_attention_config rebuild. Previously the V4
    rebuild silently dropped these back to subclass defaults, so a
    user setting enable_heuristic_topk: true in
    --extra_llm_api_options had no effect on the V4 model.
  - test_indexer_topk.py: dist-parameterized correctness test for V4
    (cr=4) covering MTP windows {1,2,3}, all GVR-supported K
    {512,1024,2048}, and fp32/bf16/fp16 dtypes; plus
    apply_mtp_structure_compressed and generate_pre_idx_v4 helpers
    that honor the cr=4 compressed-index space semantics.

Perf evidence (8x B300 SXM6 AC, indexer kernel-isolated microbench,
numColumns=16642 = §4 production [Scheme X] gate hit, K and pre_idx
hit-rate aligned with V4 HF config & test convention):

  Flash (index_topk=512, success_ratio=0.6):
    bf16:  R/H 1.55-1.67x over nR ∈ {4,8,12,16}
    fp32:  R/H 2.01-2.08x

  Pro (index_topk=1024, success_ratio=0.6):
    bf16:  R/H 1.40-1.57x
    fp32:  R/H 2.02-2.17x

  Attribution (sr=0 pessimistic baseline; sr = prev-step pre_idx hit
  rate, 0.6 matches generate_pre_idx_v4 test convention):
    K axis  (K=2048 → 512/1024 @ sr=0):  +0.27x bf16 / +0.42x fp32 (dominant)
    sr axis (sr=0 → 0.6 @ V4 K):         +0.04 to +0.17x (bf16 K=512 strongest)

NCU SOL/occupancy at nR=16 bf16 (legacy K=2048 capture; SOL/occupancy
conclusions apply equally at V4 K): both kernels latency-bound
(Compute SOL 2.7-3.8%, Achieved Occupancy ~25%, 0.05 full waves on
148 SMs). Heuristic wins by doing less work per row (pre_idx hints
skip the Radix histogram), not by better HW utilization.

End-to-end trtllm-bench (Flash, ISL=65536 BS=4 MTP=3, autotuner-off
greedy-seeded "clean" conditions): Heuristic +6.4% req/s vs Radix.
The earlier autotuner-on result (-4.4%) was fully attributed to
autotuner tactic interaction + sampling-induced DAR drift.

Accuracy: Flash GSM8K 96.85 (matches prior baseline; Heuristic is an
exact algorithm, no precision loss).

Full report and reproduction scripts:
perf_logs/long_isl/REPORT.md (NCU report, V4-faithful microbench JSONs
under microbench_v4_K/, YAML configs, bench scripts all archived).

Made-with: Claude Code (Opus 4.7, 1M context)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>
@longcheng-nv

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #48757 [ run ] triggered by Bot. Commit: 4a84ea5 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #48759 [ run ] triggered by Bot. Commit: 4a84ea5 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #48757 [ run ] completed with state ABORTED. Commit: 4a84ea5

Link to invocation

@longcheng-nv

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #48767 [ run ] triggered by Bot. Commit: 4a84ea5 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #48759 [ run ] completed with state ABORTED. Commit: 4a84ea5

Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #48767 [ run ] completed with state SUCCESS. Commit: 4a84ea5
/LLM/main/L0_MergeRequest_PR pipeline #38532 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@longcheng-nv

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #48776 [ run ] triggered by Bot. Commit: 4a84ea5 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #48776 [ run ] completed with state SUCCESS. Commit: 4a84ea5
/LLM/main/L0_MergeRequest_PR pipeline #38541 completed with status: 'SUCCESS'

CI Report

Link to invocation

@HuiGao-NV HuiGao-NV left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Adds a one-shot write-back at the end of the prefill block that seeds
`metadata.heuristic_prev_topk` with each finishing-prefill sequence's
last-context-token top-K, before the first decode step of that sequence
reads it as `preIdx`. Mirrors the design intent of the original GVR /
DeepSeek-V3.2 reference: the previous step's top-K is the warm-start
for the next step's secant, and there is no good reason for "previous"
to skip the prefill→decode boundary specifically.

Without this, decode step 0 of every sequence runs from the all-zero
init left behind by `heuristic_prev_topk.zero_()`, which after the
kernel's +1 temporal offset resolves to all-index-1 — a benign but
uninformative seed that forces full P2 secant bisection from scratch.
Captured real-loop data on DSv4-Flash (1 prompt × BS=1 × K=512 ×
TP=8 EP=8 MTP=0) shows the prefill last-row top-K shares
~60-75 % of indices with the eventual decode step-0 top-K per layer
(layers 20 / 22 / 40 / 42 measured); this is comparable to the
steady-state preIdx ∩ topk hit-rate on subsequent decode steps
(0.61-0.84 mean per layer), so the bridge gives step 0 a warm start
indistinguishable in quality from a "normal" mid-decode step.

Slot convention mirrors the existing decode write-back at the bottom
of the decode block: newly-promoted gens append after currently-active
gens, i.e., `[num_generations : num_generations + num_contexts]`.

Gated on:
  * `self._enable_heuristic_topk` (no-op if GVR disabled)
  * `has_prefill` (no-op for pure-decode iters)
  * `not metadata.skip_indexer_for_ctx_reqs` (no-op when the indexer
    is being short-circuited for ctx requests)

Numerical safety: same dtype, same buffer storage, same compressed-
index space as the existing decode write-back. No new collectives, no
new graph nodes for CUDA Graph compatibility.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Made-with: Claude Code
Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>
@longcheng-nv
longcheng-nv requested a review from a team as a code owner May 18, 2026 09:05
@longcheng-nv
longcheng-nv requested review from QiJune and removed request for a team May 18, 2026 09:05
@longcheng-nv

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #48887 [ run ] triggered by Bot. Commit: b6291ef Link to invocation

@mingyangHao mingyangHao left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Apply the exact reformatting that pre-commit/yapf produced on
b6291ef in CI, so the GitHub Actions Pre-commit Check passes.

Made-with: Claude Code (Opus 4.7)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>
@longcheng-nv

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #48907 [ run ] triggered by Bot. Commit: 4199524 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #48887 [ run ] completed with state ABORTED. Commit: b6291ef

Link to invocation

@lfr-0531

Copy link
Copy Markdown
Collaborator

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #48933 [ run ] triggered by Bot. Commit: 4199524 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #48907 [ run ] completed with state ABORTED. Commit: 4199524

Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #48933 [ run ] completed with state FAILURE. Commit: 4199524
/LLM/main/L0_MergeRequest_PR pipeline #38679 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@longcheng-nv

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #49030 [ run ] triggered by Bot. Commit: 4199524 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #49030 [ run ] completed with state SUCCESS. Commit: 4199524
/LLM/main/L0_MergeRequest_PR pipeline #38767 completed with status: 'SUCCESS'

CI Report

Link to invocation

@lfr-0531
lfr-0531 merged commit bef645d into NVIDIA:feat/deepseek_v4 May 19, 2026
6 checks passed
limin2021 added a commit to limin2021/TensorRT-LLM that referenced this pull request May 28, 2026
Mirrors heuristicTopKDecode.cu PR NVIDIA#14219 cr-aware branch in the DSL
GVR Top-K kernel. compress_ratio=1 (default) preserves DSv3.2 behavior
exactly; compress_ratio=4 enables the DSv4 (overlap-compressor) indexer
path:
  * pre_idx_offset = 0 (vs (row % next_n) + 1 for cr=1) — in compressed-
    index space, new entries append at the end so prev-step indices
    remain valid as-is.
  * N = actual_kv_len / cr — logits/preIdx live in compressed-token-
    index space when cr > 1.

GvrParams TABLE is also keyed by (dtype, K, cr) so V3.2 and V4 use
their respectively tuned kFTarget values:
  cr=1 (V3.2): kFTarget = 384 (K=512) / 2560 (K=1024), pre-NVIDIA#14413.
  cr=4 (V4):   kFTarget = kK   = 512 (K=512) / 1024 (K=1024), PR NVIDIA#14413.
  K=2048: identical across cr (V4 doesn't natively use K=2048).

Cache key includes compress_ratio so different cr settings compile
separate kernels. assert restricts compress_ratio in {1, 4}.

Verified: 1152/1152 pytest configs pass on cr=1 default path.
Signed-off-by: Mindy Li <11663212+limin2021@users.noreply.github.com>
lfr-0531 pushed a commit to lfr-0531/TensorRT-LLM that referenced this pull request May 29, 2026
…VIDIA#14219)

Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
(cherry picked from commit bef645d)
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
lfr-0531 pushed a commit to lfr-0531/TensorRT-LLM that referenced this pull request Jun 1, 2026
…VIDIA#14219)

Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
(cherry picked from commit bef645d)
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
lfr-0531 pushed a commit to lfr-0531/TensorRT-LLM that referenced this pull request Jun 3, 2026
…VIDIA#14219)

Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
(cherry picked from commit bef645d)
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
lfr-0531 pushed a commit to lfr-0531/TensorRT-LLM that referenced this pull request Jun 7, 2026
…VIDIA#14219)

Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
(cherry picked from commit bef645d)
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
lfr-0531 pushed a commit to lfr-0531/TensorRT-LLM that referenced this pull request Jun 10, 2026
…VIDIA#14219)

Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
lfr-0531 pushed a commit to lfr-0531/TensorRT-LLM that referenced this pull request Jun 11, 2026
…VIDIA#14219)

Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
lfr-0531 pushed a commit to lfr-0531/TensorRT-LLM that referenced this pull request Jun 12, 2026
…VIDIA#14219)

Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
lfr-0531 pushed a commit to lfr-0531/TensorRT-LLM that referenced this pull request Jun 12, 2026
…VIDIA#14219)

Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
(cherry picked from commit bef645d)
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
lfr-0531 pushed a commit to lfr-0531/TensorRT-LLM that referenced this pull request Jun 13, 2026
…VIDIA#14219)

Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
(cherry picked from commit bef645d)
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
lfr-0531 pushed a commit to lfr-0531/TensorRT-LLM that referenced this pull request Jun 16, 2026
…VIDIA#14219)

Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
(cherry picked from commit bef645d)
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
lfr-0531 pushed a commit to lfr-0531/TensorRT-LLM that referenced this pull request Jun 17, 2026
…VIDIA#14219)

Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
(cherry picked from commit bef645d)
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants