Skip to content

test(c_api): skip flaky TimeLimitTestFixture MIP case (#1135) - #1136

Merged
ramakrishnap-nv merged 2 commits into
NVIDIA:mainfrom
ramakrishnap-nv:fix/disable-flaky-time-limit-mip-1135
Apr 22, 2026
Merged

test(c_api): skip flaky TimeLimitTestFixture MIP case (#1135)#1136
ramakrishnap-nv merged 2 commits into
NVIDIA:mainfrom
ramakrishnap-nv:fix/disable-flaky-time-limit-mip-1135

Conversation

@ramakrishnap-nv

Copy link
Copy Markdown
Collaborator

Summary

  • Skip the c_api/TimeLimitTestFixture.time_limit/2 parametric instance (/mip/supportcase22.mps, 15s, CUOPT_METHOD_DUAL_SIMPLEX) via GTEST_SKIP().
  • LP cases /0 and /1 remain enabled.

Why

The MIP case intermittently overshoots the 3s excess_allowed_time tolerance on CPU-thread-constrained CI runners. cuOptGetSolveTime() for MIP returns mip_solution_t::get_total_solve_time(), which is set in cpp/src/mip_heuristics/solver.cu:489 from a timer started in cpp/src/mip_heuristics/solve.cu:306before Papilo presolve. So solve_time includes Papilo presolve (OpenMP-parallel) and the post-B&B serial wind-down (branch_and_bound_status_future.get(), compute_feasibility, test_variable_bounds), neither of which is bounded by the user's time_limit setting.

On a 12-thread-visible CI runner, observed solve_time = 18.398s vs the 18s bar (15s + 3s tolerance). Solver itself respected the limit (Explored 0 nodes in 15.01s).

Full investigation and proposed fixes (measure inside the time-limited region only, or per-case tolerances) are in the issue.

Tracking: #1135.

Test plan

  • Pre-commit clean (clang-format, verify-copyright, etc.)
  • Local rebuild of C_API_TEST succeeds
  • Verified /2 is reported as [ SKIPPED ] Disabled pending NVIDIA/cuopt#1135; /0 and /1 are unchanged in behavior
  • CI green

The /2 parametric instance (supportcase22.mps, 15s, dual simplex)
intermittently overshoots the 3s tolerance on CPU-thread-constrained
CI runners because solve_time includes Papilo presolve and post-B&B
serial wind-down, neither bounded by the time_limit setting.

Skipping the MIP case via GTEST_SKIP while NVIDIA#1135 is
investigated. The LP /0 and /1 cases remain enabled.
@ramakrishnap-nv
ramakrishnap-nv requested a review from a team as a code owner April 22, 2026 18:27
@ramakrishnap-nv ramakrishnap-nv self-assigned this Apr 22, 2026
@ramakrishnap-nv ramakrishnap-nv added bug Something isn't working non-breaking Introduces a non-breaking change labels Apr 22, 2026
@ramakrishnap-nv ramakrishnap-nv added this to the 26.06 milestone Apr 22, 2026
@coderabbitai

coderabbitai Bot commented Apr 22, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: e619a227-7a3d-4278-82cc-f3f0371e16aa

📥 Commits

Reviewing files that changed from the base of the PR and between 16fb3fb and 548f611.

📒 Files selected for processing (1)
  • cpp/tests/linear_programming/c_api_tests/c_api_tests.cpp
🚧 Files skipped from review as they are similar to previous changes (1)
  • cpp/tests/linear_programming/c_api_tests/c_api_tests.cpp

📝 Walkthrough

Walkthrough

A parameterized test in TimeLimitTestFixture.time_limit now skips a single parameter instance by calling GTEST_SKIP() when the first GetParam() string equals "/mip/supportcase22.mps", leaving other parameter cases and solver logic unchanged.

Changes

Cohort / File(s) Summary
Test Skip Logic
cpp/tests/linear_programming/c_api_tests/c_api_tests.cpp
Added a conditional GTEST_SKIP() in TimeLimitTestFixture.time_limit to skip the specific parameter instance whose GetParam() equals "/mip/supportcase22.mps", bypassing subsequent solve and assertions for that case.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~7 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the main change: skipping a flaky MIP test case in the C API tests with a reference to the tracking issue.
Description check ✅ Passed The description provides clear context about why the test is being skipped, including the root cause, observed behavior on CI runners, and references to the tracking issue.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
cpp/tests/linear_programming/c_api_tests/c_api_tests.cpp (1)

54-56: Scope the skip to the single flaky parameter, not all MIP paths.

Line 54 currently skips any case with "/mip/" in the path. That’s broader than the PR intent (only /mip/supportcase22.mps with dual simplex) and could hide future MIP regressions if more params are added.

Proposed narrowing
-  if (filename.find("/mip/") != std::string::npos) {
+  const auto& relative_path = std::get<0>(GetParam());
+  if (relative_path == "/mip/supportcase22.mps" && method == CUOPT_METHOD_DUAL_SIMPLEX) {
     GTEST_SKIP() << "Disabled pending NVIDIA/cuopt#1135";
   }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@cpp/tests/linear_programming/c_api_tests/c_api_tests.cpp` around lines 54 -
56, The current skip checks any path containing "/mip/" via the filename
variable and is too broad; change it to only skip the specific flaky case by
checking filename for "supportcase22.mps" AND the test parameter that selects
the dual-simplex solver/method (e.g., inspect the test's solver/method parameter
or option that indicates "dual simplex" or DUAl_SIMPLEX) so only when
filename.find("supportcase22.mps") != std::string::npos && method==DUAL_SIMPLEX
(or parameter string contains "dual") you call GTEST_SKIP().
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@cpp/tests/linear_programming/c_api_tests/c_api_tests.cpp`:
- Around line 54-56: The current skip checks any path containing "/mip/" via the
filename variable and is too broad; change it to only skip the specific flaky
case by checking filename for "supportcase22.mps" AND the test parameter that
selects the dual-simplex solver/method (e.g., inspect the test's solver/method
parameter or option that indicates "dual simplex" or DUAl_SIMPLEX) so only when
filename.find("supportcase22.mps") != std::string::npos && method==DUAL_SIMPLEX
(or parameter string contains "dual") you call GTEST_SKIP().

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 21773cf7-f9e1-4ab9-920d-f5d4fc3a0fa1

📥 Commits

Reviewing files that changed from the base of the PR and between 0c002b6 and 16fb3fb.

📒 Files selected for processing (1)
  • cpp/tests/linear_programming/c_api_tests/c_api_tests.cpp

Comment thread cpp/tests/linear_programming/c_api_tests/c_api_tests.cpp Outdated
Substring "/mip/" suggested there might be other MIP cases in the
fixture; only supportcase22.mps is parametrized.

@chris-maes chris-maes left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ramakrishnap-nv

Copy link
Copy Markdown
Collaborator Author

/merge

@ramakrishnap-nv
ramakrishnap-nv merged commit 55fb9b9 into NVIDIA:main Apr 22, 2026
104 of 106 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working non-breaking Introduces a non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants