Faster engine compile time - #316

Kh4ster · 2025-08-18T15:07:38Z

This PR aims at making the compile time faster by adding several options:

build lp only (extracted MIP files necessary in LP), also meant we needed a skip c and python adapter option
put back no fetch rapids
put back not building test option
added skip routing
added skip write of fatbin
adding ccache to libcuopt target
added two benchmarking targets, one for LP and for MIP

ramakrishnap-nv

Thank you @Kh4ster for the PR, I have requested few changes and clarification.

ramakrishnap-nv · 2025-08-18T15:14:10Z

+SKIP_C_PYTHON_ADAPTERS=0
+SKIP_TESTS_BUILD=0
+SKIP_ROUTING_BUILD=0
+WRITE_FATBIN=0


WRITE FATBIN should be by default enabled, and disabled with option

ramakrishnap-nv · 2025-08-18T15:14:54Z

 option(BUILD_TESTS "Configure CMake to build tests" ON)
 option(DISABLE_OPENMP "Disable OpenMP" OFF)
 option(CUDA_STATIC_RUNTIME "Statically link the CUDA toolkit runtime and libraries" OFF)
+option(BUILD_LP_ONLY "Build only linear programming components, exclude routing and MIP-specific files" OFF)


Missing BUILD_TESTS
WRITE_FATBIN and BUILD_FATBIN is being used.

Build test is above. We had it already but it was not used and not working in practice

Good catch for the BUILD_FATBIN!

ramakrishnap-nv · 2025-08-18T15:15:08Z

 BUILD_CI_ONLY=0
+BUILD_LP_ONLY=0
+SKIP_C_PYTHON_ADAPTERS=0
+SKIP_TESTS_BUILD=0


Dow we need to add option to skip BUILD_MIP_BENCHMARKS

I think we can keep this for now and see if someone needs it

ramakrishnap-nv · 2025-08-18T15:16:29Z

-  PRIVATE "$<$<COMPILE_LANGUAGE:CXX>:${CUOPT_CXX_FLAGS}>"
-  "$<$<COMPILE_LANGUAGE:CUDA>:${CUOPT_CUDA_FLAGS}>"
-)
+if(NOT BUILD_LP_ONLY)


Should this be

Suggested change

if(NOT BUILD_LP_ONLY)

if(BUILD_LP_ONLY)

No I confirm we don't want to have cuopt_cli if we are in lp_only mode since there is a call to the MIP solver

ramakrishnap-nv · 2025-08-18T15:17:31Z

-add_library(cuopttestutils STATIC
-    routing/utilities/check_constraints.cu
-)
+if(BUILD_TESTS)


Is there an option where we can skip directly from cpp/CMakefiles.txt

I don't know any. That's the only way I found to truely diable the compilation of tests

rg20 · 2025-08-18T17:54:02Z

-}
-]=])
-target_link_options(mps_parser PRIVATE "${MPS_PARSER_BINARY_DIR}/fatbin.ld")
+if(WRITE_FATBIN)


Remove this always?

What do you mean? Remove the fatbin writing completly?

I think he is asking, by default currently this was disabled, so do we need to disable it by default might be the question. Think I also had asked this question.

And you have changed it if I am not wrong.

copy-pr-bot · 2025-08-19T08:53:27Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Kh4ster · 2025-08-19T08:56:15Z

/ok to test 0e3191a

Kh4ster · 2025-08-19T09:13:01Z

/ok to test 3a9bac7

Kh4ster · 2025-08-19T09:37:27Z

/ok to test 7bc2ce4

Kh4ster · 2025-08-20T08:56:46Z

/ok to test 2ad2c73

copy-pr-bot · 2025-08-20T08:56:49Z

/ok to test 2ad2c73

@Kh4ster, there was an error processing your request: E2

See the following link for more information: https://docs.gha-runners.nvidia.com/cpr/e/2/

Kh4ster · 2025-08-20T09:01:20Z

/ok to test 33e9344

Kh4ster · 2025-08-20T11:29:58Z

/ok to test 3a3e7c9

rg20 · 2025-08-25T12:53:27Z

    instance_name=$(basename $instance)
    echo "Parsing ${instance_name}.mps then solving"
-    ${CUOPT_HOME}/cpp/build/solve_MPS_file --path ${CUOPT_HOME}/benchmarks/linear_programming/datasets/${instance_name}/${instance_name}.mps --time-limit 3600 # --solution-path $CUOPT_HOME/benchmarks/linear_programming/datasets/$instance.sol
+    ${CUOPT_HOME}/cpp/build/solve_LP --path ${CUOPT_HOME}/benchmarks/linear_programming/datasets/${instance_name}/${instance_name}.mps --time-limit 3600 # --solution-path $CUOPT_HOME/benchmarks/linear_programming/datasets/$instance.sol


Let's start using cuopt_cli here? That's what Hans would be using as well.

same comment as below

Here, you are not using any of those additional settings/options from run_pdlp right? We can still use run_pdlp for when benchmarking with those settings.

rg20 · 2025-08-25T12:56:58Z

-  target_compile_options(solve_MPS_file
+option(BUILD_LP_BENCHMARKS "Build LP benchmarks" OFF)
+if(BUILD_LP_BENCHMARKS)
+  add_executable(solve_LP ../benchmarks/linear_programming/cuopt/run_pdlp.cu)


For benchmarking, I think we should directly use the cuopt_cli, because that's what everyone will use

The issue I have is run_pdlp has more options that cuopt_cli

Kh4ster · 2025-09-01T07:43:03Z

/ok to test 3a3e7c9

ramakrishnap-nv · 2025-09-01T19:12:32Z

/ok to test 50e56b8

Kh4ster · 2025-09-03T07:50:22Z

/ok to test c241602

aliceb-nv

Approving :) Awesome work Nicolas, this is going to be super useful
Is it ready to merge?

rg20 · 2025-09-03T14:00:43Z

/merge

rg20 · 2025-09-03T14:14:16Z

/merge

This PR aims at making the compile time faster by adding several options: - build lp only (extracted MIP files necessary in LP), also meant we needed a skip c and python adapter option - put back no fetch rapids - put back not building test option - added skip routing - added skip write of fatbin - adding ccache to libcuopt target - added two benchmarking targets, one for LP and for MIP Authors: - Nicolas Blin (https://github.com/Kh4ster) Approvers: - Alice Boucher (https://github.com/aliceb-nv) - Rajesh Gandham (https://github.com/rg20) - Ramakrishnap (https://github.com/rgsl888prabhu) URL: #316

This PR aims at making the compile time faster by adding several options: - build lp only (extracted MIP files necessary in LP), also meant we needed a skip c and python adapter option - put back no fetch rapids - put back not building test option - added skip routing - added skip write of fatbin - adding ccache to libcuopt target - added two benchmarking targets, one for LP and for MIP Authors: - Nicolas Blin (https://github.com/Kh4ster) Approvers: - Alice Boucher (https://github.com/aliceb-nv) - Rajesh Gandham (https://github.com/rg20) - Ramakrishnap (https://github.com/rgsl888prabhu) URL: NVIDIA#316

several improvements to make the compile time faster

fa32ca4

Kh4ster requested review from a team as code owners August 18, 2025 15:07

Kh4ster requested review from akifcorduk, jakirkham, ramakrishnap-nv and rg20 and removed request for jakirkham August 18, 2025 15:07

Kh4ster added non-breaking Introduces a non-breaking change improvement Improves an existing functionality labels Aug 18, 2025

anandhkb added this to the 25.10 milestone Aug 18, 2025

ramakrishnap-nv reviewed Aug 18, 2025

View reviewed changes

rg20 reviewed Aug 18, 2025

View reviewed changes

changed fatbin write to be on by default

4baece1

Merge branch 'branch-25.10' into faster-build

0e3191a

disable lp benchmark build by default

3a9bac7

put back correct link library and update lp benchmark script

7bc2ce4

Kh4ster and others added 3 commits August 20, 2025 10:43

Merge branch 'branch-25.10' into faster-build

2ad2c73

update names in argparse

ec33c0f

disable mip benchmark binary building by default

33e9344

akifcorduk reviewed Aug 20, 2025

View reviewed changes

Comment thread cpp/tests/CMakeLists.txt

skip correct tests accoring to flags

3a3e7c9

rg20 reviewed Aug 25, 2025

View reviewed changes

Merge branch 'branch-25.10' into faster-build

50e56b8

Merge branch 'branch-25.10' into faster-build

c241602

aliceb-nv approved these changes Sep 3, 2025

View reviewed changes

rg20 approved these changes Sep 3, 2025

View reviewed changes

ramakrishnap-nv approved these changes Sep 3, 2025

View reviewed changes

rapids-bot Bot merged commit eac3d69 into branch-25.10 Sep 3, 2025
73 checks passed

Kh4ster deleted the faster-build branch October 22, 2025 07:59

Uh oh!

Conversation

Kh4ster commented Aug 18, 2025

Uh oh!

ramakrishnap-nv left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

copy-pr-bot Bot commented Aug 19, 2025

Uh oh!

Kh4ster commented Aug 19, 2025

Uh oh!

Kh4ster commented Aug 19, 2025

Uh oh!

Kh4ster commented Aug 19, 2025

Uh oh!

Kh4ster commented Aug 20, 2025

Uh oh!

copy-pr-bot Bot commented Aug 20, 2025

Uh oh!

Kh4ster commented Aug 20, 2025

Uh oh!

Uh oh!

Kh4ster commented Aug 20, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Kh4ster commented Sep 1, 2025

Uh oh!

ramakrishnap-nv commented Sep 1, 2025

Uh oh!

Kh4ster commented Sep 3, 2025

Uh oh!

aliceb-nv left a comment

Choose a reason for hiding this comment

Uh oh!

rg20 commented Sep 3, 2025

Uh oh!

rg20 commented Sep 3, 2025

Uh oh!

Uh oh!

Reviewers

Assignees