[BUG] Running with barrier from python api sometimes gets segfault

[ramakrishnap-nv]

Describe the bug

When running LP with barrier enabled, segfault intermittently when only run from python API, and we are expecting this could be related to cudss or cusparse in a race condition with python.

Steps/Code to reproduce bug
Attached a test case to reproduce this, this only happens when run in l4 without debug symbols and with restricted to 0-8 cpu threads

As part of this few tests have been disabled to keep CI running smoothly. Please search with this bug number to find those.

Expected behavior
There shouldn't be any segfault

Environment details (please complete the following information):

• In CI container for wheel testing

test_lp_barrier_python_api.py

[anandhkb]

@rgsl888prabhu Can we close this issue? I believe you have a PR open to re-enable these tests, which were earlier switched off, correct? Thanks.

[ramakrishnap-nv]

No, this is still happening

[anandhkb]

Challenge is in repro.

Risk mitigation: @tmckayus please take a stab at a repro

[tmckayus]

@rgsl888prabhu are you still seeing this?

Here is a script that will build a container based on the most recent nightly (cuda 13, py13) and then run the barrier test 100 times. Iterations and CPU threads can be passed as arguments but default to 100 iterations and 3 threads.

I ran this on an L4 multiple times and cannot get it to fail.

Just unzip and

$ ./run_barrier_test_loop.sh

lp_barrier_test.zip

[tmckayus]

@rgsl888prabhu tested again separately with a CI container and was unable to reproduce. Assuming this has been fixed by recent changes, closing.