Add a busy list to CUDAStreamPool. Don't return busy streams from the pool. #6072
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Collaborator
|
CI MESSAGE: [36983125]: BUILD STARTED |
JanuszL
reviewed
Oct 21, 2025
| @@ -1,4 +1,4 @@ | |||
| // Copyright (c) 2021-2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved. | |||
| // Copyright (c) 2021-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved. | |||
Contributor
There was a problem hiding this comment.
Is there any additional testing needed or the one which exists is sufficient?
Contributor
Author
There was a problem hiding this comment.
Well, theoretically I could add a test that schedules some work on a stream, returns it, picks a new stream and verifies that it's not busy.
JanuszL
approved these changes
Oct 21, 2025
Collaborator
|
CI MESSAGE: [36983125]: BUILD FAILED |
Collaborator
|
CI MESSAGE: [36983125]: BUILD PASSED |
mzient
force-pushed
the
stream_pool_with_busy_list
branch
from
d5c23c4 to
5f97acc
Compare
Collaborator
|
CI MESSAGE: [37043200]: BUILD STARTED |
JanuszL
reviewed
Oct 22, 2025
JanuszL
approved these changes
Oct 22, 2025
Collaborator
|
CI MESSAGE: [37046256]: BUILD STARTED |
JanuszL
approved these changes
Oct 22, 2025
JanuszL
approved these changes
Oct 22, 2025
rostan-t
approved these changes
Oct 22, 2025
Collaborator
|
CI MESSAGE: [37043200]: BUILD PASSED |
Collaborator
|
CI MESSAGE: [37046256]: BUILD PASSED |
…ead pool. Signed-off-by: Michal Zientkiewicz <[email protected]>
Signed-off-by: Michał Zientkiewicz <[email protected]>
Signed-off-by: Michał Zientkiewicz <[email protected]>
mzient
force-pushed
the
stream_pool_with_busy_list
branch
from
a90c08d to
52179fe
Compare
Collaborator
|
CI MESSAGE: [37060756]: BUILD STARTED |
Collaborator
|
CI MESSAGE: [37060756]: BUILD PASSED |
mdabek-nvidia
pushed a commit
to mdabek-nvidia/DALI
that referenced
this pull request
Nov 27, 2025
… pool. (NVIDIA#6072) * Add a busy list to CUDAStreamPool. Don't return busy streams from thread pool. * Extend stream pool tests. --------- Signed-off-by: Michal Zientkiewicz <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Category:
New feature (non-breaking change which adds functionality)
Description:
Prior to this change, the CUDAStreamPool used a per-device stream stack. This caused the most recently returned stream to be used, increasing the likelihood of getting a stream with pending work scheduled. This PR adds another "busy" list to which the streams are initially returned if they're not ready. Then, if the ready list is empty, the busy list will be traversed in search of ready streams.
Additional information:
Affected modules and functionalities:
Key points relevant for the review:
Tests:
Checklist
Documentation
DALI team only
Requirements
REQ IDs: N/A
JIRA TASK: N/A