[node-modules] Fixes race between mkdirp and remove#1108
Merged
Conversation
…ng all `remove` first.
…uring all the operations
larixer
force-pushed
the
nm-fix-mkdirp-race
branch
from
a6358a5 to
b78ca4d
Compare
Contributor
|
@larixer I saw that you pinged me but I don't remember where the race condition came up 😅 |
Member
Author
|
@nicolo-ribaudo I have pinged you, to keep you in the loop, that the problem has been spotted, that might affect you. |
Contributor
|
Oh ok, thank you! Btw, I haven't found any recent new issue with |
arcanis
reviewed
Mar 23, 2020
Comment on lines
+690
to
+694
| deleteList.add(curLocation); | ||
| // If previous install had inner node_modules folder, we should explicitely list it for | ||
| // `removeDir` to delete it, but we need to delete it first, so we add it to inner delete list | ||
| if (prevNode && prevNode.children.has(NODE_MODULES)) | ||
| innerDeleteList.add(ppath.join(curLocation, NODE_MODULES)); |
Member
There was a problem hiding this comment.
Wouldn't it be enough to process the delete list in the reverse insertion order?
Member
Author
There was a problem hiding this comment.
No, since we run all delete operations in parallel
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What's the problem this PR addresses?
We have observed one case of a race condition in
node-moduleslinker betweenmkdirpandremoveoperations, during persisting of packages intonode_modulesThe error message looks like:
How did you fix it?
I have tweaked the order of operations, so that all
remove's were executed first and only after they had been finished all the other file operations were performed.Another change in this PR is simplification of the code for inner
node_modulesdirectories handling for the cases likenode_modules/foo/node_modules/bar. In all theaddModule,deleteModule,copyDir,removeDiroperations that targetnode_modules/foowe now do not remove/copynode_modules/foo/node_modulesand to actually deletenode_modules/foo/node_modules, theremoveDir('node_modules/foo/node_modules')should be called. All the innernode_modulesdeletions are performed at the very beginning of linking before all other copying/cloning operations. This way the code is seriously simplified.cc @nicolo-ribaudo