GH-115060: Speed up pathlib.Path.glob() by not scanning literal parts - #117732
Merged
Conversation
…al parts Don't bother calling `os.scandir()` to scan for literal pattern segments, like `foo` in `foo/*.py`. Instead, append the segment(s) as-is and call through to the next selector with `exists=False`, which signals that the path might not exist. Subsequent selectors will call `os.scandir()` or `os.lstat()` to filter out missing paths as needed.
diegorusso
pushed a commit
to diegorusso/cpython
that referenced
this pull request
Apr 17, 2024
…al parts (python#117732) Don't bother calling `os.scandir()` to scan for literal pattern segments, like `foo` in `foo/*.py`. Instead, append the segment(s) as-is and call through to the next selector with `exists=False`, which signals that the path might not exist. Subsequent selectors will call `os.scandir()` or `os.lstat()` to filter out missing paths as needed.
SirVer
added a commit
to SirVer/ultisnips
that referenced
this pull request
Sep 6, 2026
`find_all_snippet_directories()` expands the wildcards Vim allows in `'runtimepath'` entries. Since the pathlib migration (c84bb5a) it did so with `Path(pth.anchor).glob(str(pth.relative_to(pth.anchor)))`, i.e. it globbed the whole path starting at `/`. Python 3.12's pathlib has no fast path for literal pattern components: every component goes through _WildcardSelector`, which `scandir`s the parent directory and calls `is_dir()` on each entry. With the ~100 entries of a plugin-heavy runtimepath that lists `/`, `/home`, `/home/user`, ... once per entry (595 directory listings and 5763 stats in the report), and on WSL with symlinked network folders the first snippet expansion froze Vim for seconds. Neither 3.11 nor 3.13 have this issue. 3.11 pathlib resolved literal pattern segments through a single `stat`. Python 3.12 removed that path in python/cpython#102710. Python 3.13 brought a literal fast path back in the glob rewrite (python/cpython#117732). We work around this by finding globbing in the snippet paths and only start globbing from the segment that has globbing characters. While strictly only needed for 3.12 it does not hurt on other Pythons. Fixes #1694.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Don't bother calling
os.scandir()to scan for literal pattern segments, likefooinfoo/*.py. Instead, append the segment(s) as-is and call through to the next selector withexists=False, which signals that the path might not exist. Subsequent selectors will callos.scandir()oros.lstat()to filter out missing paths as needed.Timings:
pathlib.Path.glob()by removing redundant regex matching #115060