Skip to content

Support python 3.13 with updated dependencies - #4430

Merged
AyanSinhaMahapatra merged 84 commits into
developfrom
test-pyahocorasick
Jun 26, 2025
Merged

Support python 3.13 with updated dependencies#4430
AyanSinhaMahapatra merged 84 commits into
developfrom
test-pyahocorasick

Conversation

@AyanSinhaMahapatra

@AyanSinhaMahapatra AyanSinhaMahapatra commented Jun 17, 2025 •
edited
Loading

Copy link
Copy Markdown
Member

Tasks

  • Reviewed contribution guidelines
  • PR is descriptively titled 📑 and links the original issue above 🔗
  • Tests pass -- look for a green checkbox ✔️ a few minutes after opening your PR
    Run tests locally to check for errors.
  • Commits are in uniquely-named feature branch and has no merge conflicts 📁
  • Updated documentation pages (if applicable)
  • Updated CHANGELOG.rst (if applicable)

JonoYang and others added 30 commits July 17, 2023 13:28
@JonoYang
Create script to update repo skeleton #80
ea5044d 
Signed-off-by: Jono Yang <jyang@nexb.com>
@JonoYang
Replace black and isort with ruff
320ec21 
    * Use ruff config and Make commands from scancode.io

Signed-off-by: Jono Yang <jyang@nexb.com>
@pombredanne
Use org standard 100 line length
d4e29c3 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Lint all common code directories
6c028f7 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Remove unused targets
233f3ed 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Improve import sorting
55545bf 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Apply small code updates
0b63e50 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Format code
092f545 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Apply cosmetic refactorings
d05665a 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Reformat test code
63bcbf5 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Format code
9d1393a 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Refine ruff configuration
f10b783 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Format doc
1d6c8f3 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Run doc8 on all rst files
0213c1e 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Enable doc style checks
c112f2a 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Add support for new OS versions
944b6c5 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Merge pull request #94 from aboutcode-org/new-osses
a2ced96 
Add support for new OS versions
@pombredanne
Update scripts aboutcode references
136af39 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Do not format more test data
da8eff0 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Do not treat rst as Python
4f9e936 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Combine testing and docs extra for simplicity
a2809fb 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Refine checking of docs with doc8
43b96c2 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Refine doc handling
b7194c8 
* remove CI scripts and use Makefile targets instead
* ensure doc8 runs quiet
* add new docs-check make target to run documentation and links checks
* update oudated doc for docs contribution

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Add twine check to release publication
a5bcdbd 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Refine doc contribution docs
a6c25fb 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@JonoYang
Merge pull request #95 from aboutcode-org/80-update-skeleton-script
dc123f1 
Update skeleton script
@JonoYang
Replace black and isort with ruff
68daae1 
    * Use ruff config and Make commands from scancode.io

Signed-off-by: Jono Yang <jyang@nexb.com>
@pombredanne @JonoYang
Use org standard 100 line length
6a8c9ae 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne @JonoYang
Lint all common code directories
2fd31d5 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne @JonoYang
Remove unused targets
eb5fc82 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@AyanSinhaMahapatra
Merge branch 'develop' into test-pyahocorasick
0d5772c
@AyanSinhaMahapatra
Update dependency requirements
4049083 
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra
Rollback dependencies based on detection issues
97bc9b5 
Reference: #4447
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra
Update deprecated windows action runner 2019->2025
a4e6e11 
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra
Update CI runners and scripts
af87cfa 
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra
Merge pull request #97 from aboutcode-org/update-ci-and-runners
c1002c2 
Update CI runners and scripts
@AyanSinhaMahapatra
Merge remote-tracking branch 'skeleton/main' into test-pyahocorasick
d03a558 
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
github-advanced-security[bot]
github-advanced-security AI found potential problems Jun 24, 2025
View reviewed changes
Comment thread etc/scripts/utils_requirements.py
"--all", "--path", site_packages_dir]
return subprocess.check_output(args, encoding="utf-8")
args = ["pip", "freeze", "--exclude-editable", "--all", "--path", site_packages_dir]
return subprocess.check_output(args, encoding="utf-8") # noqa: S603

Check failure

Code scanning / CodeQL

Uncontrolled command line

This command line depends on a [user-provided value](1). This command line depends on a [user-provided value](2).

Copilot Autofix

AI about 1 year ago

To fix the issue, we need to validate and sanitize the site_packages_dir input before using it in the subprocess.check_output() call. The best approach is to ensure that the input is a valid directory path and restrict it to a predefined allowlist or sanitize it to prevent command injection.

Steps to fix:

  1. Add a validation function to check that site_packages_dir is a valid directory path and does not contain any malicious characters or patterns.
  2. Use shlex.quote() to safely escape the site_packages_dir value before including it in the args list for the subprocess call.
  3. Update the get_installed_reqs function in utils_requirements.py to incorporate these changes.
Suggested changeset 1
etc/scripts/utils_requirements.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/etc/scripts/utils_requirements.py b/etc/scripts/utils_requirements.py
--- a/etc/scripts/utils_requirements.py
+++ b/etc/scripts/utils_requirements.py
@@ -115,9 +115,14 @@
     Return the installed pip requirements as text found in `site_packages_dir`
-    as a text.
+    as a text. Validates and sanitizes the input to prevent command injection.
     """
-    if not os.path.exists(site_packages_dir):
-        raise Exception(f"site_packages directory: {site_packages_dir!r} does not exists")
+    if not os.path.exists(site_packages_dir) or not os.path.isdir(site_packages_dir):
+        raise Exception(f"site_packages directory: {site_packages_dir!r} does not exist or is not a directory")
+    
+    # Sanitize the input to prevent command injection
+    import shlex
+    sanitized_dir = shlex.quote(str(site_packages_dir))
+    
     # Also include these packages in the output with --all: wheel, distribute,
     # setuptools, pip
-    args = ["pip", "freeze", "--exclude-editable", "--all", "--path", site_packages_dir]
+    args = ["pip", "freeze", "--exclude-editable", "--all", "--path", sanitized_dir]
     return subprocess.check_output(args, encoding="utf-8")  # noqa: S603
EOF
@@ -115,9 +115,14 @@
Return the installed pip requirements as text found in `site_packages_dir`
as a text.
as a text. Validates and sanitizes the input to prevent command injection.
"""
if not os.path.exists(site_packages_dir):
raise Exception(f"site_packages directory: {site_packages_dir!r} does not exists")
if not os.path.exists(site_packages_dir) or not os.path.isdir(site_packages_dir):
raise Exception(f"site_packages directory: {site_packages_dir!r} does not exist or is not a directory")

# Sanitize the input to prevent command injection
import shlex
sanitized_dir = shlex.quote(str(site_packages_dir))

# Also include these packages in the output with --all: wheel, distribute,
# setuptools, pip
args = ["pip", "freeze", "--exclude-editable", "--all", "--path", site_packages_dir]
args = ["pip", "freeze", "--exclude-editable", "--all", "--path", sanitized_dir]
return subprocess.check_output(args, encoding="utf-8") # noqa: S603
Copilot is powered by AI and may make mistakes. Always verify output.
@AyanSinhaMahapatra
AyanSinhaMahapatra force-pushed the test-pyahocorasick branch 8 times, most recently from 9789e0c to 32421cb Compare June 25, 2025 13:20
@AyanSinhaMahapatra
Update scancode release script for py3.13
600759d 
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra

AyanSinhaMahapatra commented Jun 25, 2025

Copy link
Copy Markdown
Member Author

All green, release tests also passing at https://github.com/AyanSinhaMahapatra/scancode-toolkit/actions/runs/15879003647

@AyanSinhaMahapatra
Fix release smoke tests on posix
be51a43 
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
JonoYang
JonoYang approved these changes Jun 25, 2025
View reviewed changes

@JonoYang JonoYang left a comment •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! I am able to run scancode and the tests using Python 3.13 on my machine, there's just a merge issue with the contrib doc.

@AyanSinhaMahapatra AyanSinhaMahapatra mentioned this pull request Jun 26, 2025
Closed
@AyanSinhaMahapatra
Bump scancode version to v32.4.0
2a7a0b9 
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra
Merge branch 'develop' into test-pyahocorasick
03cf4b5 
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra
Bump scancode version to v32.4.0 in setup.cfg
0fe7fc8 
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra

AyanSinhaMahapatra commented Jun 26, 2025

Copy link
Copy Markdown
Member Author

Merging, test failures seem to be heisenbugs, release tests passing https://github.com/AyanSinhaMahapatra/scancode-toolkit/actions/runs/15902292932

@AyanSinhaMahapatra
AyanSinhaMahapatra merged commit c0ce98b into develop Jun 26, 2025
@AyanSinhaMahapatra
AyanSinhaMahapatra deleted the test-pyahocorasick branch December 3, 2025 16:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@JonoYang JonoYang JonoYang approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@AyanSinhaMahapatra @JonoYang @github-advanced-security @pombredanne