Skip to content

Use memchr to speedup newline search on x86 #3985

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
MichaReiser merged 3 commits into from
Apr 26, 2023
Merged

Use memchr to speedup newline search on x86 #3985

MichaReiser merged 3 commits into from
Apr 26, 2023

Conversation

@MichaReiser
Copy link
Member

@MichaReiser MichaReiser commented Apr 16, 2023 •
edited
Loading

Use memchr to find the newline characters in strings. I expect this to improve performance on X86 processors because memchr uses SIMD internally.

group                                      bytes                                  bytes-logical                          memchr                                 memchr-logical
-----                                      -----                                  -------------                          ------                                 --------------
linter/all-rules/large/dataset.py          1.02      8.6±0.08ms     4.7 MB/sec    1.06      8.9±0.17ms     4.6 MB/sec    1.00      8.4±0.08ms     4.9 MB/sec    1.04      8.7±0.13ms     4.7 MB/sec
linter/all-rules/numpy/ctypeslib.py        1.02      2.0±0.07ms     8.3 MB/sec    1.08      2.1±0.03ms     7.9 MB/sec    1.00  1957.1±37.12µs     8.5 MB/sec    1.04      2.0±0.04ms     8.2 MB/sec
linter/all-rules/numpy/globals.py          1.05    220.7±4.63µs    13.4 MB/sec    1.11    232.3±4.36µs    12.7 MB/sec    1.00    209.3±2.68µs    14.1 MB/sec    1.07    223.2±1.44µs    13.2 MB/sec
linter/all-rules/pydantic/types.py         1.00      3.5±0.04ms     7.3 MB/sec    1.08      3.8±0.03ms     6.7 MB/sec    1.00      3.5±0.01ms     7.3 MB/sec    1.04      3.7±0.10ms     7.0 MB/sec
linter/default-rules/large/dataset.py      1.03      4.3±0.07ms     9.5 MB/sec    1.11      4.6±0.08ms     8.8 MB/sec    1.00      4.2±0.07ms     9.7 MB/sec    1.09      4.5±0.03ms     9.0 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.00    870.1±5.69µs    19.1 MB/sec    1.10   956.6±10.45µs    17.4 MB/sec    1.00   866.6±19.79µs    19.2 MB/sec    1.12    972.7±2.62µs    17.1 MB/sec
linter/default-rules/numpy/globals.py      1.03     95.9±0.72µs    30.8 MB/sec    1.13    105.4±1.49µs    28.0 MB/sec    1.00     93.3±0.54µs    31.6 MB/sec    1.09    101.7±1.74µs    29.0 MB/sec
linter/default-rules/pydantic/types.py     1.01  1883.0±10.58µs    13.5 MB/sec    1.13      2.1±0.03ms    12.2 MB/sec    1.00  1855.8±20.37µs    13.7 MB/sec    1.12      2.1±0.00ms    12.3 MB/sec

@MichaReiser
Copy link
Member Author

MichaReiser commented Apr 16, 2023 •
edited
Loading

Current dependencies on/for this PR:

This comment was auto-generated by Graphite.

@MichaReiser MichaReiser mentioned this pull request Apr 16, 2023
Merged
6 tasks
@github-actions
Copy link
Contributor

github-actions bot commented Apr 16, 2023 •
edited
Loading

PR Check Results

Ecosystem

✅ ecosystem check detected no changes.

Benchmark

Linux

group                                      main                                   pr
-----                                      ----                                   --
linter/all-rules/large/dataset.py          1.01     16.9±0.07ms     2.4 MB/sec    1.00     16.8±0.10ms     2.4 MB/sec
linter/all-rules/numpy/ctypeslib.py        1.01      4.0±0.02ms     4.2 MB/sec    1.00      4.0±0.04ms     4.2 MB/sec
linter/all-rules/numpy/globals.py          1.01    496.2±6.52µs     5.9 MB/sec    1.00    491.2±0.91µs     6.0 MB/sec
linter/all-rules/pydantic/types.py         1.01      7.0±0.03ms     3.7 MB/sec    1.00      6.9±0.03ms     3.7 MB/sec
linter/default-rules/large/dataset.py      1.00      8.3±0.03ms     4.9 MB/sec    1.00      8.3±0.03ms     4.9 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.00   1785.4±4.26µs     9.3 MB/sec    1.00  1792.9±12.81µs     9.3 MB/sec
linter/default-rules/numpy/globals.py      1.00    202.2±4.03µs    14.6 MB/sec    1.00    202.7±4.65µs    14.6 MB/sec
linter/default-rules/pydantic/types.py     1.00      3.7±0.01ms     6.8 MB/sec    1.00      3.7±0.01ms     6.8 MB/sec
parser/large/dataset.py                    1.02      6.6±0.04ms     6.1 MB/sec    1.00      6.5±0.03ms     6.2 MB/sec
parser/numpy/ctypeslib.py                  1.02   1290.0±5.98µs    12.9 MB/sec    1.00   1268.6±0.92µs    13.1 MB/sec
parser/numpy/globals.py                    1.02    130.9±0.44µs    22.5 MB/sec    1.00    128.8±0.65µs    22.9 MB/sec
parser/pydantic/types.py                   1.02      2.8±0.00ms     9.1 MB/sec    1.00      2.8±0.02ms     9.3 MB/sec

Windows

group                                      main                                   pr
-----                                      ----                                   --
linter/all-rules/large/dataset.py          1.02     24.2±0.55ms  1720.8 KB/sec    1.00     23.7±0.60ms  1759.1 KB/sec
linter/all-rules/numpy/ctypeslib.py        1.05      6.0±0.34ms     2.8 MB/sec    1.00      5.7±0.14ms     2.9 MB/sec
linter/all-rules/numpy/globals.py          1.02   676.4±32.15µs     4.4 MB/sec    1.00   663.6±31.65µs     4.4 MB/sec
linter/all-rules/pydantic/types.py         1.01     10.0±0.32ms     2.5 MB/sec    1.00     10.0±0.36ms     2.6 MB/sec
linter/default-rules/large/dataset.py      1.01     12.1±0.35ms     3.4 MB/sec    1.00     11.9±0.29ms     3.4 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.00      2.5±0.08ms     6.6 MB/sec    1.00      2.5±0.13ms     6.6 MB/sec
linter/default-rules/numpy/globals.py      1.00   295.5±15.18µs    10.0 MB/sec    1.01   299.0±21.64µs     9.9 MB/sec
linter/default-rules/pydantic/types.py     1.00      5.4±0.18ms     4.8 MB/sec    1.00      5.4±0.18ms     4.8 MB/sec
parser/large/dataset.py                    1.00      9.5±0.32ms     4.3 MB/sec    1.01      9.6±0.24ms     4.2 MB/sec
parser/numpy/ctypeslib.py                  1.01  1856.6±59.69µs     9.0 MB/sec    1.00  1841.1±92.16µs     9.0 MB/sec
parser/numpy/globals.py                    1.01    189.4±8.41µs    15.6 MB/sec    1.00   187.9±10.09µs    15.7 MB/sec
parser/pydantic/types.py                   1.01      4.1±0.18ms     6.2 MB/sec    1.00      4.0±0.13ms     6.3 MB/sec

@MichaReiser MichaReiser force-pushed the Use_memchr_to_speedup_newline_search_on_x86 branch from 9791361 to ebfeb11 Compare April 17, 2023 06:07
@MichaReiser MichaReiser mentioned this pull request Apr 17, 2023
Merged
@MichaReiser MichaReiser force-pushed the byte-offset-parser branch 2 times, most recently from dc30757 to 815f484 Compare April 17, 2023 14:53
@MichaReiser MichaReiser force-pushed the Use_memchr_to_speedup_newline_search_on_x86 branch 2 times, most recently from 63fed61 to 322b800 Compare April 17, 2023 15:22
@MichaReiser MichaReiser force-pushed the byte-offset-parser branch 3 times, most recently from 8eca22d to c29914a Compare April 17, 2023 16:08
@MichaReiser MichaReiser force-pushed the Use_memchr_to_speedup_newline_search_on_x86 branch from 322b800 to 6d1311c Compare April 17, 2023 16:08
@MichaReiser MichaReiser force-pushed the byte-offset-parser branch 2 times, most recently from cfe0fe1 to 78b9a89 Compare April 18, 2023 08:30
@MichaReiser MichaReiser force-pushed the Use_memchr_to_speedup_newline_search_on_x86 branch from 6d1311c to 4b3c569 Compare April 18, 2023 08:30
@MichaReiser MichaReiser mentioned this pull request Apr 19, 2023
Merged
charliermarsh
charliermarsh approved these changes Apr 20, 2023
View reviewed changes
Copy link
Member

@charliermarsh charliermarsh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Needs to be rebased but I reviewed the new commits individually.)

charliermarsh
charliermarsh reviewed Apr 20, 2023
View reviewed changes
@MichaReiser MichaReiser force-pushed the Use_memchr_to_speedup_newline_search_on_x86 branch from 4b3c569 to c1b40a3 Compare April 20, 2023 05:56
@MichaReiser MichaReiser force-pushed the byte-offset-parser branch 4 times, most recently from 90fc963 to 5995306 Compare April 26, 2023 17:22
@MichaReiser MichaReiser force-pushed the Use_memchr_to_speedup_newline_search_on_x86 branch from c1b40a3 to d22ae89 Compare April 26, 2023 17:23
This was referenced Apr 26, 2023
Closed
Closed
Base automatically changed from byte-offset-parser to main April 26, 2023 18:11
@MichaReiser MichaReiser force-pushed the Use_memchr_to_speedup_newline_search_on_x86 branch from d22ae89 to 1df35db Compare April 26, 2023 18:17
@MichaReiser MichaReiser marked this pull request as ready for review April 26, 2023 19:03
@MichaReiser MichaReiser added the internal An internal refactor or improvement label Apr 26, 2023
@MichaReiser MichaReiser merged commit e04ef42 into main Apr 26, 2023
@MichaReiser MichaReiser deleted the Use_memchr_to_speedup_newline_search_on_x86 branch April 26, 2023 19:15
@Tenzer Tenzer mentioned this pull request May 2, 2023
Merged
renovate bot referenced this pull request in ixm-one/pytest-cmake-presets May 2, 2023
@renovate
⬆️ Update dependency ruff to ^0.0.264 (#93)
132cf33 
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [ruff](https://togithub.com/charliermarsh/ruff) | `^0.0.263` ->
`^0.0.264` |
[![age](https://badges.renovateapi.com/packages/pypi/ruff/0.0.264/age-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://badges.renovateapi.com/packages/pypi/ruff/0.0.264/adoption-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://badges.renovateapi.com/packages/pypi/ruff/0.0.264/compatibility-slim/0.0.263)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://badges.renovateapi.com/packages/pypi/ruff/0.0.264/confidence-slim/0.0.263)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>charliermarsh/ruff</summary>

###
[`v0.0.264`](https://togithub.com/charliermarsh/ruff/releases/tag/v0.0.264)

[Compare
Source](https://togithub.com/charliermarsh/ruff/compare/v0.0.263...v0.0.264)

<!-- Release notes generated using configuration in .github/release.yml
at 8cb76f85eba1c970a8c800348fd1e0c874621a57 -->

#### What's Changed

##### Rules

- Autofix `EM101`, `EM102`, `EM103` if possible by
[@&#8203;dhruvmanila](https://togithub.com/dhruvmanila) in
[https://github.com/charliermarsh/ruff/pull/4123](https://togithub.com/charliermarsh/ruff/pull/4123)
- Add bugbear immutable functions as allowed in dataclasses by
[@&#8203;mosauter](https://togithub.com/mosauter) in
[https://github.com/charliermarsh/ruff/pull/4122](https://togithub.com/charliermarsh/ruff/pull/4122)

##### Settings

- Add support for providing command-line arguments via `argfile` by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/4087](https://togithub.com/charliermarsh/ruff/pull/4087)

##### Bug Fixes

- Make D410/D411 autofixes mutually exclusive by
[@&#8203;evanrittenhouse](https://togithub.com/evanrittenhouse) in
[https://github.com/charliermarsh/ruff/pull/4110](https://togithub.com/charliermarsh/ruff/pull/4110)
- Remove `pyright` comment prefix from PYI033 checks by
[@&#8203;evanrittenhouse](https://togithub.com/evanrittenhouse) in
[https://github.com/charliermarsh/ruff/pull/4152](https://togithub.com/charliermarsh/ruff/pull/4152)
- Fix F811 false positive with match by
[@&#8203;JonathanPlasse](https://togithub.com/JonathanPlasse) in
[https://github.com/charliermarsh/ruff/pull/4161](https://togithub.com/charliermarsh/ruff/pull/4161)
- Fix `E713` and `E714` false positives for multiple comparisons by
[@&#8203;JonathanPlasse](https://togithub.com/JonathanPlasse) in
[https://github.com/charliermarsh/ruff/pull/4083](https://togithub.com/charliermarsh/ruff/pull/4083)
- Fix B023 shadowed variables in nested functions by
[@&#8203;MichaReiser](https://togithub.com/MichaReiser) in
[https://github.com/charliermarsh/ruff/pull/4111](https://togithub.com/charliermarsh/ruff/pull/4111)
- Preserve star-handling special-casing for force-single-line by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/4129](https://togithub.com/charliermarsh/ruff/pull/4129)
- Respect parent-scoping rules for `NamedExpr` assignments by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/4145](https://togithub.com/charliermarsh/ruff/pull/4145)
- Fix UP032 auto-fix by
[@&#8203;JonathanPlasse](https://togithub.com/JonathanPlasse) in
[https://github.com/charliermarsh/ruff/pull/4165](https://togithub.com/charliermarsh/ruff/pull/4165)
- Allow boolean parameters for `pytest.param` by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/4176](https://togithub.com/charliermarsh/ruff/pull/4176)

##### Internal

- Replace row/column based `Location` with byte-offsets. by
[@&#8203;MichaReiser](https://togithub.com/MichaReiser) in
[https://github.com/charliermarsh/ruff/pull/3931](https://togithub.com/charliermarsh/ruff/pull/3931)
- perf(logical-lines): Various small perf improvements by
[@&#8203;MichaReiser](https://togithub.com/MichaReiser) in
[https://github.com/charliermarsh/ruff/pull/4022](https://togithub.com/charliermarsh/ruff/pull/4022)
- Use `memchr` to speedup newline search on x86 by
[@&#8203;MichaReiser](https://togithub.com/MichaReiser) in
[https://github.com/charliermarsh/ruff/pull/3985](https://togithub.com/charliermarsh/ruff/pull/3985)
- Remove `ScopeStack` in favor of child-parent `ScopeId` pointers by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/4138](https://togithub.com/charliermarsh/ruff/pull/4138)

**Full Changelog**:
astral-sh/ruff@v0.0.263...v0.0.264

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR is behind base branch, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://app.renovatebot.com/dashboard#github/ixm-one/pytest-cmake-presets).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNS42Ni4zIiwidXBkYXRlZEluVmVyIjoiMzUuNjYuMyIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==-->

Signed-off-by: Renovate Bot <[email protected]>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
renovate bot referenced this pull request in allenporter/flux-local May 3, 2023
@renovate
Update dependency ruff to v0.0.264 (#177)
b63f70e 
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [ruff](https://togithub.com/charliermarsh/ruff) | `==0.0.263` ->
`==0.0.264` |
[![age](https://badges.renovateapi.com/packages/pypi/ruff/0.0.264/age-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://badges.renovateapi.com/packages/pypi/ruff/0.0.264/adoption-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://badges.renovateapi.com/packages/pypi/ruff/0.0.264/compatibility-slim/0.0.263)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://badges.renovateapi.com/packages/pypi/ruff/0.0.264/confidence-slim/0.0.263)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>charliermarsh/ruff</summary>

###
[`v0.0.264`](https://togithub.com/charliermarsh/ruff/releases/tag/v0.0.264)

[Compare
Source](https://togithub.com/charliermarsh/ruff/compare/v0.0.263...v0.0.264)

<!-- Release notes generated using configuration in .github/release.yml
at 8cb76f85eba1c970a8c800348fd1e0c874621a57 -->

#### What's Changed

##### Rules

- Autofix `EM101`, `EM102`, `EM103` if possible by
[@&#8203;dhruvmanila](https://togithub.com/dhruvmanila) in
[https://github.com/charliermarsh/ruff/pull/4123](https://togithub.com/charliermarsh/ruff/pull/4123)
- Add bugbear immutable functions as allowed in dataclasses by
[@&#8203;mosauter](https://togithub.com/mosauter) in
[https://github.com/charliermarsh/ruff/pull/4122](https://togithub.com/charliermarsh/ruff/pull/4122)

##### Settings

- Add support for providing command-line arguments via `argfile` by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/4087](https://togithub.com/charliermarsh/ruff/pull/4087)

##### Bug Fixes

- Make D410/D411 autofixes mutually exclusive by
[@&#8203;evanrittenhouse](https://togithub.com/evanrittenhouse) in
[https://github.com/charliermarsh/ruff/pull/4110](https://togithub.com/charliermarsh/ruff/pull/4110)
- Remove `pyright` comment prefix from PYI033 checks by
[@&#8203;evanrittenhouse](https://togithub.com/evanrittenhouse) in
[https://github.com/charliermarsh/ruff/pull/4152](https://togithub.com/charliermarsh/ruff/pull/4152)
- Fix F811 false positive with match by
[@&#8203;JonathanPlasse](https://togithub.com/JonathanPlasse) in
[https://github.com/charliermarsh/ruff/pull/4161](https://togithub.com/charliermarsh/ruff/pull/4161)
- Fix `E713` and `E714` false positives for multiple comparisons by
[@&#8203;JonathanPlasse](https://togithub.com/JonathanPlasse) in
[https://github.com/charliermarsh/ruff/pull/4083](https://togithub.com/charliermarsh/ruff/pull/4083)
- Fix B023 shadowed variables in nested functions by
[@&#8203;MichaReiser](https://togithub.com/MichaReiser) in
[https://github.com/charliermarsh/ruff/pull/4111](https://togithub.com/charliermarsh/ruff/pull/4111)
- Preserve star-handling special-casing for force-single-line by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/4129](https://togithub.com/charliermarsh/ruff/pull/4129)
- Respect parent-scoping rules for `NamedExpr` assignments by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/4145](https://togithub.com/charliermarsh/ruff/pull/4145)
- Fix UP032 auto-fix by
[@&#8203;JonathanPlasse](https://togithub.com/JonathanPlasse) in
[https://github.com/charliermarsh/ruff/pull/4165](https://togithub.com/charliermarsh/ruff/pull/4165)
- Allow boolean parameters for `pytest.param` by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/4176](https://togithub.com/charliermarsh/ruff/pull/4176)

##### Internal

- Replace row/column based `Location` with byte-offsets. by
[@&#8203;MichaReiser](https://togithub.com/MichaReiser) in
[https://github.com/charliermarsh/ruff/pull/3931](https://togithub.com/charliermarsh/ruff/pull/3931)
- perf(logical-lines): Various small perf improvements by
[@&#8203;MichaReiser](https://togithub.com/MichaReiser) in
[https://github.com/charliermarsh/ruff/pull/4022](https://togithub.com/charliermarsh/ruff/pull/4022)
- Use `memchr` to speedup newline search on x86 by
[@&#8203;MichaReiser](https://togithub.com/MichaReiser) in
[https://github.com/charliermarsh/ruff/pull/3985](https://togithub.com/charliermarsh/ruff/pull/3985)
- Remove `ScopeStack` in favor of child-parent `ScopeId` pointers by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/4138](https://togithub.com/charliermarsh/ruff/pull/4138)

**Full Changelog**:
astral-sh/ruff@v0.0.263...v0.0.264

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://app.renovatebot.com/dashboard#github/allenporter/flux-local).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNS42OS4zIiwidXBkYXRlZEluVmVyIjoiMzUuNjkuMyIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
renovate bot referenced this pull request in allenporter/pyrainbird May 3, 2023
@renovate
Update dependency ruff to v0.0.264 (#167)
a5bbcba 
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [ruff](https://togithub.com/charliermarsh/ruff) | `==0.0.263` ->
`==0.0.264` |
[![age](https://badges.renovateapi.com/packages/pypi/ruff/0.0.264/age-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://badges.renovateapi.com/packages/pypi/ruff/0.0.264/adoption-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://badges.renovateapi.com/packages/pypi/ruff/0.0.264/compatibility-slim/0.0.263)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://badges.renovateapi.com/packages/pypi/ruff/0.0.264/confidence-slim/0.0.263)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>charliermarsh/ruff</summary>

###
[`v0.0.264`](https://togithub.com/charliermarsh/ruff/releases/tag/v0.0.264)

[Compare
Source](https://togithub.com/charliermarsh/ruff/compare/v0.0.263...v0.0.264)

<!-- Release notes generated using configuration in .github/release.yml
at 8cb76f85eba1c970a8c800348fd1e0c874621a57 -->

#### What's Changed

##### Rules

- Autofix `EM101`, `EM102`, `EM103` if possible by
[@&#8203;dhruvmanila](https://togithub.com/dhruvmanila) in
[https://github.com/charliermarsh/ruff/pull/4123](https://togithub.com/charliermarsh/ruff/pull/4123)
- Add bugbear immutable functions as allowed in dataclasses by
[@&#8203;mosauter](https://togithub.com/mosauter) in
[https://github.com/charliermarsh/ruff/pull/4122](https://togithub.com/charliermarsh/ruff/pull/4122)

##### Settings

- Add support for providing command-line arguments via `argfile` by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/4087](https://togithub.com/charliermarsh/ruff/pull/4087)

##### Bug Fixes

- Make D410/D411 autofixes mutually exclusive by
[@&#8203;evanrittenhouse](https://togithub.com/evanrittenhouse) in
[https://github.com/charliermarsh/ruff/pull/4110](https://togithub.com/charliermarsh/ruff/pull/4110)
- Remove `pyright` comment prefix from PYI033 checks by
[@&#8203;evanrittenhouse](https://togithub.com/evanrittenhouse) in
[https://github.com/charliermarsh/ruff/pull/4152](https://togithub.com/charliermarsh/ruff/pull/4152)
- Fix F811 false positive with match by
[@&#8203;JonathanPlasse](https://togithub.com/JonathanPlasse) in
[https://github.com/charliermarsh/ruff/pull/4161](https://togithub.com/charliermarsh/ruff/pull/4161)
- Fix `E713` and `E714` false positives for multiple comparisons by
[@&#8203;JonathanPlasse](https://togithub.com/JonathanPlasse) in
[https://github.com/charliermarsh/ruff/pull/4083](https://togithub.com/charliermarsh/ruff/pull/4083)
- Fix B023 shadowed variables in nested functions by
[@&#8203;MichaReiser](https://togithub.com/MichaReiser) in
[https://github.com/charliermarsh/ruff/pull/4111](https://togithub.com/charliermarsh/ruff/pull/4111)
- Preserve star-handling special-casing for force-single-line by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/4129](https://togithub.com/charliermarsh/ruff/pull/4129)
- Respect parent-scoping rules for `NamedExpr` assignments by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/4145](https://togithub.com/charliermarsh/ruff/pull/4145)
- Fix UP032 auto-fix by
[@&#8203;JonathanPlasse](https://togithub.com/JonathanPlasse) in
[https://github.com/charliermarsh/ruff/pull/4165](https://togithub.com/charliermarsh/ruff/pull/4165)
- Allow boolean parameters for `pytest.param` by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/4176](https://togithub.com/charliermarsh/ruff/pull/4176)

##### Internal

- Replace row/column based `Location` with byte-offsets. by
[@&#8203;MichaReiser](https://togithub.com/MichaReiser) in
[https://github.com/charliermarsh/ruff/pull/3931](https://togithub.com/charliermarsh/ruff/pull/3931)
- perf(logical-lines): Various small perf improvements by
[@&#8203;MichaReiser](https://togithub.com/MichaReiser) in
[https://github.com/charliermarsh/ruff/pull/4022](https://togithub.com/charliermarsh/ruff/pull/4022)
- Use `memchr` to speedup newline search on x86 by
[@&#8203;MichaReiser](https://togithub.com/MichaReiser) in
[https://github.com/charliermarsh/ruff/pull/3985](https://togithub.com/charliermarsh/ruff/pull/3985)
- Remove `ScopeStack` in favor of child-parent `ScopeId` pointers by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/4138](https://togithub.com/charliermarsh/ruff/pull/4138)

**Full Changelog**:
astral-sh/ruff@v0.0.263...v0.0.264

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://app.renovatebot.com/dashboard#github/allenporter/pyrainbird).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNS42OS4zIiwidXBkYXRlZEluVmVyIjoiMzUuNjkuMyIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
BurntSushi added a commit that referenced this pull request Nov 10, 2023
@BurntSushi
ruff_source_file: remove bound check elision
9ad0efc 
Eliding bounds checks very rarely results in a meaningful performance
improvement. This is usually because branch predictors are very good,
and in cases like this, the branch predictor likely predicts perfectly
given that the code is correct.

It looks like this use of unsafe was added in #3985 as part of an
optimization to use `memchr`. But `memchr` is where the real win is.

Benchmarks for before and after:

    $ critcmp main test
    group                                       main                                   test
    -----                                       ----                                   ----
    linter/all-rules/large/dataset.py           1.01      5.2±0.03ms     7.8 MB/sec    1.00      5.2±0.01ms     7.9 MB/sec
    linter/all-rules/numpy/ctypeslib.py         1.00   1375.3±6.38µs    12.1 MB/sec    1.00  1379.3±10.77µs    12.1 MB/sec
    linter/all-rules/numpy/globals.py           1.01    157.1±0.63µs    18.8 MB/sec    1.00    155.5±0.73µs    19.0 MB/sec
    linter/all-rules/pydantic/types.py          1.00      2.6±0.00ms     9.9 MB/sec    1.00      2.6±0.00ms     9.9 MB/sec
    linter/all-rules/unicode/pypinyin.py        1.00    647.7±1.27µs     6.5 MB/sec    1.00    647.5±3.46µs     6.5 MB/sec
    linter/default-rules/large/dataset.py       1.00      2.3±0.00ms    17.5 MB/sec    1.00      2.3±0.00ms    17.6 MB/sec
    linter/default-rules/numpy/ctypeslib.py     1.00    456.1±1.13µs    36.5 MB/sec    1.00    458.3±1.53µs    36.3 MB/sec
    linter/default-rules/numpy/globals.py       1.00     39.7±0.21µs    74.2 MB/sec    1.00     39.7±0.23µs    74.3 MB/sec
    linter/default-rules/pydantic/types.py      1.01   1005.8±5.49µs    25.4 MB/sec    1.00    995.5±2.99µs    25.6 MB/sec
    linter/default-rules/unicode/pypinyin.py    1.01    139.9±0.91µs    30.0 MB/sec    1.00    138.0±0.22µs    30.5 MB/sec

Typically, eliding bounds checks makes the most sense when it unlocks
some other kind of optimization (e.g., autovectorization).
BurntSushi added a commit that referenced this pull request Nov 10, 2023
@BurntSushi
ruff_source_file: remove bound check elision
34d6bea 
Eliding bounds checks very rarely results in a meaningful performance
improvement. This is usually because branch predictors are very good,
and in cases like this, the branch predictor likely predicts perfectly
given that the code is correct.

It looks like this use of unsafe was added in #3985 as part of an
optimization to use `memchr`. But `memchr` is where the real win is.

Benchmarks for before and after:

    $ critcmp main test
    group                                       main                                   test
    -----                                       ----                                   ----
    linter/all-rules/large/dataset.py           1.01      5.2±0.03ms     7.8 MB/sec    1.00      5.2±0.01ms     7.9 MB/sec
    linter/all-rules/numpy/ctypeslib.py         1.00   1375.3±6.38µs    12.1 MB/sec    1.00  1379.3±10.77µs    12.1 MB/sec
    linter/all-rules/numpy/globals.py           1.01    157.1±0.63µs    18.8 MB/sec    1.00    155.5±0.73µs    19.0 MB/sec
    linter/all-rules/pydantic/types.py          1.00      2.6±0.00ms     9.9 MB/sec    1.00      2.6±0.00ms     9.9 MB/sec
    linter/all-rules/unicode/pypinyin.py        1.00    647.7±1.27µs     6.5 MB/sec    1.00    647.5±3.46µs     6.5 MB/sec
    linter/default-rules/large/dataset.py       1.00      2.3±0.00ms    17.5 MB/sec    1.00      2.3±0.00ms    17.6 MB/sec
    linter/default-rules/numpy/ctypeslib.py     1.00    456.1±1.13µs    36.5 MB/sec    1.00    458.3±1.53µs    36.3 MB/sec
    linter/default-rules/numpy/globals.py       1.00     39.7±0.21µs    74.2 MB/sec    1.00     39.7±0.23µs    74.3 MB/sec
    linter/default-rules/pydantic/types.py      1.01   1005.8±5.49µs    25.4 MB/sec    1.00    995.5±2.99µs    25.6 MB/sec
    linter/default-rules/unicode/pypinyin.py    1.01    139.9±0.91µs    30.0 MB/sec    1.00    138.0±0.22µs    30.5 MB/sec

Typically, eliding bounds checks makes the most sense when it unlocks
some other kind of optimization (e.g., autovectorization).
BurntSushi added a commit that referenced this pull request Nov 10, 2023
@BurntSushi
ruff_source_file: remove bound check elision
ca56485 
Eliding bounds checks very rarely results in a meaningful performance
improvement. This is usually because branch predictors are very good,
and in cases like this, the branch predictor likely predicts perfectly
given that the code is correct.

It looks like this use of unsafe was added in #3985 as part of an
optimization to use `memchr`. But `memchr` is where the real win is.

Benchmarks for before and after:

    $ critcmp main test
    group                                       main                                   test
    -----                                       ----                                   ----
    linter/all-rules/large/dataset.py           1.01      5.2±0.03ms     7.8 MB/sec    1.00      5.2±0.01ms     7.9 MB/sec
    linter/all-rules/numpy/ctypeslib.py         1.00   1375.3±6.38µs    12.1 MB/sec    1.00  1379.3±10.77µs    12.1 MB/sec
    linter/all-rules/numpy/globals.py           1.01    157.1±0.63µs    18.8 MB/sec    1.00    155.5±0.73µs    19.0 MB/sec
    linter/all-rules/pydantic/types.py          1.00      2.6±0.00ms     9.9 MB/sec    1.00      2.6±0.00ms     9.9 MB/sec
    linter/all-rules/unicode/pypinyin.py        1.00    647.7±1.27µs     6.5 MB/sec    1.00    647.5±3.46µs     6.5 MB/sec
    linter/default-rules/large/dataset.py       1.00      2.3±0.00ms    17.5 MB/sec    1.00      2.3±0.00ms    17.6 MB/sec
    linter/default-rules/numpy/ctypeslib.py     1.00    456.1±1.13µs    36.5 MB/sec    1.00    458.3±1.53µs    36.3 MB/sec
    linter/default-rules/numpy/globals.py       1.00     39.7±0.21µs    74.2 MB/sec    1.00     39.7±0.23µs    74.3 MB/sec
    linter/default-rules/pydantic/types.py      1.01   1005.8±5.49µs    25.4 MB/sec    1.00    995.5±2.99µs    25.6 MB/sec
    linter/default-rules/unicode/pypinyin.py    1.01    139.9±0.91µs    30.0 MB/sec    1.00    138.0±0.22µs    30.5 MB/sec

Typically, eliding bounds checks makes the most sense when it unlocks
some other kind of optimization (e.g., autovectorization).
BurntSushi added a commit that referenced this pull request Nov 27, 2023
@BurntSushi
ruff_source_file: remove bound check elision
c791dc3 
Eliding bounds checks very rarely results in a meaningful performance
improvement. This is usually because branch predictors are very good,
and in cases like this, the branch predictor likely predicts perfectly
given that the code is correct.

It looks like this use of unsafe was added in #3985 as part of an
optimization to use `memchr`. But `memchr` is where the real win is.

Benchmarks for before and after:

    $ critcmp main test
    group                                       main                                   test
    -----                                       ----                                   ----
    linter/all-rules/large/dataset.py           1.01      5.2±0.03ms     7.8 MB/sec    1.00      5.2±0.01ms     7.9 MB/sec
    linter/all-rules/numpy/ctypeslib.py         1.00   1375.3±6.38µs    12.1 MB/sec    1.00  1379.3±10.77µs    12.1 MB/sec
    linter/all-rules/numpy/globals.py           1.01    157.1±0.63µs    18.8 MB/sec    1.00    155.5±0.73µs    19.0 MB/sec
    linter/all-rules/pydantic/types.py          1.00      2.6±0.00ms     9.9 MB/sec    1.00      2.6±0.00ms     9.9 MB/sec
    linter/all-rules/unicode/pypinyin.py        1.00    647.7±1.27µs     6.5 MB/sec    1.00    647.5±3.46µs     6.5 MB/sec
    linter/default-rules/large/dataset.py       1.00      2.3±0.00ms    17.5 MB/sec    1.00      2.3±0.00ms    17.6 MB/sec
    linter/default-rules/numpy/ctypeslib.py     1.00    456.1±1.13µs    36.5 MB/sec    1.00    458.3±1.53µs    36.3 MB/sec
    linter/default-rules/numpy/globals.py       1.00     39.7±0.21µs    74.2 MB/sec    1.00     39.7±0.23µs    74.3 MB/sec
    linter/default-rules/pydantic/types.py      1.01   1005.8±5.49µs    25.4 MB/sec    1.00    995.5±2.99µs    25.6 MB/sec
    linter/default-rules/unicode/pypinyin.py    1.01    139.9±0.91µs    30.0 MB/sec    1.00    138.0±0.22µs    30.5 MB/sec

Typically, eliding bounds checks makes the most sense when it unlocks
some other kind of optimization (e.g., autovectorization).
BurntSushi added a commit that referenced this pull request Nov 28, 2023
@BurntSushi
ruff_source_file: remove bound check elision
b72603d 
Eliding bounds checks very rarely results in a meaningful performance
improvement. This is usually because branch predictors are very good,
and in cases like this, the branch predictor likely predicts perfectly
given that the code is correct.

It looks like this use of unsafe was added in #3985 as part of an
optimization to use `memchr`. But `memchr` is where the real win is.

Benchmarks for before and after:

    $ critcmp main test
    group                                       main                                   test
    -----                                       ----                                   ----
    linter/all-rules/large/dataset.py           1.01      5.2±0.03ms     7.8 MB/sec    1.00      5.2±0.01ms     7.9 MB/sec
    linter/all-rules/numpy/ctypeslib.py         1.00   1375.3±6.38µs    12.1 MB/sec    1.00  1379.3±10.77µs    12.1 MB/sec
    linter/all-rules/numpy/globals.py           1.01    157.1±0.63µs    18.8 MB/sec    1.00    155.5±0.73µs    19.0 MB/sec
    linter/all-rules/pydantic/types.py          1.00      2.6±0.00ms     9.9 MB/sec    1.00      2.6±0.00ms     9.9 MB/sec
    linter/all-rules/unicode/pypinyin.py        1.00    647.7±1.27µs     6.5 MB/sec    1.00    647.5±3.46µs     6.5 MB/sec
    linter/default-rules/large/dataset.py       1.00      2.3±0.00ms    17.5 MB/sec    1.00      2.3±0.00ms    17.6 MB/sec
    linter/default-rules/numpy/ctypeslib.py     1.00    456.1±1.13µs    36.5 MB/sec    1.00    458.3±1.53µs    36.3 MB/sec
    linter/default-rules/numpy/globals.py       1.00     39.7±0.21µs    74.2 MB/sec    1.00     39.7±0.23µs    74.3 MB/sec
    linter/default-rules/pydantic/types.py      1.01   1005.8±5.49µs    25.4 MB/sec    1.00    995.5±2.99µs    25.6 MB/sec
    linter/default-rules/unicode/pypinyin.py    1.01    139.9±0.91µs    30.0 MB/sec    1.00    138.0±0.22µs    30.5 MB/sec

Typically, eliding bounds checks makes the most sense when it unlocks
some other kind of optimization (e.g., autovectorization).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@charliermarsh charliermarsh charliermarsh approved these changes

Assignees

No one assigned

Labels

internal An internal refactor or improvement

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@MichaReiser @charliermarsh