Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid nested loops in missing_whitespace #3688

Merged
merged 1 commit into from
Mar 23, 2023
Merged

Conversation

charliermarsh
Copy link
Member

Summary

Small performance oversight: on my machine, this checks takes about 70 seconds to run on CPython. With these changes, it's down to a few hundred milliseconds.

@charliermarsh charliermarsh added the performance Potential performance improvement label Mar 23, 2023
@charliermarsh charliermarsh merged commit 71c0da2 into main Mar 23, 2023
@charliermarsh charliermarsh deleted the charlie/whitespace branch March 23, 2023 18:19
@github-actions
Copy link
Contributor

github-actions bot commented Mar 23, 2023

PR Check Results

Ecosystem

✅ ecosystem check detected no changes.

Benchmark

Linux

group                                      main                                   pr
-----                                      ----                                   --
linter/all-rules/large/dataset.py          1.00     13.8±0.06ms     3.0 MB/sec    1.03     14.3±0.06ms     2.9 MB/sec
linter/all-rules/numpy/ctypeslib.py        1.00      3.6±0.01ms     4.7 MB/sec    1.03      3.7±0.01ms     4.6 MB/sec
linter/all-rules/numpy/globals.py          1.00    489.8±1.23µs     6.0 MB/sec    1.03    502.3±1.90µs     5.9 MB/sec
linter/all-rules/pydantic/types.py         1.00      6.0±0.02ms     4.2 MB/sec    1.03      6.2±0.03ms     4.1 MB/sec
linter/default-rules/large/dataset.py      1.00      7.3±0.02ms     5.6 MB/sec    1.08      7.8±0.02ms     5.2 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.00   1634.3±2.00µs    10.2 MB/sec    1.05   1720.3±6.97µs     9.7 MB/sec
linter/default-rules/numpy/globals.py      1.00    180.1±0.35µs    16.4 MB/sec    1.05    189.0±1.22µs    15.6 MB/sec
linter/default-rules/pydantic/types.py     1.00      3.4±0.01ms     7.5 MB/sec    1.06      3.6±0.01ms     7.1 MB/sec

Windows

group                                      main                                   pr
-----                                      ----                                   --
linter/all-rules/large/dataset.py          1.00     12.7±0.05ms     3.2 MB/sec    1.00     12.7±0.07ms     3.2 MB/sec
linter/all-rules/numpy/ctypeslib.py        1.00      3.3±0.01ms     5.1 MB/sec    1.00      3.3±0.01ms     5.1 MB/sec
linter/all-rules/numpy/globals.py          1.00    366.9±1.89µs     8.0 MB/sec    1.02    373.1±1.90µs     7.9 MB/sec
linter/all-rules/pydantic/types.py         1.00      5.5±0.05ms     4.7 MB/sec    1.00      5.5±0.02ms     4.7 MB/sec
linter/default-rules/large/dataset.py      1.00      7.0±0.02ms     5.8 MB/sec    1.00      7.0±0.06ms     5.8 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.01   1458.0±8.27µs    11.4 MB/sec    1.00   1443.2±9.14µs    11.5 MB/sec
linter/default-rules/numpy/globals.py      1.02    153.8±1.44µs    19.2 MB/sec    1.00    151.1±1.80µs    19.5 MB/sec
linter/default-rules/pydantic/types.py     1.00      3.2±0.01ms     8.1 MB/sec    1.00      3.2±0.02ms     8.1 MB/sec

renovate bot referenced this pull request in ixm-one/pytest-cmake-presets Mar 23, 2023
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [ruff](https://github.com/charliermarsh/ruff) | `^0.0.258` ->
`^0.0.259` |
[![age](https://badges.renovateapi.com/packages/pypi/ruff/0.0.259/age-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://badges.renovateapi.com/packages/pypi/ruff/0.0.259/adoption-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://badges.renovateapi.com/packages/pypi/ruff/0.0.259/compatibility-slim/0.0.258)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://badges.renovateapi.com/packages/pypi/ruff/0.0.259/confidence-slim/0.0.258)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>charliermarsh/ruff</summary>

###
[`v0.0.259`](https://github.com/charliermarsh/ruff/releases/tag/v0.0.259)

[Compare
Source](https://github.com/charliermarsh/ruff/compare/v0.0.258...v0.0.259)

<!-- Release notes generated using configuration in .github/release.yml
at main -->

#### Summary

Follow-up release to `v0.0.258` to fix an issue related to rule
resolution via `select` and `ignore`.

#### What's Changed

##### Bug Fixes

- Fix RuleSet.remove by
[@&#8203;MichaReiser](https://github.com/MichaReiser) in
[https://github.com/charliermarsh/ruff/pull/3685](https://github.com/charliermarsh/ruff/pull/3685)
- Respect all rule-exemption sources when suppressing parser errors by
[@&#8203;charliermarsh](https://github.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3665](https://github.com/charliermarsh/ruff/pull/3665)
- Avoid nested loops in `missing_whitespace` by
[@&#8203;charliermarsh](https://github.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3688](https://github.com/charliermarsh/ruff/pull/3688)

**Full Changelog**:
astral-sh/ruff@v0.0.258...v0.0.259

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR is behind base branch, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://app.renovatebot.com/dashboard#github/ixm-one/pytest-cmake-presets).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNS4xNy4xIiwidXBkYXRlZEluVmVyIjoiMzUuMTcuMSJ9-->

Signed-off-by: Renovate Bot <[email protected]>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
let mut prev_lsqb = None;
let mut prev_lbrace = None;
for (idx, (char, next_char)) in line.chars().tuple_windows().enumerate() {
if char == '[' {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not. I like to use match statements if I have many branches matching on constant values. It gives you compile time evaluation that no teo branches are overlapping (and can be shorter than changing char == comparisons)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I think a lot of this code is a line-by-line port of pycodestyle, for correctness purposes as we were trying to get the test suite passing. Doesn't mean it has to stay that way :))

renovate bot referenced this pull request in allenporter/flux-local Mar 25, 2023
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [ruff](https://github.com/charliermarsh/ruff) | `==0.0.257` ->
`==0.0.259` |
[![age](https://badges.renovateapi.com/packages/pypi/ruff/0.0.259/age-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://badges.renovateapi.com/packages/pypi/ruff/0.0.259/adoption-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://badges.renovateapi.com/packages/pypi/ruff/0.0.259/compatibility-slim/0.0.257)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://badges.renovateapi.com/packages/pypi/ruff/0.0.259/confidence-slim/0.0.257)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>charliermarsh/ruff</summary>

###
[`v0.0.259`](https://github.com/charliermarsh/ruff/releases/tag/v0.0.259)

[Compare
Source](https://github.com/charliermarsh/ruff/compare/v0.0.258...v0.0.259)

<!-- Release notes generated using configuration in .github/release.yml
at main -->

#### Summary

Follow-up release to `v0.0.258` to fix an issue related to rule
resolution via `select` and `ignore`.

#### What's Changed

##### Bug Fixes

- Fix RuleSet.remove by
[@&#8203;MichaReiser](https://github.com/MichaReiser) in
[https://github.com/charliermarsh/ruff/pull/3685](https://github.com/charliermarsh/ruff/pull/3685)
- Respect all rule-exemption sources when suppressing parser errors by
[@&#8203;charliermarsh](https://github.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3665](https://github.com/charliermarsh/ruff/pull/3665)
- Avoid nested loops in `missing_whitespace` by
[@&#8203;charliermarsh](https://github.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3688](https://github.com/charliermarsh/ruff/pull/3688)

**Full Changelog**:
astral-sh/ruff@v0.0.258...v0.0.259

###
[`v0.0.258`](https://github.com/charliermarsh/ruff/releases/tag/v0.0.258)

[Compare
Source](https://github.com/charliermarsh/ruff/compare/v0.0.257...v0.0.258)

<!-- Release notes generated using configuration in .github/release.yml
at main -->

#### What's Changed

##### Rules

- \[`flake8-comprehensions`] Update `C416` with dict comprehension
(autofixable) by [@&#8203;dhruvmanila](https://github.com/dhruvmanila)
in
[https://github.com/charliermarsh/ruff/pull/3605](https://github.com/charliermarsh/ruff/pull/3605)
- \[`pylint`]: Implement `assert-on-string-literal` (`W0129`) by
[@&#8203;latonis](https://github.com/latonis) in
[https://github.com/charliermarsh/ruff/pull/3610](https://github.com/charliermarsh/ruff/pull/3610)
- \[`pyupgrade`] Convert single-argument %-style format calls by
[@&#8203;charliermarsh](https://github.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3600](https://github.com/charliermarsh/ruff/pull/3600)
- \[`pyupgrade`] Flag PEP 585 and PEP 604 violations in quoted
annotations by
[@&#8203;charliermarsh](https://github.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3593](https://github.com/charliermarsh/ruff/pull/3593)
- \[`pyupgrade`] Enable autofix for annotations within 'simple' string
literals by [@&#8203;charliermarsh](https://github.com/charliermarsh)
in
[https://github.com/charliermarsh/ruff/pull/3657](https://github.com/charliermarsh/ruff/pull/3657)
- \[`pyflakes`] Add autofix functionality for `F523`
([#&#8203;3613](https://github.com/charliermarsh/ruff/issues/3613)) by
[@&#8203;JonathanPlasse](https://github.com/JonathanPlasse) in
[https://github.com/charliermarsh/ruff/pull/3613](https://github.com/charliermarsh/ruff/pull/3613)
- \[`flake8-bandit`]: Implement deny-list rules for suspicious member
calls by [@&#8203;colin99d](https://github.com/colin99d) in
[https://github.com/charliermarsh/ruff/pull/3239](https://github.com/charliermarsh/ruff/pull/3239)
- \[`flake8-annotations`] Add autofix for `ANN204` with magic methods by
[@&#8203;JonathanPlasse](https://github.com/JonathanPlasse) in
[https://github.com/charliermarsh/ruff/pull/3633](https://github.com/charliermarsh/ruff/pull/3633)
- \[`pylint`] Implement `binary-op-exception` (`PLW0711`) by
[@&#8203;latonis](https://github.com/latonis) in
[https://github.com/charliermarsh/ruff/pull/3639](https://github.com/charliermarsh/ruff/pull/3639)
- \[`flake8-django`]: Implement rule DJ012 by
[@&#8203;dhruvmanila](https://github.com/dhruvmanila) in
[https://github.com/charliermarsh/ruff/pull/3659](https://github.com/charliermarsh/ruff/pull/3659)

##### Bug Fixes

- Check exclusions prior to resolving `pyproject.toml` files by
[@&#8203;charliermarsh](https://github.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3588](https://github.com/charliermarsh/ruff/pull/3588)
- Fix D417 false positive by
[@&#8203;JonathanPlasse](https://github.com/JonathanPlasse) in
[https://github.com/charliermarsh/ruff/pull/3596](https://github.com/charliermarsh/ruff/pull/3596)
- Avoid removing comment hash for noqa's with trailing content by
[@&#8203;charliermarsh](https://github.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3589](https://github.com/charliermarsh/ruff/pull/3589)
- Avoid panics for implicitly-concatenated docstrings by
[@&#8203;charliermarsh](https://github.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3584](https://github.com/charliermarsh/ruff/pull/3584)
- Fix infinite loop due to rules `D207` & `W605` by
[@&#8203;vlindhol](https://github.com/vlindhol) in
[https://github.com/charliermarsh/ruff/pull/3609](https://github.com/charliermarsh/ruff/pull/3609)
- Avoid trimming escaped whitespace in D210 by
[@&#8203;charliermarsh](https://github.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3635](https://github.com/charliermarsh/ruff/pull/3635)
- Handle `UP032` autofix with adjacent keywords by
[@&#8203;JonathanPlasse](https://github.com/JonathanPlasse) in
[https://github.com/charliermarsh/ruff/pull/3636](https://github.com/charliermarsh/ruff/pull/3636)
- Consider same-site fixes to be overlapping by
[@&#8203;charliermarsh](https://github.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3638](https://github.com/charliermarsh/ruff/pull/3638)
- Avoid `RUF007` fixes for more than two arguments by
[@&#8203;JonathanPlasse](https://github.com/JonathanPlasse) in
[https://github.com/charliermarsh/ruff/pull/3654](https://github.com/charliermarsh/ruff/pull/3654)
- Allow `pairwise` diagnostics for `zip(..., strict=True)` by
[@&#8203;charliermarsh](https://github.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3669](https://github.com/charliermarsh/ruff/pull/3669)
- isort: fix bad interaction between `force-sort-within-sections` and
`force-to-top` by [@&#8203;bluetech](https://github.com/bluetech) in
[https://github.com/charliermarsh/ruff/pull/3645](https://github.com/charliermarsh/ruff/pull/3645)
- Gracefully handle lint panics by
[@&#8203;MichaReiser](https://github.com/MichaReiser) in
[https://github.com/charliermarsh/ruff/pull/3509](https://github.com/charliermarsh/ruff/pull/3509)
- Fix TRY300 false positive by
[@&#8203;JonathanPlasse](https://github.com/JonathanPlasse) in
[https://github.com/charliermarsh/ruff/pull/3634](https://github.com/charliermarsh/ruff/pull/3634)
- Avoid raising PEP 604 errors with forward-referenced members by
[@&#8203;charliermarsh](https://github.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3640](https://github.com/charliermarsh/ruff/pull/3640)
- Avoid attempting infinite `open` fix with re-bound builtin by
[@&#8203;charliermarsh](https://github.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3650](https://github.com/charliermarsh/ruff/pull/3650)
- Check indentation level when executing E231 by
[@&#8203;kyoto7250](https://github.com/kyoto7250) in
[https://github.com/charliermarsh/ruff/pull/3668](https://github.com/charliermarsh/ruff/pull/3668)
- Flag, but don't fix, unused imports (`F401`) in `ModuleNotFoundError`
blocks by [@&#8203;charliermarsh](https://github.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3658](https://github.com/charliermarsh/ruff/pull/3658)

#### New Contributors

- [@&#8203;Rogdham](https://github.com/Rogdham) made their first
contribution in
[https://github.com/charliermarsh/ruff/pull/3607](https://github.com/charliermarsh/ruff/pull/3607)
- [@&#8203;vlindhol](https://github.com/vlindhol) made their first
contribution in
[https://github.com/charliermarsh/ruff/pull/3609](https://github.com/charliermarsh/ruff/pull/3609)
- [@&#8203;dhruvmanila](https://github.com/dhruvmanila) made their
first contribution in
[https://github.com/charliermarsh/ruff/pull/3605](https://github.com/charliermarsh/ruff/pull/3605)
- [@&#8203;luke396](https://github.com/luke396) made their first
contribution in
[https://github.com/charliermarsh/ruff/pull/3604](https://github.com/charliermarsh/ruff/pull/3604)
- [@&#8203;fuziontech](https://github.com/fuziontech) made their first
contribution in
[https://github.com/charliermarsh/ruff/pull/3641](https://github.com/charliermarsh/ruff/pull/3641)

**Full Changelog**:
astral-sh/ruff@v0.0.257...v0.0.258

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://app.renovatebot.com/dashboard#github/allenporter/flux-local).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNS4xNC4yIiwidXBkYXRlZEluVmVyIjoiMzUuMTQuMiJ9-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Potential performance improvement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants