[`pylint`] Implement `too-many-lines` (`PLC0302`) #10262

augustelalande · 2024-03-07T01:45:46Z

Summary

Implements the too-many-lines rule (C0302) from pylint.

Test Plan

New fixtures have been added

Part of #970

github-actions · 2024-03-07T02:12:03Z

`ruff-ecosystem` results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

ℹ️ ecosystem check detected linter changes. (+143 -2 violations, +0 -0 fixes in 9 projects; 34 projects unchanged)

DisnakeDev/disnake (+10 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ disnake/abc.py:1:1: PLC0302 Too many lines in module (2013>2000)
+ disnake/channel.py:1:1: PLC0302 Too many lines in module (5053>2000)
+ disnake/client.py:1:1: PLC0302 Too many lines in module (3213>2000)
+ disnake/ext/commands/core.py:1:1: PLC0302 Too many lines in module (2683>2000)
+ disnake/flags.py:1:1: PLC0302 Too many lines in module (2567>2000)
+ disnake/guild.py:1:1: PLC0302 Too many lines in module (5341>2000)
+ disnake/http.py:1:1: PLC0302 Too many lines in module (2775>2000)
+ disnake/message.py:1:1: PLC0302 Too many lines in module (2513>2000)
+ disnake/state.py:1:1: PLC0302 Too many lines in module (2324>2000)
+ disnake/webhook/async_.py:1:1: PLC0302 Too many lines in module (2039>2000)

aiven/aiven-client (+2 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ aiven/client/cli.py:1:1: PLC0302 Too many lines in module (6024>2000)
+ aiven/client/client.py:1:1: PLC0302 Too many lines in module (2763>2000)

apache/airflow (+30 -1 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview --select ALL

+ airflow/cli/cli_config.py:1:1: PLC0302 Too many lines in module (2128>2000)
+ airflow/configuration.py:1:1: D100 Missing docstring in public module
- airflow/configuration.py:1:1: D100 Missing docstring in public module
+ airflow/configuration.py:1:1: PLC0302 Too many lines in module (2348>2000)
+ airflow/models/dag.py:1:1: PLC0302 Too many lines in module (4199>2000)
+ airflow/models/taskinstance.py:1:1: PLC0302 Too many lines in module (3772>2000)
+ airflow/providers/fab/auth_manager/security_manager/override.py:1:1: PLC0302 Too many lines in module (2765>2000)
+ airflow/providers/google/cloud/hooks/bigquery.py:1:1: PLC0302 Too many lines in module (3539>2000)
... 24 additional changes omitted for rule PLC0302
... 23 additional changes omitted for project

aws/aws-sam-cli (+10 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ samcli/hook_packages/terraform/hooks/prepare/resource_linking.py:1:1: PLC0302 Too many lines in module (2447>2000)
+ tests/integration/buildcmd/test_build_cmd.py:1:1: PLC0302 Too many lines in module (3146>2000)
+ tests/integration/local/start_api/test_start_api.py:1:1: PLC0302 Too many lines in module (3253>2000)
+ tests/unit/commands/init/test_cli.py:1:1: PLC0302 Too many lines in module (3195>2000)
+ tests/unit/commands/local/lib/test_sam_api_provider.py:1:1: PLC0302 Too many lines in module (2273>2000)
+ tests/unit/commands/local/lib/test_sam_function_provider.py:1:1: PLC0302 Too many lines in module (2560>2000)
+ tests/unit/hook_packages/terraform/hooks/prepare/test_resource_linking.py:1:1: PLC0302 Too many lines in module (3132>2000)
+ tests/unit/lib/build_module/test_app_builder.py:1:1: PLC0302 Too many lines in module (3015>2000)
+ tests/unit/lib/package/test_artifact_exporter.py:1:1: PLC0302 Too many lines in module (2024>2000)
+ tests/unit/local/apigw/test_local_apigw_service.py:1:1: PLC0302 Too many lines in module (2039>2000)

freedomofpress/securedrop (+1 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ securedrop/tests/test_journalist.py:1:1: PLC0302 Too many lines in module (3774>2000)

ibis-project/ibis (+6 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ ibis/backends/tests/test_generic.py:1:1: PLC0302 Too many lines in module (2023>2000)
+ ibis/backends/tests/test_temporal.py:1:1: PLC0302 Too many lines in module (2493>2000)
+ ibis/expr/api.py:1:1: PLC0302 Too many lines in module (2410>2000)
+ ibis/expr/types/generic.py:1:1: PLC0302 Too many lines in module (2232>2000)
+ ibis/expr/types/relations.py:1:1: PLC0302 Too many lines in module (4424>2000)
+ ibis/tests/expr/test_table.py:1:1: PLC0302 Too many lines in module (2077>2000)

pandas-dev/pandas (+54 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ pandas/conftest.py:1:1: PLC0302 Too many lines in module (2027>2000)
+ pandas/core/arrays/arrow/array.py:1:1: PLC0302 Too many lines in module (2981>2000)
+ pandas/core/arrays/base.py:1:1: PLC0302 Too many lines in module (2604>2000)
+ pandas/core/arrays/categorical.py:1:1: PLC0302 Too many lines in module (3122>2000)
+ pandas/core/arrays/datetimelike.py:1:1: PLC0302 Too many lines in module (2615>2000)
+ pandas/core/arrays/datetimes.py:1:1: PLC0302 Too many lines in module (2835>2000)
+ pandas/core/dtypes/dtypes.py:1:1: PLC0302 Too many lines in module (2353>2000)
+ pandas/core/frame.py:1:1: PLC0302 Too many lines in module (12924>2000)
+ pandas/core/generic.py:1:1: PLC0302 Too many lines in module (12875>2000)
+ pandas/core/groupby/generic.py:1:1: PLC0302 Too many lines in module (2814>2000)
+ pandas/core/groupby/groupby.py:1:1: PLC0302 Too many lines in module (5623>2000)
+ pandas/core/indexes/base.py:1:1: PLC0302 Too many lines in module (7513>2000)
+ pandas/core/indexes/multi.py:1:1: PLC0302 Too many lines in module (4099>2000)
+ pandas/core/indexing.py:1:1: PLC0302 Too many lines in module (2778>2000)
+ pandas/core/internals/blocks.py:1:1: PLC0302 Too many lines in module (2371>2000)
+ pandas/core/internals/managers.py:1:1: PLC0302 Too many lines in module (2524>2000)
+ pandas/core/resample.py:1:1: PLC0302 Too many lines in module (2703>2000)
+ pandas/core/reshape/merge.py:1:1: PLC0302 Too many lines in module (2751>2000)
... 36 additional changes omitted for project

rotki/rotki (+8 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ rotkehlchen/api/rest.py:1:1: PLC0302 Too many lines in module (4799>2000)
+ rotkehlchen/api/v1/resources.py:1:1: PLC0302 Too many lines in module (3199>2000)
+ rotkehlchen/api/v1/schemas.py:1:1: PLC0302 Too many lines in module (3466>2000)
+ rotkehlchen/db/dbhandler.py:1:1: PLC0302 Too many lines in module (3559>2000)
+ rotkehlchen/globaldb/handler.py:1:1: PLC0302 Too many lines in module (2120>2000)
+ rotkehlchen/tests/db/test_db_upgrades.py:1:1: PLC0302 Too many lines in module (2496>2000)
+ rotkehlchen/tests/unit/decoders/test_makerdao_sai.py:1:1: PLC0302 Too many lines in module (2487>2000)
+ rotkehlchen/tests/utils/dataimport.py:1:1: PLC0302 Too many lines in module (2516>2000)

zulip/zulip (+22 -1 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview --select ALL

+ analytics/tests/test_counts.py:1:1: PLC0302 Too many lines in module (2139>2000)
+ corporate/lib/stripe.py:1:1: PLC0302 Too many lines in module (5161>2000)
+ corporate/tests/test_stripe.py:1:1: PLC0302 Too many lines in module (9178>2000)
+ tools/setup/emoji/emoji_names.py:1:1: PLC0302 Too many lines in module (2081>2000)
+ zerver/actions/message_send.py:1:1: PLC0302 Too many lines in module (2008>2000)
+ zerver/lib/export.py:1:1: PLC0302 Too many lines in module (2443>2000)
... 16 additional changes omitted for rule PLC0302
+ zerver/lib/test_classes.py:1:1: D100 Missing docstring in public module
- zerver/lib/test_classes.py:1:1: D100 Missing docstring in public module
... 15 additional changes omitted for project

Changes by rule (2 rules affected)

code	total	+ violation	- violation	+ fix	- fix
PLC0302	141	141	0	0	0
D100	4	2	2	0	0

Formatter (stable)

✅ ecosystem check detected no format changes.

Formatter (preview)

✅ ecosystem check detected no format changes.

augustelalande · 2024-03-07T02:28:36Z

~~Not sure if docs should be considered a KNOWN_FORMATTING_VIOLATIONS, or formatting should be changed. I did my best to copy the docs from the pylint docs.~~

Nevermind

eslamodeh · 2024-03-07T14:36:20Z

Would be great to have this merged 🙌 🙌

T-256 · 2024-03-21T21:41:02Z

crates/ruff_linter/src/rules/pylint/rules/too_many_lines.rs

+impl Violation for TooManyLines {
+    #[derive_message_formats]
+    fn message(&self) -> String {
+        format!("Too many lines in module")


Could it also show the number? e.g. pylint shows Too many lines in module (3000/2000)
I think use > sign is more proper there:
Too many lines in module (3000>2000)

T-256 · 2024-03-21T21:44:05Z

crates/ruff_linter/src/rules/pylint/rules/too_many_lines.rs

+    let lines = locator.contents().universal_newlines();
+    let length = lines.count() + 1;
+
+    if length > settings.pylint.max_module_lines {


Suggested change

let lines = locator.contents().universal_newlines();

let length = lines.count() + 1;

if length > settings.pylint.max_module_lines {

let lines_count = locator.contents().universal_newlines().count();

if lines_count > settings.pylint.max_module_lines {

Is there any reason that you increased the lines count by one?

universal_newlines splits on newline characters, so if the last line is empty it won't be counted. Really what I want to count is the number of newline characters, but I didn't find anything to do that.

It's a good point though that if the last line is not empty, then this will be off-by-one in the other direction.

Correct, it should increase by one when the last line is not empty.

I think lines count of all below contents is 1:

first

first\n

first\r\n

In this PR, first\n consumed as 2 lines, I'm not sure if it is expected behavior.

I'll check against pylint

Ok pylint doesn't count the final newline, but it does count multiple final new lines, so first\n\n should be 2 but first\n should be one. I will update it.

MichaReiser · 2024-03-22T07:38:15Z

Thanks for your contribution. Unfortunately, the rule is incompatible with the formatter because the formatter can introduce new violations by splitting lines across multiple lines, making the module exceed the configured max lines. We currently want to hold off from adding new rules that conflict with the formatter because everyone using --select ALL automatically opts into these rules, making ruff print an ugly warning that the rules are incompatible. We can reconsider adding this rule after we've finished #1774.

I'm sorry we didn't point this out on the PyLint issue before you started implementing it. This must feel frustrating to you and we could have done better. I'm trying to catch up with rules and plan to first go through the open PRs and later triage open issues to ensure we communicate clearly which rules will be accepted today.

augustelalande · 2024-03-22T14:08:22Z

@MichaReiser No worries. I do hope we can merge this eventually though, because I think it is an important check.

implement too-many-lines

b37358d

augustelalande changed the title ~~[pylint] implement too-many-lines (PLC0302)~~ [pylint] Implement too-many-lines (PLC0302) Mar 7, 2024

add docs

f99704f

augustelalande added 3 commits March 6, 2024 21:12

update docs

3fe9659

linting

fb1307c

Merge branch 'main' into too-many-lines

7470e83

reformat docs

38b3f2e

T-256 reviewed Mar 21, 2024

View reviewed changes

improve error message

b59d4b6

augustelalande force-pushed the too-many-lines branch from 477cf1a to b59d4b6 Compare March 21, 2024 22:15

do not count last line if empty

c41b325

MichaReiser closed this Mar 22, 2024

augustelalande deleted the too-many-lines branch March 31, 2024 15:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[`pylint`] Implement `too-many-lines` (`PLC0302`) #10262

[`pylint`] Implement `too-many-lines` (`PLC0302`) #10262

augustelalande commented Mar 7, 2024

github-actions bot commented Mar 7, 2024 •

edited

Loading

augustelalande commented Mar 7, 2024 •

edited

Loading

eslamodeh commented Mar 7, 2024

T-256 Mar 21, 2024

augustelalande Mar 21, 2024

T-256 Mar 21, 2024

augustelalande Mar 21, 2024

T-256 Mar 21, 2024

augustelalande Mar 21, 2024

augustelalande Mar 21, 2024

augustelalande Mar 21, 2024

MichaReiser commented Mar 22, 2024 •

edited

Loading

augustelalande commented Mar 22, 2024

[pylint] Implement too-many-lines (PLC0302) #10262

[pylint] Implement too-many-lines (PLC0302) #10262

Conversation

augustelalande commented Mar 7, 2024

Summary

Test Plan

github-actions bot commented Mar 7, 2024 • edited Loading

ruff-ecosystem results

Linter (stable)

Linter (preview)

Formatter (stable)

Formatter (preview)

augustelalande commented Mar 7, 2024 • edited Loading

eslamodeh commented Mar 7, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MichaReiser commented Mar 22, 2024 • edited Loading

augustelalande commented Mar 22, 2024

[`pylint`] Implement `too-many-lines` (`PLC0302`) #10262

[`pylint`] Implement `too-many-lines` (`PLC0302`) #10262

github-actions bot commented Mar 7, 2024 •

edited

Loading

`ruff-ecosystem` results

augustelalande commented Mar 7, 2024 •

edited

Loading

MichaReiser commented Mar 22, 2024 •

edited

Loading