Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add testifylint linter #4103

Merged
merged 2 commits into from
Oct 9, 2023
Merged

Add testifylint linter #4103

merged 2 commits into from
Oct 9, 2023

Conversation

Antonboom
Copy link
Contributor

@Antonboom Antonboom commented Sep 24, 2023

Hi!

https://github.com/Antonboom/testifylint

  1. Supports analysis.SuggestedFix (but no integration in golangci-lint? Add support for SuggestedFixes #1779)
  2. Supports analysis.Diagnostic.Category & URL (but no usage of this feature in golangci-lint?)

P.S. For local dev you need 1.21, but the module itself I returned to 1.20 by replacing useful package (slices) with own code 😢


Recreated from #4072, because of
Screenshot 2023-09-24 at 21 58 16

@Antonboom Antonboom requested a review from ldez September 24, 2023 18:58
@ldez
Copy link
Member

ldez commented Sep 24, 2023

In order for a pull request adding a linter to be reviewed, the linter and the PR must follow some requirements.

  • The CLA must be signed

Pull Request Description

  • It must have a link to the linter repository.
  • It must provide a short description of the linter.

Linter

  • It must not be a duplicate of another linter or a rule of a linter. (the team will help to verify that)
  • It must have a valid license (AGPL is not allowed) and the file must contain the required information by the license, ex: author, year, etc.
  • The linter repository must have a CI and tests.
  • It must use go/analysis.
  • It must have a valid tag, ex: v1.0.0, v0.1.0.
  • It must not contain init().
  • It must not contain panic().
  • It must not contain log.fatal(), os.exit(), or similar.
  • It must not modify the AST.
  • It must not have false positives/negatives. (the team will help to verify that)
  • It must have tests inside golangci-lint.

The Linter Tests Inside Golangci-lint

  • They must have at least one std lib import.
  • They must work with T=<name of the linter test file>.go make test_linters. (the team will help to verify that)

.golangci.reference.yml

  • The linter must be added to the list of available linters (alphabetical case-insensitive order).
    • enable and disable options
  • If the linter has a configuration, the exhaustive configuration of the linter must be added (alphabetical case-insensitive order)
    • The values must be different from the default ones.
    • The default values must be defined in a comment.
    • The option must have a short description.

Others Requirements

  • The files (tests and linter) inside golangci-lint must have the same name as the linter.
  • The .golangci.yml of golangci-lint itself must not be edited and the linter must not be added to this file.
  • The linters must be sorted in the alphabetical order (case-insensitive) in the Manager.GetAllSupportedLinterConfigs(...) and .golangci.reference.yml.
  • The load mode (WithLoadMode(...)):
    • if the linter doesn't use types: goanalysis.LoadModeSyntax
    • goanalysis.LoadModeTypesInfo required WithLoadForGoAnalysis() in the Manager.GetAllSupportedLinterConfigs(...)
  • The version in WithSince(...) must be the next minor version (v1.X.0) of golangci-lint.
  • WithURL() must contain the URL of the repository.

Recommendations

  • The linter should not use SSA. (SSA can be a problem with generics)
  • The linter repository should have a readme and linting.
  • The linter should be published as a binary. (useful to diagnose bug origins)

The golangci-lint team will edit this comment to check the boxes before and during the review.

The code review will start after the completion of those checkboxes (except for the specific items that the team will help to verify).

If the author of the PR is a member of the golangci-lint team, he should not edit this message.

This checklist does not imply that we will accept the linter.

@ldez ldez added the linter: new Support new linter label Sep 24, 2023
@Antonboom Antonboom force-pushed the feat/testifylint branch 2 times, most recently from 9875ef3 to 95e419f Compare October 3, 2023 13:35
@ldez
Copy link
Member

ldez commented Oct 3, 2023

I will answer here about this #4072 (comment)

Yes, some linters have historically init/panic because they were not designed to be used as a lib.

Not problem, but just for clarification – why the linter doesn't cover PresetStyle and PresetMetaLinter?

not PresetStyle: because the reports of your linter are not (mainly) related to the "taste" of a user or just styling.

not PresetMetaLinter: because you are handling only one topic: testify.

In this case, the tests will not work because the linter ignores non-test files for performance and sanity reasons.

The choice of the scope should rely on golangci-lint configuration if possible.
If I create a dedicated test package, I will not suffix the file with _test because I want to use it outside of the package but your linter can also be useful in this context. Also, golangci-lint controls what the linter will analyze or not.

@ldez ldez added the feedback required Requires additional feedback label Oct 3, 2023
@Antonboom
Copy link
Contributor Author

Antonboom commented Oct 4, 2023

@ldez, thank u for feedback

If I create a dedicated test package

Nice point, I completely forgot about this case.

Summarizing:

  1. No init, no panics, analyze all files. Changelog here.
  2. T=testifylint.go make test_linters
Output
GL_TEST_RUN=1 go test -v ./test -count 1 -run TestSourcesFromTestdata/testifylint.go
# github.com/golangci/golangci-lint/test.test
ld: warning: -bind_at_load is deprecated on macOS
=== RUN   TestSourcesFromTestdata
    linters_test.go:21: testdata/*.go
=== RUN   TestSourcesFromTestdata/testifylint.go
=== PAUSE TestSourcesFromTestdata/testifylint.go
=== CONT  TestSourcesFromTestdata/testifylint.go
level=info msg="[test] ran [/Users/anthony/golangci-lint/golangci-lint run --internal-cmd-test --no-config --allow-parallel-runners --disable-all --out-format=json --max-same-issues=100 -Etestifylint testifylint.go] in 979.000166ms"
level=info msg="[test] ran [/Users/anthony/golangci-lint/golangci-lint run --internal-cmd-test --no-config --allow-parallel-runners --disable-all --out-format=json --max-same-issues=100 -Etypecheck -Etestifylint testifylint.go] in 294.706667ms"
--- PASS: TestSourcesFromTestdata (10.18s)
    --- PASS: TestSourcesFromTestdata/testifylint.go (1.28s)
=== RUN   TestSourcesFromTestdataSubDir
--- PASS: TestSourcesFromTestdataSubDir (0.00s)
PASS
ok      github.com/golangci/golangci-lint/test  12.382s
  1. .golangci.reference.yml updated
  2. Run on large projects and happy with the result
testifylint --enable-all -json ./... | jq '.[] | .[] | .[].category' | sort | uniq -c
# Consul
  30 "bool-compare"
 183 "compares"
 295 "empty"
  11 "error-is-as"
 463 "error-nil"
 636 "expected-actual"
  54 "float-compare"
  30 "len"
 106 "require-error"
 181 "suite-dont-use-pkg"

# Grafana
 145 "bool-compare"
  17 "compares"
 152 "empty"
  36 "error-is-as"
 737 "error-nil"
 885 "expected-actual"
 160 "float-compare"
 384 "len"
 362 "require-error"

# Kubernetes
  34 "bool-compare"
  16 "compares"
  46 "empty"
 119 "error-nil"
  74 "expected-actual"
   5 "float-compare"
  89 "len"
 502 "require-error"

Based on the analysis I opened Antonboom/testifylint#5 but it doesn't seem to be a blocker.

@Antonboom Antonboom removed the feedback required Requires additional feedback label Oct 4, 2023
Copy link
Member

@ldez ldez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ldez ldez merged commit 2d5d29f into golangci:master Oct 9, 2023
11 checks passed
@ldez ldez modified the milestone: v1.55 Mar 4, 2024
@ldez ldez added the enhancement New feature or improvement label Apr 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or improvement linter: new Support new linter
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants