dev-tools: run golint/reviewdog in Jenkins #3832
Conversation
|
For non-PR builds we could setup Jenkins to track the changes to the linter reports. This could be done by using gometalinter's checkstyle output and have Jenkins parse that report. No action needed, just a thought for the new Jenkins setup. |
|
|
||
| .PHONY: fmt | ||
| fmt: python-env | ||
| $(foreach var,$(PROJECTS),$(MAKE) -C $(var) fmt || exit 1;) | ||
| # Cleans also python files which are not part of the beats | ||
| find . -type f -name *.py -not -path "*/vendor/*" -not -path "*/build/*" -not -path "*/.git/*" -exec autopep8 --in-place --max-line-length 120 {} \; | ||
|
|
||
| .PHONY: lint |
There was a problem hiding this comment.
It would be nice to be able to run make lint for any individual Beat and for community Beats. Would it be possible to have a lint target in libbeat/scripts/Makefile that's is common to all Beats?
There was a problem hiding this comment.
I thought it would be nice to mimic what Jenkins does, i.e., run it on the diff. Are you suggesting running it on everything by default or having the ability to restrict it to the diff of a single beat?
There was a problem hiding this comment.
I'm suggesting moving the implementation of the lint target to libbeat/scripts/Makefile. This provides two things:
- Developers of community beats can run
make lintto see new linter warnings that they introduced while making changes. - Developers of elastic/beats can call
make lintinside of a beat sub-dir and the operation is limited to changes to that specific beat.
I would still want to have the ability to run make lint at the top level in elastic/beats to check the whole repo. This is consistent with other commands like make fmt or make check.
There was a problem hiding this comment.
Cool that makes sense, thanks!
Makefile
Outdated
| @@ -71,15 +78,20 @@ check: python-env | |||
| # Corrects spelling errors | |||
| .PHONY: misspell | |||
| misspell: | |||
| go get github.com/client9/misspell | |||
| ${FIND} -name '*' -exec misspell -w {} \; | |||
| go get $(MISSPELL_REPO) | |||
There was a problem hiding this comment.
Is there a specific reason you introduce a variable here?
There was a problem hiding this comment.
I tend to prefer doing this so the actual logic is more readable and easier to maintain (ideally also have $(GO)). If you don't like it I can remove it though.
There was a problem hiding this comment.
I kind of worry that # of variables in the Makefile is going to explode. For me the two reasons for variables are:
- Needs to be overwritable by beat x
- Multiple use of the same
Perhaps we can figure out a better way to "store" the variables.
There was a problem hiding this comment.
A different way to store would be nice, but I don't know any. Do you? Either way I'll remove this change.
There was a problem hiding this comment.
The mostly constant variables like names and urls could be moved to a separate makefile and then be included by libbeat/scripts/Makefile.
There was a problem hiding this comment.
I like this idea, libbeat/scripts/Makefile-env?
There was a problem hiding this comment.
I'd prefer something like xyz.mk.
There was a problem hiding this comment.
The nicest thing would be if we could somehow load a yaml file. So we would have things like beat_name or beat_version also available in other tools. But I don't know if there is a way in Makefiles to load env variables from a file.
There was a problem hiding this comment.
I reverted the misspell change, should we pick up this discussion in a different issue?
Makefile
Outdated
| .PHONY: lint | ||
| lint: | ||
| @go get $(GOLINT_REPO) $(REVIEWDOG_REPO) | ||
| $(GOLINT) ./... | $(REVIEWDOG) $(REVIEWDOG_OPTIONS) |
There was a problem hiding this comment.
Is this going to give lint warnings when vendored files are updated? Do we need to filter vendor files?
reviewdog.yml
Outdated
| @@ -0,0 +1,3 @@ | |||
| runner: | |||
| golint: | |||
| cmd: golint ./... | |||
There was a problem hiding this comment.
Does this command also need to filter out vendor? I'm not really sure what this command is used for TBH.
There was a problem hiding this comment.
You're right, I think we can just plug the whole golint command here and just run reviewdog directly. I'm not sure why its doc shows the usage as golint | reviewdog, since if you don't specify golint somewhere in the reviewdog options it complains.
| @@ -24,7 +24,7 @@ type Log struct { | |||
| config prospectorConfig | |||
| } | |||
|
|
|||
| // NewLog instantiates a new Log | |||
| // NewProspectorLog instantiates a new Log | |||
There was a problem hiding this comment.
[golint] reported by reviewdog 🐶
comment on exported function NewLog should be of the form "NewLog ..."
| @@ -8,6 +8,11 @@ SNAPSHOT?=yes | |||
| PYTHON_ENV?=${BUILD_DIR}/python-env | |||
| VIRTUALENV_PARAMS?= | |||
| FIND=find . -type f -not -path "*/vendor/*" -not -path "*/build/*" -not -path "*/.git/*" | |||
| GOLINT=golint | |||
There was a problem hiding this comment.
I think these variables are not needed anymore here because they are in the libbeat Makefile?
There was a problem hiding this comment.
We still have a lint target in this Makefile and I don't see it including the one from libbeat/scripts, did I miss that?
There was a problem hiding this comment.
my fault, I assume this lint target calls all the other lint targets like we do for update and other things.
There was a problem hiding this comment.
Not really. The reason is I didn't want to risk running it on overlapping code, nor keep track of what's a beat and what's not. So if you run it at the top level it runs on everything, if you run it inside a beat it runs only on the beat.
There was a problem hiding this comment.
Ok for me. My main worry is that we have quite a few duplicated lines and will change them in the future in one place but not the other.
reviewdogallows to restrictgolintruns to the current diff. It also integrates with GitHub, so it can comment inline on a PR and leave it up to the reviewers to decide wether or not the warnings are worth fixing (the build will still pass). This would allow the kind of flexibility we need in deciding which comments to add to functions etc.