Build binaries using Github actions. #181

remgodow · 2020-09-07T08:37:22Z

A proposal of fix for #46.
Saves debug binaries for all OS in Github Action Artifact.
Branched from Windows pull request to test Windows OS builds.
Based on ripgrep

…plementation based on sysinfo and netstat2.

# Conflicts: # Cargo.lock

# Conflicts: # Cargo.lock # Cargo.toml

… formatting.

…p SDK.

# Conflicts: # Cargo.lock # Cargo.toml

imsnif

Awesome. :) I left some minor comments, and after we merge the opensockets PR I think we can definitely go ahead with this one and even use this instead of travis.

imsnif · 2020-09-07T13:31:26Z

.github/workflows/ci.yaml

+  pull_request:
+  push:
+    branches:
+      - master


We use main as our main branch

imsnif · 2020-09-07T13:31:47Z

.github/workflows/ci.yaml

+    strategy:
+      matrix:
+        build:
+          # We test ripgrep on a pinned version of Rust, along with the moving


ripgrep => bandwhich :)

imsnif · 2020-09-07T13:33:29Z

.github/workflows/ci.yaml

+  push:
+    branches:
+      - master
+      - actions


I think we can remove this once we're ready to merge and don't need to debug anymore, right?

imsnif · 2020-09-07T13:34:28Z

.github/workflows/ci.yaml

+            rust: stable
+          - build: nightly
+            os: ubuntu-18.04
+            rust: nightly


Right now on travis, our nightly branch is allowed to fail and we just use it as an FYI. Is there a way to do this here too?

Not now unfortunately. I'm not familiar with setting up checks for pull requests, but I guess that we could do ci action with only stable as a gate for pull requests and ci_nightly which would build but would not guard anything.

imsnif · 2020-09-10T17:04:48Z

Hey @remgodow, let me know if there's anything I can pick up on this side. Maybe I can work on the binary release part a bit? (eg. keep our naming convention, make it build/release a generic musl linux file, etc.)?

Also, what do we have left after this? (I know I've asked before somehwere, but I can't seem to find where just now - sorry :) ).
I was also thinking of including some basic textual instructions (or at least a link?) for the winpcap installation, what do you think?

remgodow · 2020-09-10T17:31:40Z

@imsnif Sure, feel free to adapt it for project needs. We could add an information for Windows users that WinPcap or NpCap in WinPcap compatible mode is required for bandwhich to work, with a link to NpCap, as WinPcap is depracated. An extra for windows users would be a package for chocolatey/scoop package managers.

remgodow · 2020-09-10T19:05:58Z

Scoop looks easy, its uses a simple json with project description and links to binaries as a package manifest. To add it to the repository you make a pull to scoop repo. Here is an example for ccache

imsnif · 2020-09-11T08:14:35Z

Hey @remgodow, there seems to be a git hitch that I'm not sure I have the permissions to fix here. Could you please merge main into here? I did this locally but am not able to push. I think it's just getting everything from main except the actions themselves.

imsnif · 2020-09-11T08:35:22Z

Also, @remgodow - as a side note, I would very much like to add you as an author in Cargo.toml - since you made a very significant contributions. If you'd like that, please add yourself to the bottom of the authors with your email address (if you'd like to include it).

# Conflicts: # Cargo.lock # Cargo.toml # build.rs # src/os/linux.rs # src/os/lsof.rs # src/os/lsof_utils.rs # src/os/shared.rs

remgodow · 2020-09-11T16:53:04Z

The issue with pushing to the branch is caused by release action - right now, it creates /refs/heads/action tags. Therefore it is neccesary to delete release and tag on the fork page to allow pushes. I've added you as collaborator to my fork, I hope that it will allow you to test actions and remove test releases.

imsnif · 2020-09-13T14:55:01Z

I played around with this a little. Github actions look cool, but tbh - while I do see the benefit of releasing additional executables, this is too much of a time sink for me right now. Seeing as this isn't blocking anything (to my knowledge) at the moment, I'm going to leave this open and get back to it if the need becomes more pressing. Thanks again for the work on this and the windows port!

remgodow · 2020-09-13T19:17:02Z

@imsnif I have adopted release action to current release versioning, and added musl target. Would you need anything else?
You can see the effects here: https://github.com/remgodow/bandwhich/releases/tag/0.19.7

imsnif · 2020-09-14T13:36:22Z

This looks great! Thanks :)

A few things:

When unpacking the tar.gz file, I get all the folders of the release in the repository (target/x86_64-unknown-linux-musl/release), to fix this I usually tar from the same folder as the original file, but I think there's also a tar flag for this.
Would it be possible to make the release a manual step (I think I saw this is possible in github actions) rather than a regex on the commit?
How would we get a visual indication for these in the PR? Like we get the pass/fail now with travis and a link to the log? Or will it just happen once we merge?

…d on github, tar executable without directories

remgodow · 2020-09-14T20:00:47Z

Fixed by using working-directory parameter
I changed release action so that: it no longer creates release by itself, instead it triggers on "release: created" event - when a release is created using Github, then it builds binaries and uploads them to created release. https://github.com/remgodow/bandwhich/releases/tag/0.20.0
When you configure branch protection, it automatically detects github actions and you can select actions which will be required to succeed to allow merging. Checks indicator looks like travis one - Eventresize remgodow/bandwhich#1 (I guess it does not run because actions are not in main, but you can see the UI).

imsnif · 2020-09-15T13:57:35Z

This looks really great. I don't have a lot of time to devote to it right now, but I plan on releasing a new version in the next few days, so will definitely merge this and use it for the release beforehand.

imsnif · 2020-09-20T11:15:53Z

So, unfortunately working with github acations is a no-go for me at the moment. I merged this and tried to work with them, but they fail consistently on macos because of some sort of race condition.

I realize that the fault here is in the tests themselves and that a proper fix would be to find the race condition and fix it there, but after spending some time on it I can see that this might not be trivial. At the moment, the travis tests do what we want them to (they sometimes also fail here as well, but much more rarely and almost always run successfully when re-run). So, for our current needs I think they are enough.

Thanks very much for your work on this. I realize you put a lot of effort here. We gave it a good go, but I guess at least in our case, travis is a better option. The actions are still in main but I disabled github actions on this repository so the output will not confuse those wanting to contribute. Maybe in the future I'll give them another go.

remgodow · 2020-09-20T17:57:37Z

Sure thing, the benefit of nigtly and ci actions over travis was having the artifacts, but its not a deal breaker when it comes to developent builds, hovever there is also a release actions which works on all platforms and is triggered manually by creating a release (as it builds and uploads the artifacts, without testing). Therefore I would suggest deleting nigtly.yaml and ci.yaml, keeping the release.yaml. This way you could benefit from artifacts for releases, while keeping travis for CI/pull requests.

remgodow · 2020-09-20T18:00:18Z

By the way, if there is anything I can help with, let me know. I may have some time next week, and I've seen that the windows port caused issues for other systems through the dependencies.

imsnif · 2020-09-20T20:49:34Z

By the way, if there is anything I can help with, let me know. I may have some time next week, and I've seen that the windows port caused issues for other systems through the dependencies.

Uff, yeah - it was a hot mess but I think I took care of it here: #190 :)

If you'd like something else to work on, this has resurfaced recently: #150
Also, I've been thinking of adding a flag that would let you manually specify a DNS server instead of autodetecting it.
Only if you feel like it of course. Your contributions are very much welcome.

Remigiusz Godowicz and others added 28 commits August 16, 2020 23:03

Remove connections vector from OpenSockets, use common OpenSockets im…

6333b33

…plementation based on sysinfo and netstat2.

Replace termion backend with crossterm, which works on Windows as well.

f09ad10

Merge branch 'crossterm' into windows

c9e748f

# Conflicts: # Cargo.lock

More fixes for windows build.

56d47ea

Remove tui default-features (termion), update unit tests for crossterm.

8c07e77

Merge remote-tracking branch 'origin/crossterm' into windows

d6bdcae

# Conflicts: # Cargo.lock # Cargo.toml

Windows compilation fixes.

9f0f0d9

Remove unused get_open_sockets implementations for linux and mac. Fix…

20f7c9c

… formatting.

Add build.rs for windows to download and extract Packet.lib from npca…

1772bdb

…p SDK.

Merge remote-tracking branch 'upstream/main' into windows

f7d9a65

# Conflicts: # Cargo.lock # Cargo.toml

Resolve Cargo.lock after merging main.

743150f

Merge branch 'main' into windows

ae5d79d

fix(tests): adjust snapshots new location of the dns resolution

336fde4

style(clippy): clippy

3fda7ef

style(clippy): remove dead code

db1966a

Test github actions.

b4908d3

Fix action syntax.

644afec

Fix "Show build.rs stderr" step.

0f12de9

Add artifact upload step.

a7a20fe

style(clippy): use write_all in build.rs

9acb8a3

Merge branch 'windows' into actions

186c982

Fix path in upload binary step

b3afa3d

style(clippy): remove unused import added by Intellij

77704d2

Merge branch 'windows' into actions

0d854c6

Separate upload artifacts actions for Windows.

921bbc6

Troubleshooting windows artifacts upload.

6b0253a

Fix typo

fb33bc8

Bump upload-artifact to v2.

5f969c4

imsnif requested changes Sep 7, 2020

View reviewed changes

fix(review): change branch name to main, remove nightly

4d3c1dd

remgodow marked this pull request as ready for review September 10, 2020 17:31

Remigiusz Godowicz added 2 commits September 11, 2020 18:08

Merge remote-tracking branch 'upstream/main' into actions

1df5f08

# Conflicts: # Cargo.lock # Cargo.toml # build.rs # src/os/linux.rs # src/os/lsof.rs # src/os/lsof_utils.rs # src/os/shared.rs

fix: merge issues

7203331

fix(release): add linux generic

69fe4d1

Remigiusz Godowicz added 8 commits September 13, 2020 19:00

feat(infra): build musl on release

82ce838

feat(infra): fix release verioning

e963010

feat(infra): another try for fixing versioning

3e79868

fix(infra): use version from version step instead of release id

7b9dd5d

fix(infra): fix add macos target variable, use --target for cargo build

adf65b8

fix(infra): add musl-tools required by musl target

7eac88b

fix(infra): fix action

c4b8cae

fix(infra): fix action

4ecf85d

fix(build): trigger build-release action when release has been create…

003294d

…d on github, tar executable without directories

fix(debug): remove actions branch

5436af4

imsnif merged commit 58187b4 into imsnif:main Sep 20, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build binaries using Github actions. #181

Build binaries using Github actions. #181

remgodow commented Sep 7, 2020

imsnif left a comment

imsnif Sep 7, 2020

imsnif Sep 7, 2020

imsnif Sep 7, 2020

remgodow Sep 8, 2020

imsnif Sep 7, 2020

remgodow Sep 8, 2020

imsnif commented Sep 10, 2020

remgodow commented Sep 10, 2020

remgodow commented Sep 10, 2020

imsnif commented Sep 11, 2020

imsnif commented Sep 11, 2020

remgodow commented Sep 11, 2020

imsnif commented Sep 13, 2020

remgodow commented Sep 13, 2020 •

edited

Loading

imsnif commented Sep 14, 2020

remgodow commented Sep 14, 2020

imsnif commented Sep 15, 2020

imsnif commented Sep 20, 2020

remgodow commented Sep 20, 2020

remgodow commented Sep 20, 2020

imsnif commented Sep 20, 2020

Build binaries using Github actions. #181

Build binaries using Github actions. #181

Conversation

remgodow commented Sep 7, 2020

imsnif left a comment

Choose a reason for hiding this comment

imsnif Sep 7, 2020

Choose a reason for hiding this comment

imsnif Sep 7, 2020

Choose a reason for hiding this comment

imsnif Sep 7, 2020

Choose a reason for hiding this comment

remgodow Sep 8, 2020

Choose a reason for hiding this comment

imsnif Sep 7, 2020

Choose a reason for hiding this comment

remgodow Sep 8, 2020

Choose a reason for hiding this comment

imsnif commented Sep 10, 2020

remgodow commented Sep 10, 2020

remgodow commented Sep 10, 2020

imsnif commented Sep 11, 2020

imsnif commented Sep 11, 2020

remgodow commented Sep 11, 2020

imsnif commented Sep 13, 2020

remgodow commented Sep 13, 2020 • edited Loading

imsnif commented Sep 14, 2020

remgodow commented Sep 14, 2020

imsnif commented Sep 15, 2020

imsnif commented Sep 20, 2020

remgodow commented Sep 20, 2020

remgodow commented Sep 20, 2020

imsnif commented Sep 20, 2020

remgodow commented Sep 13, 2020 •

edited

Loading