SSH Connect back related to #555 and discussion #51

[NHAS]

This is a rather large and moderately work in progress pull request to add ssh connecting back to the beszel server.

I am going to attempt to describe what I've done and what needs to be done.

Things I have done:

Agents

• ADDR as the env variable to determine listen/connect address
• Client mode (connects back):
  • The client Args[0] argument instructs the client to attempt to connect to ADDR or PORT env variables rather than open listener
• Connect to remove service over SSH (golang standard library)
• Back off in case of connection failure
• Generate own keypair and save to local directory
• Will send stats every 15 seconds +/- 30 seconds to stop waves of clients crushing beszel

Hub

• Added --sshd, -s flag to set where the SSH server port listens
• Starts an ssh server startSSHServer
• Generates ssh keys a bit better
• Added record creation awareness for new systems to either poll/not poll based on client type
• Added 3 new tables
  • blocked_systems: Connected servers that are blocked from ever showing up in the new systems table
  • new_systems: The new systems table that contains all new connection attempts with valid API keys
  • connection_settings: Settings regarding what the beszel hub should do when a new connection occurs
• Modified a number of existing tables:
  • user_settings: Now has additional column connection_key for use with the agent authorization process
  • systems: Now has type and fingerprint for the client type and ssh fingerprint respectively
• Refactor some internal functions such as updateSystems to more apt names likes pollServerSystems
• Added mechanism to record and update agent statistics via the ssh server
• Added default join "actions" that can be configured via connection_settings. I.e when a client joins you can say "if it has a valid api key add it immediately, deny, or wait for confirmation before addeding

Site/Server/UI:

• Added UI to for connection settings, i.e what the beszel hub should do when a new client holding a valid api key connects
• Added UI in settings for the users API key
• Created new table under "Add System" button to show pending system join requests (and retain old connection method by "Connect To" button
• Add controls for new systems connections to accept, deny or block the systems (blocking is done via fingerprint)

Screenshots:

Add System/New System table:

Connections setting:

Things left to do:

• Add a notification indicator on the "+ Add System" button for when a pending connection is there
• Potentially add notifications when pending connections exist
• Link up Settings -> Connections "Max number of waiting connections" and "External address" to actual values in the DB
• Add a copy paste button for the new way of connecting
• Probably a fair amount of web UI fixes

Some caveats

I am not a particularly good web developer so it is quite likely the majority of the web stuff I've written for this is subtly wrong. So please give it a check!

This is roughly a wip as I've been making about 300 migrations (humor) because I've also unfamiliar with pocketbase. So please give that a look through as well.

Other than that, if you guys want to take it from here that'd be fantastic, otherwise its largely just struggling through web development which isnt my favorite (or cleaning up or writing docs)

Final note

Feel free to reach out and ask for changes on this. I do plan to clean it up somewhat so I havent changed a bunch of language files and migrations but am running out of steam for now.

I hope this is what you guys were looking for!

@henrygd

[henrygd]

That was quick! Thank you so much for your work on this. I'll check it out this weekend and let you know if I have any questions.

As far as timeline, I think we'll make a release with existing / in progress changes first, then get this out in its own release afterward.

Please feel free to make any changes if you think of something, but at this point I should be able to get it across the finish line. No need to worry about the migrations, language files, or docs. I can handle that, as well as any merge conflicts that may come up.

Thanks again! 👍

[NHAS]

Sweet I've just made a small change to tidy up some of the SSHD code in the hub.
Effectively just moving the connection key checking into the "new systems" check portion, so that a user can rotate their connection key without having a bunch of systems be dropped off.

Authorisation is done with the ssh key fingerprint after first use anyhow.

[christiaangoossens]

Love this PR. I worked with NATS and websockets for similar things before, but reverse ssh is really smart here and introduces no extra apps you need to host.

Great job.

[zeegeeguy]

is it possible to deploy this yet ? would like to see if i can get it going
cheers

[henrygd]

Not yet, but I'm working on it and it's still planned for the next minor release.

Haven't had as much spare time this year, especially over the past couple of months. I moved up a few things that I get lots of questions about to hopefully decrease the load on that front.

There will be another patch release this week and it shouldn't be too much longer after that.

[NHAS]

Not yet, but I'm working on it and it's still planned for the next minor release.

Haven't had as much spare time this year, especially over the past couple of months. I moved up a few things that I get lots of questions about to hopefully decrease the load on that front.

There will be another patch release this week and it shouldn't be too much longer after that.

Keep it up man you're doing good work!

[henrygd]

Should have this finally wrapped up over the weekend or early next week 🤞

Edit: Sidetracked by other issues again 😑. I will finish with this asap.

[NHAS]

Hey man good work for keeping up with it.

Shoot me a message if ya need any help reconciling the changes you've made with my pr

[henrygd]

I ended up taking a lot of things you did and adapting it to use WebSocket instead of SSH. I think it's a little more straightforward from a user perspective and fit in well with the other logic.

I'll try to have a beta out later today, and I'll add you as a coauthor unless you don't want that.

Also what you did with the SSH channel was really nice. I totally overlooked that before, but it's much better than just creating sessions and capturing the output. I'll definitely switch to your way at some point.

[NHAS]

Awesome! Glad to hear, I think websockets are also a good choice for this.

Thanks for adding me as a coauthor I really appreciate it.

[irnutsmurt]

Let me know if you need a beta tester. I'm down to try it.

[henrygd]

@irnutsmurt Thanks, you can check it out here: https://github.com/henrygd/beszel/releases/tag/v0.12.0-beta1