Skip to content

Conversation

tale
Copy link
Owner

@tale tale commented Jun 20, 2025

This is the start of the next release cycle (version 0.6.1), which aims to do a few things. These changes are to address lingering issues, implement SSH agent capability through the web browser, and focus on data migration to SQLite.

As changes are committed, they will be listed below

  • Headplane now supports connecting to machines via SSH in the web browser.
    • This is an experimental feature and requires the integration.agent section to be set up in the config file.
    • This is built on top of a Go binary that runs in WebAssembly, using Xterm.js for the terminal interface.
  • Begin using a new SQLite database file in /var/lib/headplane/hp_persist.db.
    • The database is created automatically if it does not exist.
    • It currently stores SSH connection details and will migrate older data.
  • The docker container now runs in a non-root, distroless image (closes Use a distroless docker image #255).
    • Due to this change you may need to run chown -R 65532:65532 . on the volume mount.
    • A debug version of the container that runs as root and has a shell is available as ghcr.io/tale/headplane:<version>-shell.
  • Removing a Split DNS record will no longer make the split domain unresolvable by clients (closes Removing Split-DNS nameserver via web-ui not functioning correctly #231).
  • Reintroduce the toggle for overriding local DNS settings in the Headscale config (closes DNS edition causes the override_local_dns switch to be set to false. #236).
  • Prefer cross-compiling in the Dockerfile to speed up builds while still supporting multiple architectures.
  • Add a build attestation to validate SLSA provenance for the Docker image.
  • Implement more accurate guessing on the PID with the /proc integration (via Proc mode: try to guess PID based on Parent PID when multiple found #219).
  • Usernames will now correctly fall back to emails if not provided (via fix: username never falling back to email #257).
  • Configuration loading via paths is now supported for sensitive values (via [next] Re-Implement optional paths in the config loader #283)
    • Options like server.cookie_secret_path can override server.cookie_secret
    • Environment variables are interpolatable into these paths
    • See the full reference in the docs
  • The nix overlay build is fixed for the SSH module (via [next] Nix #282))
  • OIDC profile pictures are now available from Gravatar by setting oidc.profile_picture_source to gravatar (closes Support gravatar profile #232).
  • OIDC now allows passing many custom parameters:
    • oidc.authorization_endpoint, oidc.token_endpoint, and oidc.userinfo_endpoint can be overridden to support non-standard providers or scenarios without discovery (closes Support GitHub OIDC #117).
    • oidc.scope can be set to specify custom scopes (defaults to openid email profile).
    • oidc.extra_params can be set to pass arbitrary query parameters to the authorization endpoint (closes Support specifying OIDC query parameters via the config #197).

This PR is now complete and I will be merging this in very soon. As a final note, it addresses outdated dependency vulnerabilities, tons of robust changes, support for SSH via the web, and a switch to SQLite for the backing datastore that we use.

My last few steps include:

  • Validating that the config between 0.6.0 and 0.6.1 doesn't have breaking changes
  • Updating documentation and the example config to reflect best practices
  • Small little UI tweaks and visual glitches (nothing to do with actual logic)
  • That said, expect this PR to land sometime in the weekend and cut a release shortly after!

Interested in beta testing? Subscribe to the Pre-release testing discussion to be notified about future testing release. As always, pre-release versions are accessible on Docker via ghcr.io/tale/headplane:next or the next tag on Git.

@tale tale added this to the 0.6.1 milestone Jun 20, 2025
@tale tale self-assigned this Jun 20, 2025
@tale tale added help wanted Extra attention is needed pre release Next release to cut labels Jun 20, 2025
@tale
Copy link
Owner Author

tale commented Jun 20, 2025

And in typical fashion the build is broken. Will address this shortly.

@losthorizon84
Copy link

losthorizon84 commented Jun 20, 2025

Ufff.. Headplane docker is down.. I can't see nothing in log causing this behavior:
Captura de pantalla 2025-06-20 a las 18 47 54

After a while, I could see following error:

Error: ConnectionFailed("Unable to open connection to local database /var/lib/headplane/hp_persist.db: 14")
2025-06-20T16:44:23.132Z [config] DEBUG: Validating Headplane configuration
    at new Database (/app/node_modules/.pnpm/[email protected]/node_modules/libsql/index.js:93:17)

I understood that db was created by default if it didn't exist

@tecosaur
Copy link

I've just tested out the latest. After the headplane page loads (with OIDC login), and I see the onboarding page (no devices shown), I click "I know what I'm doing" and then some sort of redirect loop seems to happen:

image

@tale
Copy link
Owner Author

tale commented Aug 21, 2025

I've just tested out the latest. After the headplane page loads (with OIDC login), and I see the onboarding page (no devices shown), I click "I know what I'm doing" and then some sort of redirect loop seems to happen:

image

Can you try manually clearing a bunch of cookies and caches on your Headscale domain and probably also on Authelia for good measure? I've tried everything and I cannot reproduce your issue.

@tecosaur
Copy link

tecosaur commented Aug 21, 2025

I've just run this in a private window (so there should be no cookies at all). I get redirected to login, but then (without seeing the onboarding page this time) I hit a redirect loop again 😕.

This is my config:

headscale:
  config_path: /nix/store/r3cljz68gagl3ss0lij4nq113qvn4kf8-headscale.yaml
  config_strict: true
  url: http://localhost:8174
integration:
  agent:
    enabled: false
    pre_authkey_path: /run/agenix/headplane-preauth
  proc:
    enabled: true
oidc:
  client_id: headscale
  client_secret_path: /run/agenix/headscale-oidc
  disable_api_key_login: true
  headscale_api_key: --set in env file--
  issuer: https://auth.tecosaur.net
  redirect_uri: https://headscale.tecosaur.net/admin/oidc/callback
  token_endpoint_auth_method: client_secret_basic
server:
  cookie_secret: '--------set in env file---------'
  cookie_secure: true
  host: 0.0.0.0
  port: 8175

I'll be AFK for the next ~18h, but happy to help debug after then

@igor-ramazanov
Copy link
Contributor

@tecosaur This is likely something with OIDC setup between Authelia and Headplane.
I had same issue before and solved it somehow, will post if I find out how exactly.

@tale
Copy link
Owner Author

tale commented Aug 22, 2025

I think it also sounds like the cookie isn't setting correctly. You can validate by checking for the hp_session or hp_auth cookies in the application storage in DevTools.

@tecosaur
Copy link

@igor-ramazanov if that's the case, it could be helpful if we compared our Headplane/headscale and Authelia configs. My headscale config is unchanged from my last message, and with Authelia I have:

    - access_token_signed_response_alg: none
      authorization_policy: headscale
      client_id: headscale
      client_name: Headscale
      client_secret: $argon2id$v=19$m=65536,t=3,p=4$...
      consent_mode: implicit
      public: false
      redirect_uris:
      - https://headscale.tecosaur.net/oidc/callback
      - https://headscale.tecosaur.net/admin/oidc/callback
      scopes:
      - openid
      - email
      - profile
      - groups
      token_endpoint_auth_method: client_secret_basic
      userinfo_signed_response_alg: none

@tale: I cannot find any indication of those cookies being set if I inspect the site storage. If it's of any use, in the logs I just see:

2025-08-21T17:23:32.163Z [server] INFO: Running Node.js 22.17.0
2025-08-21T17:23:32.180Z [config] INFO: Found a valid configuration file at /etc/headplane/config.yaml
2025-08-21T17:23:32.194Z [config] INFO: Loading a .env file (if available)
[[email protected]] injecting env (0) from .env -- tip: 🔐 prevent committing .env to code: https://dotenvx.com/precommit
2025-08-21T17:23:32.212Z [config] INFO: Found a valid Headscale configuration file at /nix/store/r3cljz68gagl3ss0lij4nq113qvn4kf8-headscale.yaml
2025-08-21T17:23:32.212Z [config] WARN: Headscale configuration file at /nix/store/r3cljz68gagl3ss0lij4nq113qvn4kf8-headscale.yaml is not writable
2025-08-21T17:23:32.225Z [config] ERROR: Error validating Headscale configuration:
2025-08-21T17:23:32.225Z [config] ERROR:  - (0): tls_cert_path must be a string (was null)
2025-08-21T17:23:32.226Z [config] ERROR:  - (1): tls_key_path must be a string (was null)
2025-08-21T17:23:32.227Z [config] INFO: Migrating old user database from /var/lib/headplane/users.json
2025-08-21T17:23:32.227Z [config] INFO: Found old user database file at /var/lib/headplane/users.json
2025-08-21T17:23:32.228Z [config] INFO: Migrating user database to the new SQL database
2025-08-21T17:23:32.228Z [config] INFO: Migrating 1 users from the old database
2025-08-21T17:23:32.238Z [config] INFO: Migrated 1 users successfully
2025-08-21T17:23:32.238Z [config] INFO: Removed old user database file /var/lib/headplane/users.json
2025-08-21T17:23:32.240Z [config] INFO: Using Proc integration
2025-08-21T17:23:32.254Z [config] INFO: Found Headscale process with PID: 1922487
2025-08-21T17:23:32.890Z [config] INFO: Using https://auth.tecosaur.net as the OIDC issuer
2025-08-21T17:23:34.173Z [server] INFO: Running on 0.0.0.0:8175
No routes matched location "/favicon.ico"

@tale
Copy link
Owner Author

tale commented Aug 26, 2025

If the cookies are not being set it is 100% an issue with your reverse proxy or hosting setup. When cookie_secure is set to true you need to be running in HTTPS. The cookie also will only match the /admin path.

@igor-ramazanov
Copy link
Contributor

@tecosaur Here's my authelia config:

access_control:
  default_policy: two_factor
authentication_backend:
  file:
    path: /etc/authelia/users.yaml
    search:
      email: true
default_2fa_method: ''
identity_providers:
  oidc:
    jwks:
      - key: '{{ secret "/run/secrets/authelia/privateKey" | mindent 10 "|" | msquote }}'
    authorization_policies:
      headplane:
        default_policy: deny
        rules:
          - policy: two_factor
            subject:
              - group:admin
    clients:
      - authorization_policy: headplane
        client_id: headplane
        client_name: Headplane
        client_secret: replaceme
        consent_mode: implicit
        redirect_uris:
          - https://headscale.example.org/admin/oidc/callback
      - client_id: headscale
        client_name: Headscale
        client_secret: replaceme
        consent_mode: implicit
        redirect_uris:
          - https://headscale.example.org/oidc/callback
log:
  file_path: null
  format: text
  keep_stdout: false
  level: info
notifier:
  smtp:
    address: smtp://smtp.example.org:587
    password: '{{fileContent "/run/secrets/authelia/smtp"}}'
    sender: Authelia <[email protected]>
    startup_check_address: [email protected]
    username: [email protected]
server:
  address: tcp4://localhost:9091
session:
  cookies:
    - authelia_url: https://login.example.org
      domain: login.example.org
storage:
  local:
    path: /var/lib/authelia-main/db.sqlite3
telemetry:
  metrics:
    address: tcp://localhost:9959
    enabled: false
theme: light

@igor-ramazanov
Copy link
Contributor

@tecosaur Headscale's oidc section:

oidc:
  allowed_domains: []
  allowed_users: []
  client_id: headscale
  client_secret_path: /run/secrets/headscale/oidcClientSecret
  extra_params: {}
  issuer: https://login.example.org
  only_start_if_oidc_is_available: true
  scope:
    - openid
    - profile
    - email

@igor-ramazanov
Copy link
Contributor

@tecosaur Headplane:

headscale:
  config_path: /nix/store/fi7mbrb9glgmy20vrzn8h3k3fd53fs7r-headscale.yml
  config_strict: true
  url: https://headscale.example.org
integration:
  agent:
    cache_path: /var/lib/headplane/agent_cache.json
    cache_ttl: 180000
    enabled: true
    executable_path: /nix/store/x6sy166dzp971419ghv7882ivq21m57k-hp_agent-0.6.1/bin/hp_agent
    host_name: headplane-agent
    pre_authkey_path: /run/secrets/tailscale/authKey
    work_dir: /var/lib/headplane/agent
  proc:
    enabled: true
oidc:
  client_id: headplane
  client_secret_path: /run/secrets/headplane/oidcClientSecret
  disable_api_key_login: true
  headscale_api_key_path: /run/secrets/headscale/accessToken
  issuer: https://login.example.org
  redirect_uri: https://headscale.example.org/admin/oidc/callback
  token_endpoint_auth_method: client_secret_basic
  user_storage_file: /var/lib/headplane/users.json
server:
  cookie_secret_path: /run/secrets/headplane/cookieSecret
  cookie_secure: true
  data_path: /var/lib/headplane
  host: 127.0.0.1
  port: 3000

@igor-ramazanov
Copy link
Contributor

@tecosaur Looks like you need to split up the Authelia client into two separate Authelia clients:

      redirect_uris:
      - https://headscale.tecosaur.net/oidc/callback
      - https://headscale.tecosaur.net/admin/oidc/callback

@tecosaur
Copy link

tecosaur commented Sep 5, 2025

Ah interesting, so you've ended up with different headscale and headplane OIDC auth. Is there a particular reason why you've split them?

@igor-ramazanov
Copy link
Contributor

@tecosaur Not sure if what's the correct way of doing it, but I assumed that's how it should be done in the first place and that it won't work otherwise. I haven't tried reusing the same client for both.

@igor-ramazanov
Copy link
Contributor

@tale

Wdym hangs, is there a connection established? Sort of surprised this stuff is happening, is your reverse proxy stripping the mime type?

So, the issue was in Nginx, it didn't serve the /assets/..., so it couldn't download the necessary *.wasm file. Didn't find /assets/... mentions in reverse-proxy docs.

Added a valid Nginx' location directive, still doesn't connect, but it's another problem.

Devtools logs from web ssh connection page:

2025/09/06 17:05:49 Attempting SSH dial to host: <REDACTED: INSTANCE NAME>:22 console-BYUlunva.js:2:27
2025/09/06 17:05:49 wgengine: Reconfig: configuring userspace WireGuard config (with 1/22 peers) console-BYUlunva.js:2:27
2025/09/06 17:05:49 wg: [v2] [C41WV] - UAPI: Created console-BYUlunva.js:2:27
2025/09/06 17:05:49 wg: [v2] [C41WV] - UAPI: Updating endpoint console-BYUlunva.js:2:27
2025/09/06 17:05:49 [unexpected] magicsock: ParseEndpoint: unknown node key=[C41WV] console-BYUlunva.js:2:27
2025/09/06 17:05:49 wg: IPC error 2: failed to set endpoint <REDACTED: BASE32 NODE_KEY>: magicsock: ParseEndpoint: unknown peer "[C41WV]" console-BYUlunva.js:2:27
2025/09/06 17:05:49 wgcfg.Reconfig failed: multiple errors:
	IPC error 2: failed to set endpoint <REDACTED: BASE32 NODE_KEY>: magicsock: ParseEndpoint: unknown peer "[C41WV]"
	ToUAPI: io: read/write on closed pipe console-BYUlunva.js:2:27
2025/09/06 17:05:49 wgdev.Reconfig: multiple errors:
	IPC error 2: failed to set endpoint <REDACTED: BASE32 NODE_KEY>: magicsock: ParseEndpoint: unknown peer "[C41WV]"
	ToUAPI: io: read/write on closed pipe console-BYUlunva.js:2:27
Terminal resized to 238x23 console-BYUlunva.js:51:2099
2025/09/06 17:05:49 control: [v1] successful lite map update in 51ms console-BYUlunva.js:2:27
2025/09/06 17:05:49 control: [v1] PollNetMap: stream=false ep=[[fe80:123:456:789::1]:12345] console-BYUlunva.js:2:27
2025/09/06 17:05:49 health(warnable=no-derp-connection): ok console-BYUlunva.js:2:27
2025/09/06 17:05:49 NOTIFY: Notify{Health{...}} console-BYUlunva.js:2:27
2025/09/06 17:05:49 websocket: connected to https://headplane.example.org/derp console-BYUlunva.js:2:27
2025/09/06 17:05:49 control: [v1] successful lite map update in 24ms console-BYUlunva.js:2:27
2025/09/06 17:05:49 health(warnable=no-derp-connection): ok console-BYUlunva.js:2:27
2025/09/06 17:05:49 NOTIFY: Notify{Health{...}} console-BYUlunva.js:2:27
2025/09/06 17:05:49 magicsock: derp-999 connected; connGen=1 console-BYUlunva.js:2:27
2025/09/06 17:05:49 health(warnable=no-derp-connection): ok 2 console-BYUlunva.js:2:27
2025/09/06 17:05:49 NOTIFY: Notify{Health{...}} 2 console-BYUlunva.js:2:27
2025/09/06 17:05:49 netcheck: [v1] report: udp=true v4=false v6=false v6os=false mapvarydest= portmap=? derp=999 derpdist= console-BYUlunva.js:2:27
2025/09/06 17:05:49 control: [v�JSON]1{"controltime":"2025-09-06T17:05:49.404302014Z"} console-BYUlunva.js:2:27
2025/09/06 17:05:49 control: [v1] mapRoutine: netmap received: state:synchronized console-BYUlunva.js:2:27
2025/09/06 17:05:49 control: [v1] sendStatus: mapRoutine-got-netmap: state:synchronized console-BYUlunva.js:2:27
2025/09/06 17:05:49 network-lock unavailable; no state directory console-BYUlunva.js:2:27
2025/09/06 17:05:49 [v1] netmap diff: (none) console-BYUlunva.js:2:27
2025/09/06 17:05:49 [v1] wgengine: Reconfig done console-BYUlunva.js:2:27
2025/09/06 17:05:49 [v1] authReconfig: ra=false dns=false 0x00: <nil> console-BYUlunva.js:2:27
2025/09/06 17:05:49 [v1] initPeerAPIListener: entered console-BYUlunva.js:2:27
2025/09/06 17:05:49 NOTIFY: Notify{NetMap{...}} console-BYUlunva.js:2:27
2025/09/06 17:05:49 [v1] initPeerAPIListener: 2 netmap addresses match existing listeners console-BYUlunva.js:2:27
2025/09/06 17:05:51 control: [v�JSON]1{"controltime":"2025-09-06T17:05:51.809623553Z"} console-BYUlunva.js:2:27
2025/09/06 17:05:51 control: [v1] mapRoutine: netmap received: state:synchronized console-BYUlunva.js:2:27
2025/09/06 17:05:51 control: [v1] sendStatus: mapRoutine-got-netmap: state:synchronized console-BYUlunva.js:2:27
2025/09/06 17:05:51 network-lock unavailable; no state directory console-BYUlunva.js:2:27
2025/09/06 17:05:51 [v1] netmap diff: (none) console-BYUlunva.js:2:27
2025/09/06 17:05:51 [v1] wgengine: Reconfig done console-BYUlunva.js:2:27
2025/09/06 17:05:51 [v1] authReconfig: ra=false dns=false 0x00: <nil> console-BYUlunva.js:2:27
2025/09/06 17:05:51 [v1] initPeerAPIListener: entered console-BYUlunva.js:2:27
2025/09/06 17:05:51 [v1] initPeerAPIListener: 2 netmap addresses match existing listeners console-BYUlunva.js:2:27
2025/09/06 17:05:51 NOTIFY: Notify{NetMap{...}} console-BYUlunva.js:2:27
2025/09/06 17:05:56 control: [v�JSON]1{"controltime":"2025-09-06T17:05:56.60222517Z"} console-BYUlunva.js:2:27
2025/09/06 17:05:56 control: [v1] mapRoutine: netmap received: state:synchronized console-BYUlunva.js:2:27
2025/09/06 17:05:56 control: [v1] sendStatus: mapRoutine-got-netmap: state:synchronized console-BYUlunva.js:2:27
2025/09/06 17:05:56 network-lock unavailable; no state directory console-BYUlunva.js:2:27
2025/09/06 17:05:56 [v1] netmap diff: (none) console-BYUlunva.js:2:27
2025/09/06 17:05:56 [v1] wgengine: Reconfig done console-BYUlunva.js:2:27
2025/09/06 17:05:56 [v1] authReconfig: ra=false dns=false 0x00: <nil> console-BYUlunva.js:2:27
2025/09/06 17:05:56 [v1] initPeerAPIListener: entered console-BYUlunva.js:2:27
2025/09/06 17:05:56 [v1] initPeerAPIListener: 2 netmap addresses match existing listeners console-BYUlunva.js:2:27
2025/09/06 17:05:56 NOTIFY: Notify{NetMap{...}} console-BYUlunva.js:2:27
2025/09/06 17:05:57 control: [v�JSON]1{"controltime":"2025-09-06T17:05:57.404758819Z"} console-BYUlunva.js:2:27
2025/09/06 17:05:57 control: [v1] mapRoutine: netmap received: state:synchronized console-BYUlunva.js:2:27
2025/09/06 17:05:57 control: [v1] sendStatus: mapRoutine-got-netmap: state:synchronized console-BYUlunva.js:2:27
2025/09/06 17:05:57 network-lock unavailable; no state directory console-BYUlunva.js:2:27
2025/09/06 17:05:57 [v1] netmap diff: (none) console-BYUlunva.js:2:27
2025/09/06 17:05:57 [v1] wgengine: Reconfig done console-BYUlunva.js:2:27
2025/09/06 17:05:57 [v1] authReconfig: ra=false dns=false 0x00: <nil> console-BYUlunva.js:2:27
2025/09/06 17:05:57 [v1] initPeerAPIListener: entered console-BYUlunva.js:2:27
2025/09/06 17:05:57 [v1] initPeerAPIListener: 2 netmap addresses match existing listeners console-BYUlunva.js:2:27
2025/09/06 17:05:57 NOTIFY: Notify{NetMap{...}}

@kaanaldemir
Copy link

kaanaldemir commented Sep 23, 2025

@tale I'm on 0.6.1 and when I removed all the nameservers while override dns servers toggle was still on, headplane stopped responding (probably more than just headplane) until I manually set it to false via file editing.

@StealthBadger747
Copy link
Contributor

@igor-ramazanov #319 should fix your issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed pre release Next release to cut
Projects
None yet
7 participants