V2 certificate format #1216

nbrownus · 2024-09-13T04:05:57Z

This is a near complete implementation of an ipv6 enabled overlay network. There are a handful of pull requests targeting this branch in addition to a number of incomplete issues within this PR.

I have tried to highlight the main trouble spots with comments on the PR and in addition here are my internal notes:

nebula-cert is defaulting to mint both v1 and v2 certificates and nebula config is defaulting to transmit v1 certificates if both are present. This will lead to a situation where net new adopters will be stuck using v1 certificates. Should this be the default behavior?
Currently nebula wont allow you to remove a v1 certificate on reload but this means a restart is required to complete a migration to v2. A comment on this review highlights this, we should allow it.
Currently there are a few spots (control, ssh) where we return the certificate in use but they only return 1 cert, should we leave it returning the default or force folks to consume both? How do we indicate which one is default?
Every time we encounter a certificate we need to sort the networks as well as only record the union of applicable addresses. See [cert-v2] punchy-respond on an address in common with the querying host #1261
We need to take extra care to ensure we never allow a ipv4 mapped ipv6 address. This is not currently addressed in this PR.
There may be an opportunity to improve the lighthouse protocol by directly using the MarshalBinary and UnmarshalBinary functions for netip objects. This is more of a nit than anything but we should agree on this before merging.
nebula-cert sign help text needs to talk about primary vpn address (or the union of effective addresses) being the first one after being sorted and what that means for tunnels.
The firewall config still uses naming like ca-sha should we alias this to ca-fingerprint to normalize the verbiage?
The firewall config still uses naming like ip should we alias this to network to normalize the verbiage
Normalize all names for Marshal* and Unmarshal*, we have some From*/To* and others without From/To.
Should we use this moment to swap p256 keys to use their compressed form? Saves ~32 bytes.

Once this is merged the upgrade path should likely be:

Give every host dual certs and leave the default pki.default_version to 1. Everything continues to use v1 protocols.
Switch all am_relay: true hosts pki.default_version to 2. Relays can now initiate v2 tunnels, but this is an uncommon flow.
Switch all non lighthouse hosts pki.default_version to 2. Lighthouses will reply to v2 hosts using v2 protocols.
Switch all lighthouses pki.default_version to 2.
Remove v1 certs from all hosts

cert/cert_v2.asn1

cert/cert_v2.go

cert/cert_v2.asn1

nebula.proto

cert/pem.go

cert/cert_v2.asn1

lighthouse.go

overlay/tun_linux.go

nbrownus · 2024-10-17T01:35:04Z

udp/udp_rio_windows.go

-			q,
-			cache.Get(u.l),
-		)
+		r(netip.AddrPortFrom(netip.AddrFrom16(rua.Addr).Unmap(), (rua.Port>>8)|((rua.Port&0xff)<<8)), buffer[:n])


use binary to avoid endianness issues

nbrownus · 2024-10-17T04:55:58Z

cmd/nebula-cert/sign.go

-	}
+	if *sf.networks != "" {
+		for _, rs := range strings.Split(*sf.networks, ",") {
+			//TODO: error on duplicates? Mainly only addr matters, having two of the same addr in the same or different prefix space is strange


Also no 4in6

Addressed in #1266

nbrownus · 2024-10-17T04:56:54Z

cmd/nebula-cert/sign.go

 				if err != nil {
-					return newHelpErrorf("invalid subnet definition: %s", rs)
+					return newHelpErrorf("invalid -unsafe-networks definition: %s", rs)


And no 4in6

nbrownus · 2024-10-17T04:57:28Z

cmd/nebula-cert/sign.go

-			return fmt.Errorf("error while signing with PKCS#11: %w", err)
+
+		if version == cert.Version1 {
+			// If we are asked to mint a v1 certificate only then we cant just ignore any v6 addresses


We arent ignoring them

--------- Co-authored-by: Jack Doan <[email protected]>

cert/cert_v1.go

cert/cert_v2.go

nbrownus · 2024-10-29T03:06:10Z

overlay/tun_darwin.go

+			Vltime: 0xffffffff,
+			Pltime: 0xffffffff,
+		},
+		//TODO: should we disable DAD (duplicate address detection) and mark this as a secured address?


Resolve TODO

at least on Linux, my unsubstantiated opinion is that nebula should not do this, but our docs should suggest appropriate sysctls and systemd-networkd settings.

nbrownus · 2024-10-29T03:07:14Z

overlay/tun_linux.go

+
+	//add all new addresses
+	for i := range newAddrs {
+		//todo do we want to stack errors and try as many ops as possible?


Resolve TODO

overlay/tun_linux.go

nbrownus · 2024-10-29T03:10:36Z

overlay/tun_netbsd.go

@@ -213,7 +223,8 @@ func (t *tun) removeRoutes(routes []Route) error {
 			continue
 		}

-		cmd := exec.Command("/sbin/route", "-n", "delete", "-net", r.Cidr.String(), t.cidr.Addr().String())
+		//todo is this right?


Resolve TODO

nbrownus · 2024-10-29T03:11:04Z

overlay/tun_openbsd.go

@@ -181,8 +191,8 @@ func (t *tun) removeRoutes(routes []Route) error {
 		if !r.Install {
 			continue
 		}
-
-		cmd := exec.Command("/sbin/route", "-n", "delete", "-inet", r.Cidr.String(), t.cidr.Addr().String())
+		//todo is this right?


Resolve TODO

nbrownus · 2024-10-29T03:11:43Z

ssh.go

@@ -785,7 +785,8 @@ func sshPrintCert(ifce *Interface, fs interface{}, a []string, w sshd.StringWrit
 		return nil
 	}

-	cert := ifce.pki.GetCertState().Certificate
+	//TODO: This should return both certs


Resolve TODO

nbrownus · 2024-10-29T03:12:40Z

relay_manager.go

@@ -91,40 +92,60 @@ func AddRelay(l *logrus.Logger, relayHostInfo *HostInfo, hm *HostMap, vpnIp neti
 func (rm *relayManager) EstablishRelay(relayHostInfo *HostInfo, m *NebulaControl) (*Relay, error) {
 	relay, ok := relayHostInfo.relayState.CompleteRelayByIdx(m.InitiatorRelayIndex, m.ResponderRelayIndex)
 	if !ok {
-		rm.l.WithFields(logrus.Fields{"relay": relayHostInfo.vpnIp,
+		//TODO: we need to handle possibly logging deprecated fields as well


Resolve TODO

nbrownus · 2024-10-29T03:12:57Z

relay_manager.go

+	if msg.OldRelayFromAddr > 0 || msg.OldRelayToAddr > 0 {
+		v = cert.Version1
+
+		//TODO: yeah this is junk but maybe its less junky than the other options


Resolve TODO

nbrownus · 2024-10-29T03:13:16Z

relay_manager.go

+		if v == cert.Version1 {
+			peer := peerHostInfo.vpnAddrs[0]
+			if !peer.Is4() {
+				//TODO: log cant do it


Resolve TODO

nbrownus · 2024-10-29T03:13:46Z

relay_manager.go

@@ -310,11 +357,11 @@ func (rm *relayManager) handleCreateRelayRequest(h *HostInfo, f *Interface, m *N
 				f.SendMessageToHostInfo(header.Control, 0, peer, msg, make([]byte, 12), make([]byte, mtu))
 				rm.l.WithFields(logrus.Fields{
 					//TODO: IPV6-WORK another lazy used to use the req object


Resolve TODO

nbrownus · 2024-10-29T03:14:02Z

relay_manager.go

+
+				if v == cert.Version1 {
+					if !h.vpnAddrs[0].Is4() {
+						//TODO: log it


Resolve TODO

nbrownus · 2024-10-29T03:14:16Z

relay_manager.go

@@ -360,11 +419,11 @@ func (rm *relayManager) handleCreateRelayRequest(h *HostInfo, f *Interface, m *N
 					f.SendMessageToHostInfo(header.Control, 0, h, msg, make([]byte, 12), make([]byte, mtu))
 					rm.l.WithFields(logrus.Fields{
 						//TODO: IPV6-WORK more lazy, used to use resp object


Resolve TODO

nbrownus · 2024-10-29T03:14:39Z

pki.go

 	return p.cs.Load()
 }

-func (p *PKI) GetCAPool() *cert.CAPool {
-	return p.caPool.Load()
+// TODO: We should remove this


Resolve TODO

nbrownus · 2024-10-29T03:14:47Z

pki.go

+	return p.cs.Load().GetDefaultCertificate()
+}
+
+// TODO: We should remove this


Resolve TODO

nbrownus · 2024-10-29T03:15:03Z

pki.go

+			}
+
+		} else if currentState.v1Cert != nil {
+			//TODO: we should be able to tear this down


Resolve TODO

nbrownus · 2024-10-29T03:15:21Z

pki.go

-	p.cs.Store(cs)
+	p.cs.Store(newState)
+
+	//TODO: newState needs a stringer that does json


Resolve TODO

pki.go

nbrownus · 2024-10-29T03:16:02Z

pki.go

+			return nil, util.NewContextualError("v1 and v2 curve are not the same, ignoring", nil, nil)
+		}
+
+		//TODO: make sure v2 has v1s address


Resolve TODO

overlay/tun_windows.go

…didn't work on the first try (#1268)

…st (#1261)

nbrownus · 2024-11-13T03:16:13Z

lighthouse.go

+			if ok {
+				whereToPunch = newDest
+			} else {
+				//TODO this means the destination will have no addresses in common with the punch-ee


salesforce-cla bot added the cla:signed label Sep 13, 2024

johnmaguire reviewed Sep 13, 2024

View reviewed changes

cert/cert_v2.asn1 Show resolved Hide resolved

JackDoanRivian reviewed Sep 13, 2024

View reviewed changes

cert/cert_v2.go Show resolved Hide resolved

JackDoanRivian reviewed Sep 16, 2024

View reviewed changes

cert/cert_v2.asn1 Outdated Show resolved Hide resolved

JackDoanRivian reviewed Sep 16, 2024

View reviewed changes

nebula.proto Show resolved Hide resolved

JackDoanRivian reviewed Sep 16, 2024

View reviewed changes

cert/pem.go Show resolved Hide resolved

JackDoanRivian reviewed Sep 20, 2024

View reviewed changes

cert/cert_v2.asn1 Show resolved Hide resolved

JackDoanRivian reviewed Sep 21, 2024

View reviewed changes

lighthouse.go Outdated Show resolved Hide resolved

JackDoanRivian reviewed Sep 21, 2024

View reviewed changes

overlay/tun_linux.go Show resolved Hide resolved

nbrownus mentioned this pull request Oct 4, 2024

Cert interface #1212

Merged

nbrownus force-pushed the cert-v2 branch from 9d15808 to f1fca4d Compare October 8, 2024 03:40

Base automatically changed from cert-interface to master October 10, 2024 23:00

nbrownus force-pushed the cert-v2 branch from c4ef068 to d6f1b51 Compare October 11, 2024 21:44

nbrownus commented Oct 17, 2024

View reviewed changes

nbrownus force-pushed the cert-v2 branch 3 times, most recently from 9f75b80 to a09f39f Compare October 24, 2024 03:17

Support for ipv6 in the overlay with v2 certificates

f2c3242

--------- Co-authored-by: Jack Doan <[email protected]>

nbrownus force-pushed the cert-v2 branch from a09f39f to f2c3242 Compare October 24, 2024 03:25