When you SO_MARK a socket to get it to use a different routing table, it doesn't currently reset the dst so until something else triggers that dst reset the new routing table is not used for the connection, this will force a route lookup next time you try to send something

Author: crucially

Diff for: net/core/sock.c

@@ -918,8 +918,10 @@ int sock_setsockopt(struct socket *sock, int level, int optname,
 	case SO_MARK:
 		if (!ns_capable(sock_net(sk)->user_ns, CAP_NET_ADMIN))
 			ret = -EPERM;
-		else
+		else {
 			sk->sk_mark = val;
+			sk_dst_reset(sk);
+		}
 		break;
 
 		/* We implement the SO_SNDLOWAT etc to