Traffic seems to break when it has to go between FRRs

[PrimeYeti]

Description

Current setup overview:

• Juniper router (Core) that connects to
  • 2x Mikrotik distribution switches
    • Each of which then connects to 2x Dell servers that host
      • an instance of FRR per server.

I am hoping that I am asking this in the right place.

The current problem I am having is that when traffic has to route via one FRR to the next FRR to get to its destination it seems to break. For example if I attempt to SSH into FRR1 and the route it takes is via FRR2, I can't connect to FRR1. I can see the messages on a TCPDump on both FRRs but it never negotiates. I can also never SSH into FRR2 as the route it takes always ends up going via FRR1 first. If I disable one of the FRRs I can get to the other one perfectly fine.

Each FRR has 3 interfaces:

• Public Management IP on 195.a.a.a/28 - ens18 (MTU 1578) that runs OSPF/OSPF6 and connects to the Mikrotik switches and the other FRR
• Local service access IP on 10.a.a.a/24 - ens19 (MTU 1450)
• Public service access IP on 149.a.a.a/28 - ens20 (MTU 1450) runs OSPF/OSPF6 but configured as passive

FRR config is below. Both FRRs are the same except with different router IDs and hostnames:
frr version 8.4.4
frr defaults traditional
hostname FRR1
service integrated-vtysh-config
log file /var/log/frr/ospfd.log debugging
!
interface ens18
ip ospf message-digest-key 1 md5 *****
ip ospf area 0.0.0.0
ipv6 ospf6 area 0.0.0.0
exit
!
interface ens20
ip ospf area 0.0.0.0
ip ospf passive
ipv6 ospf6 area 0.0.0.0
ipv6 ospf6 passive
exit
!
router ospf
ospf router-id 195.a.a.a
area 0.0.0.0 authentication message-digest
exit
!
router ospf6
ospf6 router-id 195.a.a.a
exit
!

The VMs are hosted on Proxmox and I am using their built-in SDN VXLAN facility for the 10.a.a.a and 149.a.a.a subnets.

I have checked the routing table and both FRRs use their connected route to the 149.a.a.a subnet.

Version

8.4.4

How to reproduce

Attempt to get to a destination that has to route via two FRRs to get there.

Expected behavior

I can reach any VM regardless of what route is being taken.

Actual behavior

See above.

Additional context

No response

Checklist

• I have searched the open issues for this bug.
• I have not included sensitive information in this report.

[donaldsharp]

can we get the output of show zebra?

[PrimeYeti]

Please see below from FRR1.

_OS Linux(6.1.0-27-amd64)
ECMP Maximum 256
v4 Forwarding On
v6 Forwarding On
MPLS Off
EVPN Off
Kernel socket buffer size 90000000
VRF l3mdev Available
ASIC offload Unavailable
RA Compiled in
RFC 5549 BGP is not using
Kernel NHG Available
Allow Non FRR route deletion No
v4 All LinkDown Routes Off
v4 Default LinkDown Routes Off
v6 All LinkDown Routes Off
v6 Default LinkDown Routes Off
v4 All MC Forwarding Off
v4 Default MC Forwarding Off
v6 All MC Forwarding Off
v6 Default MC Forwarding Off

                        Route      Route      Neighbor   LSP        LSP

VRF Installs Removals Updates Installs Removals
default 34376 20864 0 0 0_

[donaldsharp]

and a show ip route

[PrimeYeti]

Is there anything in particular you want to see from there? I only ask as there are thousands of routes

[PrimeYeti]

Is anyone able to assist here, please?