-
Notifications
You must be signed in to change notification settings - Fork 0
Pipeline Debugging (DPIPE)
While performing the hardware offloading process, much of the hardware specifics cannot be presented. Those details can be useful for debugging and should be exposed in a standardized way. This section provides a description of the DPIPE's (Pipeline Debug) model which is used in order to provide visibility into the offloading process.
The hardware’s pipeline is modeled as graph of match/action tables, where each table represents a specific hardware block. This model is not new and was first introduced by the P4 language, yet here it is used for visibility and not for configuration. The basic blocks of the model will be discussed in the following sections.
In a similar way to P4 the headers/fields are used to describe the table behavior. The headers can be observed per device:
$ devlink dpipe header show pci/0000:03:00.0
pci/0000:03:00.0:
name mlxsw_meta
field:
name erif_port bitwidth 32 mapping_type ifindex
name l3_forward bitwidth 1
name l3_drop bitwidth 1
name adj_index bitwidth 32
name adj_hash_index bitwidth 32
name ipv6
field:
name destination ip bitwidth 128
name ipv4
field:
name destination ip bitwidth 32
name ethernet
field:
name destination mac bitwidth 48
Each driver can introduce special metadata headers as well as standard protocol headers like Ethernet. Metadata fields may be mapped to kernel objects. For example, the RIF's (Router Interface) index (erif_index), which is a driver's internal object index, is logically mapped to a netdevice's ifindex.
The table is an abstraction of a hardware process which can be described by a match/action behavior. In order to observe a specific table:
$ devlink dpipe table show pci/0000:03:00.0 name mlxsw_host6
pci/0000:03:00.0:
name mlxsw_host6 size 0 counters_enabled false
match:
type field_exact header mlxsw_meta field erif_port mapping ifindex
type field_exact header ipv6 field destination ip
action:
type field_modify header ethernet field destination mac
In case no name is specified all the device's tables are provided. The above described table is an IPv6 host table, which is a hash table. It does match on the output RIF index and destination IPv6 address and modifies the destination mac address. The user can furthermore enable/disable counters by:
$ devlink dpipe table set pci/0000:03:00.0 name mlxsw_host6 counters enable
The entries are the content of the table. They can be obtained by:
$ devlink dpipe table dump pci/0000:03:00.0 name mlxsw_host6
pci/0000:03:00.0:
index 0 counter 123045
match_value:
type field_exact header mlxsw_meta field erif_port mapping ifindex mapping_value 224 value 0
type field_exact header ipv6 field destination ip value 2001:db81::2
action_value:
type field_modify header ethernet field destination mac value e4:1d:2d:a5:f1:7e
index 1 counter 0
match_value:
type field_exact header mlxsw_meta field erif_port mapping ifindex mapping_value 223 value 1
type field_exact header ipv6 field destination ip value 2001:db82::2
action_value:
type field_modify header ethernet field destination mac value e4:1d:2d:a5:f0:4a
index 2 counter 1253
match_value:
type field_exact header mlxsw_meta field erif_port mapping ifindex mapping_value 225 value 2
type field_exact header ipv6 field destination ip value 2001:db83::2
action_value:
type field_modify header ethernet field destination mac value e4:1d:2d:a5:f0:4b
This sections covers the available tables for the Spectrum ASIC. For more detailed description of the tables, please refer to DPIPE's Netdev2.1 conference paper.
The ERIF (Egress Router Interface) table controls the output of the router. It contains the L3 statistics for each RIF. Example of a dump:
$ devlink dpipe table dump pci/0000:03:00.0 name mlxsw_erif
pci/0000:03:00.0:
index 0
match_value:
type field_exact header mlxsw_meta field erif_port mapping ifindex mapping_value 704 value 0
action_value:
type field_modify header mlxsw_meta field l3_forward value 1
index 1
match_value:
type field_exact header mlxsw_meta field erif_port mapping ifindex mapping_value 703 value 1
action_value:
type field_modify header mlxsw_meta field l3_forward value 1
Currently only statistics regarding forwarded packets are available.
The IPv4 host table is a hash table that matches on the IPv4 destination address and egress RIF and sets the destination MAC address. Example of a dump:
$ devlink dpipe table dump pci/0000:03:00.0 name mlxsw_host4
pci/0000:03:00.0:
index 0 counter 156
match_value:
type field_exact header mlxsw_meta field erif_port mapping ifindex mapping_value 224 value 0
type field_exact header ipv4 field destination ip value 10.0.0.2
action_value:
type field_modify header ethernet field destination mac value e4:1d:2d:a5:f1:7e
index 1 counter 5896
match_value:
type field_exact header mlxsw_meta field erif_port mapping ifindex mapping_value 223 value 1
type field_exact header ipv4 field destination ip value 10.0.0.2
action_value:
type field_modify header ethernet field destination mac value e4:1d:2d:a5:f0:4a
The IPv6 host table is very similar the IPv4 one. The only difference is that it matches on the IPv6 destination address. Example of a dump:
$ devlink dpipe table dump pci/0000:03:00.0 name mlxsw_host6
pci/0000:03:00.0:
index 0 counter 123045
match_value:
type field_exact header mlxsw_meta field erif_port mapping ifindex mapping_value 224 value 0
type field_exact header ipv6 field destination ip value 2001:db81::2
action_value:
type field_modify header ethernet field destination mac value e4:1d:2d:a5:f1:7e
index 1 counter 0
match_value:
type field_exact header mlxsw_meta field erif_port mapping ifindex mapping_value 223 value 1
type field_exact header ipv6 field destination ip value 2001:db82::2
action_value:
type field_modify header ethernet field destination mac value e4:1d:2d:a5:f0:4a
The adjacency table is used in case of a remote route. This table performs direct lookup for the nexthop group. The local offset of the nexthop is calculated based on a per-packet hash value. Then, the table sets the egress RIF and destination MAC of the packet. Example of a dump:
$ devlink dpipe table dump pci/0000:03:00.0 name mlxsw_adj
pci/0000:03:00.0:
index 0 counter 8247263
match_value:
type field_exact header mlxsw_meta field adj_index value 65536
type field_exact header mlxsw_meta field adj_size value 512
type field_exact header mlxsw_meta field adj_hash_index value 0
action_value:
type field_modify header ethernet field destination mac value e4:1d:2d:a5:f3:64
type field_modify header mlxsw_meta field erif_port mapping ifindex mapping_value 3 value 1
index 1 counter 81135
match_value:
type field_exact header mlxsw_meta field adj_index value 65536
type field_exact header mlxsw_meta field adj_size value 512
type field_exact header mlxsw_meta field adj_hash_index value 507
action_value:
type field_modify header ethernet field destination mac value e4:1d:2d:a5:f3:65
type field_modify header mlxsw_meta field erif_port mapping ifindex mapping_value 4 value 2
The above example shows configuration with one adjacency group which contains two nexthops. The first nexthop occupies 507 adjacency entries whereas the second one occupies the remaining 5 entries. This is because the nexthops were configured with weights of 100 to 1.
Note: The adjacency group's size (adj_size) match parameter had
been added in Linux 4.15.
For further information and examples please refer to the DPIPE's Netdev2.1 conference paper.
Installation
System Maintenance
Network Interface Configuration
- Switch Port Configuration
- Persistent Configuration
- Quality of Service
- Queues Management
- Port Mirroring
- ACLs
- OVS
- Resource Management
Layer 2
Layer 3
Debugging