EVPN+M-LAG group VLAN Underlay Communication Failure

Devices and Versions: S6850 R6616P01

Networking: The diagram below shows the networking setup. Two S6850 switches are configured with DRNI. The upstream connection is connected to the spine device, and the downstream DR ports are connected to the access devices.

In an on-site EVPN+DRNI network, after adding a new business VXLAN 10, it was discovered that the underlay communication between the two DRNI devices was not working. Checking both DR devices, there were no ARP entries for the peer VLAN interface 11.

The DR devices did not learn the ARP entries from each other. Packet capture on the IPP port revealed that the devices sent out ARP request packets but did not receive any reply packets.

So why didn't they receive the ARP reply packets?

In an EVPN+DRNI network, when a user-side interface creates an Ethernet service instance and associates it with a corresponding VSI, the Leaf device will automatically create a dynamic AC on the IPL with the same packet matching rules and associate it with the same VSI. This allows packets to be bypassed through the IPL for link backup. Additionally, to prevent conflicts between dynamically created ACs on the IPL due to the same packet matching rules on different user-side interfaces, the "l2vpn drni peer-link ac-match-rule vxlan-mapping" command needs to be configured on the device.

After configuring the "l2vpn drni peer-link ac-match-rule vxlan-mapping" command, the AC packet matching rules on the IPL will be generated based on the VXLAN ID mapping. When a VXLAN is created on the VTEP, the IPL will automatically generate an AC associated with the corresponding VSI. The packet matching rules for this AC are as follows:

The packet matching rules are:

· The outer VLAN tag (s-vid) is obtained by dividing the VXLAN ID by 4094, rounding up, and adding 1, i.e., VXLAN ID/4094+1.

· The inner VLAN tag (c-vid) is obtained by dividing the VXLAN ID by 4094, taking the remainder, and adding 1, i.e., VXLAN ID%4094+1.

· If the calculated outer VLAN tag is the PVID of the IPP port, then the s-vid is changed to VXLAN ID%4094+1, and there is no c-vid.

Note: After executing this command, the ID of the VXLAN created on the VTEP cannot exceed 16000000.

If the "l2vpn drni peer-link ac-match-rule vxlan-mapping" command is not configured, there will be an error when configuring service instance matching rules on user-side interfaces with the same configuration.

PS: You can review the above knowledge points by clicking on "Exchange Matters | Technical Special Topic - DRNI+EVPN Networking Advanced Introduction".

A new VXLAN 10 was added to the live network. According to the above packet matching rules, the calculated outer VLAN tag (s-vid) should be 10/4094+1=1. However, the PVID of the IPP port on-site has not been modified and remains the default value of 1. As a result, the calculated outer VLAN tag is the same as the PVID. According to the third rule, the s-vid will be changed to 10%4094+1=11, which means the AC packet matching rule on the IPL is s-vid=11. Therefore, when a packet with VLAN tag 11 passes through the IPP port, it will match the rule with s-vid=11 and go through the overlay, resulting in underlay communication failure.

Checking the automatically created dynamic AC information on the IPP port of the two DR devices, the s-vid of the IPP port is indeed 11. When an ARP packet with VLAN tag 11 arrives at the device through the IPL link, it matches the rule with s-vid=11 and goes through the overlay. Therefore, no ARP reply packet is sent, resulting in the on-site failure.

We can modify the PVID of the IPP port to avoid having the calculated s-vid the same as the PVID. Considering that after executing the "l2vpn drni peer-link ac-match-rule vxlan-mapping" command, the ID of the VXLAN created on the VTEP cannot exceed 16000000 (i.e., the maximum s-vid is 16000000/4094+1=3909), we can set the PVID of the IPP port to 4094 to avoid underlay packet matching with s-vid when the PVID is the same as the outer VLAN tag.