S7503X keepalive keeps up/down in VXLAN network

Spine (S10506 configured DHCP relay) ------ Leaf (S7503X) -------Access -------Client

Unstandard ADCampus VXLAN network, because DHCP relay on Spine, not on leaf.

S7503X keepalive state keeps up/down

1. Check the customer site, we found that client can not get an IP address. Spine was configured dhcp relay. Leaf was configured dhcp snooping on it.

2. There are too many rxtx softcar drop log about DHCP packet, and S7503X keepalive keeps up/down. 

3. We thought the issue may related on DHCP, so we try to undo dhcp snooping trust of VXLAN 4094 on Leaf, and the issue was resolved.

4. The source client of the dhcp packet is only access via vlan 4094, and debug DHCP had nak packets

Message type: REQUEST (1)

    Hardware type: 1, Hardware address length: 6

    Hops: 0, Transaction ID: 2273587435

    Seconds: 0, Broadcast flag: 0

    Client IP address: 0.0.0.0   Your IP address: 0.0.0.0

    Server IP address: 0.0.0.0   Relay agent IP address: 0.0.0.0

    Client hardware address: c465-1690-2cb3

    Server host name: not configured

    Boot file name: not configured

    DHCP message type: DHCPREQUEST (3)

*Jul 10 22:28:12:146 2023 BJ-CD-5-leaf-BGW-2 DHCPS/7/EVENT: -MDC=1; Receive a DHCPREQUEST message for 172.16.212.8 from Vsi-interface4094; The server identifier is (none).

*Jul 10 22:28:12:146 2023 BJ-CD-5-leaf-BGW-2 DHCPS/7/PACKET: -MDC=1; To 255.255.255.255 port 68, interface Vsi-interface4094

    Message type: REPLY (2)

    Hardware type: 1, Hardware address length: 6

    Hops: 0, Transaction ID: 2273587435

    Seconds: 0, Broadcast flag: 1

    Client IP address: 0.0.0.0   Your IP address: 0.0.0.0

    Server IP address: 0.0.0.0   Relay agent IP address: 0.0.0.0

    Client hardware address: c465-1690-2cb3

    Server host name: not configured

    Boot file name: not configured

    DHCP message type: DHCPNAK (6)

5. Trouble shooting:

Client enter VXLAN 4094, but there is a DHCP Snooping trust config on VXLAN 4094 interface. So DHCP packets will be broadcasted to other Leaf through VXLAN tunnel, and other leaf will send the DHCP packets back to this Leaf, which lead to a loop in the network.

Tips: VXLAN 4094 is used for management devices by Controller, do not use it to work for business traffic.

Work around:

undo dhcp snooping trust of VXLAN 4094

Solution:

Using other VXLAN for service traffic. Don't using VXLAN 4094, which is used for management.