VXLAN Routing In and Out of Tunnels (RIOT)

VXLAN Routing In and Out of Tunnels (RIOT) allows hosts in different VXLAN Network Identifiers (VNIs) to communicate by performing Layer 3 routing on the inner packets at the Virtual Tunnel Endpoint (VTEP). Packets are routed between VXLAN segments without requiring a separate physical router.

Only unicast routing is supported with VXLAN RIOT.

Enable VXLAN RIOT before configuring a VE interface on a VLAN that is mapped to a VNI (overlay VE interface).

The overlay VE interface must belong to a non-default user Virtual Routing and Forwarding (VRF) instance.

ARP (broadcast and unicast) and OSPF (multicast) packets received over the tunnel are processed and mapped to an overlay VE interface.

Protocol packets are copied to the CPU; data packets from access and tunnel ports are Layer 3 routed in the user VRF assigned to the overlay VE interface.

Unicast Reverse Path Forwarding (uRPF) is not supported on overlay VE interfaces mapped to VXLAN VNIs.

Hardware Considerations

  • VXLAN RIOT is supported only on RUCKUS ICX 7550 and ICX 7850 devices.
  • VXLAN RIOT requires two lookups (overlay and underlay) for the same packet.
  • The egress Layer 3 interface and next-hop tables (EGR_L3_INTF and EGR_L3_NEXT_HOP) are partitioned between overlay and underlay.
  • By default, VXLAN RIOT is disabled, and all resources are allocated to the underlay.
  • When enabled, the overlay next-hop table partition is configurable in increments of 4000 entries (for example, 4000, 8000, 12000, 16000).
  • The overlay egress interface table defaults to 2000 entries when VXLAN RIOT is enabled; this value is not configurable.
  • Partitioning reduces Layer 3 scale numbers.

Inter‑VNI Routing Using VXLAN RIOT

VXLAN Routing In and Out of Tunnels

1. Unicast packet to H3 is sent to VTEP=1 (default gateway for H1) 3. VTEP-1 send the encapsulated packet to VTEP-2
2. VTEP-1 performs inter-VLAN routing and VXLAN encapsulation 4. VTEP-2 decapsulates the packet and forwards it to H3

The grahic illustrates the sequence of operations for inter‑VNI routing using VXLAN RIOT on ICX devices. The process in a FastIron VXLAN implementation unfolds as follows:

  1. Host H1 in VLAN 100 sends traffic to Host H3 in VLAN 101. VTEP‑1 receives the VXLAN‑encapsulated packet on its tunnel port.
  2. VTEP-1 performs a Layer 3 lookup in the overlay VRF and determines the next hop for VLAN 101.
  3. VTEP-1 encapsulates the routed packet in VXLAN with the VNI for VLAN 101 and sends it across the underlay to VTEP-2.
  4. VTEP-2 removes the VXLAN header, restores VLAN 101 context, and bridges the frame to Host H3 on the correct access port.

If a VXLAN device supports RIOT, VXLAN RIOT must be enabled before the switch allows for associating a VE interface with a VLAN associated with a VNI.

Packets received on the VXLAN tunnel with that VNI with the gateway router's MAC address as the DMAC address will be decapsulated and processed by the routing engine.

Packets routed to the VE interface or VLAN with a MAC address (known or unknown) destined for the VXLAN tunnel will be encapsulated and forwarded using the SMAC address of the switch associated with that VE interface, and the DMAC address of the destination device.

In this section:

  1. Enabling VXLAN RIOT
    Complete the following steps to enable VXLAN RIOT.
  2. Configuring the Overlay Next-Hop Partition Size
Parent topic: VXLAN