1. What Is VXLAN?

VXLAN stands for Virtual Extensible LAN. It is an overlay networking technology that encapsulates Layer 2 Ethernet frames inside UDP/IP packets so they can travel across an underlying IP network.

This allows devices or virtual machines in the same logical network to communicate even when they are connected to different physical servers or locations within a data-center fabric.

Why do we need VXLAN?

Traditional VLANs use a 12-bit VLAN ID, providing 4,094 commonly usable VLAN IDs in standard Ethernet deployments, excluding reserved values. This can become a limitation in large multi-tenant data centers.

VXLAN uses a 24-bit VXLAN Network Identifier (VNI), supporting approximately 16.7 million possible identifiers. The actual number of deployed networks depends on the design and platform.

• Extends logical Layer 2 connectivity over a Layer 3 network.
• Supports large-scale network segmentation.
• Helps isolate tenants and applications.
• Allows workloads to communicate across different physical hosts.
• Supports scalable data-center and cloud network architectures.

2. How Does VXLAN Work?

VXLAN adds an outer network header around an original Ethernet frame. The resulting packet travels through the existing IP network to another VXLAN endpoint.

Original Ethernet Frame
+-------------------------------+
| Destination MAC               |
| Source MAC                    |
| Original Ethernet Payload     |
+-------------------------------+
                |
                v
VXLAN Encapsulation
+--------------------------------+
| Outer Ethernet Header          |
| Outer IP Header                |
| UDP Header                     |
| VXLAN Header (VNI)             |
| Original Ethernet Frame        |
+--------------------------------+
                |
                v
       Underlay IP Network
                |
                v
      Remote VXLAN Endpoint
                |
                v
      Decapsulated Ethernet Frame
        

Simple example

Imagine a virtual machine on Server A needs to communicate with a virtual machine on Server B. Both belong to the same logical VXLAN network.

1. The source virtual machine sends an Ethernet frame.
2. The local VXLAN endpoint identifies the destination and the appropriate VNI.
3. The endpoint encapsulates the original frame inside a VXLAN packet.
4. The underlay IP network forwards the packet toward the remote endpoint.
5. The remote endpoint removes the outer headers.
6. The original Ethernet frame is delivered to the destination workload.

The physical network forwards the outer IP packet. The overlay provides the logical Ethernet connectivity between workloads.

3. What Is a VTEP?

VTEP stands for VXLAN Tunnel Endpoint. It is the device or software function that encapsulates and decapsulates VXLAN traffic.

A VTEP can be implemented in a physical network switch, a virtual switch, or another platform that supports VXLAN.

Virtual Machine A
        |
        v
+------------------+
| VTEP A           |
| Encapsulates     |
+------------------+
        |
        | VXLAN over IP
        v
+------------------+
| IP Underlay      |
| Routers / Links  |
+------------------+
        |
        v
+------------------+
| VTEP B           |
| Decapsulates     |
+------------------+
        |
        v
Virtual Machine B
        

Important VTEP responsibilities

• Encapsulating Ethernet frames into VXLAN packets.
• Adding the relevant VNI and outer network headers.
• Sending packets toward remote VTEPs.
• Removing VXLAN encapsulation at the receiving endpoint.
• Forwarding the original Ethernet frame to the appropriate destination.

4. What Is a VNI?

A VXLAN Network Identifier (VNI) identifies a logical VXLAN segment. It is a 24-bit field in the VXLAN header.

For example, a data center might map VLAN 10 to VNI 10010 and VLAN 20 to VNI 10020. These are example mappings; actual values depend on the network design.

A VNI is not the same thing as an IP subnet. Network designers define how Layer 2 segments, IP subnets, and routing services relate to each other.

5. VXLAN Underlay vs Overlay

VXLAN networks are often explained using two layers: the underlay and the overlay.

Underlay network

The underlay is the physical or routed IP network that provides IP reachability between VTEPs. It includes the physical links, switches, routers, IP addresses, and routing protocols that transport the encapsulated packets.

Overlay network

The overlay is the logical network created using VXLAN. It connects workloads through tunnels over the underlay, allowing logical network segments to extend across the infrastructure.

          OVERLAY NETWORK
   VM A -------- VXLAN -------- VM B
                  VNI
                    |
                    v
          UNDERLAY IP NETWORK
     Router ---- Switch ---- Router
        

Remember: The underlay transports VXLAN packets. The overlay connects the workloads. A working overlay depends on correct underlay IP reachability.

6. How Does VXLAN Encapsulation Work?

VXLAN commonly uses UDP destination port 4789. This is the standard VXLAN UDP destination port, although implementations may support configurable alternatives.

The encapsulated packet includes:

• An outer Ethernet header for delivery on the local underlay link.
• An outer IP header identifying the source and destination VTEPs.
• A UDP header identifying the VXLAN transport.
• A VXLAN header containing the VNI.
• The original Ethernet frame.

Because VXLAN adds headers, the packet becomes larger than the original frame. The underlay MTU must account for this overhead, or oversized packets may be dropped or fragmented depending on the implementation and configuration.

For many Ethernet/IP deployments, engineers use a larger underlay MTU to accommodate encapsulated traffic. Always verify end-to-end MTU requirements for the specific platform and packet path.

7. How Are Remote VTEPs Discovered?

VXLAN needs a way to determine where remote destinations are located. Two broad approaches are common.

Flood and Learn

In traditional flood-and-learn VXLAN designs, unknown unicast, broadcast, and certain multicast traffic may be distributed through configured replication mechanisms. VTEPs learn MAC-to-VTEP associations from observed traffic.

EVPN Control Plane

In VXLAN EVPN designs, BGP EVPN advertises reachability information, such as MAC addresses, IP addresses, and VTEP-related information, depending on the route type and configuration. This can reduce reliance on flooding and improve scalable endpoint learning.

EVPN is covered in the next article in this series.

8. VXLAN Use Cases

• Data centers: Connect workloads across multiple racks and servers.
• Cloud infrastructure: Build logical networks for different applications and tenants.
• Multi-tenancy: Separate logical network segments for different customers or business units.
• Virtualization: Extend logical connectivity between virtualized workloads.
• Network modernization: Build scalable overlay networks on routed IP fabrics.
• Workload mobility: Support network designs in which workloads may move between hosts while retaining their logical network membership, subject to the platform and architecture.

9. VXLAN vs VLAN

VXLAN does not make VLANs obsolete. VLANs remain useful and may be used at the edge of a VXLAN network, with VLAN-to-VNI mappings connecting local segments to overlay networks.

10. VXLAN Troubleshooting Checklist

If workloads cannot communicate over VXLAN, investigate each layer systematically.

1. Check underlay reachability: Verify that VTEPs can reach each other through the routed IP network.
2. Verify VTEP addresses: Confirm the source and destination VTEP IPs.
3. Check VNI mapping: Make sure the correct VLAN-to-VNI mapping is configured where required.
4. Inspect UDP traffic: Check whether VXLAN traffic can pass between VTEPs.
5. Check MTU: Confirm that encapsulated packets can traverse the full path.
6. Inspect MAC learning: Verify that the destination MAC is known through the expected mechanism.
7. Check EVPN, if deployed: Verify BGP sessions, advertised routes, and route import/export policies.
8. Review security policies: Check ACLs, firewalls, and other filters that may block required traffic.

11. Quick Quiz

12. Key Takeaways

• VXLAN creates logical Layer 2 networks over a Layer 3 IP underlay.
• VTEPs encapsulate and decapsulate Ethernet frames.
• The 24-bit VNI identifies a VXLAN overlay segment.
• VXLAN commonly uses UDP destination port 4789.
• Underlay reachability and MTU are important to reliable VXLAN operation.
• VXLAN can use flood-and-learn or a control plane such as EVPN for endpoint reachability.