VxLAN and VxLAN Routing Configuration Guide in ICOS

Get the PDF version here.

This article shows how to configure VxLAN networks, and VxLAN routing using Netberg Aurora switches with ICOS NOS.

Use Case – VxLAN

Service providers allow a tenant to have multiple Layer-2 networks. These Layer-2 networks are bridge domains in the overlay network. The VNIs which are associated with them are often referred to as Layer-2 (L2) VNIs.

  • Configure the VxLAN tunnel for VNI
  • Associate VNI to VLAN
  • Associate VLAN to Access Port

Network Diagram

net_diag_1

Topology

vxlan_topology_1
Table 1. Tenants List

VM

VNID

VLAN

IP

VM1

1000

10

100.1.1.1/8

VM3

1000

10

100.1.1.2/8

VM5

1000

10

100.1.1.3/8

VM7

1000

10

100.1.1.4/8

VM2

2000

20

200.1.1.1/8

VM4

2000

20

200.1.1.2/8

VM6

2000

20

200.1.1.3/8

VM8

2000

20

200.1.1.4/8

Configuration

Leaf 1

!--- Create VLAN for VNI association
vlan database
vlan 10,20
exit
configure
ip routing
!--- Enable VxLAN mode
vxlan enable
!--- Configure source loopback interface for soure VTEP address
vxlan source-interface loopback 0
!--- Associate VNI 1000 to VLAN 10
vxlan 1000 vlan 10
!--- Configure remote VTEP 8.8.8.8 for VNI 1000
vxlan 1000 vtep 8.8.8.8
!--- Associate VNI 2000 to VLAN 20
vxlan 2000 vlan 20
!--- Configure remote VTEP 8.8.8.8 for VNI 2000
vxlan 2000 vtep 8.8.8.8
!--- Create loopback for VxLAN source interface and source VTEP address
interface loopback 0
ip address 7.7.7.7 255.255.255.255
ip ospf area 0
exit
!--- Create the underlay routing interface to reach remote VTEP
interface 0/1
routing
ip address 2.2.2.2 255.255.255.0
ip ospf area 0
exit
!--- Create the underlay routing interface to reach remote VTEP
interface 0/2
routing
ip address 2.2.6.2 255.255.255.0
ip ospf area 0
exit
!--- Associate the VLAN 10 to configure the access port for VNI 1000
interface 0/3
vlan pvid 10
vlan participation exclude 1
vlan participation include 10
exit
!--- Associate the VLAN 20 to configure the access port for VNI 2000
interface 0/4
vlan pvid 20
vlan participation exclude 1
vlan participation include 20
exit
!--- Associate the VLAN 10 to configure the access port for VNI 1000
interface 0/5
vlan pvid 10
vlan participation exclude 1
vlan participation include 10
exit
!--- Associate the VLAN 20 to configure the access port for VNI 2000
interface 0/6
vlan pvid 20
vlan participation exclude 1
vlan participation include 20
exit
router ospf
router-id 2.2.2.2
exit
exit

Leaf 2

!--- Create VLAN for VNI association
vlan database
vlan 10,20
exit
configure
ip routing
!--- Enable VxLAN mode
vxlan enable
!--- Configure source loopback interface for soure VTEP address
vxlan source-interface loopback 0
!--- Associate VNI 1000 to VLAN 10
vxlan 1000 vlan 10
!--- Configure remote VTEP 7.7.7.7 for VNI 1000
vxlan 1000 vtep 7.7.7.7
!--- Associate VNI 2000 to VLAN 20
vxlan 2000 vlan 20
!--- Configure remote VTEP 7.7.7.7 for VNI 2000
vxlan 2000 vtep 7.7.7.7
!--- Add port to the port-channel
interface 0/3
addport 3/1
exit
interface 0/4
addport 3/2
exit
interface 0/49
addport 3/10
exit
!--- Create loopback for VxLAN source interface and source VTEP address
interface loopback 0
ip address 8.8.8.3 255.255.255.255
ip address 8.8.8.8 255.255.255.255 secondary
ip ospf area 0
exit
!--- Create the underlay routing interface to reach remote VTEP
interface 0/1
routing
ip address 3.3.3.3 255.255.255.0
ip ospf area 0
exit
!--- Create the underlay routing interface to reach remote VTEP
interface 0/2
routing
ip address 3.3.6.3 255.255.255.0
ip ospf area 0
exit
!--- Associate the VLAN 10 to configure the access port for VNI 1000
interface 0/5
vlan pvid 10
vlan participation include 10
exit
!--- Associate the VLAN 10 to configure the access port for VNI 1000
interface lag 1
no port-channel static
switchport mode access
switchport access vlan 10
!--- Configure the port-channel to VPC 1
vpc 1
exit
!--- Associate the VLAN 20 to configure the access port for VNI 2000
interface lag 2
no port-channel static
switchport mode access
switchport access vlan 20
!--- Configure the port-channel to VPC 2
vpc 2
exit
interface lag 10
no port-channel static
switchport mode trunk
!--- Allow the VLAN on peer-link for the access port
switchport trunk allowed vlan 10,20
vlan acceptframe vlanonly
!--- Configure the port-channel as the VPC peer-link
vpc peer-link
exit
router ospf
router-id 3.3.3.3
exit
!--- Enable VPC
feature vpc
!--- Creaet the VPC domain
vpc domain 1
!--- Enable the peer keepalive
peer-keepalive enable
!--- Configure the peer keepalive connection
peer-keepalive destination 10.58.151.57 source 10.58.151.56
!--- Configure the system MAC address for VPC domain
system-mac 00:00:00:33:33:33
!--- Configure the peer detection interval and timeout
peer detection interval 600 timeout 2000
!--- Enable the peer detection
peer detection enable
exit
exit

Leaf 3

!--- Create VLAN for VNI association
vlan database
vlan 10,20
exit
configure
ip routing
!--- Enable VxLAN mode
vxlan enable
!--- Configure source loopback interface for soure VTEP address
vxlan source-interface loopback 0
!--- Associate VNI to VLAN
!--- Configure remote VTEP for VNI
vxlan 1000 vlan 10
vxlan 1000 vtep 7.7.7.7
vxlan 2000 vlan 20
vxlan 2000 vtep 7.7.7.7
!--- Add port to the port-channel
interface 0/3
addport 3/1
exit
interface 0/4
addport 3/2
exit
interface 0/49
addport 3/10
exit
!--- Create loopback for VxLAN source interface and source VTEP address
interface loopback 0
ip address 8.8.8.4 255.255.255.255
ip address 8.8.8.8 255.255.255.255 secondary
ip ospf area 0
exit
!--- Create the underlay routing interface to reach remote VTEP
interface 0/1
routing
ip address 4.4.4.4 255.255.255.0
ip ospf area 0
exit
!--- Create the underlay routing interface to reach remote VTEP
interface 0/2
routing
ip address 4.4.6.4 255.255.255.0
ip ospf area 0
exit
!--- Associate the VLAN 20 to configure the access port for VNI 2000
interface 0/5
vlan pvid 20
vlan participation include 20
exit
interface lag 1
no port-channel static
switchport mode access
!--- Associate the VLAN 10 to configure the access port for VNI 1000
switchport access vlan 10
!--- Configure the port-channel to VPC ID 1
vpc 1
exit
interface lag 2
no port-channel static
switchport mode access
!--- Associate the VLAN 20 to configure the access port for VNI 2000
switchport access vlan 20
!--- Configure the port-channel to VPC ID 2
vpc 2
exit
interface lag 10
no port-channel static
switchport mode trunk
!--- Allow the VLAN on peer-link for the access port
switchport trunk allowed vlan 10,20
vlan acceptframe vlanonly
!--- Configure the port-channel as the VPC peer-link
vpc peer-link
exit
router ospf
router-id 4.4.4.4
exit
!--- Enable VPC
feature vpc
!--- Creaet the VPC domain
vpc domain 1
!--- Enable the peer keepalive
peer-keepalive enable
!--- Configure the peer keepalive connection
peer-keepalive destination 10.58.151.56 source 10.58.151.57
!--- Configure the system MAC address for VPC domain
system-mac 00:00:00:33:33:33
!--- Configure the peer detection
peer detection interval 600 timeout 2000
!--- Enable the peer detection
peer detection enable
exit
exit

Spine 1

!--- Configure the underlay network
configure
ip routing
interface 0/2
routing
ip address 2.2.6.5 255.255.255.0
ip ospf area 0
exit
interface 0/3
routing
ip address 3.3.6.5 255.255.255.0
ip ospf area 0
exit
interface 0/4
routing
ip address 4.4.6.5 255.255.255.0
ip ospf area 0
exit
router ospf
router-id 1.1.1.1
exit
exit

Spine 2

!--- Configure the underlay network
configure
ip routing
interface 0/2
routing
ip address 2.2.2.5 255.255.255.0
ip ospf area 0
exit
interface 0/3
routing
ip address 3.3.3.5 255.255.255.0
ip ospf area 0
exit
interface 0/4
routing
ip address 4.4.4.5 255.255.255.0
ip ospf area 0
exit
router ospf
router-id 5.5.5.5
exit
exit

Use Case – VxLAN RIOT

Each tenant also needs a Layer-3 routing in itself VNIs for inter-VXLAN routing in its service domain.

  • Configure the ethernet loopback LAG mode (ELBL)
  • Configure the distributed anycast gateway
  • Configure the VRF instance

Network Diagram

net_diag_riot

Topology

vxlan_topology_riot
Table 2. Tenants List

VRF

VM

VNID

VLAN

IP

GateWay

VRF1

VM1

1000

10

100.1.1.1/8

100.0.0.2

VRF1

VM5

1000

10

100.1.1.3/8

100.0.0.2

VRF1

VM2

2000

20

200.1.1.1/8

200.0.0.2

VRF1

VM6

2000

20

200.1.1.3/8

200.0.0.2

VRF2

VM3

4000

100

100.1.1.2/8

100.0.0.2

VRF2

VM7

4000

100

100.1.1.4/8

100.0.0.2

VRF2

VM4

5000

200

200.1.1.2/8

200.0.0.2

VRF2

VM8

5000

200

200.1.1.4/8

200.0.0.2

Configuration

Leaf 1

vlan database
!--- Create VLAN for VNI association
vlan 10,20,100,200
!--- Create VLAN routing as the gateway for RIOT host
vlan routing 10 1
vlan routing 20 2
vlan routing 200 4
vlan routing 100 5
exit
configure
ip routing
!--- Enable VxLAN mode
vxlan enable
!--- Configure source loopback interface for soure VTEP address
vxlan source-interface loopback 0
!--- Associate VNI to VLAN
!--- Configure the remote VTEP for VNI
vxlan 1000 vlan 10
vxlan 1000 vtep 8.8.8.8
vxlan 2000 vlan 20
vxlan 2000 vtep 8.8.8.8
vxlan 4000 vlan 100
vxlan 4000 vtep 8.8.8.8
vxlan 5000 vlan 200
vxlan 5000 vtep 8.8.8.8
!--- Configure the MAC address of anycast-gateway across all leaf nodes
fabric forwarding mode anycast-gateway 00:00:00:BB:BB:BB
!--- Add port to the port-channel
interface 0/48
addport 3/11
exit
!--- Create VRF for multi-tenancy
ip vrf "aaa"
exit
ip vrf "bbb"
exit
!--- Create loopback for VxLAN source interface and source VTEP address
interface loopback 0
ip address 7.7.7.7 255.255.255.255
ip ospf area 0
exit
!--- Create the underlay routing interface to reach remote VTEP
interface 0/1
routing
ip address 2.2.2.2 255.255.255.0
ip ospf area 0
exit
!--- Create the underlay routing interface to reach remote VTEP
interface 0/2
routing
ip address 2.2.6.2 255.255.255.0
ip ospf area 0
exit
!--- Associate the VLAN 10 to configure the access port for VNI 1000
interface 0/3
vlan pvid 10
vlan participation exclude 1
vlan participation include 10
exit
!--- Associate the VLAN 20 to configure the access port for VNI 2000
interface 0/4
vlan pvid 20
vlan participation exclude 1
vlan participation include 20
exit
!--- Associate the VLAN 100 to configure the access port for VNI 4000
interface 0/5
vlan pvid 100
vlan participation include 100
exit
!--- Associate the VLAN 200 to configure the access port for VNI 5000
interface 0/6
vlan pvid 200
vlan participation include 200
exit
interface lag 11
!--- Enable the ethernet loopback LAG mode on the port-channel for RIOT
host routing
vxlan elbl mode
no spanning-tree port mode
vlan participation include 10,20,100,200
vlan tagging 10,20,100,200
exit
!--- Configure the vlan routing
interface vlan 10
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF aaa
ip vrf forwarding "aaa"
!--- Configure IP address
ip address 100.0.0.2 255.0.0.0
exit
interface vlan 20
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF aaa
ip vrf forwarding "aaa"
!--- Configure IP address
ip address 200.0.0.2 255.0.0.0
exit
interface vlan 200
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
! --- Configure VRF instance for VRF bbb
ip vrf forwarding "bbb"
!--- Configure IP address
ip address 200.0.0.2 255.0.0.0
exit
interface vlan 100
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF bbb
ip vrf forwarding "bbb"
!--- Configure IP address
ip address 100.0.0.2 255.0.0.0
exit
router ospf
router-id 2.2.2.2
exit
!--- Enable the ARP dynamic renew mode for RIOT host
arp dynamicrenew
exit

Leaf 2

vlan database
!--- Create VLAN for VNI association
vlan 10,20,100,200
!--- Create VLAN routing as the gateway for RIOT host
vlan routing 10 1
vlan routing 20 2
vlan routing 200 4
vlan routing 100 5
exit
configure
ip routing
!--- Enable VxLAN mode
vxlan enable
!--- Configure source loopback interface for soure VTEP address
vxlan source-interface loopback 0
!--- Associate VNI to VLAN
!--- Configure the remote VTEP for VNI
vxlan 1000 vlan 10
vxlan 1000 vtep 7.7.7.7
vxlan 2000 vlan 20
vxlan 2000 vtep 7.7.7.7
vxlan 4000 vlan 100
vxlan 4000 vtep 7.7.7.7
vxlan 5000 vlan 200
vxlan 5000 vtep 7.7.7.7
!--- Configure the MAC address of anycast-gateway across all leaf nodes
fabric forwarding mode anycast-gateway 00:00:00:BB:BB:BB
!--- Add port to the port-channel
interface 0/3
addport 3/1
exit
interface 0/4
addport 3/2
exit
interface 0/49
addport 3/10
exit
interface 0/48
addport 3/11
exit
!--- Create VRF for multi-tenancy
ip vrf "aaa"
exit
ip vrf "bbb"
exit
!--- Create loopback for VxLAN source interface and source VTEP address
interface loopback 0
ip address 8.8.8.3 255.255.255.255
ip address 8.8.8.8 255.255.255.255 secondary
ip ospf area 0
exit
!--- Create the underlay routing interface to reach remote VTEP
interface 0/1
routing
ip address 3.3.3.3 255.255.255.0
ip ospf area 0
exit
!--- Create the underlay routing interface to reach remote VTEP
interface 0/2
routing
ip address 3.3.6.3 255.255.255.0
ip ospf area 0
exit
!--- Associate the VLAN 100 to configure the access port for VNI 4000
interface 0/5
vlan pvid 100
vlan participation include 100
exit
interface lag 1
no port-channel static
switchport mode access
!--- Associate the VLAN 10 to configure the access port for VNI 1000
switchport access vlan 10
!--- Configure the port-channel to VPC ID 1
vpc 1
exit
interface lag 2
no port-channel static
switchport mode access
!--- Associate the VLAN 20 to configure the access port for VNI 2000
switchport access vlan 20
!--- Configure the port-channel to VPC ID 2
vpc 2
exit
interface lag 10
no port-channel static
switchport mode trunk
!--- Allow the VLAN on peer-link for the access port
switchport trunk allowed vlan 10,20,100,200
vlan acceptframe vlanonly
!--- Configure the port-channel as the VPC peer-link
vpc peer-link
exit
interface lag 11
!--- Enable the ethernet loopback LAG mode on the port-channel for RIOT
host routing
vxlan elbl mode
no spanning-tree port mode
vlan participation include 10,20,100,200
vlan tagging 10,20,100,200
exit
interface vlan 10
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF aaa
ip vrf forwarding "aaa"
ip address 100.0.0.2 255.0.0.0
exit
interface vlan 20
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF aaa
ip vrf forwarding "aaa"
ip address 200.0.0.2 255.0.0.0
exit
interface vlan 200
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF bbb
ip vrf forwarding "bbb"
ip address 200.0.0.2 255.0.0.0
exit
interface vlan 100
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF bbb
ip vrf forwarding "bbb"
ip address 100.0.0.2 255.0.0.0
exit
router ospf
router-id 3.3.3.3
exit
!--- Enable the ARP dynamic renew mode for RIOT host
arp dynamicrenew
!--- Enable VPC
feature vpc
vpc domain 1
role priority 1
peer-keepalive enable
peer-keepalive destination 10.58.151.57 source 10.58.151.56
system-mac 00:00:00:33:33:33
peer detection interval 600 timeout 2000
peer detection enable
exit
exit

Leaf 3

vlan database
!--- Create VLAN for VNI association
vlan 10,20,100,200
!--- Create VLAN routing as the gateway for RIOT host
vlan routing 10 1
vlan routing 20 2
vlan routing 200 4
vlan routing 100 5
exit
configure
ip routing
!--- Enable VxLAN mode
vxlan enable
!--- Configure source loopback interface for soure VTEP address
vxlan source-interface loopback 0
!--- Associate VNI to VLAN
!--- Configure remote VTEP for VNI
vxlan 1000 vlan 10
vxlan 1000 vtep 7.7.7.7
vxlan 2000 vlan 20
vxlan 2000 vtep 7.7.7.7
vxlan 4000 vlan 100
vxlan 4000 vtep 7.7.7.7
vxlan 5000 vlan 200
vxlan 5000 vtep 7.7.7.7
!--- Configure the MAC address of anycast-gateway across all leaf nodes
fabric forwarding mode anycast-gateway 00:00:00:BB:BB:BB
!--- Add port to the port-channel
interface 0/3
addport 3/1
exit
interface 0/4
addport 3/2
exit
interface 0/49
addport 3/10
exit
interface 0/48
addport 3/11
exit
!--- Create VRF for multi-tenancy
ip vrf "aaa"
exit
ip vrf "bbb"
exit
!--- Create loopback for VxLAN source interface and source VTEP address
interface loopback 0
ip address 8.8.8.4 255.255.255.255
ip address 8.8.8.8 255.255.255.255 secondary
ip ospf area 0
exit
!--- Create the underlay routing interface to reach remote VTEP
interface 0/1
routing
ip address 4.4.4.4 255.255.255.0
ip ospf area 0
exit
!--- Create the underlay routing interface to reach remote VTEP
interface 0/2
routing
ip address 4.4.6.4 255.255.255.0
ip ospf area 0
exit
!--- Associate the VLAN 200 to configure the access port for VNI 5000
interface 0/5
vlan pvid 200
vlan participation include 200
exit
interface lag 1
no port-channel static
switchport mode access
!--- Associate the VLAN 10 to configure the access port for VNI 1000
switchport access vlan 10
!--- Configure the port-channel to VPC ID 1
vpc 1
exit
interface lag 2
no port-channel static
switchport mode access
!--- Associate the VLAN 20 to configure the access port for VNI 2000
switchport access vlan 20
!--- Configure the port-channel to VPC ID 2
vpc 2
exit
interface lag 10
no port-channel static
switchport mode trunk
!--- Allow the VLAN on peer-link for the access port
switchport trunk allowed vlan 10,20,100,200
vlan acceptframe vlanonly
!--- Configure the port-channel as the VPC peer-link
vpc peer-link
exit
interface lag 11
!--- Enable the ethernet loopback LAG mode on the port-channel for RIOT
host routing
vxlan elbl mode
no spanning-tree port mode
vlan participation include 10,20,100,200
vlan tagging 10,20,100,200
exit
interface vlan 10
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF aaa
ip vrf forwarding "aaa"
ip address 100.0.0.2 255.0.0.0
exit
interface vlan 20
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF aaa
ip vrf forwarding "aaa"
ip address 200.0.0.2 255.0.0.0
exit
interface vlan 200
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF bbb
ip vrf forwarding "bbb"
ip address 200.0.0.2 255.0.0.0
exit
interface vlan 100
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF bbb
ip vrf forwarding "bbb"
ip address 100.0.0.2 255.0.0.0
exit
router ospf
router-id 4.4.4.4
exit
!--- Enable the ARP dynamic renew mode for RIOT host
arp dynamicrenew
!--- Enable VPC
feature vpc
vpc domain 1
role priority 2
peer-keepalive enable
peer-keepalive destination 10.58.151.56 source 10.58.151.57
system-mac 00:00:00:33:33:33
peer detection interval 600 timeout 2000
peer detection enable
exit
exit

Spine 1

!--- Configure the underlay network
configure
ip routing
interface 0/2
routing
ip address 2.2.6.5 255.255.255.0
ip ospf area 0
exit
interface 0/3
routing
ip address 3.3.6.5 255.255.255.0
ip ospf area 0
exit
interface 0/4
routing
ip address 4.4.6.5 255.255.255.0
ip ospf area 0
exit
router ospf
router-id 1.1.1.1
exit
exit

Spine 2

!--- Configure the underlay network
configure
ip routing
interface 0/2
routing
ip address 2.2.2.5 255.255.255.0
ip ospf area 0
exit
interface 0/3
routing
ip address 3.3.3.5 255.255.255.0
ip ospf area 0
exit
interface 0/4
routing
ip address 4.4.4.5 255.255.255.0
ip ospf area 0
exit
router ospf
router-id 5.5.5.5
exit
exit

Use Case – VxLAN RIOT with an External Router

The data center is not isolated from the rest of the network. When a VXLAN fabric is deployed in the data center, it needs to maintain connectivity with these networks that are external to the VXLAN fabric. With the standard spine-and-leaf fabric architecture, external connectivity can be achieved by using border leaf nodes to connect to the outside routing devices.

  • Configure the network connectivity to the external router on border leaf
  • Configure to import the route into VxLAN fabric network on border leaf
  • Configure the VNI for external routing

Network Diagram

net_diag_riot_ext

Topology

vxlan_topology_riot_ext
Table 3. Tenants List

VRF

VM

VNID

VLAN

IP

GateWay

VRF1

VM1

1000

10

100.1.1.1/8

100.0.0.2

VRF1

VM5

1000

10

100.1.1.3/8

100.0.0.2

VRF1

VM2

2000

20

200.1.1.1/8

200.0.0.2

VRF1

VM6

2000

20

200.1.1.3/8

200.0.0.2

VRF1

External Router

3000

30

55.55.55.1/24

VRF2

VM3

4000

100

100.1.1.2/8

100.0.0.2

VRF2

VM7

4000

100

100.1.1.4/8

100.0.0.2

VRF2

VM4

5000

200

200.1.1.2/8

200.0.0.2

VRF2

VM8

5000

200

200.1.1.4/8

200.0.0.2

Configuration

Leaf 1

vlan database
!--- Create VLAN for VNI association
!--- Creaet VLAN 30 to associate VNI 3000 for external routing
vlan 10,20,30,100,200
!--- Create VLAN routing as the gateway for RIOT host
vlan routing 10 1
vlan routing 20 2
vlan routing 30 3
vlan routing 200 4
vlan routing 100 5
exit
configure
ip routing
!--- Enable VxLAN mode
vxlan enable
!--- Configure source loopback interface for soure VTEP address
vxlan source-interface loopback 0
!--- Associate VNI to VLAN
!--- Configure remote VTEP for VNI
vxlan 1000 vlan 10
vxlan 1000 vtep 8.8.8.8
vxlan 2000 vlan 20
vxlan 2000 vtep 8.8.8.8
!--- VNI 3000 is used for external routing
vxlan 3000 vlan 30
vxlan 3000 vtep 8.8.8.8
vxlan 4000 vlan 100
vxlan 4000 vtep 8.8.8.8
vxlan 5000 vlan 200
vxlan 5000 vtep 8.8.8.8
!--- Configure the MAC address of anycast-gateway across all leaf nodes
fabric forwarding mode anycast-gateway 00:00:00:BB:BB:BB
!--- Add port to the port-channel
interface 0/48
addport 3/11
exit
!--- Create VRF for multi-tenancy
ip vrf "aaa"
!--- Import the routes from VRF aaa into VxLAN fabirc on Border Leaf
ip import vxlan fabric
exit
ip vrf "bbb"
exit
!--- Create loopback for VxLAN source interface and source VTEP address
interface loopback 0
ip address 7.7.7.7 255.255.255.255
ip ospf area 0
exit
!--- Create the underlay routing interface to reach remote VTEP
interface 0/1
routing
ip address 2.2.2.2 255.255.255.0
ip ospf area 0
exit
!--- Create the underlay routing interface to reach remote VTEP
interface 0/2
routing
ip address 2.2.6.2 255.255.255.0
ip ospf area 0
exit
!--- Associate the VLAN 10 to configure the access port for VNI 1000
interface 0/3
vlan pvid 10
vlan participation exclude 1
vlan participation include 10
exit
!--- Associate the VLAN 20 to configure the access port for VNI 2000
interface 0/4
vlan pvid 20
vlan participation exclude 1
vlan participation include 20
exit
!--- Associate the VLAN 100 to configure the access port for VNI 4000
interface 0/5
vlan pvid 100
vlan participation include 100
exit
!--- Associate the VLAN 200 to configure the access port for VNI 5000
interface 0/6
vlan pvid 200
vlan participation include 200
exit
!--- Associate the VLAN 30 to configure the access port for VNI 3000
!--- VNI 3000 is used for external routing
interface 0/11
vlan pvid 30
vlan participation include 30
exit
interface lag 11
!--- Enable the ethernet loopback LAG mode on the port-channel for RIOT
host routing
vxlan elbl mode
no spanning-tree port mode
vlan participation include 10,20,30,100,200
vlan tagging 10,20,30,100,200
exit
interface vlan 10
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF aaa
ip vrf forwarding "aaa"
!--- Configure IP address
ip address 100.0.0.2 255.0.0.0
exit
interface vlan 20
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF aaa
ip vrf forwarding "aaa"
!--- Configure IP address
ip address 200.0.0.2 255.0.0.0
exit
!--- Create the vlan30 routing on Border Leaf for external routing
interface vlan 30
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF aaa
ip vrf forwarding "aaa"
!--- Configure IP address
ip address 55.55.55.2 255.255.255.0
exit
interface vlan 200
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF bbb
ip vrf forwarding "bbb"
!--- Configure IP address
ip address 200.0.0.2 255.0.0.0
exit
interface vlan 100
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF bbb
ip vrf forwarding "bbb"
!--- Configure IP address
ip address 100.0.0.2 255.0.0.0
exit
router ospf
router-id 2.2.2.2
exit
!--- Configure BGP
router bgp 2222
bgp router-id 2.2.2.2
address-family ipv4 vrf "aaa"
!--- Configure the external router as a neighbor
neighbor 55.55.55.1 remote-as 1111
redistribute connected
exit
exit
!--- Enable the ARP dynamic renew mode for RIOT host
arp dynamicrenew
exit

Leaf 2

vlan database
!--- Create VLAN for VNI association
!--- Creaet VLAN 30 to associate VNI 3000 for external routing
vlan 10,20,30,100,200
!--- Create VLAN routing as the gateway for RIOT host
vlan routing 10 1
vlan routing 20 2
vlan routing 30 3
vlan routing 200 4
vlan routing 100 5
exit
configure
ip routing
!--- Enable VxLAN mode
vxlan enable
!--- Configure source loopback interface for soure VTEP address
vxlan source-interface loopback 0
!--- Associate VNI to VLAN
!--- Configure remote VTEP for VNI
vxlan 1000 vlan 10
vxlan 1000 vtep 7.7.7.7
vxlan 2000 vlan 20
vxlan 2000 vtep 7.7.7.7
!--- VNI 3000 is used for external routing
vxlan 3000 vlan 30
vxlan 3000 vtep 7.7.7.7
vxlan 4000 vlan 100
vxlan 4000 vtep 7.7.7.7
vxlan 5000 vlan 200
vxlan 5000 vtep 7.7.7.7
!--- Configure the MAC address of anycast-gateway across all leaf nodes
fabric forwarding mode anycast-gateway 00:00:00:BB:BB:BB
!--- Add port to the port-channel
interface 0/3
addport 3/1
exit
interface 0/4
addport 3/2
exit
interface 0/49
addport 3/10
exit
interface 0/48
addport 3/11
exit
!--- Create VRF for multi-tenancy
ip vrf "aaa"
exit
ip vrf "bbb"
exit
!--- Create loopback for VxLAN source interface and source VTEP address
interface loopback 0
ip address 8.8.8.3 255.255.255.255
ip address 8.8.8.8 255.255.255.255 secondary
ip ospf area 0
exit
!--- Create the underlay routing interface to reach remote VTEP
interface 0/1
routing
ip address 3.3.3.3 255.255.255.0
ip ospf area 0
exit
!--- Create the underlay routing interface to reach remote VTEP
interface 0/2
routing
ip address 3.3.6.3 255.255.255.0
ip ospf area 0
exit
!--- Associate the VLAN 100 to configure the access port for VNI 4000
interface 0/5
vlan pvid 100
vlan participation include 100
exit
interface lag 1
no port-channel static
switchport mode access
 !--- Associate the VLAN 10 to configure the access port for VNI 1000
switchport access vlan 10
!--- Configure the port-channel to VPC ID 1
vpc 1
exit
interface lag 2
no port-channel static
switchport mode access
!--- Associate the VLAN 20 to configure the access port for VNI 2000
switchport access vlan 20
!--- Configure the port-channel to VPC ID 2
vpc 2
exit
interface lag 10
no port-channel static
switchport mode trunk
!--- Allow the VLAN on peer-link for the access port
switchport trunk allowed vlan 10,20,30,100,200
vlan acceptframe vlanonly
!--- Configure the port-channel as the VPC peer-link
vpc peer-link
exit
interface lag 11
!--- Enable the ethernet loopback LAG mode on the port-channel for RIOT
routing
vxlan elbl mode
no spanning-tree port mode
vlan participation include 10,20,30,100,200
vlan tagging 10,20,30,100,200
exit
interface vlan 10
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF aaa
ip vrf forwarding "aaa"
ip address 100.0.0.2 255.0.0.0
exit
interface vlan 20
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF aaa
ip vrf forwarding "aaa"
ip address 200.0.0.2 255.0.0.0
exit
!--- Create the vlan30 routing on Border Leaf for external routing
interface vlan 30
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF aaa
ip vrf forwarding "aaa"
ip address 55.55.55.2 255.255.255.0
exit
interface vlan 200
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF bbb
ip vrf forwarding "bbb"
ip address 200.0.0.2 255.0.0.0
exit
interface vlan 100
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF bbb
ip vrf forwarding "bbb"
ip address 100.0.0.2 255.0.0.0
exit
router ospf
router-id 3.3.3.3
exit
!--- Enable the ARP dynamic renew mode for RIOT host
arp dynamicrenew
!--- Enable VPC
feature vpc
vpc domain 1
role priority 1
peer-keepalive enable
peer-keepalive destination 10.58.151.104 source 10.58.151.56
system-mac 00:00:00:33:33:33
peer detection interval 600 timeout 2000
peer detection enable
exit
exit

Leaf 3

vlan database
!--- Create VLAN for VNI association
!--- Creaet VLAN 30 to associate VNI 3000 for external routing
vlan 10,20,30,100,200
!--- Create VLAN routing as the gateway for RIOT host
vlan routing 10 1
vlan routing 20 2
vlan routing 30 3
vlan routing 200 4
vlan routing 100 5
exit
configure
ip routing
!--- Enable VxLAN mode
vxlan enable
!--- Configure source loopback interface for soure VTEP address
vxlan source-interface loopback 0
!--- Associate VNI to VLAN
!--- Configure remote VTEP for VNI
vxlan 1000 vlan 10
vxlan 1000 vtep 7.7.7.7
vxlan 2000 vlan 20
vxlan 2000 vtep 7.7.7.7
!--- VNI 3000 is used for external routing
vxlan 3000 vlan 30
vxlan 3000 vtep 7.7.7.7
vxlan 4000 vlan 100
vxlan 4000 vtep 7.7.7.7
vxlan 5000 vlan 200
vxlan 5000 vtep 7.7.7.7
!--- Configure the MAC address of anycast-gateway across all leaf nodes
fabric forwarding mode anycast-gateway 00:00:00:BB:BB:BB
!--- Add port to the port-channel
interface 0/3
addport 3/1
exit
interface 0/4
addport 3/2
exit
interface 0/49
addport 3/10
exit
interface 0/48
addport 3/11
exit
!--- Create VRF for multi-tenancy
ip vrf "aaa"
exit
ip vrf "bbb"
exit
!--- Create loopback for VxLAN source interface and source VTEP address
interface loopback 0
ip address 8.8.8.4 255.255.255.255
ip address 8.8.8.8 255.255.255.255 secondary
ip ospf area 0
exit
!--- Create the underlay routing interface to reach remote VTEP
interface 0/1
routing
ip address 4.4.4.4 255.255.255.0
ip ospf area 0
exit
!--- Create the underlay routing interface to reach remote VTEP
interface 0/2
routing
ip address 4.4.6.4 255.255.255.0
ip ospf area 0
exit
!--- Associate the VLAN 200 to configure the access port for VNI 5000
interface 0/5
vlan pvid 200
vlan participation include 200
exit
interface lag 1
no port-channel static
switchport mode access
!--- Associate the VLAN 10 to configure the access port for VNI 1000
switchport access vlan 10
!--- Configure the port-channel to VPC ID 1
vpc 1
exit
interface lag 2
no port-channel static
switchport mode access
!--- Associate the VLAN 20 to configure the access port for VNI 2000
switchport access vlan 20
!--- Configure the port-channel to VPC ID 2
vpc 2
exit
interface lag 10
no port-channel static
switchport mode trunk
!--- Allow the VLAN on peer-link for the access port
switchport trunk allowed vlan 10,20,30,100,200
vlan acceptframe vlanonly
!--- Configure the port-channel as the VPC peer-link
vpc peer-link
exit
interface lag 11
!--- Enable the ethernet loopback LAG mode on the port-channel for RIOT
routing
vxlan elbl mode
no spanning-tree port mode
vlan participation include 10,20,30,100,200
vlan tagging 10,20,30,100,200
exit
interface vlan 10
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF aaa
ip vrf forwarding "aaa"
ip address 100.0.0.2 255.0.0.0
exit
interface vlan 20
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF aaa
ip vrf forwarding "aaa"
ip address 200.0.0.2 255.0.0.0
exit
!--- Create the vlan30 routing on Border Leaf for external routing
interface vlan 30
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF aaa
ip vrf forwarding "aaa"
ip address 55.55.55.2 255.255.255.0
exit
interface vlan 200
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF bbb
ip vrf forwarding "bbb"
ip address 200.0.0.2 255.0.0.0
exit
interface vlan 100
routing
!--- Enable the anycast-gateway on vlan routing interface
fabric forwarding mode anycast-gateway
!--- Configure VRF instance for VRF bbb
ip vrf forwarding "bbb"
ip address 100.0.0.2 255.0.0.0
exit
router ospf
router-id 4.4.4.4
exit
!--- Enable the ARP dynamic renew mode for RIOT host
arp dynamicrenew
!--- Enable VPC
feature vpc
vpc domain 1
role priority 2
peer-keepalive enable
peer-keepalive destination 10.58.151.56 source 10.58.151.104
system-mac 00:00:00:33:33:33
peer detection interval 600 timeout 2000
peer detection enable
exit
exit

Spine 1

!--- Configure the underlay network
configure
ip routing
interface 0/2
routing
ip address 2.2.6.5 255.255.255.0
ip ospf area 0
exit
interface 0/3
routing
ip address 3.3.6.5 255.255.255.0
ip ospf area 0
exit
interface 0/4
routing
ip address 4.4.6.5 255.255.255.0
ip ospf area 0
exit
router ospf
router-id 1.1.1.1
exit
exit

Spine 2

!--- Configure the underlay network
configure
ip routing
interface 0/2
routing
ip address 2.2.2.5 255.255.255.0
ip ospf area 0
exit
interface 0/3
routing
ip address 3.3.3.5 255.255.255.0
ip ospf area 0
exit
interface 0/4
routing
ip address 4.4.4.5 255.255.255.0
ip ospf area 0
exit
router ospf
router-id 5.5.5.5
exit
exit