Virtual Routing and Forwarding – VRF

Virtual routing and forwarding (VRF) provides for the presence of multiple independent routing tables working simultaneously on the same router or switch. This provides for multiple network paths without the need for multiple switches. VRFs are like VLANs for layer 3. But unlike VLANs, there is no field in the IP header carrying it.

The primary use cases for VRFs in a data center are similar to VLANs at layer 2: using common physical infrastructure to carry multiple isolated traffic streams for multi-tenant environments, where these streams are allowed to cross over only at configured boundary points, typically firewalls or IDS. You can also use it to burst traffic from private clouds to enterprise networks where the burst point is at layer 3.

VRF is fully supported in the Linux kernel, so it has the following characteristics:

  • The VRF is presented as a layer 3 master network device with its own associated routing table.
  • The layer 3 interfaces (VLAN interfaces, bonds) associated with the VRF are enslaved to that VRF; IP rules direct FIB (forwarding information base) lookups to the routing table for the VRF device.
  • The VRF device can have its own IP address, known as a VRF-local loopback.
  • Connected and local routes are placed in appropriate VRF tables.
  • Neighbor entries continue to be per-interface, and you can view all entries associated with the VRF device.
  • A VRF does not map to its own network namespace; however, you can nest VRFs in a network namespace.

Example model & SONiC version:

  • Aurora 615/715
  • Netberg SONiC: sonic-202012-nb-inno-211121

Configure a VRF

By default, all layer 3 interfaces are placed in the default VRF. This command is used to bind an interface to a VRF.

  • The VRF name must always start with Vrf.
  • If you assign IP to the interface without VRF instance binding, it will use the default VRF for this routing entry.
  • If there’s an IP address on the interface, it will be removed after binding the VRF.
  • When the MGMT VRF is enabled, if you want to use out-of-band management network, you need to assign MGMT VRF to run the command.

Create the VRF:

admin@sonic:~$ sudo config vrf add Vrf01

Then bind an interface to the VRF:

admin@sonic:~$ sudo config interface vrf bind Ethernet0 Vrf01

Assign an IP address to Ethernet0

admin@sonic:~$ sudo config interface ip add Ethernet0 172.18.0.1/24

Save your changes to the configuration:

admin@switch:~$ sudo config save -y

To see the VRFs configured on the switch, and the interfaces bound to them, run:

admin@sonic:~$ show vrf
VRF    Interfaces
-----  ------------
Vrf01  Ethernet0
admin@sonic:~$ show ip interfaces
Interface    Master    IPv4 address/mask    Admin/Oper    BGP Neighbor    Neighbor IP
-----------  --------  -------------------  ------------  --------------  -------------
Ethernet0    Vrf01     172.18.0.1/24        up/down       N/A             N/A

Check the routing table:

admin@sonic:~$ show ip route vrf Vrf01
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup

VRF Vrf01:
C>* 172.18.0.0/24 is directly connected, Ethernet0, 00:00:26

admin@sonic:~$ show ip route vrf all
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup

VRF Vrf01:
C>* 172.18.0.0/24 is directly connected, Ethernet0, 00:00:12

VRF default:
K>* 0.0.0.0/0 [0/0] via 192.168.0.1, eth0, 00:50:46
C>* 10.0.0.2/31 is directly connected, Ethernet4, 00:00:28
C>* 10.1.0.1/32 is directly connected, Loopback0, 00:50:32
C>* 192.168.0.0/24 is directly connected, eth0, 00:50:46

Or modify the INTERFACE and VRF hierarchy statements in the /etc/sonic/config_db.json file.

"INTERFACE": {
    "Ethernet0": {
        "vrf_name": "Vrf01"
    },
"VRF": {
    "Vrf01": {}
},

Management VRF

Management VRF is a subset of VRF that separates the out-of-band management network and the in-band data plane network. For all VRFs, the main routing table is the default table for all data plane switch ports. With management VRF, a second table called mgmt is used for routing through the Ethernet ports of the switch.

SONiC only supports eth0 for out-of-band management. VLAN subinterfaces, bonds, bridges, and the front panel switch ports are not supported as OOB management interfaces.

The ‘ interfaces-config ‘ service restarts when you enable the management VRF on a switch, regenerating the /etc/network/interfaces file. This creates a new interface and l3mdev CGROUP with the name mgmt and enslaves the management interface eth0 into this master interface mgmt.

The management VRF is not enabled by default.

Note the VRF names mgmt and management are reserved for the management VRF; data plane VRFs should not use either of these reserved VRF names.

Create the management VRF:

admin@sonic:~$ sudo config vrf add mgmt

Save the configuratio:

admin@sonic:~$ sudo config save -y

Or modify the MGMT_VRF_CONFIG hierarchy statements in the /etc/sonic/config_db.json file.

"MGMT_VRF_CONFIG": {
    "vrf_global": {
        "mgmtVrfEnabled": "true"
    }
},

Check the result:

admin@sonic:~$ show mgmt-vrf

ManagementVRF : Enabled

Management VRF interfaces in Linux:
42: mgmt: <NOARP,MASTER,UP,LOWER_UP> mtu 65536 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 8e:e4:6d:df:5d:bf brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu 68 maxmtu 1500
    vrf table 5000 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master mgmt state UP mode DEFAULT group default qlen 1000
    link/ether 70:b3:d5:cc:f7:f3 brd ff:ff:ff:ff:ff:ff
43: lo-m: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue master mgmt state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 3a:de:5e:70:c0:3e brd ff:ff:ff:ff:ff:ff

admin@sonic:~$ show ip interfaces
Interface    Master    IPv4 address/mask    Admin/Oper    BGP Neighbor    Neighbor IP
-----------  --------  -------------------  ------------  --------------  -------------
Ethernet0    Vrf01     172.18.0.1/24        up/up         N/A             N/A
Loopback0              10.1.0.1/32          up/up         N/A             N/A
d1ocker0                240.127.1.1/24       up/down       N/A             N/A
eth0         mgmt      192.168.0.126/24     up/up         N/A             N/A
lo                     127.0.0.1/16         up/up         N/A             N/A
lo-m         mgmt      127.0.0.1/16         up/up         N/A             N/A

The show mgmt-vrf routes command displays the routes that are present in the routing table 5000, which is dedicated to the management VRF.

admin@sonic:~$ show mgmt-vrf routes

Routes in Management VRF Routing Table:
default via 192.168.0.1 dev eth0
broadcast 127.0.0.0 dev lo-m proto kernel scope link src 127.0.0.1
127.0.0.0/16 dev lo-m proto kernel scope link src 127.0.0.1
local 127.0.0.1 dev lo-m proto kernel scope host src 127.0.0.1
broadcast 127.0.255.255 dev lo-m proto kernel scope link src 127.0.0.1
broadcast 192.168.0.0 dev eth0 proto kernel scope link src 192.168.0.126
192.168.0.0/24 dev eth0 proto kernel scope link src 192.168.0.126
local 192.168.0.126 dev eth0 proto kernel scope host src 192.168.0.126
broadcast 192.168.0.255 dev eth0 proto kernel scope link src 192.168.0.126
Note If you didn’t assign the MGMT VRF to run the traffic, it would use in-band data plane network (default VRF). 
admin@sonic:~$ ping 8.8.8.8
connect: Network is unreachable
admin@sonic:~$ sudo ip vrf exec mgmt ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=117 time=3.65 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=117 time=3.58 ms
^C
--- 8.8.8.8 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 3ms
rtt min/avg/max/mdev = 3.576/3.615/3.654/0.039 ms

111123, г. Москва, Шоссе Энтузиастов 56, стр. 26
Tel: +7-495-150-4050
Email: sales@netberg.ru