乾颐堂安德华为DC实现指南新书一瞥-VXLAN的静态部署-基础VXLAN实验案例
乾颐堂安德华为DC实现指南新书一瞥-VXLAN的静态部署-基础VXLAN实验案例
乾颐堂网络实验室-安德（周亚军）
VXLAN（Virtual eXtensible Local Area Network）采用MAC in UDP（User Datagram Protocol）封装方式，一种网络虚拟化技术。已经变成当下SDN以及大2层的业界标准，当然也是不折不扣的网红技术。是当下华为数据中心的当红炸子鸡，明年初的DATACOM HCIE也需要对应知识。
一【实验目的】
通过部署VXLAN基础实验了解VXLAN的工作原理
了解VXLAN的业务接入方式和数据转发
二【实验环境】
本实验采用ENSP完成部署，其中Leaf设备和SPINE设备采用ENSP中CE设备，下联的传统接入网络采用ENSP中的5700设备。
CE设备的底层IGP采用OSPF即部署Underlay网络。
三【实验原理】
此处省略，请参考正式书籍
四【实验步骤】
1）完成底层的IGP（Underlay网络），为后续部署其他协议做前置准备
SPINE:
ospf router 3.3.3.3 //配置OSPF的RID
area 0 //配置区域0，在本例中仅仅存在area0
int g1/0/0
un shu
undo portswitch //接口切换为3层接口
ip address 10.1.13.3 24
ospf enable a 0 //接口开启OSPF，并配置在区域0中
ospf network-type p2p //为了加速OSPF邻居关系建立，全网改用了点到点的网络类型
int g1/0/1
un shu
undo portswitch
ip address 10.1.23.3 24
ospf enable a 0
ospf network-type p2p
int lo0
ip address 3.3.3.3 32
ospf enable a 0 //环回接口配置OSPF并运行在区域0中
Leaf设备：
Leaf1：
ospf router-id 1.1.1.1
area 0
int lo0
ip address 1.1.1.1 32
ospf en a 0
int g1/0/0
undo shutdown
undo portswitch
ip address 10.1.13.1 24
ospf en a 0
ospf network-type p2p
Leaf2：ospf router-id 2.2.2.2
area 0
int g1/0/1
un shutdown
undo portswitch
ip address 10.1.23.2 24
ospf enable a 0
ospf network-type p2p
int lo0
ip address 2.2.2.2 32
ospf enable a 0
ospf network-type p2p
配置完毕OSPF来验证邻居以及路由情况：
<SPINE>dis ospf peer b
rief //OSPF邻居正常，SPINE分别和LEAF1、2建立了邻居
OSPF Process 1 with Router ID 3.3.3.3
Peer Statistic Information
Total number of peer(s): 2
Peer(s) in full state: 2
-----------------------------------------------------------------------------
Area Id Interface Neighbor id State
0.0.0.0 GE1/0/0 1.1.1.1 Full
0.0.0.0 GE1/0/1 2.2.2.2 Full
------------------------------------------------------------
<SPINE>display ip routing-table protocol ospf
//查看OSPF的路由表，SPINE得到了1.1.1.1和2.2.2.2的路由信息
Proto: Protocol Pre: Preference
Route Flags: R - relay, D - download to fib, T - to vp嗯-instance, B - black hole route
------------------------------------------------------------------------------
_public_ Routing Table : OSPF
Destinations : 5 Routes : 5
OSPF routing table status : <Active>
Destinations : 2 Routes : 2
Destination/Mask Proto Pre Cost Flags NextHop Interface
1.1.1.1/32 OSPF 10 1 D 10.1.13.1 GE1/0/0
2.2.2.2/32 OSPF 10 1 D 10.1.23.2 GE1/0/1
验证Underlay网络的数据通信：
<SPINE>ping -a 3.3.3.3 1.1.1.1
PING 1.1.1.1: 56 data bytes, press CTRL_C to break
Reply from 1.1.1.1: bytes=56 Sequence=1 ttl=255 time=8 ms
Reply from 1.1.1.1: bytes=56 Sequence=2 ttl=255 time=4 ms
Reply from 1.1.1.1: bytes=56 Sequence=3 ttl=255 time=6 ms
Reply from 1.1.1.1: bytes=56 Sequence=4 ttl=255 time=3 ms
Reply from 1.1.1.1: bytes=56 Sequence=5 ttl=255 time=5 ms
--- 1.1.1.1 ping statistics ---
5 packet(s) transmitted
5 packet(s) received
0.00% packet loss
round-trip min/avg/max = 3/5/8 ms
<SPINE>ping -a 3.3.3.3 2.2.2.2
PING 2.2.2.2: 56 data bytes, press CTRL_C to break
Reply from 2.2.2.2: bytes=56 Sequence=1 ttl=255 time=8 ms
Reply from 2.2.2.2: bytes=56 Sequence=2 ttl=255 time=5 ms
Reply from 2.2.2.2: bytes=56 Sequence=3 ttl=255 time=4 ms
Reply from 2.2.2.2: bytes=56 Sequence=4 ttl=255 time=10 ms
Reply from 2.2.2.2: bytes=56 Sequence=5 ttl=255 time=5 ms
--- 2.2.2.2 ping statistics ---
5 packet(s) transmitted
5 packet(s) received
0.00% packet loss
round-trip min/avg/max = 4/6/10 ms
2)业务接入点实施：创建桥接域
Leaf1和Leaf2作为业务接入点，连接了传统接入交换机5700，流量需要经由Leaf设备的2层子接口进入VXLAN，之后完成大2层互访。在本步骤中采用了相同的桥接域（brideg-domain）10，后续实验为了理解这个概念，我们会采用不同的桥接域。
<Leaf1>system-view immediately
Enter system view, return user view with return command.
[Leaf1]int g1/0/2
[Leaf1-GE1/0/2]description Conn2ACCESS
[Leaf1-GE1/0/2]undo shutdown
[Leaf1-GE1/0/2]quit
[Leaf1]bridge-domain 10 //创建桥接域10
[Leaf1-bd10]vxlan vni ?
INTEGER<1-16777215> Value of VXLAN network identifier //VNI，即VXLAN网络标记，此处标记了范围
[Leaf1-bd10]vxlan vni 10 //桥接域10的VXLAN VNI标记为10
！
<Leaf2>system-view immediately
Enter system view, return user view with return command.
[Leaf2]int g1/0/2
[Leaf2-GE1/0/2]description Conn2Access
[Leaf2-GE1/0/2]undo shutdown
[Leaf2-GE1/0/2]q
[Leaf2]bridge-domain 10
//创建桥接域10
[Leaf2-bd10]vxlan vni 10 //桥接域10的VXLAN VNI标记为10
查看两个设备的桥接域
[Leaf1]display bridge-domain
The total number of bridge-domains is : 1
--------------------------------------------------------------------------------
MAC_LRN: MAC learning; STAT: Statistics; SPLIT: Split-horizon;
BC: Broadcast; MC: Unknown multicast; UC: Unknown unicast;
*down: Administratively down; FWD: Forward; DSD: Discard;
--------------------------------------------------------------------------------
BDID State MAC-LRN STAT BC MC UC SPLIT Description
--------------------------------------------------------------------------------
10 up enable disable FWD FWD FWD disable
[Leaf2]dis bridge-domain
The total number of bridge-domains is : 1
--------------------------------------------------------------------------------
MAC_LRN: MAC learning; STAT: Statistics; SPLIT: Split-horizon;
BC: Broadcast; MC: Unknown multicast; UC: Unknown unicast;
*down: Administratively down; FWD: Forward; DSD: Discard;
--------------------------------------------------------------------------------
BDID State MAC-LRN STAT BC MC UC SPLIT Description
--------------------------------------------------------------------------------
10 up enable disable FWD FWD FWD disable
在配置完毕桥接域之后，把桥接域和二层子接口进行绑定
Leaf1：
[Leaf1]int g1/0/2.1 mode l2 //创建2层子接口，用于和BD绑定，业务接入点定义为二层子接口，只有二层子接口才能接入业务。二层子接口只能接入BD，不能直接接入三层网络。每个二层子接口唯一属于一个BD
[Leaf1-GE1/0/2.1]encapsulation dot1q vid 10 //dot1q在封装到vxlan之前要先拿掉帧中的VLAN TAG（解耦，即VLAN不在有意义）。此处的VID为从子接口发出帧的TAG
[Leaf1-GE1/0/2.1]bridge-domain 10 //BD域和子接口绑定，以方便携带TAG10的帧可以通过子接口转发，而后续BD会和VNI结合
Leaf2：
[Leaf2]int g1/0/2.1 mode l2
[Leaf2-GE1/0/2.1]bridge-domain 10
[Leaf2-GE1/0/2.1]encapsulation dot1q vid 10
查看桥接域
[Leaf1]display bridge-domain 10
--------------------------------------------------------------------------------
MAC_LRN: MAC learning; STAT: Statistics; SPLIT: Split-horizon;
BC: Broadcast; MC: Unknown multicast; UC: Unknown unicast;
*down: Administratively down; FWD: Forward; DSD: Discard;
U: Up; D: Down;
--------------------------------------------------------------------------------
BDID Ports
--------------------------------------------------------------------------------
10 GE1/0/2.1(U) //桥接域已经和子接口绑定
BDID State MAC-LRN STAT BC MC UC SPLIT Description
--------------------------------------------------------------------------------
10 up enable disable FWD FWD FWD disable
BDID VLANIDs
--------------------------------------------------------------------------------
10
到此桥接域，以及桥接域和二层子接口完成了绑定
3）
Leaf1：
[Leaf1]int Nve 1 //创建网络虚拟化边缘接口，即VXLAN隧道
[Leaf1-Nve1]source 1.1.1.1 //源自Leaf1的Underlay网络的环回接口，即VTEP的源地址
[Leaf1-Nve1]vni 10 head-end peer-list 2.2.2.2 //VNI10的头尾列表为2.2.2.2
Leaf2：
[Leaf2]int Nve 1
[Leaf2-Nve1]source 2.2.2.2
[Leaf2-Nve1]vni 10 head-end peer-list 1.1.1.1
验证VXLAN的VNI以及详细信息情况
[Leaf1]display vxlan vni
Number of vxlan vni : 1
VNI BD-ID State
---------------------------------------
10 10 up //VNI已经和桥接域绑定，且已经正常工作
[Leaf1]display vxlan vni 10 verbose //查看VXLAN VNI10的详细信息
BD ID : 10
State : up
NVE : 18
Source Address : 1.1.1.1 //VXLAN隧道的源地址
Source IPv6 Address : -
UDP Port : 4789 //VXLAN的UDP端口为4789
BUM Mode : head-end //广播单播组播模式为头尾复制
Group Address : -
Peer List : 2.2.2.2
IPv6 Peer List : -
[Leaf2]display vxlan vni
Number of vxlan vni : 1
VNI BD-ID State
---------------------------------------
10 10 up
[Leaf2]display vxlan vni 10 verbose
BD ID : 10
State : up
NVE : 18
Source Address : 2.2.2.2
Source IPv6 Address : -
UDP Port : 4789
BUM Mode : head-end
Group Address : -
Peer List : 1.1.1.1
IPv6 Peer List : -
查看VXLAN隧道以及详细信息
[Leaf1]display vxlan tunnel //查看VXLAN隧道
Number of vxlan tunnel : 1
Tunnel ID Source Destination State Type Uptime
-----------------------------------------------------------------------------------
4026531841 1.1.1.1 2.2.2.2 up static 02:43:31 //此处的VXLAN隧道为静态方式，该隧道的源1.1.1.1，目的为2.2.2.2
[Leaf1]display vxlan tunnel verbose //查看VXLAN隧道详细情况
Tunnel ID : 4026531841
Source : 1.1.1.1
Destination : 2.2.2.2
State : up
Type : static
Uptime : 02:43:34
[Leaf2]display vxlan tunnel
Number of vxlan tunnel : 1
Tunnel ID Source Destination State Type Uptime
-----------------------------------------------------------------------------------
4026531841 2.2.2.2 1.1.1.1 up static 02:43:15
[Leaf2]display vxlan tunnel ver
[Leaf2]display vxlan tunnel verbose
Tunnel ID : 4026531841
Source : 2.2.2.2
Destination : 1.1.1.1
State : up
Type : static
Uptime : 02:43:18
至此从Leaf1到Leaf2的VXLAN隧道已经建立。
可以在设备上开启VXLAN的检测功能
[Leaf1]nqa vxlanecho enable udp-port 6000
[Leaf2]nqa vxlanecho enable udp-port 6000 //以6000接口作为VXLAN回声测试报文的响应端口
[SPINE]nqa vxlanecho enable udp-port 6000
测试：
<Leaf1>ping vxlan vni 10 source 1.1.1.1 peer 2.2.2.2 udp-port 6000
//VXLAN通信成功
PING VXLAN: vni 10 source 1.1.1.1 peer 2.2.2.2, press CTRL_C to break
Reply from 2.2.2.2: bytes=40 Sequence=1 time=191 ms
Reply from 2.2.2.2: bytes=40 Sequence=2 time=9 ms
Reply from 2.2.2.2: bytes=40 Sequence=3 time=16 ms
Reply from 2.2.2.2: bytes=40 Sequence=4 time=9 ms
Reply from 2.2.2.2: bytes=40 Sequence=5 time=8 ms
--ping vxlan statistics--
5 packet(s) transmitted
5 packet(s) received
0.00% packet loss
round-trip min/avg/max = 8/46/191 ms
[Leaf1]tracert vxlan vni 10 source 1.1.1.1 peer 2.2.2.2 udp-port 6000
TRACERT VXLAN: vni 10 source 1.1.1.1 peer 2.2.2.2, press CTRL_C to break
TTL Replier Time Ingress Port Egress Port
1 10.1.13.3 5 ms unknown unknown
2 2.2.2.2 722 ms GE1/0/1 --
VXLAN部署告一段落。
4）
为了完成完整的数据通信，我们来配置当下情况的传统接入网络。
在SW1和SW2（5700）配置VLAN
vlan batch 10 20
配置端口常规的接入和Trunk模式
<Acces1>dis cu int g0/0/2
#
interface GigabitEthernet0/0/2
port link-type trunk
port trunk allow-pass vlan 2 to 4094
#
return
<Acces1>dis cu int g0/0/10
#
interface GigabitEthernet0/0/10
port link-type access
port default vlan 10
！
[Acces2]dis cu int g0/0/10
#
interface GigabitEthernet0/0/10
port link-type access
port default vlan 10
[Acces2]dis cu int g0/0/2
#
interface GigabitEthernet0/0/2
port link-type trunk
port trunk allow-pass vlan 2 to 4094
验证配置结果
SW1：
<Acces1>display port vlan active
T=TAG U=UNTAG
-------------------------------------------------------------------------------
Port Link Type PVID VLAN List
-------------------------------------------------------------------------------
GE0/0/1 hybrid 1 U: 1
GE0/0/2 trunk 1 U: 1
T: 10 20
GE0/0/3 hybrid 1 U: 1
GE0/0/4 hybrid 1 U: 1
GE0/0/5 hybrid 1 U: 1
GE0/0/6 hybrid 1 U: 1
GE0/0/7 hybrid 1 U: 1
GE0/0/8 hybrid 1 U: 1
GE0/0/9 hybrid 1 U: 1
GE0/0/10 access 10 U: 10
SW2：
[Acces2]dis port vlan ac
T=TAG U=UNTAG
-------------------------------------------------------------------------------
Port Link Type PVID VLAN List
-------------------------------------------------------------------------------
GE0/0/1 hybrid 1 U: 1
GE0/0/2 trunk 1 U: 1
T: 10 20
GE0/0/3 hybrid 1 U: 1
GE0/0/4 hybrid 1 U: 1
GE0/0/5 hybrid 1 U: 1
GE0/0/6 hybrid 1 U: 1
GE0/0/7 hybrid 1 U: 1
GE0/0/8 hybrid 1 U: 1
GE0/0/9 hybrid 1 U: 1
GE0/0/10 access 10 U: 10
在终端PC上完成数据测试，PC1可以和PC2完成通信。
可以在SPINE设备上完成抓包，来观察流量，当然您只能看到隧道封装，而无法看到172.16.1.0网络的流量。如下所示
接下来我们来查看Leaf1和Leaf2的MAC地址表，这一点很关键，因为VXLAN是大2层网络，所以需要观察同一子网下的2层转发表。
[Leaf1]display mac-address
Flags: * - Backup
BD : bridge-domain Age : dynamic MAC learned time in seconds
-------------------------------------------------------------------------------
MAC Address VLAN/VSI/BD Learned-From Type Age
-------------------------------------------------------------------------------
5489-9874-1819 -/-/10 GE1/0/2.1 dynamic - //此处1819结尾的MAC地址为Leaf1 在Bridge-domain 10的G1/0/2.1 2层子接口下通过动态方式学习到的PC1的MAC地址。
5489-98ba-53de -/-/10 2.2.2.2 dynamic - //此处53de结尾的MAC地址为Leaf1在Bridge-domain 10从VXLAN远端节点2.2.2.2学习到的MAC地址
5489-9874-1819 -/-/10 GE1/0/2.1 dynamic -
5489-98ba-53de -/-/10 2.2.2.2 dynamic -
-------------------------------------------------------------------------------
[Leaf2]dis mac-address
Flags: * - Backup
BD : bridge-domain Age : dynamic MAC learned time in seconds
-------------------------------------------------------------------------------
MAC Address VLAN/VSI/BD Learned-From Type Age
-------------------------------------------------------------------------------
5489-9874-1819 -/-/10 1.1.1.1 dynamic -
5489-98ba-53de -/-/10 GE1/0/2.1 dynamic -
5489-9874-1819 -/-/10 1.1.1.1 dynamic -
5489-98ba-53de -/-/10 GE1/0/2.1 dynamic -
-------------------------------------------------------------------------------
其通信示意图如下所示
从业务终端172.16.1.1到172.16.1.2的流量，经过传统接入网络，携带TAG10的帧从Leaf1的2层子接口允许进入VXLAN隧道后，此时剥离掉所有TAG，转发到VXLAN隧道内，即VNI10，VXLAN隧道在Leaf1和Leaf2之间部署，此时没有携带任何TAG的帧到达Leaf2，Leaf2会剥离VXLAN封装准备从2层子接口发送。注意此时会增加2层子接口下配置的DOT1Q封装的VLAN ID（10），发送到传统接入交换机。
5）
为了理解桥接域，我们来修改步骤2中的配置，本步骤在不同的Leaf设备上配置不同的Bridge-domain。VNI能标识VXLAN网络中的2层域，而Bridge-Domain是VXLAN网络的实体，只具有本地意义，所以在此步骤中，我们修改Leaf2的桥接域，但VNI不做改变，依旧为20
[Leaf2]bridge-domain 20
//创建新的桥接域20
[Leaf2-bd20]vxlan vni 10
//试图把桥接域20和VNI10绑定，但此时出现报错，该错误表明，VNI10已经和桥接域10完成了绑定，而桥接域和VNI是1:1的绑定关系。
Error: The VNI has already been bound to another bridge-domain.
解决方案：
bridge-domain 10
[Leaf2-bd10]undo vxlan vni 10 //在桥接域10下，解除VNI10和桥接域10的绑定关系
！
重新完成绑定：
bridge-domain 20
[Leaf2-bd20] vxlan vni 10
Info: Please disable dynamic ARP learning when the controller is used to deliver ARP entries.
除此之外，桥接域20还要和业务2层子接口进行绑定，如下所示
[Leaf2]int g1/0/2.1 mode l2
[Leaf2-GE1/0/2.1]dis th
#
interface GE1/0/2.1 mode l2
encapsulation dot1q vid 10
bridge-domain 10
#
[Leaf2-GE1/0/2.1]undo bridge-domain
[Leaf2-GE1/0/2.1]bridge-domain 20 //Leaf2的2层子接口和桥接域20绑定
验证配置结果，注意观察桥接域20和VNI10以及2层子接口的绑定关系
[Leaf2]display vxlan vni 10
VNI BD-ID State
---------------------------------------
10 20 up
//此处可以看到VNI10和桥接域20做了绑定，为工作状态
[Leaf2]display vxlan vni 10 verbose
BD ID : 20
State : up
NVE : 18
Source Address : 2.2.2.2
Source IPv6 Address : -
UDP Port : 4789
BUM Mode : head-end
Group Address : -
Peer List : 1.1.1.1
IPv6 Peer List : -
测试VXLAN数据通信
[Leaf2]ping vxlan vni 10 source 2.2.2.2 peer 1.1.1.1 udp-port 6000 //VTEP即业务接入点的测试
PING VXLAN: vni 10 source 2.2.2.2 peer 1.1.1.1, press CTRL_C to break
Reply from 1.1.1.1: bytes=40 Sequence=1 time=102 ms
Reply from 1.1.1.1: bytes=40 Sequence=2 time=7 ms
Reply from 1.1.1.1: bytes=40 Sequence=3 time=18 ms
Reply from 1.1.1.1: bytes=40 Sequence=4 time=6 ms
Reply from 1.1.1.1: bytes=40 Sequence=5 time=7 ms
--ping vxlan statistics--
5 packet(s) transmitted
5 packet(s) received
0.00% packet loss
round-trip min/avg/max = 6/28/102 ms
PC>ping 172.16.1.2 //终端的测试表明，不属于同一桥接域，但属于同一个VNI的终端完成了通信
Ping 172.16.1.2: 32 data bytes, Press Ctrl_C to break
From 172.16.1.2: bytes=32 seq=1 ttl=128 time=63 ms
From 172.16.1.2: bytes=32 seq=2 ttl=128 time=62 ms
From 172.16.1.2: bytes=32 seq=3 ttl=128 time=78 ms
--- 172.16.1.2 ping statistics ---
3 packet(s) transmitted
3 packet(s) received
0.00% packet loss
round-trip min/avg/max = 62/67/78 ms
本步骤测试完毕，到此基本的静态方式部署VXLAN实验案例完成。
乾颐堂网络实验室
安德（周亚军）
