We are attempting get macvlan and ipv6 play nicely in our Virtualization setup but are having difficulty with ConnectX-3 cards and the associated mlx-4 driver on CentOS 7.2. Replacing the ConnectX-3 with a ConnectX-4 card (100g/mlx5) works just fine.
Setup description and our test is as below. Any thoughts/ideas/comments appreciated. Thanks.
Setup :
1 Physical host with a NIC that supports SR-IOV like mellanox ConnectX-3 running CentOS 7.2
Setup 2 kvm guests on it also running CentOS 7.2
Enable SR-IOV and create multiple Virtual functions on the nic on the host.
Passthrough 1 VF each to the guests. It will show up a ethX device inside the guest.
Test:
Lets say the device name is ens4. We are going to create a VLAN 467
on top of this device and then a macvlan on top of that.
Guest#1 :
--------
1- ip link set ens4 down
2- ip link set ens.467 down
3- ip link add link ens4 name ens4.467 type vlan id 467
4- ip link add link ens4.467 name macvlan0 type macvlan mode vepa
5- ip addr add 5.5.5.201/30 dev macvlan0
6- ip -6 addr add 2614:f0d0:1002:12::1/64 dev macvlan0
7- ip link set ens4 up
8- ip link set ens4.467 up
9- ip link set macvlan0 up
Guest#2 :
--------
1- ip link set ens4 down
2- ip link set ens.467 down
3- ip link add link ens4 name ens4.467 type vlan id 467
4- ip link add link ens4.467 name macvlan0 type macvlan mode vepa
5- ip addr add 5.5.5.202/30 dev macvlan0
6- ip -6 addr add 2614:f0d0:1002:12::3/64 dev macvlan0
7- ip link set ens4 up
8- ip link set ens4.467 up
9- ip link set macvlan0 up
From Guest#1 - ping6 2614:f0d0:1002:12::3/64. This does not work.
From Guest#1 - ping 5.5.5.202. This works.
Any ideas?