[Bridge] VLAN tag being incorreclty removed from ethernet frame when sent on the wire

Paulo Estante estantep at gmail.com
Sun Nov 20 15:24:29 UTC 2016


Hello

I have the following setup:

+------------------------------------+
|  KVM CENTOS7 3.10.0-327.el7.x86_64 |
| +------+                           |  +--------+
| | VM1  |         +--------+        |  | SWITCH |
| | vio0-+--vnet0--+mytrunk0+--enp1s0+--+ TRUNK  |
| +------+         +--------+        |  |  PORT  |
+------------------------------------+  +--------+

The enp1s0 physical interface dump from ethtool is pasted at the end of
this email as well as brctl output and interfaces config (please scroll
down).

I am trying to achieve transparent VLAN bridging without defining any
subinterfaces on the KVM box. The goal is for the KVM box just forward
frames from physical port to VM and vice-versa. Any VM connected to
mytrunk0 should be able to use any VLAN ID without the hypervisor having to
be configured on it as well. The VMs (to be) connected to mytrunk0 will be
a part of several dozen VLANs therefore having to manually define one
bridge for each one of those just won't scale from a management perspective.

What I am seeing is:

0 - After KVM host boot i add enp1s0 to the bridge using the command "brctl
addif mytrunk0 enp1s0"

1 - VM1 (OpenBSD 6.0) is properly tagging packets out on vnet0 (tcpdump
output below):

# tcpdump -n -e -i vnet0
<...>
09:52:42.786031 52:54:00:40:24:a5 > Broadcast, ethertype 802.1Q (0x8100),
length 46: vlan 1000, p 3, ethertype ARP, Request who-has 10.0.0.19 tell
10.0.0.222, length 28

2 - enp2s0 tells me (INCORRECTLY) it is forwarding this packet out enp1s0
*WITH* the tag:
# tcpdump -n -e -i enp1s0
<...>
09:54:47.107468 52:54:00:40:24:a5 > Broadcast, ethertype 802.1Q (0x8100),
length 46: vlan 1000, p 3, ethertype ARP, Request who-has 10.0.0.19 tell
10.0.0.222, length 28

3 - Switch trunk port learns no MAC address on vlan 1000. As soon as I
configure the switch as ACCESS port I learn the VM MAC address. This tells
me the VLAN tag is being removed from the ethernet frame before sending it
to the switch. VM1 learns NO mac address from anywhere with switch being
configured either as trunk or access (makes no difference).

4 - I tried disabling all possible NIC offloads with ethtool with no luck.
It seems the NIC is somehow stripping the VLAN tag out of the packet before
putting on the wire.

# ethtool -K enp1s0 gro off gso off rxvlan off txvlan off rx off tx off sg
off rxhash off

5 - I also tried Ubuntu 16.04 and 16.10, and also tried replacing e1000e
NIC with a realtek one, getting the exact same results.

Any help figuring this out is greatly appreciated. Not sure I am
missing/skipping a step or if this is a bug.

Thank you

Paulo

# ethtool -i enp1s0
driver: e1000e
version: 3.2.5-k
firmware-version: 1.8-0
bus-info: 0000:01:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: no

# ethtool -d enp1s0
MAC Registers
-------------
0x00000: CTRL (Device control register)  0x18100248
      Endian mode (buffers):             little
      Link reset:                        reset
      Set link up:                       1
      Invert Loss-Of-Signal:             no
      Receive flow control:              enabled
      Transmit flow control:             enabled
      VLAN mode:                         disabled
      Auto speed detect:                 disabled
      Speed select:                      1000Mb/s
      Force speed:                       no
      Force duplex:                      no
0x00008: STATUS (Device status register) 0x00080783
      Duplex:                            full
      Link up:                           link config
      TBI mode:                          disabled
      Link speed:                        1000Mb/s
      Bus type:                          PCI
      Bus speed:                         33MHz
      Bus width:                         32-bit
0x00100: RCTL (Receive control register) 0x0400801A
      Receiver:                          enabled
      Store bad packets:                 disabled
      Unicast promiscuous:               enabled
      Multicast promiscuous:             enabled
      Long packet:                       disabled
      Descriptor minimum threshold size: 1/2
      Broadcast accept mode:             accept
      VLAN filter:                       disabled
      Canonical form indicator:          disabled
      Discard pause frames:              filtered
      Pass MAC control frames:           don't pass
      Receive buffer size:               2048
0x02808: RDLEN (Receive desc length)     0x00001000
0x02810: RDH   (Receive desc head)       0x00000003
0x02818: RDT   (Receive desc tail)       0x00000000
0x02820: RDTR  (Receive delay timer)     0x00000020
0x00400: TCTL (Transmit ctrl register)   0x3103F0FA
      Transmitter:                       enabled
      Pad short packets:                 enabled
      Software XOFF Transmission:        disabled
      Re-transmit on late collision:     enabled
0x03808: TDLEN (Transmit desc length)    0x00001000
0x03810: TDH   (Transmit desc head)      0x00000003
0x03818: TDT   (Transmit desc tail)      0x00000003
0x03820: TIDV  (Transmit delay timer)    0x00000008
PHY type:                                unknown

# brctl show
bridge name bridge id STP enabled interfaces
mytrunk0 8000.6805ca4124ff no enp1s0
vnet0
# cat /etc/sysconfig/network-scripts/ifcfg-mytrunk0
DEVICE=mytrunk0
STP=no
TYPE=Bridge
BOOTPROTO=none
IPV4_FAILURE_FATAL=no
IPV6INIT=no
NAME=mytrunk0
ONBOOT=yes
NM_CONTROLLED=no

# cat /etc/sysconfig/network-scripts/ifcfg-enp1s0
TYPE=Ethernet
BOOTPROTO=none
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=no
IPV6_AUTOCONF=no
IPV6_DEFROUTE=no
IPV6_PEERDNS=no
IPV6_PEERROUTES=no
IPV6_FAILURE_FATAL=no
NAME=enp1s0
DEVICE=enp1s0
ONBOOT=yes
BRIDGE=mytrunk0

# virsh dumpxml VM1
<...>
    <interface type='bridge'>
      <mac address='52:54:00:40:24:a5'/>
      <source bridge='mytrunk0'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03'
function='0x0'/>
    </interface>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bridge/attachments/20161120/6b21b887/attachment.html>


More information about the Bridge mailing list