Archive for March, 2018

VMWare/ESX/vSphere 6.5 – Network Disconnect with DELL R730 Servers Having vLAN Tagging…!!!


In environment I have 13 hosts Dell R730 and R630 Dell Blade Servers. I recently upgraded Dell R730 servers with the Dell customized ESXi ISO “VMware-VMvisor-Installer-6.5.0.update01-7388607.x86_64-DellEMC_Customized-A07.iso” When the hosts finish upgrading, and exist the maintenance mode, Alert start appearing on the host related to network disconnection issues..When I started looking and the errors, and match it with the Physical Network Adapters; few googles and I came across VMWare KB about this issue and its related to the vLAN Tagging. I made a test by removing one VMNIC and created another temp vSwitch and VM Network PortGroup without vLAN Tagging and it stays UP at all the time.

The case of the group which causing both pNICs goes up and down is the Broadcom NetXtreme BCM57810 10G

vmnic4 0000:006:00.0 bnx2x Up 10000 Full 00:0e:1e:79:a2:b0 1500 Broadcom Corporation NetXtreme II BCM57810 10 Gigabit Ethernet
vmnic5 0000:006:00.1 bnx2x Up 10000 Full 00:0e:1e:79:a2:b2 1500 Broadcom Corporation NetXtreme II BCM57810 10 Gigabit Ethernet

Resolution;

By removing the network driver bnx2x from the ESXi host and reboot the host, the network becomes stable;

~ # esxcli network nic list
Name PCI Device Driver Link Speed Duplex MAC Address MTU Description
—— ————- —— —- —– —— —————– —- ————————————————————–
vmnic0 0000:001:00.0 tg3 Up 1000 Full b0:83:fe:d4:de:7c 1500 Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet
vmnic1 0000:001:00.1 tg3 Up 1000 Full b0:83:fe:d4:de:7d 1500 Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet
vmnic2 0000:002:00.0 tg3 Up 1000 Full b0:83:fe:d4:de:7e 1500 Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet
vmnic3 0000:002:00.1 tg3 Up 1000 Full b0:83:fe:d4:de:7f 1500 Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet
vmnic4 0000:006:00.0 bnx2x Up 10000 Full 00:0e:1e:79:a2:b0 1500 Broadcom Corporation NetXtreme II BCM57810 10 Gigabit Ethernet
vmnic5 0000:006:00.1 bnx2x Up 10000 Full 00:0e:1e:79:a2:b2 1500 Broadcom Corporation NetXtreme II BCM57810 10 Gigabit Ethernet
vmnic6 0000:005:00.0 bnx2 Up 1000 Full 00:10:18:64:d1:2c 1500 Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T
vmnic7 0000:005:00.1 bnx2 Up 1000 Full 00:10:18:64:d1:2e 1500 Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T

=================================================================================

vmnic0 0000:01:00.0 ntg3 Up Up 1000 Full b0:83:fe:d4:de:7c 1500 Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet
vmnic1 0000:01:00.1 ntg3 Up Up 1000 Full b0:83:fe:d4:de:7d 1500 Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet
vmnic2 0000:02:00.0 ntg3 Up Up 1000 Full b0:83:fe:d4:de:7e 1500 Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet
vmnic3 0000:02:00.1 ntg3 Up Up 1000 Full b0:83:fe:d4:de:7f 1500 Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet
vmnic4 0000:06:00.0 bnx2x Up Down 0 Half 00:0e:1e:79:a2:b0 1500 Broadcom Corporation QLogic 57810 10 Gigabit Ethernet Adapter
vmnic5 0000:06:00.1 bnx2x Up Down 0 Half 00:0e:1e:79:a2:b2 1500 Broadcom Corporation QLogic 57810 10 Gigabit Ethernet Adapter
vmnic6 0000:05:00.0 bnx2 Up Up 1000 Full 00:10:18:64:d1:2c 1500 Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T
vmnic7 0000:05:00.1 bnx2 Up Up 1000 Full 00:10:18:64:d1:2e 1500 Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T

Remove the driver which is causing this issue;

~ # esxcli software vib remove -n net-bnx2x
Removal Result
Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
Reboot Required: true
VIBs Installed:
VIBs Removed: Broadcom_bootbank_net-bnx2x_2.710.39.v55.2-1OEM.550.0.0.1331820
VIBs Skipped:

Reboot your host, and the updated and support driver will automatically be installed;

[root@esx03:~] esxcli network nic list
Name PCI Device Driver Admin Status Link Status Speed Duplex MAC Address MTU Description
—— ———— —— ———— ———– —– —— —————– —- ————————————————————-
vmnic0 0000:01:00.0 ntg3 Up Up 1000 Full b0:83:fe:d1:72:de 1500 Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet
vmnic1 0000:01:00.1 ntg3 Up Up 1000 Full b0:83:fe:d1:72:df 1500 Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet
vmnic2 0000:02:00.0 ntg3 Up Up 1000 Full b0:83:fe:d1:72:e0 1500 Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet
vmnic3 0000:02:00.1 ntg3 Up Up 1000 Full b0:83:fe:d1:72:e1 1500 Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet
vmnic4 0000:06:00.0 qfle3 Up Up 10000 Full 00:0e:1e:79:c9:20 1500 QLogic Inc. QLogic 57810 10 Gigabit Ethernet Adapter
vmnic5 0000:06:00.1 qfle3 Up Up 10000 Full 00:0e:1e:79:c9:22 1500 QLogic Inc. QLogic 57810 10 Gigabit Ethernet Adapter
vmnic6 0000:05:00.0 bnx2 Up Up 1000 Full 00:10:18:64:d1:68 1500 Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T
vmnic7 0000:05:00.1 bnx2 Up Up 1000 Full 00:10:18:64:d1:6a 1500 Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T

#HappyVirtualization

Advertisements

, , , , , , , , , ,

Leave a comment