IBM DS3500 Multipathing / vSphere MPIO


Below I’m summrizing how I have configured the IBM DS3500 SAN Storage for my DR Site.

Each SCSI port in the IBM DS3500 with different subnet,

  • Port-3 10.10.10.1
  • Port-4 10.10.20.1
  • Port-5 10.10.30.1
  • Port-6 10.10.40.1 /24

 

The Storage Processor Ports are connected to the Dell 2724 switches, 10.x subnet goes into Port-1 in pSwitch1 and subnet 20.x goes into pSwitch1 Port-2, Subnet 30.x goes into pSwitch2 into Port-1 and Subnet 40.x goes into pSwitch2 Port-2.

In the ESX side, one vSwitch attached to two vmnic’s vmnic2 and vmnic3. In this vSwitch  created four VMkernel Portgroups,

  • iSCSI-01 10.10.10.30
  • iSCSI-02 10.10.20.30
  • iSCSI-03 10.10.30.30
  • iSCSI-04 10.10.40.30

If all the nics are active at the same time and try to vmkping to the Storage Processor, (DUP!) Duplicate pings from one of the subnet, then the ping will stop working but it will work for the other Storage Processor.

To overcome this issue is to have the vmnic2 is active for iSCSI-01 and iSCSI-02 and vmnic3 unused for iSCSI-01 and iSCSI-02. The same for the other Portgroups, vmnic3 is Active only for iSCSI-03 and iSCSI-04 and vmnic2 is unused these two Portgroups. After that the ping is steady without (DUP!).

iSCSI Binding is done on the iSCSI adapter to have all the VMkernel Portgroups. esxcli swiscsi nic add -n -d vmk# vmhba33.

After that, all the four paths are appeared, and changed the Storage Path to Round Robin to have them all Active(I/O) at the same time.

Testing were done, by switching off pSwitch1 and connection were alive to the SAN Storage, and the number of paths changed from four to Two. Turned off pSwitch2 and the connection were alive too to the SAN Storage and the number of paths changed from four to two.

As per IBM, they say it’s not the recommended way of having this config and its unsupported  I don’t know the reason why it’s not supported / recommended. The only support method they say as per the IBM engineer who installed this SAN Storage is to have only two paths if only one Controller is installed and cannot put four connections!!! But he surprised of what he has seen from my configuration :). As per him, he said I can use the 8 Paths ONLY if the second controller is installed, which is logically I can’t find any answer for it why is that!!!!

Advertisements

, , , , , , , ,

  1. #1 by Pm on February 27, 2012 - 8:37 am

    Hi, i would need to setup a similar configuration (two esx 4.1 for HA and ds3500 iscsi with both controllers, from each ds3500 controller two connections – one connection to a first cisco physical switch, the second connection to a second cisco physical switch, each esx has two dedicated physical nics for iscsi) . Could you please advise me on how to configure vmkernel ports, vlans etc? Do I need 4 vlans (subnets) for iscsi available or 2 vlans are enough (two iscsi interfaces on the second controller could use the same two vlans as the first iscsi interfaces? Thank you.

  2. #2 by habibalby on February 28, 2012 - 7:33 am

    Hi, with two controllers, you will need four subnets from iSCSI port in the CTRL-A and four subnets from iSCSI port in CTRL-B.

    Make sure your pSwitches are stacked/cascaded together.
    Create vLAN for the whole switch “if these switches are only for iSCSI connectivity”
    Connect two iSCSI ports from each CTRL-A to pSwitch1 and two iSCSI ports from CTRL-A to pSwitch2.
    Do the same for CTRL-B.
    How many interfaces you have in each ESX? if they are four it’s perfect, if they are two you can do the work as well.

    Connect each ESX pNIC into one of the pSwitches. Then create vMKernel PortGroup for each subnet. And configure Nic Teaming in the vSwitch to have dedicated iSCSI Ports to map pNIC in ESX.

    iSCSI-10 “VMKernel” -> iSCSI Port 10.10.10.1 mapped to vmnic0
    iSCSI-11 “VMKernel” -> iSCSI Port 10.10.11.1 mapped to vmnic0

    iSCSI-12 “VMKernel” -> iSCSI Port 10.10.12.1 mapped to vmnic1
    iSCSI-13 “VMKernel” -> iSCSI Port 10.10.13.1 mapped to vmnic1

    And so on.

    Thanks,

    • #3 by Pm on February 29, 2012 - 7:01 pm

      Hi, I don’t need 4 connections from each ds3500 controller but only 2 from each controller (each one connected to one of the physical switches). I have two physical nics on an each ESX server. I have two ESX for vmware HA. Physical servers are not dedicated to iSCSI communication only but also for vm machines public VLAN, vMotion VLAN and service console VLAN. If I am correct you say there is one VLAN with four configured subnets for iSCSI because DS3500 controllers require to have each iSCSI interface on different subnets. Can I have 4 VLANs for iSCSI instead? Will it work? Will it be any benefits to have VLANs instead of subnets for iSCSI just because networking experts say it is recommended to have one VLAN equals one subnet? The main reason I need a redundant networking is failover not load balancing. Do I need interlink between physical switches? MRU policy would be enough. Thank you again for any hints.

  3. #4 by Nabil on November 20, 2012 - 11:03 am

    Hi ,
    i have same configuration but with ESXi 5.0 Update 1 ,that is also supported with the latest DS3500 firmware . we have two controllers installed on the array , and all the four nic are connected in redundancy schema to two DELL switch .
    we configured groupports as you did with separate vlans etc etc , and i got 132 path trough all this

    the only thing is when we reboot the switchs we lose storage accès till we reboot the DS3500 !!
    i dont understand where it can be failing ..

    can you telme if you have any idea ?

    • #5 by habibalby on November 20, 2012 - 1:49 pm

      Hello,
      How many interfaces you have in each host? Did you configure the LUNs as RR?

      Have you done with VMKernel ports binding within the vSwitch ?

      Have you made the pNICs as Active for certain Port Groups and not used for the other ports and vice versa?

      All of the above if not configured correctly it does gives the same symptoms you described.

      Pls check them back and let me know how it goes.

      Thanks

    • #6 by habibalby on November 20, 2012 - 7:57 pm

      Hi, don’t forget to put each pNIC of the hosts to each pSwitch the same for the controllers each controller iSCSI ports to pSwitch, the vmKernel portgroup must be map to the pNIC where the controller is plugged.

      Don’t split the controller ports among the two pSwitch, keep Ctrl0 ports to pSwitch0 and Ctrl1 ports xuto pSwitch1

      I;e iSCSI0 > VMNIC0 > pSwitch0 > Ctrl0 “VMNIC1 unused”.
      iSCSI1 > VMNIC1 > pSwitch1 > Ctrl1 “VMNIC0 unused”.

  4. #7 by veloura.Blogspot.com on July 19, 2013 - 8:47 pm

    Your way of describing the whole thing in this article is truly pleasant, every one
    be able to effortlessly understand it, Thanks a lot.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: