Adding and Removing SAN Disks from Linux Device Manager Multipath Systems

Add a new disk

This assumes you have created an array on the SAN and allocated space to a logical volume on it; you have mapped a LUN pointing that logical volume to that host, and that the host is correctly zoned to see the SAN in the fibre channel fabric.

• Before anything, run multipath -ll to see what is there currently.
• See how many HBAs are connected (and zoned) to the SAN – you need to repeat the commands for each one. For example:

            echo 1 > /sys/class/fc_host/host0/issue_lip
            echo 1 > /sys/class/fc_host/host1/issue_lip
            echo "- - -" > /sys/class/scsi_host/host0/scan
            echo "- - -" > /sys/class/scsi_host/host0/scan
  		  
  

• After running those commands, check that something happened by using dmesg and /var/log/messages.
• Run multipath -v2to get multipath to pick it up – you can then compare the listing to the previously run command.
  Note the scsi devices for the new disk, it will be sdX and sdY or whatever.
• Edit /etc/lvm/lvm.conf and make sure these are being filtered to remove duplicates – use vgdisplay -vvto show what LVM considers duplicate.
  FYI: device mapper / multipath create multiple device handles for the same device, this can cause delays with LVM2 and severely impact throughput.
• Now you can pvcreate /dev/dm-XX, vgextend VolGroup dev/dm-XX, etc.

Remove a disk

• Run the multipath -ll command, note the UUID (the big hex number), LUN and sdX device of the disk, eg in the example below it is LUN 2:, and they are /dev/sdf and /dev/sde– you will need this info for the procedure. Just confirm this is in fact the one you want to remove – cross-check the LUN and size of the volume on the SAN before proceeding…

        3600a0b80000fb6e50000000e487b02f5 dm-10 IBM,1742
        [size=1.6T][features=1 queue_if_no_path][hwhandler=1 rdac]
        \_ round-robin 0 [prio=6][active]
        \_ 1:0:0:2 sdf 8:80 [active][ready]
        \_ round-robin 0 [prio=1][enabled]
        \_ 0:0:0:2 sde 8:64 [active][ghost]
  

  Note – the dm-XX is not permanent and may change when you’ve added or removed disks, so don’t rely on old info – check each time.
• Also cat /proc/scsi/scsi to see match up the kernel SCSI devices with the SCSI IDs and LUNs of the SAN disks.

• First you need to remove the disk from the volume group.
  • If the disk is in use, either delete what is on it (if there is a logical volume limited to that disk), or use pvmove. (this of course assumes you have sufficient space to move everything off the original disk)
    NB – with pvmove on SUSE10 sp2, there is a bug where if you are moving from a bigger disk to smaller disk(s), it may complain there isn’t enough space. Just move as many extents as are on the first smaller disk, the you can move the rest on to the second, eg: pvmove /dev/dm-1:0-20000 /dev/dm-2.
  • Once stuff is deleted/removed: vgreduce VolGroup dev/dm-XX and pvremove /dev/dm-XX.
• Using the disk ID for the next command (the dm-xx isn’t recognised), from multipath -ll:
  dmsetup remove 3600a0b80000f7b270000000b47b15c26. (of course you need to use your own disk ID)
• Now, finally, you remove the SCSI devices from the kernel:

      echo 1 > /sys/block/sdX/device/delete
      echo 1 > /sys/block/sdY/device/delete
	  

You should now have all traces removed, you can run multipath -ll and cat /proc/scsi/scsi to cross check. You can now remove the mapping from the SAN and delete the logical volume if required.