Linux -- Disabling a path in a Multipath configuration

You may need to disable a path in the multipath enabled Linux system. Maybe you may need to simulate a path failure.. Such a need may arise to prove that multipath can handle a path failure..
Also , it may be required for making some sort of perfomance diagnostics in a active-active multipath configuration.

I have recently done such an operation , and want it to share with you..

We start with checking multipath paths using multipath -ll command.

Here is an example:
Suppose we want to disable path which is through 3:0:3:0, device /dev/sdp

Our multipath.conf

defaults {
        udev_dir                /dev
        polling_interval        10
        selector                "round-robin 0"
        path_grouping_policy    failover
        getuid_callout          "/sbin/scsi_id -g -u -s /block/%n"
        prio_callout            "/bin/true"
        path_checker            tur
        rr_min_io               100
        rr_weight               uniform
        failback                immediate
        no_path_retry           12
        user_friendly_names     yes
}

mpath9 (36006016029b0380029dbd37cf734e411) dm-4 DGC,VRAID
[size=1.5T][features=1 queue_if_no_path][hwhandler=1 emc][rw]
\_ round-robin 0 [prio=1][active]
\_ 3:0:3:0 sdp 8:240 [active][ready]
\_ round-robin 0 [prio=0][enabled]
\_ 2:0:5:0 sdh 8:112 [active][ready]

Note that:

active - Path group currently receiving I/O requests.
enabled - Path groups to try if the active path group has no paths in the ready state.
disabled - Path groups to try if the active path group and all enabled path groups have no paths in the active state.The disabled state only exists for certain storage arrays.

As you see the devices /dev/sdp and /dev/sdh  are enabled. Altough they are not working as active/active (because we use failover not multibus as path_grouping_policy) , lets proceed by executing lsscsi and lsscsi to see the source of these devices to ensure that their transport protocol is FCP.

(this step is optional)

lsscsi -H; lsscsi -g

[0]    ata_piix      

[1]    ata_piix      

[2]    qla2xxx       

[3]    qla2xxx       

[0:0:0:0]    cd/dvd  TEAC     DV-28S-W         C.2C  /dev/scd0  /dev/sg0 

[2:0:0:0]    disk    DGC      RAID 5           0223  /dev/sda   /dev/sg1 

[2:0:1:0]    storage HP       HSV300           0953  -          /dev/sg2 

[2:0:1:1]    disk    HP       HSV300           0953  /dev/sdb   /dev/sg3 

[2:0:1:2]    disk    HP       HSV300           0953  /dev/sdc   /dev/sg4 

[2:0:1:3]    disk    HP       HSV300           0953  /dev/sdd   /dev/sg5 

[2:0:2:0]    storage HP       HSV300           0953  -          /dev/sg6 

[2:0:2:1]    disk    HP       HSV300           0953  /dev/sde   /dev/sg7 

[2:0:2:2]    disk    HP       HSV300           0953  /dev/sdf   /dev/sg8 

[2:0:2:3]    disk    HP       HSV300           0953  /dev/sdg   /dev/sg9 

[2:0:3:0]    storage HP       MSA CONTROLLER   7.20  -          /dev/sg10

[2:0:4:0]    storage HP       MSA CONTROLLER   7.20  -          /dev/sg11

[2:0:5:0]    disk    DGC      VRAID            0533  /dev/sdh   /dev/sg12

[3:0:0:0]    disk    DGC      RAID 5           0223  /dev/sdi   /dev/sg13

[3:0:1:0]    storage HP       HSV300           0953  -          /dev/sg14

[3:0:1:1]    disk    HP       HSV300           0953  /dev/sdj   /dev/sg15

[3:0:1:2]    disk    HP       HSV300           0953  /dev/sdk   /dev/sg16

[3:0:1:3]    disk    HP       HSV300           0953  /dev/sdl   /dev/sg17

[3:0:2:0]    storage HP       HSV300           0953  -          /dev/sg18

[3:0:2:1]    disk    HP       HSV300           0953  /dev/sdm   /dev/sg19

[3:0:2:2]    disk    HP       HSV300           0953  /dev/sdn   /dev/sg20

[3:0:2:3]    disk    HP       HSV300           0953  /dev/sdo   /dev/sg21

[3:0:3:0]    disk    DGC      VRAID            0533  /dev/sdp   /dev/sg22

lsscsi -t
[0:0:0:0] cd/dvd ata: /dev/scd0
[2:0:0:0] disk fc:0x500601604be027be,0x010b00 /dev/sda
[2:0:1:0] storage fc:0x50014380013c6cf9,0x010000 -
[2:0:1:1] disk fc:0x50014380013c6cf9,0x010000 /dev/sdb
[2:0:1:2] disk fc:0x50014380013c6cf9,0x010000 /dev/sdc
[2:0:1:3] disk fc:0x50014380013c6cf9,0x010000 /dev/sdd
[2:0:2:0] storage fc:0x50014380013c6cfd,0x010100 -
[2:0:2:1] disk fc:0x50014380013c6cfd,0x010100 /dev/sde
[2:0:2:2] disk fc:0x50014380013c6cfd,0x010100 /dev/sdf
[2:0:2:3] disk fc:0x50014380013c6cfd,0x010100 /dev/sdg
[2:0:3:0] storage fc:0x500508b30093e201,0x010700 -
[2:0:4:0] storage fc:0x500508b30093e209,0x010900 -
[2:0:5:0] disk fc:0x5006016808603b82,0x010a00 /dev/sdh
[3:0:0:0] disk fc:0x500601684be027be,0x010700 /dev/sdi
[3:0:1:0] storage fc:0x50014380013c6cf8,0x010000 -
[3:0:1:1] disk fc:0x50014380013c6cf8,0x010000 /dev/sdj
[3:0:1:2] disk fc:0x50014380013c6cf8,0x010000 /dev/sdk
[3:0:1:3] disk fc:0x50014380013c6cf8,0x010000 /dev/sdl
[3:0:2:0] storage fc:0x50014380013c6cfc,0x010100 -
[3:0:2:1] disk fc:0x50014380013c6cfc,0x010100 /dev/sdm
[3:0:2:2] disk fc:0x50014380013c6cfc,0x010100 /dev/sdn
[3:0:2:3] disk fc:0x50014380013c6cfc,0x010100 /dev/sdo 

[3:0:3:0] disk fc:0x50014380013c6cfc,0x010100 /dev/sdp 

Fibre Channel (FC)...

Okay, their transport protocol is FCP ..

Then , we continue with disabling the multipath device by using the following;

echo "offline" > /sys/block/sdp/device/state

This will offline the disk , and multipath will see the associated path as faulty. 

To clear the path completely from multipath -ll output, we remove the device from the SCSI subsystem as follows;

echo "scsi remove-single-device 3 0 3 0" > /proc/scsi/scsi

Lastly, we execute multipath -ll command to see that mpath9 now uses only one path...

mpath9 (36006016029b0380029dbd37cf734e411) dm-4 DGC,VRAID
[size=1.5T][features=1 queue_if_no_path][hwhandler=1 emc][rw]
\_ round-robin 0 [prio=0][active]

If we need to enable the path again;

we need to make linux rescan the scsi bus  and use multipath -v2 command to make multipath detect the changes..

echo "- - -" > /sys/class/scsi_host/${HBA}/scan     -> we may scan all the HBA hosts..

multipath -v2