IO multipathing with RHEL 5

In this post, we describe working IO multipathing configuration. Used technologies: RedHat Linux 5.4 64bit, 2x SunFire x4150, Sun StorageTek 2540, 2x Qlogic FC HBA.

1. Multipath.conf is shortened (only few devices are showed). Internal disks are ignored (vendor "Sun"). Also note, that changes to uid or more on multipath devices are visible only after reboot (bug?).

# cat /etc/multipath.conf
blacklist {
       devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
       devnode "^hd[a-z][[0-9]*]"
       device {
               vendor Sun
       }
}

devices {
       device {
               vendor                  "SUN"
               product                 "LCSM100_F"
               getuid_callout          "/sbin/scsi_id -g -u -s /block/%n"
               prio_callout            "/sbin/mpath_prio_rdac /dev/%n"
               features                "0"
               hardware_handler        "1 rdac"
               path_grouping_policy    group_by_prio
               failback                immediate
               rr_weight               uniform
               no_path_retry           queue
               rr_min_io               1000
               path_checker            rdac
       }
}

multipaths {
        multipath {
                wwid    3600a0b80005b0a760000042b4ad4bf74
                alias   s2-dbdata1
                mode    660
                uid     500
                gid     500
        }
        multipath {
                wwid    3600a0b80005b15e3000004594ad4bf47
                alias   s2-dbredo1
                mode    660
                uid     500
                gid     500
        }
}

2. In lvm.conf, it is good to modify two lines, to reduce LVM discovery time.

/etc/lvm/lvm.conf:
  filter = [ "a/dev/mpath/.*/", "a/dev/sda.*/", "a/dev/sdb.*/", "a/dev/md.*/", "r/.*/" ]
  types = [ "device-mapper", 1]

3. Modify also udev rules, to shorten bootup time:

/etc/udev/rules.d/05-udev-early.rules:
##ACTION=="add", DEVPATH=="/devices/*", ENV{PHYSDEVBUS}=="?*", WAIT_FOR_SYSFS="bus"

4. Modprobe.conf:

/etc/modprobe.conf:
alias eth0 e1000e
alias eth1 e1000e
alias eth2 e1000e
alias eth3 e1000e
alias scsi_hostadapter aacraid
alias scsi_hostadapter1 ata_piix
alias scsi_hostadapter2 ahci
alias scsi_hostadapter3 usb-storage
alias scsi_hostadapter4 qla2xxx
alias scsi_hostadapter5 dm_multipath
alias scsi_hostadapter6 scsi_dh_rdac

5. Last step is critical, recreate initrd so scsi_dh_rdac is loaded first (otherwise, there will be tons of errors during boot). Also please note, that you need at least update 3 of RHEL 5.

[root@db2 boot]# mkinitrd -v -f initrd-2.6.18-164.2.1.el5.img 2.6.18-164.2.1.el5 --preload scsi_dh_rdac