Installing RHEL5 on multipath devices
After some time while I've tried to install RHEL5 on multipath devices with root on dm-multipath, I finally found the time to write down the findings I've made.
The following article assumes that you want to install RHEL5 on POWER5/POWER6 based server (IBM System p) with two VIOS servers mapping LUNs through the hypervisor to the LPAR(s).
Nevertheless the description should also work for non-virtualized environments but I have not tested it - experiences etc. welcome!
Requirements
Assuming that you have an IBM System p (POWER5/POWER6) based system you'll need:
- Two Virtual I/O Server with recommended level 1.5.1-FP10.1
- A network installation server capable of serving RHEL5 installation images for ppc to a system/LPAR
- One ore more LUNs from a SAN that you're mapping through both VIOS to the LPAR - which means, from an LPAR point of view, that the LPAR will see these LUNs over two or more paths.
- As "best-practice" use separate vscsis-vscsic pairs for the root volumes. This should be done because RHEL is critical about changes to the major-minor numbers of the devices. To be safe, use an own pair for root!
 | Don't forget to change the reserve_policy of each drive you're planning to map through more than one VIOS from single_path (default) to no_reserve.
You can check this by issuing the command "lsdev -dev hdiskN -attr" on each VIOS - pls. replace hdiskN with the correct name of the LUN.
You can change the reserve_policy for a drive by issuing the command "chdev -dev hdiskN -attr reserve_policy=no_reserve" on each VIOS - pls. replace hdiskN with the correct name of the LUN. |
Installation
The installation of a LPAR using dm-multipath on RHEL5 is much AIX like - which means you just don't have to think much - it just works.
The important part is the correct kernel boot prompt parameter you'll have to pass to the install kernel in order to get dm-multipath loaded for installation. The parameter is mpath!
So boot to the open firmware prompt and start the installation as normal (i.e. define an alias for the network card and start the installation. Below is an example, using AIX's NIM server as installation source. If you'll like using DHCP you don't need all the ip parameters behind the "net".
0 > ls
...
000000d1e1e0: /vdevice
000000d20ce0: /vty@30000000
000000d21bc0: /v-scsi@30000002
000000d2b2d0: /disk
000000d2c5a8: /tape
000000d2dc70: /v-scsi@30000003
000000d37380: /disk
000000d38658: /tape
000000d39d20: /l-lan@30000004
000000d41568: /v-scsi@30000005
000000d4ac78: /disk
000000d4bf50: /tape
000000d4d618: /v-scsi@30000006
000000d56d28: /disk
000000d58000: /tape
000000d598a8: /IBM,sp@4000
000000d5a860: /rtc@4001
000000d5b168: /nvram@4002
ok
0 > devalias net /vdevice/l-lan@30000004 ok
0 > boot net:9.154.2.90,,9.154.2.105 insmod=ibmveth mpath ip=9.154.2.105 netmask=255.255.0.0 gateway=9.154.2.1 dns=9.64.163.21,9.64.162.21
Here's an example using kickstart:
0 > ls
...
000000d1e1e0: /vdevice
000000d20ce0: /vty@30000000
000000d21bc0: /v-scsi@30000002
000000d2b2d0: /disk
000000d2c5a8: /tape
000000d2dc70: /v-scsi@30000003
000000d37380: /disk
000000d38658: /tape
000000d39d20: /l-lan@30000004
000000d41568: /v-scsi@30000005
000000d4ac78: /disk
000000d4bf50: /tape
000000d4d618: /v-scsi@30000006
000000d56d28: /disk
000000d58000: /tape
000000d598a8: /IBM,sp@4000
000000d5a860: /rtc@4001
000000d5b168: /nvram@4002
ok
0 > devalias net /vdevice/l-lan@30000004 ok
0 > boot net:9.154.2.90,,9.154.2.105 insmod=ibmveth mpath ip=9.154.2.105 netmask=255.255.0.0 gateway=9.154.2.1 dns=9.64.163.21,9.64.162.21 ks=nfs:9.154.2.90:/export/linux/autoinstall/kickstart/rhel5_dm-multipath.cfg ksdevice=eth0
After the installation started an you've entered diskdruid you'll see that RHEL has changed your disks from sda,... to mpath0... and that's what we want to see ;-D.
After you've finished the installation check on the command line of the system if multipathing is working the way you're expecting - and that's it!
Kickstart
The installation of RHEL5 with root on dm-multipath with kickstart is very easy. I've attached a working kickstart example.
Please note that you'll have to start the installation as descriped above with the parameter mpath.
Below is the relevant part of the kickstart file for dm-multipath.
...
clearpart --linux --drives=mapper/mpath0
part prepboot --fstype "PPC PReP Boot" --noformat --onpart mapper/mpath0p1
part /boot --fstype ext3 --size=100 --ondisk=mapper/mpath0
part pv.7 --size=0 --grow --ondisk=mapper/mpath0
volgroup vg00 --pesize=32768 pv.7
logvol swap --fstype swap --name=swaplv --vgname=vg00 --size=1000 --grow --maxsize=1984
logvol / --fstype ext3 --name=rootlv --vgname=vg00 --size=1024 --grow
...
Enjoy!
