Walk-thru- Deploying Lustre pre-built RPMs

Lustre server machines are simple to setup with CentOS 6. Pre-built kernel images are available from Whamcloud's build server. A number of different kernels and tools are continually built and tested. For this Walk-thru the lustre-master build will be used.

Whamcloud Lustre GA

Whamcloud produce Maintenance Releases and Feature Releases. This walk through covers running Lustre from the Master release. The Master release is subject to continuous integration testing but may have known or unknown issues. If you are evaluating Lustre for your production environment, you are advised to contact Whamcloud and substitute http://downloads.whamcloud.com/public/ for the RPMs you need for the walk-thru below.

Prerequisite.

A x86_64 machine with CentOS 6 installed. The machine has the hostname rhel6vanilla.

Procedure

Install Lustre directly with yum

Include the following entries into /etc/yum.repos.d/lustre.repo as follows:

/etc/repos.d/lustre.repo

[hpddLustreserver]
name=CentOS-$releasever - Lustre
baseurl=http://build.whamcloud.com/job/lustre-master/arch=x86_64%2Cbuild_type=server%2Cdistro=el6%2Cib_stack=inkernel/lastSuccessfulBuild/artifact/artifacts/
gpgcheck=0
 
[e2fsprogs]
name=CentOS-$releasever - Ldiskfs
baseurl=http://build.whamcloud.com/job/e2fsprogs-master/arch=x86_64%2Cdistro=el6/lastSuccessfulBuild/artifact/_topdir/RPMS/
gpgcheck=0

[hpddLustreclient]
name=CentOS-$releasever - Lustre
baseurl=http://build.whamcloud.com/job/lustre-master/arch=x86_64%2Cbuild_type=client%2Cdistro=el6%2Cib_stack=inkernel/lastSuccessfulBuild/artifact/artifacts/
gpgcheck=0

run yum install lustre-tests
provided the step above completes successfully, you can skip down to the section:Disable SELinux for Lustre patched kernel.
If you experience dependency issues with yum, the best known method is to downgrade the current kernel to the version that is needed and try again.
If you are installing on RHEL and don't have Red Hat Network enabled, you will need to use the DVD ISO as a repo to satisfy unmet dependencies. This is covered on http://serverfault.com/questions/156458/

Install Lustre patched kernel.

If you choose not to use the yum install method above, you can install Lustre as follows:

From lustre-master build, download the correct kernel RPMs for your platform
- kernel
- kernel-firmware

Install these RPMs with yum localinstall ./kernel*

[root@rhel6vanilla ~]# yum localinstall ./kernel*
Loaded plugins: fastestmirror, presto
Setting up Local Package Process
Examining ./kernel-2.6.32-279.19.1.el6_lustre.x86_64.rpm: kernel-2.6.32-279.19.1.el6_lustre.x86_64
Marking ./kernel-2.6.32-279.19.1.el6_lustre.x86_64.rpm as an update to kernel-2.6.32-279.el6.x86_64
Determining fastest mirrors
 * base: ftp.byfly.by
 * extras: ftp.byfly.by
 * updates: ftp.byfly.by
Examining ./kernel-firmware-2.6.32-279.19.1.el6_lustre.x86_64.rpm: kernel-firmware-2.6.32-279.19.1.el6_lustre.x86_64
Marking ./kernel-firmware-2.6.32-279.19.1.el6_lustre.x86_64.rpm as an update to kernel-firmware-2.6.32-279.el6.noarch
Resolving Dependencies
--> Running transaction check
---> Package kernel.x86_64 0:2.6.32-279.19.1.el6_lustre will be installed
.x86_64
--> Running transaction check
---> Package kernel-firmware.noarch 0:2.6.32-279.el6 will be updated
---> Package kernel-firmware.noarch 0:2.6.32-358.6.1.el6 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

=======================================================================================================================
 Package              Arch        Version                         Repository                                      Size
=======================================================================================================================
Installing:
 kernel               x86_64      2.6.32-279.19.1.el6_lustre      /kernel-2.6.32-279.19.1.el6_lustre.x86_64      113 M
Updating for dependencies:
 kernel-firmware      noarch      2.6.32-358.6.1.el6              updates                                         11 M

Transaction Summary
=======================================================================================================================
Install       1 Package(s)
Upgrade       1 Package(s)
Total size: 124 M
Total download size: 11 M
Is this ok [y/N]: y
Downloading Packages:
Setting up and reading Presto delta metadata
updates/prestodelta                                                                             | 339 kB     00:00     
Processing delta metadata
Download delta size: 3.4 M
kernel-firmware-2.6.32-279.el6_2.6.32-358.6.1.el6.noarch.drpm                                   | 3.4 MB     00:01     
Finishing rebuild of rpms, from deltarpms
<delta rebuild>                                                                                 |  11 MB     00:03     
Presto reduced the update size by 69% (from 11 M to 3.4 M).
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Updating   : kernel-firmware-2.6.32-358.6.1.el6.noarch                                                           1/3 
  Installing : kernel-2.6.32-279.19.1.el6_lustre.x86_64                                                            2/3 
  Cleanup    : kernel-firmware-2.6.32-279.el6.noarch                                                               3/3 
  Verifying  : kernel-2.6.32-279.19.1.el6_lustre.x86_64                                                            1/3 
  Verifying  : kernel-firmware-2.6.32-358.6.1.el6.noarch                                                           2/3 
  Verifying  : kernel-firmware-2.6.32-279.el6.noarch                                                               3/3 


Installed:
  kernel.x86_64 0:2.6.32-279.19.1.el6_lustre                                                                           

Dependency Updated:
  kernel-firmware.noarch 0:2.6.32-358.6.1.el6                                                                          

Complete!
[root@rhel6vanilla ~]#

Download e2fsprogs and Lustre server RPMSs.

Download the following e2fsprogs (and dependencies) RPMs from http://build.whamcloud.com/job/e2fsprogs-master
- e2fsprogs
- e2fsprogs-libs
- libss
- libcom_err
Download the following Lustre server RPMs from lustre-master build.
- lustre
- lustre-ldiskfs
- lustre-modules
- lustre-tests

Install Lustre.

yum localinstall ./lustre-* ./e2fsprogs-* ./libss-* ./libcom* should do it:

[root@rhel6vanilla ~]# yum localinstall ./lustre-* ./e2fsprogs-* ./libss-* ./libcom*
Loaded plugins: fastestmirror, presto
Setting up Local Package Process
Examining ./lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64.rpm: lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
Marking ./lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64.rpm to be installed
Loading mirror speeds from cached hostfile
 * base: ftp.byfly.by
 * extras: ftp.byfly.by
 * updates: ftp.byfly.by
Examining ./lustre-ldiskfs-3.3.0-2.6.32_279.19.1.el6_lustre.x86_64.x86_64.rpm: lustre-ldiskfs-3.3.0-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
Marking ./lustre-ldiskfs-3.3.0-2.6.32_279.19.1.el6_lustre.x86_64.x86_64.rpm to be installed
Examining ./lustre-modules-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64.rpm: lustre-modules-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
Marking ./lustre-modules-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64.rpm to be installed
Examining ./lustre-tests-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64.rpm: lustre-tests-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
Marking ./lustre-tests-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64.rpm to be installed
Examining ./e2fsprogs-1.42.7.wc1-7.el6.x86_64.rpm: e2fsprogs-1.42.7.wc1-7.el6.x86_64
Marking ./e2fsprogs-1.42.7.wc1-7.el6.x86_64.rpm as an update to e2fsprogs-1.41.12-12.el6.x86_64
Examining ./e2fsprogs-libs-1.42.7.wc1-7.el6.x86_64.rpm: e2fsprogs-libs-1.42.7.wc1-7.el6.x86_64
Examining ./libss-1.42.7.wc1-7.el6.x86_64.rpm: libss-1.42.7.wc1-7.el6.x86_64
Marking ./libss-1.42.7.wc1-7.el6.x86_64.rpm as an update to libss-1.41.12-12.el6.x86_64
Examining ./libcom_err-1.42.7.wc1-7.el6.x86_64.rpm: libcom_err-1.42.7.wc1-7.el6.x86_64
Marking ./libcom_err-1.42.7.wc1-7.el6.x86_64.rpm as an update to libcom_err-1.41.12-12.el6.x86_64
Resolving Dependencies
--> Running transaction check
---> Package e2fsprogs.x86_64 0:1.41.12-12.el6 will be updated
---> Package e2fsprogs.x86_64 0:1.42.7.wc1-7.el6 will be an update
---> Package e2fsprogs-libs.x86_64 0:1.41.12-12.el6 will be updated
---> Package e2fsprogs-libs.x86_64 0:1.42.7.wc1-7.el6 will be an update
---> Package libcom_err.x86_64 0:1.41.12-12.el6 will be updated
---> Package libcom_err.x86_64 0:1.42.7.wc1-7.el6 will be an update
---> Package libss.x86_64 0:1.41.12-12.el6 will be updated
---> Package libss.x86_64 0:1.42.7.wc1-7.el6 will be an update
---> Package lustre.x86_64 0:2.1.5-2.6.32_279.19.1.el6_lustre.x86_64 will be installed
--> Processing Dependency: /usr/bin/perl for package: lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
--> Processing Dependency: perl(File::Path) for package: lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
--> Processing Dependency: perl(File::Path) for package: lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
--> Processing Dependency: perl(Getopt::Long) for package: lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
--> Processing Dependency: perl(Getopt::Std) for package: lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
--> Processing Dependency: perl(POSIX) for package: lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
--> Processing Dependency: libnetsnmp.so.20()(64bit) for package: lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
--> Processing Dependency: libnetsnmpagent.so.20()(64bit) for package: lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
--> Processing Dependency: libnetsnmphelpers.so.20()(64bit) for package: lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
--> Processing Dependency: libnetsnmpmibs.so.20()(64bit) for package: lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
---> Package lustre-ldiskfs.x86_64 0:3.3.0-2.6.32_279.19.1.el6_lustre.x86_64 will be installed
---> Package lustre-modules.x86_64 0:2.1.5-2.6.32_279.19.1.el6_lustre.x86_64 will be installed
---> Package lustre-tests.x86_64 0:2.1.5-2.6.32_279.19.1.el6_lustre.x86_64 will be installed
--> Processing Dependency: libmpi.so.1()(64bit) for package: lustre-tests-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
--> Running transaction check
---> Package net-snmp-libs.x86_64 1:5.5-44.el6_4.1 will be installed
--> Processing Dependency: libsensors.so.4()(64bit) for package: 1:net-snmp-libs-5.5-44.el6_4.1.x86_64
--> Processing Dependency: libperl.so()(64bit) for package: 1:net-snmp-libs-5.5-44.el6_4.1.x86_64
---> Package openmpi.x86_64 0:1.5.4-1.el6 will be installed
--> Processing Dependency: librdmacm.so.1(RDMACM_1.0)(64bit) for package: openmpi-1.5.4-1.el6.x86_64
--> Processing Dependency: libnuma.so.1(libnuma_1.2)(64bit) for package: openmpi-1.5.4-1.el6.x86_64
--> Processing Dependency: libnuma.so.1(libnuma_1.1)(64bit) for package: openmpi-1.5.4-1.el6.x86_64
--> Processing Dependency: libibverbs.so.1(IBVERBS_1.1)(64bit) for package: openmpi-1.5.4-1.el6.x86_64
--> Processing Dependency: libibverbs.so.1(IBVERBS_1.0)(64bit) for package: openmpi-1.5.4-1.el6.x86_64
--> Processing Dependency: libgomp.so.1(OMP_1.0)(64bit) for package: openmpi-1.5.4-1.el6.x86_64
--> Processing Dependency: libgomp.so.1(GOMP_1.0)(64bit) for package: openmpi-1.5.4-1.el6.x86_64
--> Processing Dependency: environment-modules for package: openmpi-1.5.4-1.el6.x86_64
--> Processing Dependency: librdmacm.so.1()(64bit) for package: openmpi-1.5.4-1.el6.x86_64
--> Processing Dependency: libpsm_infinipath.so.1()(64bit) for package: openmpi-1.5.4-1.el6.x86_64
--> Processing Dependency: libnuma.so.1()(64bit) for package: openmpi-1.5.4-1.el6.x86_64
--> Processing Dependency: libltdl.so.7()(64bit) for package: openmpi-1.5.4-1.el6.x86_64
--> Processing Dependency: libibverbs.so.1()(64bit) for package: openmpi-1.5.4-1.el6.x86_64
--> Processing Dependency: libgomp.so.1()(64bit) for package: openmpi-1.5.4-1.el6.x86_64
--> Processing Dependency: libgfortran.so.3()(64bit) for package: openmpi-1.5.4-1.el6.x86_64
--> Processing Dependency: libesmtp.so.5()(64bit) for package: openmpi-1.5.4-1.el6.x86_64
---> Package perl.x86_64 4:5.10.1-130.el6_4 will be installed
--> Processing Dependency: perl(version) for package: 4:perl-5.10.1-130.el6_4.x86_64
--> Processing Dependency: perl(Pod::Simple) for package: 4:perl-5.10.1-130.el6_4.x86_64
--> Processing Dependency: perl(Module::Pluggable) for package: 4:perl-5.10.1-130.el6_4.x86_64
--> Running transaction check
---> Package environment-modules.x86_64 0:3.2.9c-4.el6 will be installed
--> Processing Dependency: libtcl8.5.so()(64bit) for package: environment-modules-3.2.9c-4.el6.x86_64
--> Processing Dependency: libX11.so.6()(64bit) for package: environment-modules-3.2.9c-4.el6.x86_64
---> Package infinipath-psm.x86_64 0:3.0.1-115.1015_open.1.1.el6_4 will be installed
---> Package libesmtp.x86_64 0:1.0.4-15.el6 will be installed
---> Package libgfortran.x86_64 0:4.4.7-3.el6 will be installed
---> Package libgomp.x86_64 0:4.4.7-3.el6 will be installed
---> Package libibverbs.x86_64 0:1.1.6-5.el6 will be installed
---> Package librdmacm.x86_64 0:1.0.17-0.git4b5c1aa.el6 will be installed
---> Package libtool-ltdl.x86_64 0:2.2.6-15.5.el6 will be installed
---> Package lm_sensors-libs.x86_64 0:3.1.1-17.el6 will be installed
---> Package numactl.x86_64 0:2.0.7-6.el6 will be installed
---> Package perl-Module-Pluggable.x86_64 1:3.90-130.el6_4 will be installed
---> Package perl-Pod-Simple.x86_64 1:3.13-130.el6_4 will be installed
--> Processing Dependency: perl(Pod::Escapes) >= 1.04 for package: 1:perl-Pod-Simple-3.13-130.el6_4.x86_64
---> Package perl-libs.x86_64 4:5.10.1-130.el6_4 will be installed
---> Package perl-version.x86_64 3:0.77-130.el6_4 will be installed
--> Running transaction check
---> Package libX11.x86_64 0:1.5.0-4.el6 will be installed
--> Processing Dependency: libX11-common = 1.5.0-4.el6 for package: libX11-1.5.0-4.el6.x86_64
--> Processing Dependency: libxcb.so.1()(64bit) for package: libX11-1.5.0-4.el6.x86_64
---> Package perl-Pod-Escapes.x86_64 1:1.04-130.el6_4 will be installed
---> Package tcl.x86_64 1:8.5.7-6.el6 will be installed
--> Running transaction check
---> Package libX11-common.noarch 0:1.5.0-4.el6 will be installed
---> Package libxcb.x86_64 0:1.8.1-1.el6 will be installed
--> Processing Dependency: libXau.so.6()(64bit) for package: libxcb-1.8.1-1.el6.x86_64
--> Running transaction check
---> Package libXau.x86_64 0:1.0.6-4.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved
=======================================================================================================================
 Package               Arch   Version                                 Repository                                  Size
=======================================================================================================================
Installing:
 lustre                x86_64 2.1.5-2.6.32_279.19.1.el6_lustre.x86_64 /lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
                                                                                                                  26 M
 lustre-ldiskfs        x86_64 3.3.0-2.6.32_279.19.1.el6_lustre.x86_64 /lustre-ldiskfs-3.3.0-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
                                                                                                                 8.4 M
 lustre-modules        x86_64 2.1.5-2.6.32_279.19.1.el6_lustre.x86_64 /lustre-modules-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
                                                                                                                 144 M
 lustre-tests          x86_64 2.1.5-2.6.32_279.19.1.el6_lustre.x86_64 /lustre-tests-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
Updating:
 e2fsprogs             x86_64 1.42.7.wc1-7.el6                        /e2fsprogs-1.42.7.wc1-7.el6.x86_64         2.6 M
 e2fsprogs-libs        x86_64 1.42.7.wc1-7.el6                        /e2fsprogs-libs-1.42.7.wc1-7.el6.x86_64    349 k
 libcom_err            x86_64 1.42.7.wc1-7.el6                        /libcom_err-1.42.7.wc1-7.el6.x86_64         56 k
 libss                 x86_64 1.42.7.wc1-7.el6                        /libss-1.42.7.wc1-7.el6.x86_64              67 k
Installing for dependencies:
 environment-modules   x86_64 3.2.9c-4.el6                            base                                        98 k
 infinipath-psm        x86_64 3.0.1-115.1015_open.1.1.el6_4           updates                                    160 k
 libX11                x86_64 1.5.0-4.el6                             base                                       584 k
 libX11-common         noarch 1.5.0-4.el6                             base                                       192 k
 libXau                x86_64 1.0.6-4.el6                             base                                        24 k
 libesmtp              x86_64 1.0.4-15.el6                            base                                        57 k
 libgfortran           x86_64 4.4.7-3.el6                             base                                       265 k
 libgomp               x86_64 4.4.7-3.el6                             base                                       118 k
 libibverbs            x86_64 1.1.6-5.el6                             base                                        45 k
 librdmacm             x86_64 1.0.17-0.git4b5c1aa.el6                 base                                        47 k
 libtool-ltdl          x86_64 2.2.6-15.5.el6                          base                                        44 k
 libxcb                x86_64 1.8.1-1.el6                             base                                       110 k
 lm_sensors-libs       x86_64 3.1.1-17.el6                            base                                        38 k
 net-snmp-libs         x86_64 1:5.5-44.el6_4.1                        updates                                    1.5 M
 numactl               x86_64 2.0.7-6.el6                             base                                        69 k
 openmpi               x86_64 1.5.4-1.el6                             base                                       2.1 M
 perl                  x86_64 4:5.10.1-130.el6_4                      updates      
perl-Module-Pluggable x86_64 1:3.90-130.el6_4                        updates                                     38 k
 perl-Pod-Escapes      x86_64 1:1.04-130.el6_4                        updates                                     31 k
 perl-Pod-Simple       x86_64 1:3.13-130.el6_4                        updates                                    210 k
 perl-libs             x86_64 4:5.10.1-130.el6_4                      updates                                    576 k
 perl-version          x86_64 3:0.77-130.el6_4                        updates                                     50 k
 tcl                   x86_64 1:8.5.7-6.el6                           base                                       1.9 M

Transaction Summary
=======================================================================================================================
Install      27 Package(s)
Upgrade       4 Package(s)

Total size: 213 M
Total download size: 18 M
Is this ok [y/N]: y
Downloading Packages:
Setting up and reading Presto delta metadata
Processing delta metadata
Package(s) data still to download: 18 M
(1/23): environment-modules-3.2.9c-4.el6.x86_64.rpm                                             |  98 kB     00:00     
(2/23): infinipath-psm-3.0.1-115.1015_open.1.1.el6_4.x86_64.rpm                                 | 160 kB     00:00     
(3/23): libX11-1.5.0-4.el6.x86_64.rpm                                                           | 584 kB     00:00     
(4/23): libX11-common-1.5.0-4.el6.noarch.rpm                                                    | 192 kB  
(5/23): libXau-1.0.6-4.el6.x86_64.rpm                                                           |  24 kB     00:00     
(6/23): libesmtp-1.0.4-15.el6.x86_64.rpm                                                        |  57 kB     00:00     
(7/23): libgfortran-4.4.7-3.el6.x86_64.rpm                                                      | 265 kB     00:00     
(8/23): libgomp-4.4.7-3.el6.x86_64.rpm                                                          | 118 kB     00:00     
(9/23): libibverbs-1.1.6-5.el6.x86_64.rpm                                                       |  45 kB     00:00     
(10/23): librdmacm-1.0.17-0.git4b5c1aa.el6.x86_64.rpm                                           |  47 kB     00:00     
(11/23): libtool-ltdl-2.2.6-15.5.el6.x86_64.rpm                                                 |  44 kB     00:00     
(12/23): libxcb-1.8.1-1.el6.x86_64.rpm                                                          | 110 kB     00:00     
(13/23): lm_sensors-libs-3.1.1-17.el6.x86_64.rpm                                                |  38 kB     00:00     
(14/23): net-snmp-libs-5.5-44.el6_4.1.x86_64.rpm                                                | 1.5 MB     00:01     
(15/23): numactl-2.0.7-6.el6.x86_64.rpm                                                         |  69 kB     00:00     
(16/23): openmpi-1.5.4-1.el6.x86_64.rpm                                                         | 2.1 MB     00:01     
(17/23): perl-5.10.1-130.el6_4.x86_64.rpm                                                       |  10 MB     00:08     
(18/23): perl-Module-Pluggable-3.90-130.el6_4.x86_64.rpm                                        |  38 kB     00:00     
(19/23): perl-Pod-Escapes-1.04-130.el6_4.x86_64.rpm                                             |  31 kB     00:00     
(20/23): perl-Pod-Simple-3.13-130.el6_4.x86_64.rpm                                              | 210 kB     00:00     
(21/23): perl-libs-5.10.1-130.el6_4.x86_64.rpm                                                  | 576 kB     00:00     
(22/23): perl-version-0.77-130.el6_4.x86_64.rpm                                                 |  50 kB     00:00     
(23/23): tcl-8.5.7-6.el6.x86_64.rpm                                                             | 1.9 MB     00:01     
-----------------------------------------------------------------------------------------------------------------------
Total                                                                                  1.2 MB/s |  18 MB 
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : 1:perl-Pod-Escapes-1.04-130.el6_4.x86_64                                                           1/35 
  Installing : 4:perl-libs-5.10.1-130.el6_4.x86_64                                                                2/35 
  Installing : 1:perl-Module-Pluggable-3.90-130.el6_4.x86_64                                                      3/35 
  Installing : 1:perl-Pod-Simple-3.13-130.el6_4.x86_64                                                            4/35 
  Installing : 3:perl-version-0.77-130.el6_4.x86_64                                                               5/35 
  Installing : 4:perl-5.10.1-130.el6_4.x86_64                                                                     6/35 
  Updating   : libcom_err-1.42.7.wc1-7.el6.x86_64                                                                 7/35 
  Updating   : e2fsprogs-libs-1.42.7.wc1-7.el6.x86_64                                                             8/35 
  Installing : libibverbs-1.1.6-5.el6.x86_64                                                                      9/35 
  Installing : librdmacm-1.0.17-0.git4b5c1aa.el6.x86_64                                                          10/35 
  Updating   : libss-1.42.7.wc1-7.el6.x86_64                                                                     11/35 
  Updating   : e2fsprogs-1.42.7.wc1-7.el6.x86_64                                                                 12/35 
  Installing : lustre-ldiskfs-3.3.0-2.6.32_279.19.1.el6_lustre.x86_64.x86_64                                     13/35 
  Installing : lustre-modules-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64                                     14/35 
  Installing : libgomp-4.4.7-3.el6.x86_64                                                                        15/35 
  Installing : lm_sensors-libs-3.1.1-17.el6.x86_64                                                               16/35 
  Installing : 1:net-snmp-libs-5.5-44.el6_4.1.x86_64                                                             17/35 
  Installing : lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64             
 Installing : libXau-1.0.6-4.el6.x86_64                                                                         19/35 
  Installing : libxcb-1.8.1-1.el6.x86_64                                                                         20/35 
  Installing : libX11-common-1.5.0-4.el6.noarch                                                                  21/35 
  Installing : libX11-1.5.0-4.el6.x86_64                                                                         22/35 
  Installing : libgfortran-4.4.7-3.el6.x86_64                                                                    23/35 
  Installing : libesmtp-1.0.4-15.el6.x86_64                                                                      24/35 
  Installing : numactl-2.0.7-6.el6.x86_64                                                                        25/35 
  Installing : infinipath-psm-3.0.1-115.1015_open.1.1.el6_4.x86_64                                               26/35 
  Installing : 1:tcl-8.5.7-6.el6.x86_64                                                                          27/35 
  Installing : environment-modules-3.2.9c-4.el6.x86_64                                                           28/35 
  Installing : libtool-ltdl-2.2.6-15.5.el6.x86_64                                                                29/35 
  Installing : openmpi-1.5.4-1.el6.x86_64                                                                        30/35 
  Installing : lustre-tests-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64                                       31/35 
  Cleanup    : e2fsprogs-1.41.12-12.el6.x86_64                                                                   32/35 
  Cleanup    : e2fsprogs-libs-1.41.12-12.el6.x86_64                                                              33/35 
  Cleanup    : libss-1.41.12-12.el6.x86_64                                                                       34/35 
  Cleanup    : libcom_err-1.41.12-12.el6.x86_64                                                                  35/35 
  Verifying  : libtool-ltdl-2.2.6-15.5.el6.x86_64                                                                 1/35 
  Verifying  : libss-1.42.7.wc1-7.el6.x86_64                                                                      2/35 
  Verifying  : e2fsprogs-libs-1.42.7.wc1-7.el6.x86_64                                                             3/35 
  Verifying  : 1:tcl-8.5.7-6.el6.x86_64                                                                           4/35 
  Verifying  : 4:perl-5.10.1-130.el6_4.x86_64    
 Verifying  : lustre-modules-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64                                      6/35 
  Verifying  : libcom_err-1.42.7.wc1-7.el6.x86_64                                                                 7/35 
  Verifying  : lustre-tests-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64                                        8/35 
  Verifying  : infinipath-psm-3.0.1-115.1015_open.1.1.el6_4.x86_64                                                9/35 
  Verifying  : environment-modules-3.2.9c-4.el6.x86_64                                                           10/35 
  Verifying  : libX11-1.5.0-4.el6.x86_64                                                                         11/35 
  Verifying  : numactl-2.0.7-6.el6.x86_64                                                                        12/35 
  Verifying  : lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64                                             13/35 
  Verifying  : 1:perl-Pod-Simple-3.13-130.el6_4.x86_64                                                           14/35 
  Verifying  : libesmtp-1.0.4-15.el6.x86_64                                                                      15/35 
  Verifying  : libgfortran-4.4.7-3.el6.x86_64                                                                    16/35 
  Verifying  : openmpi-1.5.4-1.el6.x86_64                                                                        17/35 
  Verifying  : libX11-common-1.5.0-4.el6.noarch                                                                  18/35 
  Verifying  : 4:perl-libs-5.10.1-130.el6_4.x86_64                                                               19/35 
  Verifying  : libxcb-1.8.1-1.el6.x86_64                                                                         20/35 
  Verifying  : libXau-1.0.6-4.el6.x86_64                                                                         21/35 
  Verifying  : 1:net-snmp-libs-5.5-44.el6_4.1.x86_64                                                             22/35 
  Verifying  : 1:perl-Module-Pluggable-3.90-130.el6_4.x86_64                                                     23/35 
  Verifying  : lm_sensors-libs-3.1.1-17.el6.x86_64                                                               24/35 
  Verifying  : 1:perl-Pod-Escapes-1.04-130.el6_4.x86_64                                                          25/35 
  Verifying  : libgomp-4.4.7-3.el6.x86_64                                                                        26/35 
  Verifying  : lustre-ldiskfs-3.3.0-2.6.32_279.19.1.el6_lustre.x86_64.x86_64                                     27/35 
  Verifying  : 3:perl-version-0.77-130.el6_4.x86_64       
 Verifying  : libibverbs-1.1.6-5.el6.x86_64                                                                     29/35 
  Verifying  : e2fsprogs-1.42.7.wc1-7.el6.x86_64                                                                 30/35 
  Verifying  : librdmacm-1.0.17-0.git4b5c1aa.el6.x86_64                                                          31/35 
  Verifying  : libss-1.41.12-12.el6.x86_64                                                                       32/35 
  Verifying  : e2fsprogs-1.41.12-12.el6.x86_64                                                                   33/35 
  Verifying  : libcom_err-1.41.12-12.el6.x86_64                                                                  34/35 
  Verifying  : e2fsprogs-libs-1.41.12-12.el6.x86_64                                                              35/35 

Installed:
  lustre.x86_64 0:2.1.5-2.6.32_279.19.1.el6_lustre.x86_64                                                              
  lustre-ldiskfs.x86_64 0:3.3.0-2.6.32_279.19.1.el6_lustre.x86_64                                                      
  lustre-modules.x86_64 0:2.1.5-2.6.32_279.19.1.el6_lustre.x86_64                                                      
  lustre-tests.x86_64 0:2.1.5-2.6.32_279.19.1.el6_lustre.x86_64                                                        


Dependency Installed:
  environment-modules.x86_64 0:3.2.9c-4.el6            infinipath-psm.x86_64 0:3.0.1-115.1015_open.1.1.el6_4           
  libX11.x86_64 0:1.5.0-4.el6                          libX11-common.noarch 0:1.5.0-4.el6                              
  libXau.x86_64 0:1.0.6-4.el6                          libesmtp.x86_64 0:1.0.4-15.el6                                  
  libgfortran.x86_64 0:4.4.7-3.el6                     libgomp.x86_64 0:4.4.7-3.el6                                    
  libibverbs.x86_64 0:1.1.6-5.el6                      librdmacm.x86_64 0:1.0.17-0.git4b5c1aa.el6                      
  libtool-ltdl.x86_64 0:2.2.6-15.5.el6                 libxcb.x86_64 0:1.8.1-1.el6                                     
  lm_sensors-libs.x86_64 0:3.1.1-17.el6                net-snmp-libs.x86_64 1:5.5-44.el6_4.1                           
  numactl.x86_64 0:2.0.7-6.el6                         openmpi.x86_64 0:1.5.4-1.el6                                    
  perl.x86_64 4:5.10.1-130.el6_4                       perl-Module-Pluggable.x86_64 1:3.90-130.el6_4                   
  perl-Pod-Escapes.x86_64 1:1.04-130.el6_4             perl-Pod-Simple.x86_64 1:3.13-130.el6_4                         
  perl-libs.x86_64 4:5.10.1-130.el6_4                  perl-version.x86_64 3:0.77-130.el6_4                            
  tcl.x86_64 1:8.5.7-6.el6                            

Updated:
  e2fsprogs.x86_64 0:1.42.7.wc1-7.el6  e2fsprogs-libs.x86_64 0:1.42.7.wc1-7.el6  libcom_err.x86_64 0:1.42.7.wc1-7.el6 
  libss.x86_64 0:1.42.7.wc1-7.el6     

Complete!
[root@rhel6vanilla ~]#

Disable SELinux for Lustre patched kernel.

Lustre does not play well with SELinux (at least, the llmount.sh script fails when SELinux is enabled), so SELinux must be disabled. One option is to disable SELinux with GRUB. With your editor of choice, open /boot/grub/grub.conf. Append the option selinux=0 to the end of the kernel line for the Lustre patched kernel entry and save. This will cause SELinux to be disabled after you reboot into the patched kernel.

Reboot.

reboot

Verify that installation is complete:

uname -a

[root@rhel6vanilla ~]# uname -a
Linux rhel6vanilla 2.6.32-279.19.1.el6_lustre.x86_64 #1 SMP Wed Mar 20 16:37:18 PDT 2013 x86_64 x86_64 x86_64 GNU/Linux

Create a local Lustre filesystem with: llmount.sh

[root@rhel6vanilla ~]# /usr/lib64/lustre/tests/llmount.sh
Stopping clients: rhel6vanilla /mnt/lustre (opts:)
Stopping clients: rhel6vanilla /mnt/lustre2 (opts:)
Loading modules from /usr/lib64/lustre/tests/..
debug=0x33f0404
subsystem_debug=0xffb7e3ff
gss/krb5 is not supported
quota/lquota options: 'hash_lqs_cur_bits=3'
Formatting mgs, mds, osts
Format mds1: /tmp/lustre-mdt1
Format ost1: /tmp/lustre-ost1
Format ost2: /tmp/lustre-ost2
Checking servers environments
Checking clients rhel6vanilla environments
Loading modules from /usr/lib64/lustre/tests/..
debug=0x33f0404
subsystem_debug=0xffb7e3ff
gss/krb5 is not supported
Setup mgs, mdt, osts
Starting mds1: -o loop,user_xattr,acl  /tmp/lustre-mdt1 /mnt/mds1
debug=0x33f0404
subsystem_debug=0xffb7e3ff
debug_mb=2
Started lustre-MDT0000
Starting ost1: -o loop  /tmp/lustre-ost1 /mnt/ost1
debug=0x33f0404
subsystem_debug=0xffb7e3ff
debug_mb=2
Started lustre-OST0000
Starting ost2: -o loop  /tmp/lustre-ost2 /mnt/ost2
debug=0x33f0404
subsystem_debug=0xffb7e3ff
debug_mb=2
Started lustre-OST0001
Starting client: rhel6vanilla: -o user_xattr,acl,flock rhel6vanilla@tcp:/lustre /mnt/lustre
debug=0x33f0404
subsystem_debug=0xffb7e3ff
debug_mb=2
Using TIMEOUT=20
disable quota as required
[root@rhel6vanilla ~]# mount

/dev/mapper/vg_rhel6vanilla-lv_root on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
/dev/vda1 on /boot type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
/dev/loop0 on /mnt/mds1 type lustre (rw,loop=/dev/loop0,user_xattr,acl)
/dev/loop1 on /mnt/ost1 type lustre (rw,loop=/dev/loop1)
/dev/loop2 on /mnt/ost2 type lustre (rw,loop=/dev/loop2)
rhel6vanilla@tcp:/lustre on /mnt/lustre type lustre (rw,user_xattr,acl,flock)

Testing

Now that you have all the components of a Lustre system installed, you can test them.
This is covered in a separate wiki page: Testing a Lustre filesystem

Page tree