Page tree
Skip to end of metadata
Go to start of metadata

Lustre server machines are simple to setup with CentOS 6. Pre-built kernel images are available from Whamcloud's build server. A number of different kernels and tools are continually built and tested. For this Walk-thru the lustre-master build will be used.

Whamcloud Lustre GA

Whamcloud produce Maintenance Releases and Feature Releases. This walk through covers running Lustre from the Master release. The Master release is subject to continuous integration testing but may have known or unknown issues. If you are evaluating Lustre for your production environment, you are advised to contact Whamcloud and substitute http://downloads.whamcloud.com/public/ for the RPMs you need for the walk-thru below.

Prerequisite.

  • A x86_64 machine with CentOS 6 installed. The machine has the hostname rhel6vanilla.

Procedure

Install Lustre directly with yum

  1. Include the following entries into /etc/yum.repos.d/lustre.repo as follows:

    /etc/repos.d/lustre.repo
    [hpddLustreserver]
    name=CentOS-$releasever - Lustre
    baseurl=http://build.whamcloud.com/job/lustre-master/arch=x86_64%2Cbuild_type=server%2Cdistro=el6%2Cib_stack=inkernel/lastSuccessfulBuild/artifact/artifacts/
    gpgcheck=0
     
    [e2fsprogs]
    name=CentOS-$releasever - Ldiskfs
    baseurl=http://build.whamcloud.com/job/e2fsprogs-master/arch=x86_64%2Cdistro=el6/lastSuccessfulBuild/artifact/_topdir/RPMS/
    gpgcheck=0
    
    [hpddLustreclient]
    name=CentOS-$releasever - Lustre
    baseurl=http://build.whamcloud.com/job/lustre-master/arch=x86_64%2Cbuild_type=client%2Cdistro=el6%2Cib_stack=inkernel/lastSuccessfulBuild/artifact/artifacts/
    gpgcheck=0
  2. run yum install lustre-tests

  3. provided the step above completes successfully, you can skip down to the section:Disable SELinux for Lustre patched kernel.
  4. If you experience dependency issues with yum, the best known method is to downgrade the current kernel to the version that is needed and try again.
  5. If you are installing on RHEL and don't have Red Hat Network enabled, you will need to use the DVD ISO as a repo to satisfy unmet dependencies. This is covered on http://serverfault.com/questions/156458/

Install Lustre patched kernel.

If you choose not to use the yum install method above, you can install Lustre as follows:

  1. From lustre-master build, download the correct kernel RPMs for your platform
    • kernel
    • kernel-firmware
  2. Install these RPMs with yum localinstall ./kernel*

    [root@rhel6vanilla ~]# yum localinstall ./kernel*
    Loaded plugins: fastestmirror, presto
    Setting up Local Package Process
    Examining ./kernel-2.6.32-279.19.1.el6_lustre.x86_64.rpm: kernel-2.6.32-279.19.1.el6_lustre.x86_64
    Marking ./kernel-2.6.32-279.19.1.el6_lustre.x86_64.rpm as an update to kernel-2.6.32-279.el6.x86_64
    Determining fastest mirrors
     * base: ftp.byfly.by
     * extras: ftp.byfly.by
     * updates: ftp.byfly.by
    Examining ./kernel-firmware-2.6.32-279.19.1.el6_lustre.x86_64.rpm: kernel-firmware-2.6.32-279.19.1.el6_lustre.x86_64
    Marking ./kernel-firmware-2.6.32-279.19.1.el6_lustre.x86_64.rpm as an update to kernel-firmware-2.6.32-279.el6.noarch
    Resolving Dependencies
    --> Running transaction check
    ---> Package kernel.x86_64 0:2.6.32-279.19.1.el6_lustre will be installed
    .x86_64
    --> Running transaction check
    ---> Package kernel-firmware.noarch 0:2.6.32-279.el6 will be updated
    ---> Package kernel-firmware.noarch 0:2.6.32-358.6.1.el6 will be an update
    --> Finished Dependency Resolution
    
    Dependencies Resolved
    
    =======================================================================================================================
     Package              Arch        Version                         Repository                                      Size
    =======================================================================================================================
    Installing:
     kernel               x86_64      2.6.32-279.19.1.el6_lustre      /kernel-2.6.32-279.19.1.el6_lustre.x86_64      113 M
    Updating for dependencies:
     kernel-firmware      noarch      2.6.32-358.6.1.el6              updates                                         11 M
    
    Transaction Summary
    =======================================================================================================================
    Install       1 Package(s)
    Upgrade       1 Package(s)
    Total size: 124 M
    Total download size: 11 M
    Is this ok [y/N]: y
    Downloading Packages:
    Setting up and reading Presto delta metadata
    updates/prestodelta                                                                             | 339 kB     00:00     
    Processing delta metadata
    Download delta size: 3.4 M
    kernel-firmware-2.6.32-279.el6_2.6.32-358.6.1.el6.noarch.drpm                                   | 3.4 MB     00:01     
    Finishing rebuild of rpms, from deltarpms
    <delta rebuild>                                                                                 |  11 MB     00:03     
    Presto reduced the update size by 69% (from 11 M to 3.4 M).
    Running rpm_check_debug
    Running Transaction Test
    Transaction Test Succeeded
    Running Transaction
      Updating   : kernel-firmware-2.6.32-358.6.1.el6.noarch                                                           1/3 
      Installing : kernel-2.6.32-279.19.1.el6_lustre.x86_64                                                            2/3 
      Cleanup    : kernel-firmware-2.6.32-279.el6.noarch                                                               3/3 
      Verifying  : kernel-2.6.32-279.19.1.el6_lustre.x86_64                                                            1/3 
      Verifying  : kernel-firmware-2.6.32-358.6.1.el6.noarch                                                           2/3 
      Verifying  : kernel-firmware-2.6.32-279.el6.noarch                                                               3/3 
    
    
    Installed:
      kernel.x86_64 0:2.6.32-279.19.1.el6_lustre                                                                           
    
    Dependency Updated:
      kernel-firmware.noarch 0:2.6.32-358.6.1.el6                                                                          
    
    Complete!
    [root@rhel6vanilla ~]#
    

Download e2fsprogs and Lustre server RPMSs.

  1. Download the following e2fsprogs (and dependencies) RPMs from http://build.whamcloud.com/job/e2fsprogs-master
    • e2fsprogs
    • e2fsprogs-libs
    • libss
    • libcom_err
  2. Download the following Lustre server RPMs from lustre-master build.
    • lustre
    • lustre-ldiskfs
    • lustre-modules
    • lustre-tests

Install Lustre.

  1. yum localinstall ./lustre-* ./e2fsprogs-*  ./libss-* ./libcom* should do it:

    [root@rhel6vanilla ~]# yum localinstall ./lustre-* ./e2fsprogs-* ./libss-* ./libcom*
    Loaded plugins: fastestmirror, presto
    Setting up Local Package Process
    Examining ./lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64.rpm: lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
    Marking ./lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64.rpm to be installed
    Loading mirror speeds from cached hostfile
     * base: ftp.byfly.by
     * extras: ftp.byfly.by
     * updates: ftp.byfly.by
    Examining ./lustre-ldiskfs-3.3.0-2.6.32_279.19.1.el6_lustre.x86_64.x86_64.rpm: lustre-ldiskfs-3.3.0-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
    Marking ./lustre-ldiskfs-3.3.0-2.6.32_279.19.1.el6_lustre.x86_64.x86_64.rpm to be installed
    Examining ./lustre-modules-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64.rpm: lustre-modules-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
    Marking ./lustre-modules-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64.rpm to be installed
    Examining ./lustre-tests-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64.rpm: lustre-tests-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
    Marking ./lustre-tests-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64.rpm to be installed
    Examining ./e2fsprogs-1.42.7.wc1-7.el6.x86_64.rpm: e2fsprogs-1.42.7.wc1-7.el6.x86_64
    Marking ./e2fsprogs-1.42.7.wc1-7.el6.x86_64.rpm as an update to e2fsprogs-1.41.12-12.el6.x86_64
    Examining ./e2fsprogs-libs-1.42.7.wc1-7.el6.x86_64.rpm: e2fsprogs-libs-1.42.7.wc1-7.el6.x86_64
    Examining ./libss-1.42.7.wc1-7.el6.x86_64.rpm: libss-1.42.7.wc1-7.el6.x86_64
    Marking ./libss-1.42.7.wc1-7.el6.x86_64.rpm as an update to libss-1.41.12-12.el6.x86_64
    Examining ./libcom_err-1.42.7.wc1-7.el6.x86_64.rpm: libcom_err-1.42.7.wc1-7.el6.x86_64
    Marking ./libcom_err-1.42.7.wc1-7.el6.x86_64.rpm as an update to libcom_err-1.41.12-12.el6.x86_64
    Resolving Dependencies
    --> Running transaction check
    ---> Package e2fsprogs.x86_64 0:1.41.12-12.el6 will be updated
    ---> Package e2fsprogs.x86_64 0:1.42.7.wc1-7.el6 will be an update
    ---> Package e2fsprogs-libs.x86_64 0:1.41.12-12.el6 will be updated
    ---> Package e2fsprogs-libs.x86_64 0:1.42.7.wc1-7.el6 will be an update
    ---> Package libcom_err.x86_64 0:1.41.12-12.el6 will be updated
    ---> Package libcom_err.x86_64 0:1.42.7.wc1-7.el6 will be an update
    ---> Package libss.x86_64 0:1.41.12-12.el6 will be updated
    ---> Package libss.x86_64 0:1.42.7.wc1-7.el6 will be an update
    ---> Package lustre.x86_64 0:2.1.5-2.6.32_279.19.1.el6_lustre.x86_64 will be installed
    --> Processing Dependency: /usr/bin/perl for package: lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
    --> Processing Dependency: perl(File::Path) for package: lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
    --> Processing Dependency: perl(File::Path) for package: lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
    --> Processing Dependency: perl(Getopt::Long) for package: lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
    --> Processing Dependency: perl(Getopt::Std) for package: lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
    --> Processing Dependency: perl(POSIX) for package: lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
    --> Processing Dependency: libnetsnmp.so.20()(64bit) for package: lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
    --> Processing Dependency: libnetsnmpagent.so.20()(64bit) for package: lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
    --> Processing Dependency: libnetsnmphelpers.so.20()(64bit) for package: lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
    --> Processing Dependency: libnetsnmpmibs.so.20()(64bit) for package: lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
    ---> Package lustre-ldiskfs.x86_64 0:3.3.0-2.6.32_279.19.1.el6_lustre.x86_64 will be installed
    ---> Package lustre-modules.x86_64 0:2.1.5-2.6.32_279.19.1.el6_lustre.x86_64 will be installed
    ---> Package lustre-tests.x86_64 0:2.1.5-2.6.32_279.19.1.el6_lustre.x86_64 will be installed
    --> Processing Dependency: libmpi.so.1()(64bit) for package: lustre-tests-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
    --> Running transaction check
    ---> Package net-snmp-libs.x86_64 1:5.5-44.el6_4.1 will be installed
    --> Processing Dependency: libsensors.so.4()(64bit) for package: 1:net-snmp-libs-5.5-44.el6_4.1.x86_64
    --> Processing Dependency: libperl.so()(64bit) for package: 1:net-snmp-libs-5.5-44.el6_4.1.x86_64
    ---> Package openmpi.x86_64 0:1.5.4-1.el6 will be installed
    --> Processing Dependency: librdmacm.so.1(RDMACM_1.0)(64bit) for package: openmpi-1.5.4-1.el6.x86_64
    --> Processing Dependency: libnuma.so.1(libnuma_1.2)(64bit) for package: openmpi-1.5.4-1.el6.x86_64
    --> Processing Dependency: libnuma.so.1(libnuma_1.1)(64bit) for package: openmpi-1.5.4-1.el6.x86_64
    --> Processing Dependency: libibverbs.so.1(IBVERBS_1.1)(64bit) for package: openmpi-1.5.4-1.el6.x86_64
    --> Processing Dependency: libibverbs.so.1(IBVERBS_1.0)(64bit) for package: openmpi-1.5.4-1.el6.x86_64
    --> Processing Dependency: libgomp.so.1(OMP_1.0)(64bit) for package: openmpi-1.5.4-1.el6.x86_64
    --> Processing Dependency: libgomp.so.1(GOMP_1.0)(64bit) for package: openmpi-1.5.4-1.el6.x86_64
    --> Processing Dependency: environment-modules for package: openmpi-1.5.4-1.el6.x86_64
    --> Processing Dependency: librdmacm.so.1()(64bit) for package: openmpi-1.5.4-1.el6.x86_64
    --> Processing Dependency: libpsm_infinipath.so.1()(64bit) for package: openmpi-1.5.4-1.el6.x86_64
    --> Processing Dependency: libnuma.so.1()(64bit) for package: openmpi-1.5.4-1.el6.x86_64
    --> Processing Dependency: libltdl.so.7()(64bit) for package: openmpi-1.5.4-1.el6.x86_64
    --> Processing Dependency: libibverbs.so.1()(64bit) for package: openmpi-1.5.4-1.el6.x86_64
    --> Processing Dependency: libgomp.so.1()(64bit) for package: openmpi-1.5.4-1.el6.x86_64
    --> Processing Dependency: libgfortran.so.3()(64bit) for package: openmpi-1.5.4-1.el6.x86_64
    --> Processing Dependency: libesmtp.so.5()(64bit) for package: openmpi-1.5.4-1.el6.x86_64
    ---> Package perl.x86_64 4:5.10.1-130.el6_4 will be installed
    --> Processing Dependency: perl(version) for package: 4:perl-5.10.1-130.el6_4.x86_64
    --> Processing Dependency: perl(Pod::Simple) for package: 4:perl-5.10.1-130.el6_4.x86_64
    --> Processing Dependency: perl(Module::Pluggable) for package: 4:perl-5.10.1-130.el6_4.x86_64
    --> Running transaction check
    ---> Package environment-modules.x86_64 0:3.2.9c-4.el6 will be installed
    --> Processing Dependency: libtcl8.5.so()(64bit) for package: environment-modules-3.2.9c-4.el6.x86_64
    --> Processing Dependency: libX11.so.6()(64bit) for package: environment-modules-3.2.9c-4.el6.x86_64
    ---> Package infinipath-psm.x86_64 0:3.0.1-115.1015_open.1.1.el6_4 will be installed
    ---> Package libesmtp.x86_64 0:1.0.4-15.el6 will be installed
    ---> Package libgfortran.x86_64 0:4.4.7-3.el6 will be installed
    ---> Package libgomp.x86_64 0:4.4.7-3.el6 will be installed
    ---> Package libibverbs.x86_64 0:1.1.6-5.el6 will be installed
    ---> Package librdmacm.x86_64 0:1.0.17-0.git4b5c1aa.el6 will be installed
    ---> Package libtool-ltdl.x86_64 0:2.2.6-15.5.el6 will be installed
    ---> Package lm_sensors-libs.x86_64 0:3.1.1-17.el6 will be installed
    ---> Package numactl.x86_64 0:2.0.7-6.el6 will be installed
    ---> Package perl-Module-Pluggable.x86_64 1:3.90-130.el6_4 will be installed
    ---> Package perl-Pod-Simple.x86_64 1:3.13-130.el6_4 will be installed
    --> Processing Dependency: perl(Pod::Escapes) >= 1.04 for package: 1:perl-Pod-Simple-3.13-130.el6_4.x86_64
    ---> Package perl-libs.x86_64 4:5.10.1-130.el6_4 will be installed
    ---> Package perl-version.x86_64 3:0.77-130.el6_4 will be installed
    --> Running transaction check
    ---> Package libX11.x86_64 0:1.5.0-4.el6 will be installed
    --> Processing Dependency: libX11-common = 1.5.0-4.el6 for package: libX11-1.5.0-4.el6.x86_64
    --> Processing Dependency: libxcb.so.1()(64bit) for package: libX11-1.5.0-4.el6.x86_64
    ---> Package perl-Pod-Escapes.x86_64 1:1.04-130.el6_4 will be installed
    ---> Package tcl.x86_64 1:8.5.7-6.el6 will be installed
    --> Running transaction check
    ---> Package libX11-common.noarch 0:1.5.0-4.el6 will be installed
    ---> Package libxcb.x86_64 0:1.8.1-1.el6 will be installed
    --> Processing Dependency: libXau.so.6()(64bit) for package: libxcb-1.8.1-1.el6.x86_64
    --> Running transaction check
    ---> Package libXau.x86_64 0:1.0.6-4.el6 will be installed
    --> Finished Dependency Resolution
    
    Dependencies Resolved
    =======================================================================================================================
     Package               Arch   Version                                 Repository                                  Size
    =======================================================================================================================
    Installing:
     lustre                x86_64 2.1.5-2.6.32_279.19.1.el6_lustre.x86_64 /lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
                                                                                                                      26 M
     lustre-ldiskfs        x86_64 3.3.0-2.6.32_279.19.1.el6_lustre.x86_64 /lustre-ldiskfs-3.3.0-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
                                                                                                                     8.4 M
     lustre-modules        x86_64 2.1.5-2.6.32_279.19.1.el6_lustre.x86_64 /lustre-modules-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
                                                                                                                     144 M
     lustre-tests          x86_64 2.1.5-2.6.32_279.19.1.el6_lustre.x86_64 /lustre-tests-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64
    Updating:
     e2fsprogs             x86_64 1.42.7.wc1-7.el6                        /e2fsprogs-1.42.7.wc1-7.el6.x86_64         2.6 M
     e2fsprogs-libs        x86_64 1.42.7.wc1-7.el6                        /e2fsprogs-libs-1.42.7.wc1-7.el6.x86_64    349 k
     libcom_err            x86_64 1.42.7.wc1-7.el6                        /libcom_err-1.42.7.wc1-7.el6.x86_64         56 k
     libss                 x86_64 1.42.7.wc1-7.el6                        /libss-1.42.7.wc1-7.el6.x86_64              67 k
    Installing for dependencies:
     environment-modules   x86_64 3.2.9c-4.el6                            base                                        98 k
     infinipath-psm        x86_64 3.0.1-115.1015_open.1.1.el6_4           updates                                    160 k
     libX11                x86_64 1.5.0-4.el6                             base                                       584 k
     libX11-common         noarch 1.5.0-4.el6                             base                                       192 k
     libXau                x86_64 1.0.6-4.el6                             base                                        24 k
     libesmtp              x86_64 1.0.4-15.el6                            base                                        57 k
     libgfortran           x86_64 4.4.7-3.el6                             base                                       265 k
     libgomp               x86_64 4.4.7-3.el6                             base                                       118 k
     libibverbs            x86_64 1.1.6-5.el6                             base                                        45 k
     librdmacm             x86_64 1.0.17-0.git4b5c1aa.el6                 base                                        47 k
     libtool-ltdl          x86_64 2.2.6-15.5.el6                          base                                        44 k
     libxcb                x86_64 1.8.1-1.el6                             base                                       110 k
     lm_sensors-libs       x86_64 3.1.1-17.el6                            base                                        38 k
     net-snmp-libs         x86_64 1:5.5-44.el6_4.1                        updates                                    1.5 M
     numactl               x86_64 2.0.7-6.el6                             base                                        69 k
     openmpi               x86_64 1.5.4-1.el6                             base                                       2.1 M
     perl                  x86_64 4:5.10.1-130.el6_4                      updates      
    perl-Module-Pluggable x86_64 1:3.90-130.el6_4                        updates                                     38 k
     perl-Pod-Escapes      x86_64 1:1.04-130.el6_4                        updates                                     31 k
     perl-Pod-Simple       x86_64 1:3.13-130.el6_4                        updates                                    210 k
     perl-libs             x86_64 4:5.10.1-130.el6_4                      updates                                    576 k
     perl-version          x86_64 3:0.77-130.el6_4                        updates                                     50 k
     tcl                   x86_64 1:8.5.7-6.el6                           base                                       1.9 M
    
    Transaction Summary
    =======================================================================================================================
    Install      27 Package(s)
    Upgrade       4 Package(s)
    
    Total size: 213 M
    Total download size: 18 M
    Is this ok [y/N]: y
    Downloading Packages:
    Setting up and reading Presto delta metadata
    Processing delta metadata
    Package(s) data still to download: 18 M
    (1/23): environment-modules-3.2.9c-4.el6.x86_64.rpm                                             |  98 kB     00:00     
    (2/23): infinipath-psm-3.0.1-115.1015_open.1.1.el6_4.x86_64.rpm                                 | 160 kB     00:00     
    (3/23): libX11-1.5.0-4.el6.x86_64.rpm                                                           | 584 kB     00:00     
    (4/23): libX11-common-1.5.0-4.el6.noarch.rpm                                                    | 192 kB  
    (5/23): libXau-1.0.6-4.el6.x86_64.rpm                                                           |  24 kB     00:00     
    (6/23): libesmtp-1.0.4-15.el6.x86_64.rpm                                                        |  57 kB     00:00     
    (7/23): libgfortran-4.4.7-3.el6.x86_64.rpm                                                      | 265 kB     00:00     
    (8/23): libgomp-4.4.7-3.el6.x86_64.rpm                                                          | 118 kB     00:00     
    (9/23): libibverbs-1.1.6-5.el6.x86_64.rpm                                                       |  45 kB     00:00     
    (10/23): librdmacm-1.0.17-0.git4b5c1aa.el6.x86_64.rpm                                           |  47 kB     00:00     
    (11/23): libtool-ltdl-2.2.6-15.5.el6.x86_64.rpm                                                 |  44 kB     00:00     
    (12/23): libxcb-1.8.1-1.el6.x86_64.rpm                                                          | 110 kB     00:00     
    (13/23): lm_sensors-libs-3.1.1-17.el6.x86_64.rpm                                                |  38 kB     00:00     
    (14/23): net-snmp-libs-5.5-44.el6_4.1.x86_64.rpm                                                | 1.5 MB     00:01     
    (15/23): numactl-2.0.7-6.el6.x86_64.rpm                                                         |  69 kB     00:00     
    (16/23): openmpi-1.5.4-1.el6.x86_64.rpm                                                         | 2.1 MB     00:01     
    (17/23): perl-5.10.1-130.el6_4.x86_64.rpm                                                       |  10 MB     00:08     
    (18/23): perl-Module-Pluggable-3.90-130.el6_4.x86_64.rpm                                        |  38 kB     00:00     
    (19/23): perl-Pod-Escapes-1.04-130.el6_4.x86_64.rpm                                             |  31 kB     00:00     
    (20/23): perl-Pod-Simple-3.13-130.el6_4.x86_64.rpm                                              | 210 kB     00:00     
    (21/23): perl-libs-5.10.1-130.el6_4.x86_64.rpm                                                  | 576 kB     00:00     
    (22/23): perl-version-0.77-130.el6_4.x86_64.rpm                                                 |  50 kB     00:00     
    (23/23): tcl-8.5.7-6.el6.x86_64.rpm                                                             | 1.9 MB     00:01     
    -----------------------------------------------------------------------------------------------------------------------
    Total                                                                                  1.2 MB/s |  18 MB 
    Running rpm_check_debug
    Running Transaction Test
    Transaction Test Succeeded
    Running Transaction
      Installing : 1:perl-Pod-Escapes-1.04-130.el6_4.x86_64                                                           1/35 
      Installing : 4:perl-libs-5.10.1-130.el6_4.x86_64                                                                2/35 
      Installing : 1:perl-Module-Pluggable-3.90-130.el6_4.x86_64                                                      3/35 
      Installing : 1:perl-Pod-Simple-3.13-130.el6_4.x86_64                                                            4/35 
      Installing : 3:perl-version-0.77-130.el6_4.x86_64                                                               5/35 
      Installing : 4:perl-5.10.1-130.el6_4.x86_64                                                                     6/35 
      Updating   : libcom_err-1.42.7.wc1-7.el6.x86_64                                                                 7/35 
      Updating   : e2fsprogs-libs-1.42.7.wc1-7.el6.x86_64                                                             8/35 
      Installing : libibverbs-1.1.6-5.el6.x86_64                                                                      9/35 
      Installing : librdmacm-1.0.17-0.git4b5c1aa.el6.x86_64                                                          10/35 
      Updating   : libss-1.42.7.wc1-7.el6.x86_64                                                                     11/35 
      Updating   : e2fsprogs-1.42.7.wc1-7.el6.x86_64                                                                 12/35 
      Installing : lustre-ldiskfs-3.3.0-2.6.32_279.19.1.el6_lustre.x86_64.x86_64                                     13/35 
      Installing : lustre-modules-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64                                     14/35 
      Installing : libgomp-4.4.7-3.el6.x86_64                                                                        15/35 
      Installing : lm_sensors-libs-3.1.1-17.el6.x86_64                                                               16/35 
      Installing : 1:net-snmp-libs-5.5-44.el6_4.1.x86_64                                                             17/35 
      Installing : lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64             
     Installing : libXau-1.0.6-4.el6.x86_64                                                                         19/35 
      Installing : libxcb-1.8.1-1.el6.x86_64                                                                         20/35 
      Installing : libX11-common-1.5.0-4.el6.noarch                                                                  21/35 
      Installing : libX11-1.5.0-4.el6.x86_64                                                                         22/35 
      Installing : libgfortran-4.4.7-3.el6.x86_64                                                                    23/35 
      Installing : libesmtp-1.0.4-15.el6.x86_64                                                                      24/35 
      Installing : numactl-2.0.7-6.el6.x86_64                                                                        25/35 
      Installing : infinipath-psm-3.0.1-115.1015_open.1.1.el6_4.x86_64                                               26/35 
      Installing : 1:tcl-8.5.7-6.el6.x86_64                                                                          27/35 
      Installing : environment-modules-3.2.9c-4.el6.x86_64                                                           28/35 
      Installing : libtool-ltdl-2.2.6-15.5.el6.x86_64                                                                29/35 
      Installing : openmpi-1.5.4-1.el6.x86_64                                                                        30/35 
      Installing : lustre-tests-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64                                       31/35 
      Cleanup    : e2fsprogs-1.41.12-12.el6.x86_64                                                                   32/35 
      Cleanup    : e2fsprogs-libs-1.41.12-12.el6.x86_64                                                              33/35 
      Cleanup    : libss-1.41.12-12.el6.x86_64                                                                       34/35 
      Cleanup    : libcom_err-1.41.12-12.el6.x86_64                                                                  35/35 
      Verifying  : libtool-ltdl-2.2.6-15.5.el6.x86_64                                                                 1/35 
      Verifying  : libss-1.42.7.wc1-7.el6.x86_64                                                                      2/35 
      Verifying  : e2fsprogs-libs-1.42.7.wc1-7.el6.x86_64                                                             3/35 
      Verifying  : 1:tcl-8.5.7-6.el6.x86_64                                                                           4/35 
      Verifying  : 4:perl-5.10.1-130.el6_4.x86_64    
     Verifying  : lustre-modules-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64                                      6/35 
      Verifying  : libcom_err-1.42.7.wc1-7.el6.x86_64                                                                 7/35 
      Verifying  : lustre-tests-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64                                        8/35 
      Verifying  : infinipath-psm-3.0.1-115.1015_open.1.1.el6_4.x86_64                                                9/35 
      Verifying  : environment-modules-3.2.9c-4.el6.x86_64                                                           10/35 
      Verifying  : libX11-1.5.0-4.el6.x86_64                                                                         11/35 
      Verifying  : numactl-2.0.7-6.el6.x86_64                                                                        12/35 
      Verifying  : lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64                                             13/35 
      Verifying  : 1:perl-Pod-Simple-3.13-130.el6_4.x86_64                                                           14/35 
      Verifying  : libesmtp-1.0.4-15.el6.x86_64                                                                      15/35 
      Verifying  : libgfortran-4.4.7-3.el6.x86_64                                                                    16/35 
      Verifying  : openmpi-1.5.4-1.el6.x86_64                                                                        17/35 
      Verifying  : libX11-common-1.5.0-4.el6.noarch                                                                  18/35 
      Verifying  : 4:perl-libs-5.10.1-130.el6_4.x86_64                                                               19/35 
      Verifying  : libxcb-1.8.1-1.el6.x86_64                                                                         20/35 
      Verifying  : libXau-1.0.6-4.el6.x86_64                                                                         21/35 
      Verifying  : 1:net-snmp-libs-5.5-44.el6_4.1.x86_64                                                             22/35 
      Verifying  : 1:perl-Module-Pluggable-3.90-130.el6_4.x86_64                                                     23/35 
      Verifying  : lm_sensors-libs-3.1.1-17.el6.x86_64                                                               24/35 
      Verifying  : 1:perl-Pod-Escapes-1.04-130.el6_4.x86_64                                                          25/35 
      Verifying  : libgomp-4.4.7-3.el6.x86_64                                                                        26/35 
      Verifying  : lustre-ldiskfs-3.3.0-2.6.32_279.19.1.el6_lustre.x86_64.x86_64                                     27/35 
      Verifying  : 3:perl-version-0.77-130.el6_4.x86_64       
     Verifying  : libibverbs-1.1.6-5.el6.x86_64                                                                     29/35 
      Verifying  : e2fsprogs-1.42.7.wc1-7.el6.x86_64                                                                 30/35 
      Verifying  : librdmacm-1.0.17-0.git4b5c1aa.el6.x86_64                                                          31/35 
      Verifying  : libss-1.41.12-12.el6.x86_64                                                                       32/35 
      Verifying  : e2fsprogs-1.41.12-12.el6.x86_64                                                                   33/35 
      Verifying  : libcom_err-1.41.12-12.el6.x86_64                                                                  34/35 
      Verifying  : e2fsprogs-libs-1.41.12-12.el6.x86_64                                                              35/35 
    
    Installed:
      lustre.x86_64 0:2.1.5-2.6.32_279.19.1.el6_lustre.x86_64                                                              
      lustre-ldiskfs.x86_64 0:3.3.0-2.6.32_279.19.1.el6_lustre.x86_64                                                      
      lustre-modules.x86_64 0:2.1.5-2.6.32_279.19.1.el6_lustre.x86_64                                                      
      lustre-tests.x86_64 0:2.1.5-2.6.32_279.19.1.el6_lustre.x86_64                                                        
    
    
    Dependency Installed:
      environment-modules.x86_64 0:3.2.9c-4.el6            infinipath-psm.x86_64 0:3.0.1-115.1015_open.1.1.el6_4           
      libX11.x86_64 0:1.5.0-4.el6                          libX11-common.noarch 0:1.5.0-4.el6                              
      libXau.x86_64 0:1.0.6-4.el6                          libesmtp.x86_64 0:1.0.4-15.el6                                  
      libgfortran.x86_64 0:4.4.7-3.el6                     libgomp.x86_64 0:4.4.7-3.el6                                    
      libibverbs.x86_64 0:1.1.6-5.el6                      librdmacm.x86_64 0:1.0.17-0.git4b5c1aa.el6                      
      libtool-ltdl.x86_64 0:2.2.6-15.5.el6                 libxcb.x86_64 0:1.8.1-1.el6                                     
      lm_sensors-libs.x86_64 0:3.1.1-17.el6                net-snmp-libs.x86_64 1:5.5-44.el6_4.1                           
      numactl.x86_64 0:2.0.7-6.el6                         openmpi.x86_64 0:1.5.4-1.el6                                    
      perl.x86_64 4:5.10.1-130.el6_4                       perl-Module-Pluggable.x86_64 1:3.90-130.el6_4                   
      perl-Pod-Escapes.x86_64 1:1.04-130.el6_4             perl-Pod-Simple.x86_64 1:3.13-130.el6_4                         
      perl-libs.x86_64 4:5.10.1-130.el6_4                  perl-version.x86_64 3:0.77-130.el6_4                            
      tcl.x86_64 1:8.5.7-6.el6                            
    
    Updated:
      e2fsprogs.x86_64 0:1.42.7.wc1-7.el6  e2fsprogs-libs.x86_64 0:1.42.7.wc1-7.el6  libcom_err.x86_64 0:1.42.7.wc1-7.el6 
      libss.x86_64 0:1.42.7.wc1-7.el6     
    
    Complete!
    [root@rhel6vanilla ~]#
    

Disable SELinux for Lustre patched kernel.

Lustre does not play well with SELinux (at least, the llmount.sh script fails when SELinux is enabled), so SELinux must be disabled. One option is to disable SELinux with GRUB. With your editor of choice, open /boot/grub/grub.conf. Append the option selinux=0 to the end of the kernel line for the Lustre patched kernel entry and save. This will cause SELinux to be disabled after you reboot into the patched kernel.

Reboot.

  1. reboot

Verify that installation is complete:

  1. uname -a

    [root@rhel6vanilla ~]# uname -a
    Linux rhel6vanilla 2.6.32-279.19.1.el6_lustre.x86_64 #1 SMP Wed Mar 20 16:37:18 PDT 2013 x86_64 x86_64 x86_64 GNU/Linux
  2. Create a local Lustre filesystem with: llmount.sh

    [root@rhel6vanilla ~]# /usr/lib64/lustre/tests/llmount.sh
    Stopping clients: rhel6vanilla /mnt/lustre (opts:)
    Stopping clients: rhel6vanilla /mnt/lustre2 (opts:)
    Loading modules from /usr/lib64/lustre/tests/..
    debug=0x33f0404
    subsystem_debug=0xffb7e3ff
    gss/krb5 is not supported
    quota/lquota options: 'hash_lqs_cur_bits=3'
    Formatting mgs, mds, osts
    Format mds1: /tmp/lustre-mdt1
    Format ost1: /tmp/lustre-ost1
    Format ost2: /tmp/lustre-ost2
    Checking servers environments
    Checking clients rhel6vanilla environments
    Loading modules from /usr/lib64/lustre/tests/..
    debug=0x33f0404
    subsystem_debug=0xffb7e3ff
    gss/krb5 is not supported
    Setup mgs, mdt, osts
    Starting mds1: -o loop,user_xattr,acl  /tmp/lustre-mdt1 /mnt/mds1
    debug=0x33f0404
    subsystem_debug=0xffb7e3ff
    debug_mb=2
    Started lustre-MDT0000
    Starting ost1: -o loop  /tmp/lustre-ost1 /mnt/ost1
    debug=0x33f0404
    subsystem_debug=0xffb7e3ff
    debug_mb=2
    Started lustre-OST0000
    Starting ost2: -o loop  /tmp/lustre-ost2 /mnt/ost2
    debug=0x33f0404
    subsystem_debug=0xffb7e3ff
    debug_mb=2
    Started lustre-OST0001
    Starting client: rhel6vanilla: -o user_xattr,acl,flock rhel6vanilla@tcp:/lustre /mnt/lustre
    debug=0x33f0404
    subsystem_debug=0xffb7e3ff
    debug_mb=2
    Using TIMEOUT=20
    disable quota as required
    [root@rhel6vanilla ~]# mount
    
    /dev/mapper/vg_rhel6vanilla-lv_root on / type ext4 (rw)
    proc on /proc type proc (rw)
    sysfs on /sys type sysfs (rw)
    devpts on /dev/pts type devpts (rw,gid=5,mode=620)
    tmpfs on /dev/shm type tmpfs (rw)
    /dev/vda1 on /boot type ext4 (rw)
    none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
    sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
    /dev/loop0 on /mnt/mds1 type lustre (rw,loop=/dev/loop0,user_xattr,acl)
    /dev/loop1 on /mnt/ost1 type lustre (rw,loop=/dev/loop1)
    /dev/loop2 on /mnt/ost2 type lustre (rw,loop=/dev/loop2)
    rhel6vanilla@tcp:/lustre on /mnt/lustre type lustre (rw,user_xattr,acl,flock)

Testing

Now that you have all the components of a Lustre system installed, you can test them.
This is covered in a separate wiki page: Testing a Lustre filesystem