Page History
Lustre server machines are simple to setup with CentOS 6. 2. Pre-built kernel images are available from Whamclouds Whamcloud's build server. A number of different kernels and tools are continually built and tested. For this Walk-thru the lustre-master build will be used.
Note | ||
---|---|---|
| ||
Whamcloud produce Maintenance Releases and Feature Releases. This walk through covers running Lustre from the Master release. The Master release is subject to continuous integration testing but may have known or unknown issues. If you are evaluating Lustre for your production environment, you are advised to contact Whamcloud and substitute http://downloads.whamcloud.com/public/ for the RPMs you need for the walk-thru below. |
Prerequisite.
- A x86_64 machine with CentOS 6 .2 installed. The machine has the hostname
rhel6_vanilla
rhel6vanilla
.
Procedure
Install Lustre
...
directly with yum
Include the following entries into
/etc/yum.repos.d/lustre.repo
as follows:Code Block title /etc/repos.d/lustre.repo [hpddLustreserver] name=CentOS-$releasever - Lustre baseurl=http://build.whamcloud.com/job/lustre-master/arch=x86_64%2Cbuild_type=server%2Cdistro=el6%2Cib_stack=inkernel/lastSuccessfulBuild/artifact/artifacts/ gpgcheck=0 [e2fsprogs] name=CentOS-$releasever - Ldiskfs baseurl=http://build.whamcloud.com/job/e2fsprogs-master/arch=x86_64%2Cdistro=el6/lastSuccessfulBuild/artifact/_topdir/RPMS/ gpgcheck=0 [hpddLustreclient] name=CentOS-$releasever - Lustre baseurl=http://build.whamcloud.com/job/lustre-master/arch=x86_64%2Cbuild_type=client%2Cdistro=el6%2Cib_stack=inkernel/lastSuccessfulBuild/artifact/artifacts/ gpgcheck=0
run
yum install lustre-tests
- provided the step above completes successfully, you can skip down to the section:Disable SELinux for Lustre patched kernel.
- If you experience dependency issues with yum, the best known method is to downgrade the current kernel to the version that is needed and try again.
- If you are installing on RHEL and don't have Red Hat Network enabled, you will need to use the DVD ISO as a repo to satisfy unmet dependencies. This is covered on http://serverfault.com/questions/156458/
Install Lustre patched kernel.
If you choose not to use the yum install method above, you can install Lustre as follows:
- From lustre-master build, download the correct kernel RPMs for your platform
kernel
kernel-firmware
Install these RPMs with
yum localinstall ./kernel*
Code Block [root@rhel6vanilla ~]# yum localinstall ./kernel* Loaded plugins: fastestmirror, presto Setting up Local Package Process Examining ./kernel-2.6.32-279.19.1.el6_lustre.x86_64.rpm: kernel-2.6.32-279.19.1.el6_lustre.x86_64 Marking ./kernel-2.6.32-279.19.1.el6_lustre.x86_64.rpm as an update to kernel-2.6.32-279.el6.x86_64 Determining fastest mirrors * base: ftp.byfly.by * extras: ftp.byfly.by * updates: ftp.byfly.by Examining ./kernel-firmware-2.6.32-279.19.1.el6_lustre.x86_64.rpm: kernel-firmware-2.6.32-279.19.1.el6_lustre.x86_64 Marking ./kernel-firmware-2.6.32-279.19.1.el6_lustre.x86_64.rpm as an update to kernel-firmware-2.6.32-279.el6.noarch Resolving Dependencies --> Running transaction check ---> Package kernel.x86_64 0:2.6.32-279.19.1.el6_lustre will be installed .x86_64 --> Running transaction check ---> Package kernel-firmware.noarch 0:2.6.32-279.el6 will be updated ---> Package kernel-firmware.noarch 0:2.6.32-358.6.1.el6 will be an update --> Finished Dependency Resolution Dependencies Resolved ======================================================================================================================= Package
- From lustre-master build, download the correct kernel RPMs for your platform
kernel
kernel-firmware
- Install these RPMs with
yum localinstall ./kernel*
Code Block [root@rhel6_vanilla ~]# yum localinstall ./kernel* Loaded plugins: fastestmirror, security Setting up Local Package Process Examining ./kernel-2.6.32-220.el6_lustre.x86_64.rpm: kernel-2.6.32-220.el6_lustre.x86_64 Marking ./kernel-2.6.32-220.el6_lustre.x86_64.rpm as an update to kernel-2.6.32-220.el6.x86_64 base | 3.7 kB 00:00 base/primary_db | 4.5 MB 00:02 extras Arch Version Repository | 3.5 kB 00:00 extras/primary_db | 6.3 kB 00:00 updates Size ======================================================================================================================= Installing: kernel x86_64 2.6.32-279.19.1.el6_lustre /kernel-2.6.32-279.19.1.el6_lustre.x86_64 113 M Updating | 3.5 kB 00:00 updates/primary_db for dependencies: kernel-firmware noarch 2.6.32-358.6.1.el6 updates | 1.2 MB 00:00 Examining ./kernel-firmware-2.6.32-220.el6_lustre.x86_64.rpm: kernel-firmware-2.6.32-220.el6_lustre.x86_64 Marking ./kernel-firmware-2.6.32-220.el6_lustre.x86_64.rpm as an update to kernel-firmware-2.6.32-220.el6.noarch Resolving Dependencies --> Running transaction check ---> Package kernel.x86_64 0:2.6.32-220.el6_lustre will be installed --> Processing Dependency: kernel-firmware >= 2.6.32-220.el6_lustre for package: kernel-2.6.32-220.el6_lustre.x86_64 --> Running transaction check ---> Package kernel-firmware.noarch 0:2.6.32-220.el6 will be updated ---> Package kernel-firmware.noarch 0:2.6.32-220.4.1.el6 will be an update --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================= Package11 M Transaction Summary ======================================================================================================================= Install 1 Package(s) Upgrade 1 Package(s) Total size: 124 M Total download size: 11 M Is this ok [y/N]: y Downloading Packages: Setting up and reading Presto delta metadata updates/prestodelta | 339 kB 00:00 Processing delta metadata Download delta size: 3.4 M kernel-firmware-2.6.32-279.el6_2.6.32-358.6.1.el6.noarch.drpm | 3.4 MB 00:01 Finishing rebuild of rpms, from deltarpms <delta rebuild> Arch Version Repository | 11 MB 00:03 Size ============================================================================================================= Installing: kernel x86_64 Presto reduced the update size by 69% (from 11 M to 3.4 M). Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Updating : kernel-firmware-2.6.32-220.el6_lustre358.6.1.el6.noarch /kernel-2.6.32-220.el6_lustre.x86_64 112 M Updating for dependencies: kernel-firmware noarch 1/3 Installing : kernel-2.6.32-220279.419.1.el6_lustre.x86_64 updates 6.2 M Transaction Summary ============================================================================================================= Install 1 Package(s) Upgrade 1 Package(s) Total size: 118 M Total download size: 6.2 M Is this ok [y/N]: y Downloading Packages: 2/3 Cleanup : kernel-firmware-2.6.32-220279.4.1.el6.noarch.rpm | 6.2 MB 00:01 warning: rpmts_HdrFromFdno: Header V3 RSA/SHA1 Signature, key ID c105b9de: NOKEY Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 Importing GPG key 0xC105B9DE: Userid : CentOS-6 Key (CentOS 6 Official Signing Key) <centos-6-key@centos.org> Package: centos-release-6-2.el6.centos.7.x86_64 (@anaconda-CentOS-201112102333.x86_64/6.2) From : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 Is this ok [y/N]: y Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Updating : kernel-firmware-2.6.32-220.4.1.el6.noarch 3/3 Verifying : kernel-2.6.32-279.19.1.el6_lustre.x86_64 1/3 Verifying Installing : kernel-firmware-2.6.32-220358.6.1.el6_lustre.x86_64.noarch 2/3 CleanupVerifying : kernel-firmware-2.6.32-220279.el6.noarch 3/3 Installed: kernel.x86_64 0:2.6.32-220279.19.1.el6_lustre Dependency Updated: kernel-firmware.noarch 0:2.6.32-220.4.1.el6 Complete! [root@rhel6_vanilla ~]#
Download e2fsprogs and Lustre server RPMSs.
- Download the following e2fsprogs (and dependencies) RPMs from http://build.whamcloud.com/job/e2fsprogs-master
e2fsprogs
e2fsprogs-libs
libcom_err
libss
- Download the following Lustre server RPMs from lustre-master build.
lustre
lustre-ldiskfs
lustre-modules
lustre-tests
Install Lustre.
Dependency Updated: kernel-firmware.noarch 0:2.6.32-358.6.1.el6 Complete! [root@rhel6vanilla ~]#
Download e2fsprogs and Lustre server RPMSs.
- Download the following e2fsprogs (and dependencies) RPMs from http://build.whamcloud.com/job/e2fsprogs-master
e2fsprogs
e2fsprogs-libs
libss
libcom_err
- Download the following Lustre server RPMs from lustre-master build.
lustre
lustre-ldiskfs
lustre-modules
lustre-tests
Install Lustre.
yum localinstall ./lustre-* ./e2fsprogs-*
./libss-* ./libcom* should do it:Code Block [root@rhel6vanilla ~]# yum localinstall ./lustre-* ./e2fsprogs-* ./libss-* ./libcom* Loaded plugins: fastestmirror, presto Setting up Local Package Process Examining ./lustre-2.1.5-2.6.32_279.19
yum localinstall ./lustre-* ./e2fsprogs-* ./lib*
should do it:Code Block [root@rhel6_vanilla ~]# yum localinstall ./lustre-* ./e2fsprogs-* ./lib* Loaded plugins: fastestmirror, refresh-packagekit, security Setting up Local Package Process Examining ./lustre-2.2.54-2.6.32_220.17.1.el6_lustre.x86_64_g4bd9bf5.x86_64.rpm: lustre-2.2.54-2.6.32_220.17.1.el6_lustre.x86_64_g4bd9bf5.x86_64 Marking ./lustre-2.2.54-2.6.32_220.17.1.el6_lustre.x86_64_g4bd9bf5.x86_64.rpm to be installed Loading mirror speeds from cached hostfile * base: centos-distro.cavecreek.net * extras: mirror.raystedman.net * updates: centosv.centos.org Examining ./lustre-ldiskfs-3.3.0-2.6.32_220.17.1.el6_lustre.x86_64_g4bd9bf5.x86_64.rpm: lustre-ldiskfs-3.3.0-2.6.32_220.17.1.el6_lustre.x86_64_g4bd9bf5.x86_64 Marking ./lustre-ldiskfs-3.3.0-2.6.32_220.17.1.el6_lustre.x86_64_g4bd9bf5.x86_64.rpm to be installed Examining ./lustre-modules-2.2.54-2.6.32_220.17.1.el6_lustre.x86_64_g4bd9bf5.x86_64.rpm: lustre-modules-2.2.54-2.6.32_220.17.1.el6_lustre.x86_64_g4bd9bf5.x86_64 Marking ./lustre-modules-2.2.54.rpm: lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64 Marking ./lustre-2.1.5-2.6.32_220279.1719.1.el6_lustre.x86_64_g4bd9bf5.x86_64.rpm to be installed Loading mirror speeds from cached hostfile * base: ftp.byfly.by * extras: ftp.byfly.by * updates: ftp.byfly.by Examining ./lustre-testsldiskfs-23.23.540-2.6.32_220279.1719.1.el6_lustre.x86_64_g4bd9bf5.x86_64.rpm: lustre-testsldiskfs-23.23.540-2.6.32_220279.1719.1.el6_lustre.x86_64_g4bd9bf5.x86_64 Marking ./lustre-testsldiskfs-23.23.540-2.6.32_220279.1719.1.el6_lustre.x86_64_g4bd9bf5.x86_64.rpm to be installed Examining ./e2fsprogslustre-modules-2.1.42.3.wc1-7.el6.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64.rpm: e2fsprogslustre-modules-2.1.42.3.wc1-7.el65-2.6.32_279.19.1.el6_lustre.x86_64.x86_64 Marking ./e2fsprogslustre-modules-2.1.42.3.wc1-7.el65-2.6.32_279.19.1.el6_lustre.x86_64.x86_64.rpm asto anbe update to e2fsprogs-1.41.12-11.el6.x86_64 installed Examining ./lustre-tests-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64.rpm: lustre-tests-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64 Marking ./lustre-tests-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64.rpm to be installed Examining ./e2fsprogs-libs-1.42.37.wc1-7.el6.x86_64.rpm: e2fsprogs-libs-1.42.37.wc1-7.el6.x86_64 Marking ./e2fsprogs-libs-1.42.37.wc1-7.el6.x86_64.rpm as an update to e2fsprogs-libs-1.41.12-1112.el6.x86_64 Examining ./libcom_erre2fsprogs-libs-1.42.37.wc1-7.el6.x86_64.rpm: libcom_erre2fsprogs-libs-1.42.7.wc1-7.el6.x86_64 Examining ./libss-1.42.7.wc1-7.el6.x86_64.rpm: libss-1.42.37.wc1-7.el6.x86_64 Marking ./libcom_errlibss-1.42.37.wc1-7.el6.x86_64.rpm as an update to libcom_errlibss-1.41.12-1112.el6.x86_64 Examining ./libsslibcom_err-1.42.37.wc1-7.el6.x86_64.rpm: libsslibcom_err-1.42.37.wc1-7.el6.x86_64 Marking ./libsslibcom_err-1.42.37.wc1-7.el6.x86_64.rpm as an update to libsslibcom_err-1.41.12-1112.el6.x86_64 Resolving Dependencies --> Running transaction check ---> Package e2fsprogs.x86_64 0:1.41.12-1112.el6 will be updated ---> Package e2fsprogs.x86_64 0:1.42.37.wc1-7.el6 will be an update ---> Package e2fsprogs-libs.x86_64 0:1.41.12-1112.el6 will be updated ---> Package e2fsprogs-libs.x86_64 0:1.42.37.wc1-7.el6 will be an update ---> Package libcom_err.x86_64 0:1.41.12-1112.el6 will be updated ---> Package libcom_err.x86_64 0:1.42.37.wc1-7.el6 will be an update ---> Package libss.x86_64 0:1.41.12-1112.el6 will be updated ---> Package libss.x86_64 0:1.42.37.wc1-7.el6 will be an update ---> Package lustre.x86_64 0:2.21.545-2.6.32_220279.1719.1.el6_lustre.x86_64_g4bd9bf5 will be installed ---> Package lustre-ldiskfs.x86_64 0:3.3.0Processing Dependency: /usr/bin/perl for package: lustre-2.1.5-2.6.32_220279.1719.1.el6_lustre.x86_64_g4bd9bf5 will be installed -.x86_64 --> Package Processing Dependency: perl(File::Path) for package: lustre-modules.x86_64 0:2.21.545-2.6.32_220279.1719.1.el6_lustre.x86_64_g4bd9bf5 will be installed -.x86_64 --> Package Processing Dependency: perl(File::Path) for package: lustre-tests.x86_64 0:2.21.545-2.6.32_220279.1719.1.el6_lustre.x86_64_g4bd9bf5 will be installed --> Processing.x86_64 --> Processing Dependency: libmpi.so.1()(64bitperl(Getopt::Long) for package: lustre-tests-2.21.545-2.6.32_220279.1719.1.el6_lustre.x86_64_g4bd9bf5.x86_64 --> Processing Running transaction check ---> Package openmpi.x86_64 0:1.5.3-3.el6 will be installedDependency: perl(Getopt::Std) for package: lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64 --> Processing Dependency: libnuma.so.1(libnuma_1.1)(64bitperl(POSIX) for package: openmpilustre-2.1.5.3-3.el62.6.32_279.19.1.el6_lustre.x86_64.x86_64 --> Processing Dependency: librdmacmlibnetsnmp.so.120(RDMACM_1.0)(64bit) for package: openmpilustre-2.1.5.3-3.el6-2.6.32_279.19.1.el6_lustre.x86_64.x86_64 --> Processing Dependency: libnumalibnetsnmpagent.so.120(libnuma_1.2)(64bit) for package: openmpilustre-2.1.5.3-3.el6-2.6.32_279.19.1.el6_lustre.x86_64.x86_64 --> Processing Dependency: libibverbslibnetsnmphelpers.so.120(IBVERBS_1.1)(64bit) for package: openmpilustre-2.1.5.3-3.el6-2.6.32_279.19.1.el6_lustre.x86_64.x86_64 --> Processing Dependency: libibverbslibnetsnmpmibs.so.120(IBVERBS_1.0)(64bit) for package: openmpilustre-2.1.5.3-3.el6-2.6.32_279.19.1.el6_lustre.x86_64.x86_64 ---> Processing Dependency: environment-modules for package: openmpi-1.5.3-3.el6.x86_64 --> Processing Dependency: libesmtp.so.5()(64bit) for package: openmpi-1.5.3-3.el6.x86_64 --> Processing Dependency: libibverbs.so.1()(64bit) for package: openmpi-1.5.3-3.el6.x86_64Package lustre-ldiskfs.x86_64 0:3.3.0-2.6.32_279.19.1.el6_lustre.x86_64 will be installed ---> Package lustre-modules.x86_64 0:2.1.5-2.6.32_279.19.1.el6_lustre.x86_64 will be installed ---> Package lustre-tests.x86_64 0:2.1.5-2.6.32_279.19.1.el6_lustre.x86_64 will be installed --> Processing Dependency: libnumalibmpi.so.1()(64bit) for package: openmpilustre-tests-2.1.5.3-32.el66.x86_64 --> Processing Dependency: librdmacm.so.1()(64bit) for package: openmpi-1.5.3-3.el632_279.19.1.el6_lustre.x86_64.x86_64 --> Running transaction check ---> Package environmentnet-snmp-moduleslibs.x86_64 01:35.2.7b-6.el65-44.el6_4.1 will be installed --> Processing Dependency: libtcl8libsensors.5so.so4()(64bit) for package: environment-modules-3.2.7b-6.el61:net-snmp-libs-5.5-44.el6_4.1.x86_64 ---> Processing PackageDependency: libesmtp.x86_64 0:1.0.4-15.el6 will be installedlibperl.so()(64bit) for package: 1:net-snmp-libs-5.5-44.el6_4.1.x86_64 ---> Package libibverbsopenmpi.x86_64 0:1.15.54-31.el6 will be installed ---> PackageProcessing Dependency: librdmacm.so.x86_64 0:1.0.14.1-3.el6 will be installed ---> Package numactl.x86_64 0:2.0.3-9.el6 will be installed1(RDMACM_1.0)(64bit) for package: openmpi-1.5.4-1.el6.x86_64 --> Processing Dependency: libnuma.so.1(libnuma_1.2)(64bit) for package: openmpi-1.5.4-1.el6.x86_64 --> RunningProcessing transaction check ---> Package tcl.x86_64 1:8.5.7-6.el6 will be installedDependency: libnuma.so.1(libnuma_1.1)(64bit) for package: openmpi-1.5.4-1.el6.x86_64 --> FinishedProcessing Dependency Resolution Dependencies Resolved ============================================================================================================================= Package Arch Version Repository Size ============================================================================================================================= Installing: lustre x86_64 2.2.54-2.6.32_220.17: libibverbs.so.1(IBVERBS_1.1)(64bit) for package: openmpi-1.5.4-1.el6.x86_64 --> Processing Dependency: libibverbs.so.1(IBVERBS_1.0)(64bit) for package: openmpi-1.5.4-1.el6.x86_64 --> Processing Dependency: libgomp.so.1(OMP_1.0)(64bit) for package: openmpi-1.5.4-1.el6.x86_64 --> Processing Dependency: libgomp.so.1(GOMP_1.0)(64bit) for package: openmpi-1.5.4-1.el6.x86_64 --> Processing Dependency: environment-modules for package: openmpi-1.5.4-1.el6.x86_64 --> Processing Dependency: librdmacm.so.1()(64bit) for package: openmpi-1.5.4-1.el6.x86_64 --> Processing Dependency: libpsm_infinipath.so.1()(64bit) for package: openmpi-1.5.4-1.el6.x86_64 --> Processing Dependency: libnuma.so.1()(64bit) for package: openmpi-1.5.4-1.el6.x86_64 --> Processing Dependency: libltdl.so.7()(64bit) for package: openmpi-1.5.4-1.el6.x86_64 --> Processing Dependency: libibverbs.so.1()(64bit) for package: openmpi-1.5.4-1.el6.x86_64 --> Processing Dependency: libgomp.so.1()(64bit) for package: openmpi-1.5.4-1.el6.x86_64 --> Processing Dependency: libgfortran.so.3()(64bit) for package: openmpi-1.5.4-1.el6.x86_64 --> Processing Dependency: libesmtp.so.5()(64bit) for package: openmpi-1.5.4-1.el6.x86_64 ---> Package perl.x86_64 4:5.10.1-130.el6_4 will be installed --> Processing Dependency: perl(version) for package: 4:perl-5.10.1-130.el6_4.x86_64 --> Processing Dependency: perl(Pod::Simple) for package: 4:perl-5.10.1-130.el6_4.x86_64 --> Processing Dependency: perl(Module::Pluggable) for package: 4:perl-5.10.1-130.el6_4.x86_64 --> Running transaction check ---> Package environment-modules.x86_64 0:3.2.9c-4.el6 will be installed --> Processing Dependency: libtcl8.5.so()(64bit) for package: environment-modules-3.2.9c-4.el6.x86_64 --> Processing Dependency: libX11.so.6()(64bit) for package: environment-modules-3.2.9c-4.el6.x86_64 ---> Package infinipath-psm.x86_64 0:3.0.1-115.1015_open.1.1.el6_4 will be installed ---> Package libesmtp.x86_64 0:1.0.4-15.el6 will be installed ---> Package libgfortran.x86_64 0:4.4.7-3.el6 will be installed ---> Package libgomp.x86_64 0:4.4.7-3.el6 will be installed ---> Package libibverbs.x86_64 0:1.1.6-5.el6 will be installed ---> Package librdmacm.x86_64 0:1.0.17-0.git4b5c1aa.el6 will be installed ---> Package libtool-ltdl.x86_64 0:2.2.6-15.5.el6 will be installed ---> Package lm_sensors-libs.x86_64 0:3.1.1-17.el6 will be installed ---> Package numactl.x86_64 0:2.0.7-6.el6 will be installed ---> Package perl-Module-Pluggable.x86_64 1:3.90-130.el6_4 will be installed ---> Package perl-Pod-Simple.x86_64 1:3.13-130.el6_4 will be installed --> Processing Dependency: perl(Pod::Escapes) >= 1.04 for package: 1:perl-Pod-Simple-3.13-130.el6_4.x86_64 ---> Package perl-libs.x86_64 4:5.10.1-130.el6_4 will be installed ---> Package perl-version.x86_64 3:0.77-130.el6_4 will be installed --> Running transaction check ---> Package libX11.x86_64 0:1.5.0-4.el6 will be installed --> Processing Dependency: libX11-common = 1.5.0-4.el6 for package: libX11-1.5.0-4.el6.x86_64 --> Processing Dependency: libxcb.so.1()(64bit) for package: libX11-1.5.0-4.el6.x86_64 ---> Package perl-Pod-Escapes.x86_64 1:1.04-130.el6_4 will be installed ---> Package tcl.x86_64 1:8.5.7-6.el6 will be installed --> Running transaction check ---> Package libX11-common.noarch 0:1.5.0-4.el6 will be installed ---> Package libxcb.x86_64 0:1.8.1-1.el6 will be installed --> Processing Dependency: libXau.so.6()(64bit) for package: libxcb-1.8.1-1.el6.x86_64 --> Running transaction check ---> Package libXau.x86_64 0:1.0.6-4.el6 will be installed --> Finished Dependency Resolution Dependencies Resolved ======================================================================================================================= Package Arch Version Repository Size ======================================================================================================================= Installing: lustre x86_64 2.1.5-2.6.32_279.19.1.el6_lustre.x86_64 /lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64 26 M lustre-ldiskfs x86_64 3.3.0-2.6.32_279.19.1.el6_lustre.x86_64 /lustre-ldiskfs-3.3.0-2.6.32_279.19.1.el6_lustre.x86_64.x86_64 8.4 M lustre-modules x86_64 2.1.5-2.6.32_279.19.1.el6_lustre.x86_64 /lustre-modules-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64 144 M lustre-tests x86_64 2.1.5-2.6.32_279.19.1.el6_lustre.x86_64 /lustre-tests-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64 Updating: e2fsprogs x86_64 1.42.7.wc1-7.el6 /e2fsprogs-1.42.7.wc1-7.el6.x86_64 2.6 M e2fsprogs-libs x86_64 1.42.7.wc1-7.el6 /e2fsprogs-libs-1.42.7.wc1-7.el6.x86_64 349 k libcom_err x86_64 1.42.7.wc1-7.el6 /libcom_err-1.42.7.wc1-7.el6.x86_64 56 k libss x86_64 1.42.7.wc1-7.el6 /libss-1.42.7.wc1-7.el6.x86_64 67 k Installing for dependencies: environment-modules x86_64 3.2.9c-4.el6 base 98 k infinipath-psm x86_64 3.0.1-115.1015_open.1.1.el6_4 updates 160 k libX11 x86_64 1.5.0-4.el6 base 584 k libX11-common noarch 1.5.0-4.el6 base 192 k libXau x86_64 1.0.6-4.el6 base 24 k libesmtp x86_64 1.0.4-15.el6 base 57 k libgfortran x86_64 4.4.7-3.el6 base 265 k libgomp x86_64 4.4.7-3.el6 base 118 k libibverbs x86_64 1.1.6-5.el6 base 45 k librdmacm x86_64 1.0.17-0.git4b5c1aa.el6 base 47 k libtool-ltdl x86_64 2.2.6-15.5.el6 base 44 k libxcb x86_64 1.8.1-1.el6 base 110 k lm_sensors-libs x86_64 3.1.1-17.el6 base 38 k net-snmp-libs x86_64 1:5.5-44.el6_4.1 updates 1.5 M numactl x86_64 2.0.7-6.el6 base 69 k openmpi x86_64 1.5.4-1.el6 base 2.1 M perl x86_64 4:5.10.1-130.el6_4 updates perl-Module-Pluggable x86_64 1:3.90-130.el6_4 updates 38 k perl-Pod-Escapes x86_64 1:1.04-130.el6_4 updates 31 k perl-Pod-Simple x86_64 1:3.13-130.el6_4 updates 210 k perl-libs x86_64 4:5.10.1-130.el6_4 updates 576 k perl-version x86_64 3:0.77-130.el6_4 updates 50 k tcl x86_64 1:8.5.7-6.el6 base 1.9 M Transaction Summary ======================================================================================================================= Install 27 Package(s) Upgrade 4 Package(s) Total size: 213 M Total download size: 18 M Is this ok [y/N]: y Downloading Packages: Setting up and reading Presto delta metadata Processing delta metadata Package(s) data still to download: 18 M (1/23): environment-modules-3.2.9c-4.el6.x86_64.rpm | 98 kB 00:00 (2/23): infinipath-psm-3.0.1-115.1015_open.1.1.el6_4.x86_64.rpm | 160 kB 00:00 (3/23): libX11-1.5.0-4.el6.x86_64.rpm | 584 kB 00:00 (4/23): libX11-common-1.5.0-4.el6.noarch.rpm | 192 kB (5/23): libXau-1.0.6-4.el6.x86_64.rpm | 24 kB 00:00 (6/23): libesmtp-1.0.4-15.el6.x86_64.rpm | 57 kB 00:00 (7/23): libgfortran-4.4.7-3.el6.x86_64.rpm | 265 kB 00:00 (8/23): libgomp-4.4.7-3.el6.x86_64.rpm | 118 kB 00:00 (9/23): libibverbs-1.1.6-5.el6.x86_64.rpm | 45 kB 00:00 (10/23): librdmacm-1.0.17-0.git4b5c1aa.el6.x86_64.rpm | 47 kB 00:00 (11/23): libtool-ltdl-2.2.6-15.5.el6.x86_64.rpm | 44 kB 00:00 (12/23): libxcb-1.8.1-1.el6.x86_64.rpm | 110 kB 00:00 (13/23): lm_sensors-libs-3.1.1-17.el6.x86_64.rpm | 38 kB 00:00 (14/23): net-snmp-libs-5.5-44.el6_4.1.x86_64.rpm | 1.5 MB 00:01 (15/23): numactl-2.0.7-6.el6.x86_64.rpm | 69 kB 00:00 (16/23): openmpi-1.5.4-1.el6.x86_64.rpm | 2.1 MB 00:01 (17/23): perl-5.10.1-130.el6_4.x86_64.rpm | 10 MB 00:08 (18/23): perl-Module-Pluggable-3.90-130.el6_4.x86_64.rpm | 38 kB 00:00 (19/23): perl-Pod-Escapes-1.04-130.el6_4.x86_64.rpm | 31 kB 00:00 (20/23): perl-Pod-Simple-3.13-130.el6_4.x86_64.rpm | 210 kB 00:00 (21/23): perl-libs-5.10.1-130.el6_4.x86_64.rpm | 576 kB 00:00 (22/23): perl-version-0.77-130.el6_4.x86_64.rpm | 50 kB 00:00 (23/23): tcl-8.5.7-6.el6.x86_64.rpm | 1.9 MB 00:01 ----------------------------------------------------------------------------------------------------------------------- Total 1.2 MB/s | 18 MB Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Installing : 1:perl-Pod-Escapes-1.04-130.el6_4.x86_64 1/35 Installing : 4:perl-libs-5.10.1-130.el6_4.x86_64 2/35 Installing : 1:perl-Module-Pluggable-3.90-130.el6_4.x86_64 3/35 Installing : 1:perl-Pod-Simple-3.13-130.el6_4.x86_64 4/35 Installing : 3:perl-version-0.77-130.el6_4.x86_64 5/35 Installing : 4:perl-5.10.1-130.el6_4.x86_64 6/35 Updating : libcom_err-1.42.7.wc1-7.el6.x86_64 7/35 Updating : e2fsprogs-libs-1.42.7.wc1-7.el6.x86_64 8/35 Installing : libibverbs-1.1.6-5.el6.x86_64 9/35 Installing : librdmacm-1.0.17-0.git4b5c1aa.el6.x86_64 10/35 Updating : libss-1.42.7.wc1-7.el6.x86_64 11/35 Updating : e2fsprogs-1.42.7.wc1-7.el6.x86_64 12/35 Installing : lustre-ldiskfs-3.3.0-2.6.32_279.19.1.el6_lustre.x86_64.x86_64 13/35 Installing : lustre-modules-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64 14/35 Installing : libgomp-4.4.7-3.el6.x86_64 15/35 Installing : lm_sensors-libs-3.1.1-17.el6.x86_64 16/35 Installing : 1:net-snmp-libs-5.5-44.el6_4.1.x86_64 17/35 Installing : lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64 Installing : libXau-1.0.6-4.el6.x86_64 19/35 Installing : libxcb-1.8.1-1.el6.x86_64 20/35 Installing : libX11-common-1.5.0-4.el6.noarch 21/35 Installing : libX11-1.5.0-4.el6.x86_64 22/35 Installing : libgfortran-4.4.7-3.el6.x86_64 23/35 Installing : libesmtp-1.0.4-15.el6.x86_64 24/35 Installing : numactl-2.0.7-6.el6.x86_64 25/35 Installing : infinipath-psm-3.0.1-115.1015_open.1.1.el6_4.x86_64 26/35 Installing : 1:tcl-8.5.7-6.el6.x86_64 27/35 Installing : environment-modules-3.2.9c-4.el6.x86_64 28/35 Installing : libtool-ltdl-2.2.6-15.5.el6.x86_64 29/35 Installing : openmpi-1.5.4-1.el6.x86_64 30/35 Installing : lustre-tests-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64 31/35 Cleanup : e2fsprogs-1.41.12-12.el6.x86_64 32/35 Cleanup : e2fsprogs-libs-1.41.12-12.el6.x86_64 33/35 Cleanup : libss-1.41.12-12.el6.x86_64 34/35 Cleanup : libcom_err-1.41.12-12.el6.x86_64 35/35 Verifying : libtool-ltdl-2.2.6-15.5.el6.x86_64 1/35 Verifying : libss-1.42.7.wc1-7.el6.x86_64 2/35 Verifying : e2fsprogs-libs-1.42.7.wc1-7.el6.x86_64 3/35 Verifying : 1:tcl-8.5.7-6.el6.x86_64 4/35 Verifying : 4:perl-5.10.1-130.el6_4.x86_64 Verifying : lustre-modules-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64 6/35 Verifying : libcom_err-1.42.7.wc1-7.el6.x86_64 7/35 Verifying : lustre-tests-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64 8/35 Verifying : infinipath-psm-3.0.1-115.1015_open.1.1.el6_4.x86_64 9/35 Verifying : environment-modules-3.2.9c-4.el6.x86_64 10/35 Verifying : libX11-1.5.0-4.el6.x86_64 11/35 Verifying : numactl-2.0.7-6.el6.x86_64 12/35 Verifying : lustre-2.1.5-2.6.32_279.19.1.el6_lustre.x86_64.x86_64 13/35 Verifying : 1:perl-Pod-Simple-3.13-130.el6_4.x86_64 14/35 Verifying : libesmtp-1.0.4-15.el6.x86_64 15/35 Verifying : libgfortran-4.4.7-3.el6.x86_64 16/35 Verifying : openmpi-1.5.4-1.el6.x86_64 17/35 Verifying : libX11-common-1.5.0-4.el6.noarch 18/35 Verifying : 4:perl-libs-5.10.1-130.el6_4.x86_64 19/35 Verifying : libxcb-1.8.1-1.el6.x86_64 20/35 Verifying : libXau-1.0.6-4.el6.x86_64 21/35 Verifying : 1:net-snmp-libs-5.5-44.el6_4.1.x86_64 22/35 Verifying : 1:perl-Module-Pluggable-3.90-130.el6_4.x86_64 23/35 Verifying : lm_sensors-libs-3.1.1-17.el6.x86_64 24/35 Verifying : 1:perl-Pod-Escapes-1.04-130.el6_4.x86_64 25/35 Verifying : libgomp-4.4.7-3.el6.x86_64 26/35 Verifying : lustre-ldiskfs-3.3.0-2.6.32_279.19.1.el6_lustre.x86_64_g4bd9bf5 /lustre-2.2.54-2.6.32_220.17.1.el6_lustre.x86_64_g4bd9bf5.x86_64 26 M lustre-ldiskfs x86_64 3.3.0-2.6.32_220.17.1.el6_lustre.x86_64_g4bd9bf5 /lustre-ldiskfs-3.3.0-2.6.32_220.17.1.el6_lustre.x86_64_g4bd9bf5.x86_64 8.8 M lustre-modules x86_64 2.2.54-2.6.32_220.17.1.el6_lustre.x86_64_g4bd9bf5 /lustre-modules-2.2.54-2.6.32_220.17.1.el6_lustre.x86_64_g4bd9bf5.x86_64 154 M lustre-tests x86_64 2.2.54-2.6.32_220.17.1.el6_lustre.x86_64_g4bd9bf5 /lustre-tests-2.2.54-2.6.32_220.17.1.el6_lustre.x86_64_g4bd9bf5.x86_64 13 M Updating: e2fsprogs x86_64 1.42.3.wc1-7.el6 /e2fsprogs-1.42.3.wc1-7.el6.x86_64 2.6 M e2fsprogs-libs x86_64 1.42.3.wc1-7.el6 /e2fsprogs-libs-1.42.3.wc1-7.el6.x86_64 344 k libcom_err x86_64 1.42.3.wc1-7.el6 /libcom_err-1.42.3.wc1-7.el6.x86_64 56 k libss x86_64 1.42.3.wc1-7.el6 /libss-1.42.3.wc1-7.el6.x86_64 67 k Installing for dependencies: environment-modules x86_64 3.2.7b-6.el6 base 96 k libesmtp x86_64 1.0.4-15.el6 base 57 k libibverbs x86_64 1.1.5-3.el6 base 44 k librdmacm x86_64 1.0.14.1-3.el6 base 27 k numactl x86_64 2.0.3-9.el6 base 55 k openmpi x86_64 1.5.3-3.el6 base 2.1 M tcl x86_64 1:8.5.7-6.el6 base 1.9 M Transaction Summary ============================================================================================================================= Install 11 Package(s) Upgrade 4 Package(s) Total size: 210 M Total download size: 4.3 M Is this ok [y/N]: y Downloading Packages: (1/7): environment-modules-3.2.7b-6.el6.x86_64.rpm | 96 kB 00:00 (2/7): libesmtp-1.0.4-15.el6.x86_64.rpm | 57 kB 00:00 (3/7): libibverbs-1.1.5-3.el6.x86_64.rpm | 44 kB 00:00 (4/7): librdmacm-1.0.14.1-3.el6.x86_64.rpm | 27 kB 00:00 (5/7): numactl-2.0.3-9.el6.x86_64.rpm | 55 kB 00:00 (6/7): openmpi-1.5.3-3.el6.x86_64.rpm | 2.1 MB 00:01 (7/7): tcl-8.5.7-6.el6.x86_64.rpm | 1.9 MB 00:01 ----------------------------------------------------------------------------------------------------------------------------- Total 849 kB/s | 4.3 MB 00:05 Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Updating : libcom_err-1.42.3.wc1-7.el6.x86_64 1/19 Updating : e2fsprogs-libs-1.42.3.wc1-7.el6.x86_64 2/19 Installing : libibverbs-1.1.5-3.el6.x86_64 3/19 Installing : librdmacm-1.0.14.1-3.el6.x86_64 4/19 Updating : libss-1.42.3.wc1-7.el6.x86_64 5/19 Updating : e2fsprogs-1.42.3.wc1-7.el6.x86_64 6/19 Installing : lustre-ldiskfs-3.3.0-2.6.32_220.17.1.el6_lustre.x86_64_g4bd9bf5.x86_64 7/19 Installing : lustre-modules-2.2.54-2.6.32_220.17.1.el6_lustre.x86_64_g4bd9bf5.x86_64 8/19 Installing : lustre-2.2.54-2.6.32_220.17.1.el6_lustre.x86_64_g4bd9bf5.x86_64 9/19 Installing : 1:tcl-8.5.7-6.el6.x86_64 10/19 Installing : environment-modules-3.2.7b-6.el6.x86_64 11/19 Installing : libesmtp-1.0.4-15.el6.x86_64 12/19 Installing : numactl-2.0.3-9.el6.x86_64 13/19 Installing : openmpi-1.5.3-3.el6.x86_64 14/19 Installing : lustre-tests-2.2.54-2.6.32_220.17.1.el6_lustre.x86_64_g4bd9bf5.x86_64 15/19 Cleanup : e2fsprogs-1.41.12-11.el6.x86_64 16/19 Cleanup : e2fsprogs-libs-1.41.12-11.el6.x86_64 17/19 Cleanup : libss-1.41.12-11.el6.x86_64 18/19 Cleanup : libcom_err-1.41.12-11.el6.x86_64 19/19 Installed: lustre.x86_64 27/35 Verifying : 3:perl-version-0.77-130.el6_4.x86_64 Verifying : libibverbs-1.1.6-5.el6.x86_64 29/35 Verifying : e2fsprogs-1.42.7.wc1-7.el6.x86_64 30/35 Verifying : librdmacm-1.0.17-0.git4b5c1aa.el6.x86_64 31/35 Verifying : libss-1.41.12-12.el6.x86_64 32/35 Verifying : e2fsprogs-1.41.12-12.el6.x86_64 33/35 Verifying : libcom_err-1.41.12-12.el6.x86_64 34/35 Verifying : e2fsprogs-libs-1.41.12-12.el6.x86_64 35/35 Installed: lustre.x86_64 0:2.1.5-2.6.32_279.19.1.el6_lustre.x86_64 lustre-ldiskfs.x86_64 0:3.3.0-2.6.32_279.19.1.el6_lustre.x86_64 lustre-modules.x86_64 0:2.1.5-2.6.32_279.19.1.el6_lustre.x86_64 lustre-tests.x86_64 0:2.1.5-2.6.32_279.19.1.el6_lustre.x86_64 Dependency Installed: environment-modules.x86_64 0:3.2.9c-4.el6 infinipath-psm.x86_64 0:3.0.1-115.1015_open.1.1.el6_4 libX11.x86_64 0:1.5.0-4.el6 libX11-common.noarch 0:1.5.0-4.el6 libXau.x86_64 0:1.0.6-4.el6 libesmtp.x86_64 0:1.0.4-15.el6 libgfortran.x86_64 0:4.4.7-3.el6 libgomp.x86_64 0:4.4.7-3.el6 libibverbs.x86_64 0:1.1.6-5.el6 librdmacm.x86_64 0:1.0.17-0.git4b5c1aa.el6 libtool-ltdl.x86_64 0:2.2.546-2.6.32_220.17.1.el6_lustre.x86_64_g4bd9bf5 lustre-ldiskfs15.5.el6 libxcb.x86_64 0:1.8.1-1.el6 lm_sensors-libs.x86_64 0:3.31.01-2.6.32_220.17.1.el6_lustre.x86_64_g4bd9bf5 lustre-modules17.el6 net-snmp-libs.x86_64 1:5.5-44.el6_4.1 numactl.x86_64 0:2.20.547-2.6.32_220.17.1.el6_lustre.x86_64_g4bd9bf5 lustre-testsel6 openmpi.x86_64 0:21.25.54-2.6.32_220.17.1.el6_lustre.x86_64_g4bd9bf5 Dependency Installed: environment-modules4-1.el6 perl.x86_64 04:35.210.7b1-6.el6 libesmtp130.el6_4 perl-Module-Pluggable.x86_64 01:13.090-130.el6_4-15.el6 libibverbs perl-Pod-Escapes.x86_64 01:1.1.504-3.el6 librdmacm130.el6_4 perl-Pod-Simple.x86_64 01:1.0.14.1-3.el6 numactl3.13-130.el6_4 perl-libs.x86_64 04:25.010.31-9.el6 openmpi130.el6_4 perl-version.x86_64 3:0:1.5.377-3.el6 130.el6_4 tcl.x86_64 1:8.5.7-6.el6 el6 Updated: e2fsprogs.x86_64 0:1.42.37.wc1-7.el6 el6 e2fsprogs-libs.x86_64 0:1.42.37.wc1-7.el6 el6 libcom_err.x86_64 0:1.42.37.wc1-7.el6 el6 libss.x86_64 0:1.42.37.wc1-7.el6 el6 Complete! [root@rhel6_vanillaroot@rhel6vanilla ~]#
Disable SELinux for Lustre patched kernel.
Lustre does not play well with SELinux (at least, the llmount.sh
script fails when SELinux is enabled), so SELinux must be disabled. One option is to disable SELinux with GRUB. With your editor of choice, open /boot/grub/grub.conf
. Append the option selinux=0
to the end of the kernel
line for the Lustre patched kernel entry and save. This will cause SELinux to be disabled after you reboot into the patched kernel.
Reboot.
reboot
Verify that installation is complete:
uname -a
Code Block [root@rhel6_vanillaroot@rhel6vanilla ~]# uname -a Linux rhel6_vanillarhel6vanilla 2.6.32-220279.19.1.el6_lustre.x86_64 #1 SMP Wed JanMar 2520 1216:2137:3618 PSTPDT 20122013 x86_64 x86_64 x86_64 GNU/Linux
Create a local Lustre filesystem with:
llmount.sh
Code Block [root@rhel6_vanillaroot@rhel6vanilla ~]# /usr/lib64/lustre/tests/llmount.sh Stopping clients: rhel6_vanillarhel6vanilla /mnt/lustre (opts:) Stopping clients: rhel6_vanillarhel6vanilla /mnt/lustre2 (opts:) Loading modules from /usr/lib64/lustre/tests/.. debug=0x33f0404 subsystem_debug=0xffb7e3ff gss/krb5 is not supported quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: /tmp/lustre-mdt1 Format ost1: /tmp/lustre-ost1 Format ost2: /tmp/lustre-ost2 Checking servers environments Checking clients rhel6_vanillarhel6vanilla environments Loading modules from /usr/lib64/lustre/tests/.. debug=0x33f0404 subsystem_debug=0xffb7e3ff gss/krb5 is not supported Setup mgs, mdt, osts Starting mds1: -o loop,user_xattr,acl /tmp/lustre-mdt1 /mnt/mds1 debug=0x33f0404 subsystem_debug=0xffb7e3ff debug_mb=102 Started lustre-MDT0000 Starting ost1: -o loop /tmp/lustre-ost1 /mnt/ost1 debug=0x33f0404 subsystem_debug=0xffb7e3ff debug_mb=102 Started lustre-OST0000 Starting ost2: -o loop /tmp/lustre-ost2 /mnt/ost2 debug=0x33f0404 subsystem_debug=0xffb7e3ff debug_mb=102 Started lustre-OST0001 Starting client: rhel6_vanillarhel6vanilla: -o user_xattr,acl,flock rhel6_vanilla@tcprhel6vanilla@tcp:/lustre /mnt/lustre debug=0x33f0404 subsystem_debug=0xffb7e3ff debug_mb=102 Using TIMEOUT=20 disable quota as required [root@rhel6_vanillaroot@rhel6vanilla ~]# mount /dev/mapper/VolGroupvg_rhel6vanilla-lv_root on / type ext4 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) tmpfs on /dev/shm type tmpfs (rw) /dev/sda1vda1 on /boot type ext4 (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) /dev/loop0 on /mnt/mds1 type lustre (rw,loop=/dev/loop0,user_xattr,acl) /dev/loop1 on /mnt/ost1 type lustre (rw,loop=/dev/loop1) /dev/loop2 on /mnt/ost2 type lustre (rw,loop=/dev/loop2) rhel6_vanilla@tcprhel6vanilla@tcp:/lustre on /mnt/lustre type lustre (rw,user_xattr,acl,flock)
Testing
Now that you have all the components of a Lustre system installed, you can test them.
This is covered in a separate wiki page: Testing a Lustre filesystem