Page History

...

This walk-thou has reportedly been used successfully on CentOS 6.0. One significant difference is that CentOS 6.0 replaces /etc/modprobe.conf with a directory: /etc/modprobe.d/. For CentOS 6.0 clients, the advice is to create a file /etc/modprobe.d/lustre.conf to contain Lustre specific module configurations.

Prerequisite

• A newly installed CentOS 5.5 x86_64 machine with the hostname: client-10.
• EPEL Repository: this is a convenient source for git.

  Code Block
  # wget http://download.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm # rpm -ivh ./epel-release-5-4.noarch.rpm

Overview

Lustre 1.8 servers require a patched and compiled kernel. Patches are readily available in the Whamcloud git source repository. A test suite is included with the Lustre 1.8 source. This document walks through the steps of patching the kernel, building Lustre and running a basic test of the complete system.

...

1. Install required kernel development tools.

   Code Block
   # yum -y groupinstall "Development Tools" # yum -y install rpm-build redhat-rpm-config unifdef gnupg quilt git

2. Create a user build with the home directory /build

   Code Block
   # useradd -d /build build

3. Switch to the user build and change to the build $HOME directory.

   Code Block
   # su build # cd $HOME

4. Get the 1.8 branch from the Whamcloud git account.

   Code Block
   # git clone git://git.whamcloud.com/fs/lustre-release.git # cd lustre-release # git checkout --track -b b1_8 origin/b1_8

5. Run sh ./autogen.sh
6. Resolve any outstanding dependencies until autogen.sh completes successfully. Success will look like:

   Code Block
   # sh ./autogen.sh Checking for a complete tree... checking for automake-1.9 >= 1.9... found 1.9.6 ... Running automake-1.9... configure.ac: installing `./install-sh' configure.ac: installing `./missing' configure.ac:9: installing `./config.guess' configure.ac:9: installing `./config.sub' Running autoconf...

...

NOTE If you are performing this walk-thru on CentOS 5.6, the kernel source can be found here:
http://vault.centos.org/5.6/os/SRPMS/kernel-2.6.18-238.el5.src.rpmImage Removed

1. Expand the source. Using rpmbuild will also apply CentOS patches.

   Code Block
   # rpmbuild -bp --target=`uname -m` ~/kernel/rpmbuild/SPECS/kernel.spec

   
   This will end with:

   Code Block
   ... + patch -p1 --fuzz=2 -s + echo 'Patch #20240 (xen-hvm-add-hvmop_get_time-hypercall.patch):' Patch #20240 (xen-hvm-add-hvmop_get_time-hypercall.patch): + patch -p1 --fuzz=2 -s + echo 'Patch #20241 (xen-fix-64-bit-pv-guest-user-mode-segv-crashing-host.patch):' Patch #20241 (xen-fix-64-bit-pv-guest-user-mode-segv-crashing-host.patch): + patch -p1 --fuzz=2 -s + exit 0

...

1. Go into the kernel source directory and issue the following commands to build a kernel rpm.

   Code Block
   # cd /build/kernel/rpmbuild/BUILD/kernel-2.6.18/linux-2.6.18.x86_64 # make oldconfig || make menuconfig # make include/asm # make include/linux/version.h # make SUBDIRS=scripts # make include/linux/utsrelease.h # make # make rpm

2. A successful build will return:

   Code Block

   ... ... Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 Checking for unpackaged file(s): /usr/lib/rpm/check-files /var/tmp/kernel-2.6.18.lustre18-root Wrote: /build/kernel/rpmbuild/SRPMS/kernel-2.6.18.lustre18-1.src.rpm Wrote: /build/kernel/rpmbuild/RPMS/x86_64/kernel-2.6.18.lustre18-1.x86_64.rpm Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.35163 + umask 022 + cd /build/kernel/rpmbuild/BUILD + cd kernel-2.6.18.lustre18 + exit 0 rm ../kernel-2.6.18.lustre18.tar.gz

...

1. run /usr/lib64/lustre/tests/llmount.sh

   Code Block

   [root@client-10 ~]# /usr/lib64/lustre/tests/llmount.sh Stopping clients: rhel5_build /mnt/lustre (opts:) Stopping clients: rhel5_build /mnt/lustre2 (opts:) Loading modules from /usr/lib64/lustre/tests/.. lnet.debug=0x33f1504 lnet.subsystem_debug=0xffb7e3ff lnet options: ' accept=all' Formatting mgs, mds, osts Checking servers environments Checking clients rhel5_build environments Setup mgs, mdt, osts Starting mds: -o loop /tmp/lustre-mdt /mnt/mds lnet.debug=0x33f1504 lnet.subsystem_debug=0xffb7e3ff lnet.debug_mb=10 Started lustre-MDT0000 Starting ost1: -o loop /tmp/lustre-ost1 /mnt/ost1 lnet.debug=0x33f1504 lnet.subsystem_debug=0xffb7e3ff lnet.debug_mb=10 Started lustre-OST0000 Starting ost2: -o loop /tmp/lustre-ost2 /mnt/ost2 lnet.debug=0x33f1504 lnet.subsystem_debug=0xffb7e3ff lnet.debug_mb=10 Started lustre-OST0001 Starting client: rhel5_build: -o user_xattr,acl,flock rhel5_build@tcp:/lustre /mnt/lustre lnet.debug=0x33f1504 lnet.subsystem_debug=0xffb7e3ff lnet.debug_mb=10 Using TIMEOUT=20 [root@client-10 ~]#

2. you will now have a Lustre filesystem available at /mnt/lustre
3. NOTE: if you receive an error: mkfs.lustre: Can't parse NID 'client-10@tcp' you'll need to associate the ip address of a non-loopback interface with name of your machine into the /etc/hosts file.

...