Page tree
Skip to end of metadata
Go to start of metadata


Purpose

This page describes all the necessary steps to:

  • Download and build the Linux kernel in the appropriate version for Lustre.
  • Create the Lustre Server DEB packages
  • Install the packages to get a Lustre Server

Prerequisites

  • A newly installed Ubuntu 20.04.1 machine connected to the internet.
  • NOTE: The build instructions below assume that SE-Linux is disabled. If you enable it, this guide should only be used as a source of inspiration, not of authoritative information.

Overview

The following kernel version will be used in this guide for Ubuntu 20.04.1 release:

  • Linux kernel 5.4.0-62-generic

1.) Installing the build dependency packages

To compile Lustre for use as a server, you need to be able to compile the Linux kernel and Lustre.
You also need to compile and enable at least one backing file-system – either ZFS or LDISKFS (or both).

The fastest way to install the needed tools is by using the debian build-dependency feature:

sed -i '/deb-src/s/^# //' /etc/apt/sources.list && sudo apt update
sudo apt install -y linux-image-$(uname -r) linux-modules-$(uname -r) linux-modules-extra-$(uname -r)
sudo apt install -y git libtool libtool-bin libyaml-dev ed libreadline-dev dpatch libsnmp-dev mpi-default-dev module-assistant quilt
sudo apt install -y libncurses5-dev libncurses-dev bison flex gnupg libelf-dev gcc libssl-dev bc bzip2 build-essential udev kmod cpio
sudo apt install -y libfuse-dev libattr1-dev libblkid-dev uuid-dev devscripts fakeroot kernel-wedge libudev-dev libpython3-dev swig
sudo apt install -y gettext texinfo debhelper dh-exec update-notifier-common sg3-utils attr mpi-default-bin selinux-utils
sudo apt install -y python python2 python2-minimal python2.7 libpython2-stdlib libsgutils2-2 libpython2.7-stdlib python2.7-minimal libpython2.7-minimal

The following instructions assume that you run in a directory that has at least 20 GB free disk space and is writable by your user.
It will be assumed, that this path is available as ${BUILDPATH}, such as /home/build.

2.) Building the Linux Kernel

Before you can build Lustre server support, you will need to have a compiled kernel source directory available, for your intended kernel version. If you intend to use LDISKFS as backing filesystem, you need to use a specific kernel version to generate the LDISKFS modules.

2.1.) Building an Unmodified Kernel

If you prefer to use an unmodified kernel on the server with LDISKFS, with some reduced functionality and performance, it is not necessary to patch the kernel or produce any Debian packages for it. You only need to compile the kernel locally to allow building modules against it.

As such, first figure out the version of the kernel on your target servers by executing:

$ uname -r
5.4.0-62-generic

Then download the correct kernel for your distribution:

$ cd ${BUILDPATH}
$ sudo apt source linux-image-unsigned-$(uname -r)

or

$ sudo wget http://ports.ubuntu.com/pool/main/l/linux/linux-source-5.4.0_5.4.0-62.70_all.deb
$ sudo dpkg -i linux-source-5.4.0_5.4.0-62.70_all.deb
$ sudo tar xjf /usr/src/linux-source-5.4.0/linux-source-5.4.0.tar.bz

The next thing you need is the kernel configuration for that version. The easiest way to get it, is by grabbing it from a server running that version.
Assuming the current compile server already runs that kernel (which is not necessary, but certainly makes things easier):

$ cd linux-5.4.0
$ cp /boot/config-5.4.0-62-generic .config

Once you have the config file in place, you need to make sure that the same version number as used by Ubuntu is embedded in the kernel:

$ touch .scmversion
$ sed -i 's/^VERSION = .*/VERSION = 5/' Makefile
$ sed -i 's/^PATCHLEVEL = .*/PATCHLEVEL = 4/' Makefile
$ sed -i 's/^SUBLEVEL = .*/SUBLEVEL = 0/' Makefile
$ sed -i 's/^EXTRAVERSION =.*/EXTRAVERSION = -62-generic/' Makefile
$ sed -i 's/^NAME =.*/NAME = Focal Fossa/' Makefile

Finally, you can compile the kernel source so that it is usable by Lustre:

$ make -j $(nproc)

Note: "nproc" returns the number of CPUs. If you don't have that tool or wish to use a different parallelism, just enter a number of parallel processes there.

3.) Building modified e2fsprogs for LDISKFS support

If you are building an LDISKFS server, you need to compile and install a modified version of the e2fsprogs package, which provides the virtual ldiskfsprogs package.

To compile the needed package, execute the following steps:

$ cd ${BUILDPATH}
$ git clone git://git.whamcloud.com/tools/e2fsprogs.git
$ cd e2fsprogs
$ git checkout master-lustre
$ sed -i 's/ext2_types-wrapper.h$//g' lib/ext2fs/Makefile.in
$ ./configure
$ dpkg-buildpackage -b -us -uc

This will create several ".deb" files in ${BUILDPATH}.
Save them somewhere sane, so that you can install the modified e2fsprogs on the target servers later.

Install the following packages:

$ sudo dpkg -i e2fsprogs_1.45.6-1_arm64.deb libext2fs-dev_1.45.6-1_arm64.deb \
comerr-dev_2.1-1.45.6-1_arm64.deb libext2fs2_1.45.6-1_arm64.deb libext2fs-dev_1.45.6-1_arm64.deb \
libcom-err2_1.45.6-1_arm64.deb libss2_1.45.6-1_arm64.deb

4.) Building Lustre Server Packages

4.1) Downloading Lustre Sources

The first step is to download the most recent Lustre sources from the Git repo:

$ cd ${BUILDPATH}
$ git clone git://git.whamcloud.com/fs/lustre-release.git
$ cd lustre-release
$ git checkout master

4.2) Building Lustre Server Packages

Compiling the Lustre server packages is quite straightforward:

$ cd ${BUILDPATH}/lustre-release
$ sh autogen.sh && ./configure --with-linux=${BUILDPATH}/linux-5.4.0/ && make debs -j $(nproc)

If the compilation succeeded, you should find the server modules again under "lustre-release/debs".
Copy these to a sane location, so that you can use them for installing Lustre Servers.



  • No labels