Page History

...

Describe the steps you need to build and test a Lustre system (MGS, MDT, MDS, OSS, OST, client) from the master branch on an x86_64 or aarch64, RHEL/Rocky linux 8.7 machine.

RHEL/Rocky Linux 8.7 aarch64 can be used inside a VM (tested on UTM 4.0.9) on an M1 Macbook Pro host. The "Use Hypervisor" option should be disabled on the QEMU page of preference in this case.

Prerequisite

A newly installed RHEL/Rocky linux 8 x86_64 or aarch64 machine connected to the internet.
NOTE It is suggested that you have at least 1GB of memory on the machine you are using for the build.
NOTE Verify that SElinux is disabled.
This manual uses the Rocky linux 8 distribution available at the link.
It is assumed that Rocky linux 8 was installed as is, and nothing else was installed on it.

...

Once RHEL/Rocky 8.7 is newly installed on an x86_64 or aarch64 machine login as user root.

...

First create the directory structure, then get the source from the RPM. Create a .rpmmacros file to install the kernel source in our user directory:
Code Block
language bash
theme Eclipse
cd $HOME && mkdir -p kernel/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS} cd kernel && echo '%_topdir %(echo $HOME)/kernel/rpmbuild' > ~/.rpmmacros

Install the kernel source:

Code Block

language	bash
theme	Eclipse

rpm -ivh https://download.rockylinux.org/pub/rocky/8/BaseOS/source/tree/Packages/k/kernel-4.18.0-425.19.2.el8_7.src.rpm

Prepare the source using rpmbuild:

Code Block

language	bash
theme	Eclipse

cd ~/kernel/rpmbuild && rpmbuild -bp --target=`uname -m` ./SPECS/kernel.spec

Copy the kernel config file into lustre tree:

Code Block

language	bash
theme	Eclipse

cp ~/kernel/rpmbuild/BUILD/kernel-4.18.0-425.19.2.el8_7/linux-4.18.0-425.19.2.el8.x86_64`uname -m`/configs/kernel-4.18.0-x86_64-`uname -m`.config ~/lustre-release/lustre/kernel_patches/kernel_configs/kernel-4.18.0-4.18-rhel8.7-x86_64`uname -m`.config

Edit the kernel config file ~/lustre-release/lustre/kernel_patches/kernel_configs/kernel-4.18.0-4.18-rhel8.7-`uname -x86_64m`.config::

Find the line with '# IO Schedulers' and insert following two lines below it:

CONFIG_IOSCHED_DEADLINE=y
CONFIG_DEFAULT_IOSCHED="deadline"

Or use cmd:

Code Block

language	bash
theme	Eclipse

sed -i '/# IO Schedulers/a CONFIG_IOSCHED_DEADLINE=y\nCONFIG_DEFAULT_IOSCHED="deadline"' ~/lustre-release/lustre/kernel_patches/kernel_configs/kernel-4.18.0-4.18-rhel8.7-x86_64`uname -m`.config

Gather all the patches from lustre tree into a single file:

Code Block

enableddl	true
language	bash
theme	Eclipse
lang	Bash
globaltitle	7. Copy the kernel patch into RPM build tree:

cd ~/lustre-release/lustre/kernel_patches/series && \
for patch in $(<"4.18-rhel8.7.series"); do \
     patch_file="$HOME/lustre-release/lustre/kernel_patches/patches/${patch}"; \
     cat "${patch_file}" >> "$HOME/lustre-kernel-x86_64-`uname -m`-lustre.patch"; \
done

Copy the kernel patch into RPM build tree:

Code Block

language	bash
theme	Eclipse

cp ~/lustre-kernel-x86_64`uname -m`-lustre.patch ~/kernel/rpmbuild/SOURCES/patch-4.18.0-lustre.patch

Edit the kernel spec file ~/kernel/rpmbuild/SPECS/kernel.spec:

Code Block

language	bash
theme	Eclipse

sed -i.inst -e '/^    find $RPM_BUILD_ROOT\/lib\/modules\/$KernelVer/a\
    cp -a fs/ext4/* $RPM_BUILD_ROOT/lib/modules/$KernelVer/build/fs/ext4\
    rm -f $RPM_BUILD_ROOT/lib/modules/$KernelVer/build/fs/ext4/ext4-inode-test*' \
-e '/^# empty final patch to facilitate testing of kernel patches/i\
Patch99995: patch-%{version}-lustre.patch' \
-e '/^ApplyOptionalPatch linux-kernel-test.patch/i\
ApplyOptionalPatch patch-%{version}-lustre.patch' \
~/kernel/rpmbuild/SPECS/kernel.spec

Overwrite the kernel config file with ~/lustre-release/lustre/kernel_patches/kernel_configs/kernel-4.18.0-4.18-rhel8.7-`uname -m`.config. Change x86_64 .config:with arm64 in the first command for aarch64 architecture.

Code Block

language	bash
theme	Eclipse

echo '# x86_64' > ~/kernel/rpmbuild/SOURCES/kernel-4.18.0-x86_64.config
cat ~/lustre-release/lustre/kernel_patches/kernel_configs/kernel-4.18.0-4.18-rhel8.7-x86_64-`uname -m`.config >> ~/kernel/rpmbuild/SOURCES/kernel-4.18.0-x86_64`uname -m`.config

Start building the kernel with rpmbuild:

Code Block

language	bash
theme	Eclipse

cd ~/kernel/rpmbuild && buildid="_lustre" && \
rpmbuild -ba --with firmware --target x86_64 `uname -m` --with baseonly \
           --without kabichk --define "buildid ${buildid}" \
           ~/kernel/rpmbuild/SPECS/kernel.spec

Install the kernel packages:

Code Block

language	bash
theme	Eclipse

cd ~/kernel/rpmbuild/RPMS/x86_64`uname -m`/
sudo rpm -Uvh --replacepkgs --force kernel-*.rpm
sudo reboot

Login system after reboot, run name -r and see:
# uname -r
4.18.0-425.19.2.el8_lustre.x86_64
or
4.18.0-425.19.2.el8_lustre.aarch64

Configure and build Lustre

Configure Lustre source:

Code Block

language	bash
theme	Eclipse

cd lustre-release
./configure --with-linux=/home/build/kernel/rpmbuild/BUILD/kernel-4.18.0-425.19.2.el8_7/linux-4.18.0-425.19.2.el8_lustre.x86_64`uname -m`/ --disable-gss --disable-shared --disable-crypto

Build and install Lustre:
Code Block
language bash
theme Eclipse
make -j8 sudo make install sudo depmod -a

Run /usr/lib64/lustre/tests/llmount.sh and tests lustre:

Code Block

language	bash
theme	Eclipse

/usr/lib64/lustre/tests/llmount.sh

Stopping clients: sr050 /mnt/lustre (opts:-f)
Stopping clients: sr050 /mnt/lustre2 (opts:-f)
sr050: executing set_hostid
Loading modules from /usr/lib64/lustre/tests/..
detected 8 online CPUs by sysfs
libcfs will create CPU partition based on online CPUs
Formatting mgs, mds, osts
Format mds1: /tmp/lustre-mdt1
Format ost1: /tmp/lustre-ost1
Format ost2: /tmp/lustre-ost2
Checking servers environments
Checking clients sr050 environments
Loading modules from /usr/lib64/lustre/tests/..
detected 8 online CPUs by sysfs
libcfs will create CPU partition based on online CPUs
Setup mgs, mdt, osts
Starting mds1: -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1
Commit the device label on /tmp/lustre-mdt1
Started lustre-MDT0000
Starting ost1: -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1
Commit the device label on /tmp/lustre-ost1
Started lustre-OST0000
Starting ost2: -o localrecov /dev/mapper/ost2_flakey /mnt/lustre-ost2
Commit the device label on /tmp/lustre-ost2
Started lustre-OST0001
Starting client: sr050: -o user_xattr,flock sr050@tcp:/lustre /mnt/lustre
UUID                   1K-blocks        Used   Available Use% Mounted on
lustre-MDT0000_UUID       125056        1644      112176   2% /mnt/lustre[MDT:0]
lustre-OST0000_UUID       313104        1244      284700   1% /mnt/lustre[OST:0]
lustre-OST0001_UUID       313104        1244      284700   1% /mnt/lustre[OST:1]

filesystem_summary:       626208        2488      569400   1% /mnt/lustre

Using TIMEOUT=20
osc.lustre-OST0000-osc-ffffce9052651800.idle_timeout=debug
osc.lustre-OST0001-osc-ffffce9052651800.idle_timeout=debug
setting jobstats to procname_uid
Setting lustre.sys.jobid_var from disable to procname_uid
Waiting 90s for 'procname_uid'
Updated after 4s: want 'procname_uid' got 'procname_uid'
disable quota as required
lod.lustre-MDT0000-mdtlov.mdt_hash=crush

You will now have a Lustre filesystem available at /mnt/lustre

...

Page tree

Versions Compared

Old Version 3

New Version 4

Key

Prerequisite

Configure and build Lustre