Describe the steps you need to build and test a Lustre system (MGS, MDT, MDS, OSS, OST, client) from the master branch on an x86_64 or aarch64, RHEL/Rocky Linux 9.6 machine.
RHEL/Rocky Linux 9.6 aarch64 can be used inside a VM on an M1 Macbook Pro host with several hypervisors such as Virtualbox/Vagrant/UTM/Lima.
There is also the option of a Lima VM or Vagrant with a provided configuration files respectively which create and install all the required packages.
This document walks through the steps of patching the kernel, building Lustre and running a basic test of the complete system.
Once RHEL/Rocky 9.6 is newly installed on an x86_64 or aarch64 machine login as user root.
Install all the required packages:
dnf config-manager --set-enabled crb dnf -y install vim git libuuid-devel libblkid-devel wget rpmdevtools dnf-plugins-core redhat-rpm-config kernel-srpm-macros python3-devel epel-release keyutils libs-devel libnl3-devel libyaml-devel libmount-devel dnf -y install bc dwarves gcc-plugin-devel glibc-static kernel-rpm-macros net-tools perl-devel rsync pesign rocky-sb-certs htop dnf -y install krb5-devel dnf config-manager --set-enabled rt nf -y install realtime-setup |
useradd -m builder echo "builder ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers.d/builder |
git clone "https://review.whamcloud.com/tools/e2fsprogs" e2fsprogs && cd e2fsprogs && git checkout v1.47.1-wc1 && ./configure --with-root-prefix=/usr --enable-elf-shlibs --disable-uuidd --disable-fsck --disable-e2initrd-helper --disable-libblkid --disable-libuuid --enable-quota --disable-fuse2fs make -j8 sudo make install cd .. |
Or download and install the following e2fsprogs packages from https://build.whamcloud.com/
If you want to use the provided configuration file for Lima VM or Vagrant:
Create and start the VM:
Lima VM:
limactl start lustre_vm.yaml.... limactl shell lustre_vm |
Vagrant:
vagrant up vagrant ssh |
and just go to the Install e2fsprogs package step.
sudo su builder |
Prepare Lustre source:
git clone "https://review.whamcloud.com/fs/lustre-release" && cd lustre-release && sh ./autogen.sh |
Create the directory structure, then get the source from the RPM. Create a .rpmmacros file to install the kernel source in our user directory:
cd $HOME && mkdir -p kernel/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
cd kernel && echo '%_topdir %(echo $HOME)/kernel/rpmbuild' > ~/.rpmmacros |
Install the kernel source:
rpm -ivh https://dl.rockylinux.org/vault/rocky/9.6/BaseOS/source/tree/Packages/k/kernel-5.14.0-570.58.1.el9_6.src.rpm |
Prepare the source using rpmbuild:
cd ~/kernel/rpmbuild && rpmbuild -bp --target=`uname -m` ./SPECS/kernel.spec |
cp ~/kernel/rpmbuild/BUILD/kernel-5.14.0-570.58.1.el9_6/linux-5.14.0-570.58.1.el9.`uname -m`/configs/kernel-5.14.0-`uname -m`.config ~/lustre-release/lustre/kernel_patches/kernel_configs/kernel-5.14.0-5.14-rhel9.6-`uname -m`.config |
Edit the kernel config file ~/lustre-release/lustre/kernel_patches/kernel_configs/kernel-5.14.0-5.14-rhel9.6-`uname -m`.config:
Find the line with '# IO Schedulers' and insert following two lines below it:CONFIG_IOSCHED_DEADLINE=y
CONFIG_DEFAULT_IOSCHED="deadline"
Or use cmd:
sed -i '/# IO Schedulers/a CONFIG_IOSCHED_DEADLINE=y\nCONFIG_DEFAULT_IOSCHED="deadline"' ~/lustre-release/lustre/kernel_patches/kernel_configs/kernel-5.14.0-5.14-rhel9.6-`uname -m`.config |
Gather all the patches from lustre tree into a single file:
cd ~/lustre-release/lustre/kernel_patches/series
for patch in $(<"5.14-rhel9.6.series"); do \
patch_file="$HOME/lustre-release/lustre/kernel_patches/patches/${patch}"; \
cat "${patch_file}" >> "$HOME/lustre-kernel-`uname -m`-lustre.patch";
done |
cp ~/lustre-kernel-`uname -m`-lustre.patch ~/kernel/rpmbuild/SOURCES/patch-5.14.0-lustre.patch |
~/kernel/rpmbuild/SPECS/kernel.spec:sed -i.inst -e '/^ find $RPM_BUILD_ROOT\/lib\/modules\/$KernelVer/a\
cp -a fs/ext4/* $RPM_BUILD_ROOT/lib/modules/$KernelVer/build/fs/ext4\
rm -f $RPM_BUILD_ROOT/lib/modules/$KernelVer/build/fs/ext4/ext4-inode-test*' \
-e '/^# empty final patch to facilitate testing of kernel patches/i\
Patch99995: patch-%{version}-lustre.patch' \
-e '/^ApplyOptionalPatch linux-kernel-test.patch/i\
ApplyOptionalPatch patch-%{version}-lustre.patch' \
~/kernel/rpmbuild/SPECS/kernel.spec |
echo '# arm64' > ~/kernel/rpmbuild/SOURCES/kernel-`uname -m`.config or echo '# x86_64' > ~/kernel/rpmbuild/SOURCES/kernel-`uname -m`.config cat ~/lustre-release/lustre/kernel_patches/kernel_configs/kernel-5.14.0-5.14-rhel9.6-`uname -m`.config >> ~/kernel/rpmbuild/SOURCES/kernel-`uname -m`.config |
Start building the kernel with rpmbuild:
cd ~/kernel/rpmbuild && buildid="_lustre"
rpmbuild -ba --with firmware --target `uname -m` --with baseonly \
--without kabichk --define "buildid ${buildid}" \
~/kernel/rpmbuild/SPECS/kernel.spec |
Install the kernel packages:
cd ~/kernel/rpmbuild/RPMS/`uname -m`/ sudo rpm -Uvh --replacepkgs --force kernel-*.rpm sudo reboot |
Login system after reboot, run name -r and see:
or 5.14.0-570.58.1_lustre.el9.aarch64 |
Configure Lustre source:
cd ~/lustre-release/ ./configure --with-linux=/home/builder/kernel/rpmbuild/BUILD/kernel-5.14.0-570.58.1.el9_6/linux-5.14.0-570.58.1_lustre.el9.`uname -m`/ |
Build and install Lustre:
make -j8 sudo make install sudo depmod -a |
Run /usr/lib64/lustre/tests/llmount.sh and tests lustre:
/usr/lib64/lustre/tests/llmount.sh |
|
/mnt/lustre