Purpose
Describe the steps you need to get a Lustre system (MGS, MDT, MDS, OSS, OST, client) up and running on a cluster of three machines.
Prerequisite
- Three machines, networked together: client-10, client-11, client-12
- Able to use fdisk to create primary partitions from unallocated disk space.
Overview
The chosen platform is:
The Lustre topology will be:
- lustre file system name is
lustrewt
(lustre walk through)
client-12
(IB: 192.168.4.12) for the OSS and OST.
client-11
(IB: 192.168.4.11) for the MGT, MDS and MDT.
client-10
(IB: 192.168.4.10) for the client.
This document is based on: section 4.1 Configuring the Lustre File System from the Lustre File System, Operations Manual - Version 2.x
Procedure
Provision Machines
This is covered in the pages:
Set up the MGS, MDT, and MDS
The node client-11
is going to be the MGS, MDT, and MDS.
- connect to client-11
- check to see which networks are suitable
This should show
options lnet networks="o2ib0(ib0)" |
configured for lnet.
- Use
to create a primary partition from the unallocated disk space. This partition is
.
- Create the MGS/MDT file system:
mkfs.lustre --fsname=lustrewt --mgs --mdt /dev/sda4 |
- Create a mount point:
- Mount the MGS/MDT file system
mkdir /mnt/mdt; mount -t lustre /dev/sda4 /mnt/mdt |
You should see:
Permanent disk data:
Target: lustrewt-MDTffff
Index: unassigned
Lustre FS: lustrewt
Mount type: ldiskfs
Flags: 0x75
(MDT MGS needs_index first_time update )
Persistent mount opts: iopen_nopriv,user_xattr,errors=remount-ro
Parameters: mdt.group_upcall=/usr/sbin/l_getgroups
checking for existing Lustre data: not found
device size = 205220MB
2 6 18
formatting backing filesystem ldiskfs on /dev/sda4
target name lustrewt-MDTffff
4k blocks 52536566
options -J size=400 -i 4096 -I 512 -q -O dir_index,uninit_groups -F
mkfs_cmd = mke2fs -j -b 4096 -L lustrewt-MDTffff -J size=400 -i 4096 -I 512 -q -O dir_index,uninit_groups -F /dev/sda4 52536566
Writing CONFIGS/mountdata |
- Install lnet module:
modprobe lnet
- Start-up lnet:
lctl network up
- Check the NID:
lctl list_nids
this should return 192.168.4.11@o2ib
- Make lnet install during boot.
Set up the OST and OSS
The node client-12
is going to be the OST and OSS.
- connect to client-12
- check to see which networks are suitable
This should show
options lnet networks="o2ib0(ib0)"
configured for lnet.
- Use
fdisk /dev/sda
to create a primary partition from the unallocated disk space. This partition is /dev/sda4
.
- Create the OST:
mkfs.lustre --ost --fsname=lustrewt --mgsnode=192.168.4.11@o2ib0 /dev/sda4 |
you should see:
Permanent disk data:
Target: lustrewt-OSTffff
Index: unassigned
Lustre FS: lustrewt
Mount type: ldiskfs
Flags: 0x72
(OST needs_index first_time update )
Persistent mount opts: errors=remount-ro,extents,mballoc
Parameters: mgsnode=192.168.4.11@o2ib
checking for existing Lustre data: not found
device size = 205220MB
2 6 18
formatting backing filesystem ldiskfs on /dev/sda4
target name lustrewt-OSTffff
4k blocks 52536566
options -J size=400 -i 16384 -I 256 -q -O dir_index,extents,uninit_groups -F
mkfs_cmd = mke2fs -j -b 4096 -L lustrewt-OSTffff -J size=400 -i 16384 -I 256 -q -O dir_index,extents,uninit_groups -F /dev/sda4 52536566
Writing CONFIGS/mountdata |
- Create a mount point:
mkdir /ostoss_mount
- Mount the OST:
mount -t lustre /dev/sda4 /ostoss_mount
Setup the client.
- connect to
client-10
- Load the Lustre client module
- NOTE: The client is available from build.whamcloud.com
- Mount the lustre filesystem
mount -t lustre 192.168.4.11@o2ib:/lustrewt /mnt |
Making Luster servers persist through a reboot.
- Add mount command for MGS, MDS, and MDT to
/etc/fstab
on client-11
/dev/sda4 /mgsmdt_mount lustre defaults,_netdev 0 0
|
- Add mount command for OST and OSS to
/etc/fstab
on client-12
/dev/sda4 /ostoss_mount lustre defaults,_netdev 0 0
|
ENDS-