You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 14 Next »

What is the Intel Cloud Edition for Lustre*?

A scalable, shared filesystem for HPC applications in the cloud. HPDD provides early access Lustre* software on Amazon Web Services (AWS). The full package includes CentOS, Lustre, ganglia, and Lustre Monitoring Tool (LMT). This is delivered in the form of an Amazon Machine Image (AMI).



 

Using Intel Cloud Edition Lustre* on AWS

The Intel Lustre AMI is designed to be used with a CloudFormation template that defines all the resources needed by the Lustre filesystem. The templates provided here can be further customized by setting parameters in the CloudFormation console when creating a new stack. There are numerous parameters to this template, but to get started you only need to set the KeyName to your ssh-key, and set the LustreAZ to be specific Availability Zone in the region you are using. The remaining default values to get a small Lustre filesystem and two test clients. The filesystem is not persistent, and is deleted when the stack is deleted.

 

EBS Template

 

This CloudFormation template will create a Lustre cluster and also a small cluster of clients that will automatically mount the newly created Lustre filesystem. This cluster uses EBS storage for Lustre,  though the default volume sizes are initially small.  The filesystem created with this template is not persistent, and is deleted when the stack is deleted.

This template is using spot pricing for all the instances except for the MGS, which will use a single, on-demand m1.medium instance.  There is a mapping table for spot bid prices in the template, and are set to the on-demand price for the corresponding instance type. 

 

The CloudFormation stack will completely successfully once the MGS and autoscaler groups for the remaining nodes have been created, and it does not wait for the actual Lustre instances to start.

 

Template Parameters

 

 

 

Parameter

Default

Description

KeyName

 

Name of and existing EC2 KeyPair to enable SSH access to the instance

FsName

scratch

Name of the lustre filesystem.

LustreAZ

 

Comma delimted list of availability zones to launch cluster in. Should all be in a single region.

ClientCount

2

Number of Client instances.

ClientInstanceType

m3.2xlarge

Instance type used for Lustre clients.(m1.xlarge, m3.2xlarge, cc2.8xlarge)

MdsCount

1

Number of MDS instances.

MdsInstanceType

m3.2xlarge

Instance type used for Lustre metadata servers. (m1.xlarge, m3.2xlarge, cc2.8xlarge)

MdsUsingEbs

true

MDT EBS volumes are created in the CFN template.

MdtVolumeSize

40

Size of EBS volumes to use for MDTs. All MDT instances have 4 EBS volumes attached.

MgsInstanceType

m1.medium

Instance type used for Lustre management service. (m1.small, m1.medium, m1.large, m1.xlarge

OssCount

4

Number of OSS instances.

OssInstanceType

m1.xlarge

Instance type used for Lustre object storage servers. (m1.xlarge, m3.2xlarge, cc2.8xlarge)

OssUsingEbs

true

OST EBS volumes are created in the CFN template.

OssVolumeSize

40

Size of EBS volumes to use for OSTs. All OSS instances have 8 EBS volumes attached.

 

 


  • No labels