Page History
High Performance Parallel storage for Virtualized Cloud-Based Infrastructure
What is
IntelCloud Edition for Lustre
*?
IntelCloud Edition for Lustre
*(
ICELCE)
Softwareis a
product on AWS Marketplace. This productsoftware-defined storage cluster that runs on scalable cloud infrastructures. Cloud Edition provides a high performance
Lustreparallel filesystem
on the Amazon Web Servicesusing
AWSvirtualized resources. The full package includes CentOS, Lustre, Ganglia, and Lustre Monitoring Tool (LMT).
The product is delivered in the form of an Amazon Machine Image (AMI) available on the AWS Marketplace.Cloud Edition
ICELis intended to be used as the working filesystem for
ana HPC or other IO intensive
workloadworkloads. It is not intended to be used as
along term storage or as an alternative to cloud storage options such as S3. We recommend that S3
in particularor "cold-storage" be used for long term data storage
on AWS, and
LustreCloud Edition be used
whereverwhenever a high-performance shared filesystem is required.
Using ICEL on AWS
Step 1 - Subscribe to a Product Version
The first step in using ICEL on AWS is to navigate to the Amazon Marketplace and subscribe to the produce you will be using. The following products are available on the marketplace:
Step 2 - Launch a Cloud Formation Template
The Intel Lustre AMI is designed to be used with a CloudFormation template that defines all the resources needed by the Lustre filesystem. The templates provided below have a minimal number of a parameters to simplify deployment. To get started you only need to set the KeyName to your ssh-key, and set the LustreAZ to the specific Availability Zone in the region you are using. There is also a Filesystem Name parameter which defaults to "scratch" but this can be changed.
Cloud Formation Templates - Global Support HVM
Cloud Edition on Amazon Web Services
Amazon Web Services is a collection of remote computing services that make up Amazon's cloud computing infrastructure. Cloud Edition uses Amazon Machine Images (AMI) and Elastic Compute Storage (EC2) to provide a parallel and highly scalable storage cluster on AWS.
Cloud Edition on Microsoft Azure
Azure is Microsoft’s cloud computing platform, a growing collection of integrated services—analytics, computing, database, mobile, networking, storage, and web—for moving faster, achieving more, and saving money.
Cloud Edition on Google Cloud Platform
Google Cloud Platform is a collection of computing resources consist of virtual machines (VMs), storage, databases, networking, Infrastructure as Service (SaaS), Platform as a Service (PaaS), and Software as a Service (SaaS).
When combined with HPC and Technical computing applications running on AWS, Cloud Edition can improve storage performance and increase scalability by eliminating storage bottlenecks.
[Click Here for More Information]
Cloud Edition combines with HPC and Technical computing applications running on Azure to improve storage performance and increase scalability by eliminating storage bottlenecks.
[Click Here for More Information]
Cloud Edition on Google Cloud Platform combines with HPC and Technical computing applications to improve user experience, storage performance and increase scalability by eliminating storage bottlenecks.
Support Details
Cloud Edition for Lustre software is supported by the Lustre experts at Whamcloud. Product support includes the latest software updates, patches, and fixes to ensure a stable, flexible, and robust storage environment that leverages the benefits of cloud-based infrastructure.
To create a new support ticket, please setup an account in our JIRA ticketing system and file your ticket in the AWSP project: https://jira.whamcloud.com/secure/Dashboard.jspa and our support team will resolve your issue promptly. Please
Product Versions
There are currently three product versions available on AWS. Below is a listing of the versions as well as the associated features.
| Features | Global Support (HVM) | Global Support | Community Version |
|---|---|---|---|
| Intel Support | no | yes | yes |
| Instance Types | M1, M3 | M1, M3, C3 | M3, C3, I2 |
| Ephemeral Storage | yes | yes | no |
| EBS Storage | no | yes | yes |
| High Availability | no | yes* | yes* |
| VPC | no | yes | yes |
| Enhanced Networking | no | no | yes** |
| Raw Initialization | no | yes | yes |
* Support for HA requires EBS storage
** Enhanced Networking is only supported in VPC with C3 and I2 instances
Global Support (HVM)
The instance types that support hardware assisted virtualization (HVM) provide higher performance on AWS. Consequently, this product offering is our high-end option with support for both Enhanced Networking and the all SSD based I3 storage nodes. Enhanced Networking provides SRIOV, which allows an physical device to be virtualized and connected directly to a virtual machine. This provides lower latency and more consistent performance.
The I3 nodes all have internal SSD drives (up to 8x 800SSD in the i3.8xlarge) and Ivy Bridge. A very high performance scratch filesystem can be created on these nodes, but note it will not support HA since it will be using internal storage.
Global Support
The Global Support offering is intended for use with HPC workloads. With HA and EBS storage enabled, this product can recover the filesystem after a Lustre server is terminated. In addition to supporting EBS for more resilient storage, this version also supports higher end instance types (C3) which enable a higher performance filesystem.
Community Version
The Community Version offering is an entry level product offered on Marketplace which can be used for proof-of-concept and development testing. We only offer ephemeral storage templates for this product, so our recommendation is to limit use to scratch filesystems (data will be lost if one of the Lustre servers is terminated). Support is not included with this version, so we recommend moving to a product which includes Global Support once you have evaluated the product.
Support Details
The Intel® Cloud Edition for Lustre* - Global Support edition is supported directly by the Lustre* experts at Intel®. Customers will get Lustre* software updates, and support is available by email. Pleaseprovide your Amazon ID and a description of the issue when filing a ticket
: hpdd-cloud-lustre@intel.com.
For other questions please contact us at info@whamcloud.com
Overview
Content Tools