Page History
High Performance Parallel storage for Virtualized Cloud-Based Infrastructure
What is Cloud Edition for Lustre?
Cloud Edition for Lustre (CE) is a software-defined storage cluster that runs on scalable cloud infrastructures. Cloud Edition provides a high performance parallel filesystem using virtualized resources. The full package includes CentOS, Lustre, Ganglia, and Lustre Monitoring Tool (LMT).
Cloud Edition is intended to be used as the working filesystem for a HPC or other IO intensive workloads. It is not intended to be used as long term storage or as an alternative to cloud storage options such as S3. We recommend that S3 or "cold-storage" be used for long term data storage, and Cloud Edition be used whenever a high-performance shared filesystem is required.
Cloud Edition on Amazon Web Services
Amazon Web Services is a collection of remote computing services that make up Amazon's cloud computing infrastructure. Cloud Edition uses Amazon Machine Images (AMI) and Elastic Compute Storage (EC2) to provide a parallel and highly scalable storage cluster on AWS.
Cloud Edition on Microsoft Azure
Azure is Microsoft’s cloud computing platform, a growing collection of integrated services—analytics, computing, database, mobile, networking, storage, and web—for moving faster, achieving more, and saving money.
Cloud Edition on Google Cloud Platform
Google Cloud Platform is a collection of computing resources consist of virtual machines (VMs), storage, databases, networking, Infrastructure as Service (SaaS), Platform as a Service (PaaS), and Software as a Service (SaaS).
When combined with HPC and Technical computing applications running on AWS, Cloud Edition can improve storage performance and increase scalability by eliminating storage bottlenecks.
[Click Here for More Information]
Cloud Edition combines with HPC and Technical computing applications running on Azure to improve storage performance and increase scalability by eliminating storage bottlenecks.
[Click Here for More Information]
Cloud Edition on Google Cloud Platform combines with HPC and Technical computing applications to improve user experience, storage performance and increase scalability by eliminating storage bottlenecks.
Support Details
Cloud Edition for Lustre software is supported by the Lustre experts at Whamcloud. Product support includes the latest software updates, patches, and fixes to ensure a stable, flexible, and robust storage environment that leverages the benefits of cloud-based infrastructure.
To create a new support ticket, please setup an account in our JIRA ticketing system and file your ticket in the AWSP project: https://jira.whamcloud.com/secure/Dashboard.jspa and our support team will resolve your issue promptly. Please provide your Amazon ID and a description of the issue when filing a ticket.
For other questions please contact us at info@whamcloud.com
Scope:
- The purpose of this page is to provide a process for submitting requests to Product Management to determine next steps, request inclusion in the product, or escalate for further approval. Requests will be submitted in a standard format (illustrated below) to the Product Manager, at which time they will be added to the agenda of the Product Management Meeting. Please make sure to include any stakeholders that will be impacted, as well as those necessary for reaching a clear decision.
Template:
Title | Improved Readability of OST Rebalance Chart |
|---|---|
| Type | Feature Request (Feature, Bug, Proof of Concept) |
| Description | The OST Balance chart in IML becomes difficult to read when the number of OST's exceeds 16. Deployments exceeding 16 OST's are officially supported (reference doc) and (reference customers) have requested this functionality. The chart will need to be redesigned to facilitate readability for deployments up to and exceeding 128 OST's. |
| Purpose | What benefit does this feature provide? |
| Next Steps | What action is needed? |
| Priority | Medium (High, Medium, Low) |
| Creator | Micah Bhakti |
| Required Attendees | Bryon Neitzel, Mark Rogers |
| Status | Open, On Hold, Closed |
This template will be submitted to the Product Manger via email (micah@intel.com).
Product Management Meeting Agenda:
TBA
Product Management Current Initiatives:
Table of Contents:
| Name | Description | Priority |
|---|---|---|
| SAM (Service Assurance Management) Integration for IML | Monitoring features for IML that tie into Intel Platform | Low |
| Lustre Client for Xeon Phi | Allows Xeon Phi to connect to lustre storage cluster | Med |
| Differentiated Storage Services | Allows SSD Caching based on defined client profiles, caching done at MDS/OSS | High |
| IML Installer Improvements | Add checks to validate install environment, improve robustness, and output functional error messages when issues are encountered | |
Title | SAM (Service Assurance Management) integration for IML |
|---|---|
| Type | Proof of Concept |
| Description | Service Assurance Management is designed to provide QoS monitoring, metering, and performance analysis of nodes on a number of metrics (CPU, memory, IO) using a common platform node agent when used with IA platforms. |
| Purpose | Provides increased feature set when used with Intel hardware, as well as giving more data about bottlenecks and server usage. |
| Next Steps | Allocate a resource to work with the SAM team to setup a SAM scheduler and Node Collectors on an IEEL system and test basic functionality. Intended outcome would be to gather information about the usefulness of the data and potention benefits to IEEL product line. |
| Priority | Low |
| Creator | Micah Bhakti |
| Required Attendees | Micah Bhakti, Mrittika Ganguli (SAM team), Dan Ferber (or SE team resource) |
| Status | Open |
Title | Xeon Phi Supported Lustre Client |
|---|---|
| Type | Feature |
| Description | Xeon Phi uses an embeded Linux OS on the card. We would like to add support for the Phi to access the Lustre FS based on Phi addoption in HPC as well as the Enterprise Technical Computing potential. To do this we will either need to support the Lustre client on the Phi Linux OS, or on the host and allow client Phi's to access the FS through the host. |
| Purpose | Provides Lustre FS access to Xeon Phi co-processors. |
| Next Steps | Lustre 2.4 working on Phi with Ethernet (MPSS 2.1), IB support waiting on new OFED stack (WW39). TT-1346 will enable automatic Lustre build with RPM's, under review now. Performance is still not where we want through virtual ethernet. |
| Priority | Med |
| Creator | Micah Bhakti |
| Required Attendees | Micah Bhakti, Dmitry Eremin, Oleg Droken, Peter Jones |
| Status | Open |
Title | Differentiated Storage Services |
|---|---|
| Type | Feature |
| Description | DSS runs as a service on the storage server (MDS or OSS) allocating data to tiered storage based on metadata. Profiles are created and client-specific, which allow data to be sorted to SSD or HDD storage based on the performance-optimized profiles. |
| Purpose | Improves lustre performance for critical workloads, as well as potentially improving small-file IO. |
| Next Steps | DSS needs to be backported to current RHEL/CentOS kernel, followed by benchmarking and evaluation of test profiles for specific workloads. Following this we can define the impact and required engineering to integrate into the product. |
| Priority | High |
| Creator | Micah Bhakti |
| Required Attendees | Micah Bhakti, Michael Mesnier, Christian Black, Mikhail Pershin |
| Status | Open |
...
Title
...
...
Overview
Content Tools