The steps below will show how to create a Lustre Cluster on Microsoft'a Azure infrastructure. It will start with a Lustre template in the Azure marketplace.
You will need an Azure account.
Login to your account and go to the dashboard. You can get to the dashboard from anywhere by clicking on the blue "Microsoft Azure" in the upper left corner of the GUI.
To get started click on "Create a resource" to get ready to start deployments of a new instance of Lustre
In the search box that appears, type "lustre" and hit return. For some reason the completions that start to show up do not work as expected
choose "Lustre on Azure" in Compute category.
click "Create"
In the "Basics" screen that appears you will fill out the fields in the template.
Lustre Filesystem name - this will be name of your lustre filesystem and also prepended to the names oVM hosts. For example here it is specified as ldemo so our MGS node will be ldemomgs.
Username - This is the username for the administor account.
Authentication type - You can choose to use a password or a key. You can cut and paste a public key here. For windows you can load your private key into puttygen and cut and paste the openssh version of the public key into the box. Whatever you do here will be propagated to all the VMs in the Lustre Cluster.
Resource Group - Is the unique name for your group of resources (the resources that make up your Lustre cluster). You can create a new resource group or use existing one (must be empty)
Location - Choose the Azure region you would like to deploy to from the drop down list.
When done hit "OK" to move to the next screen.
In the "Lustre Network Settings", fill out the fill in the template
Lustre virtual network - this is the virtual network that the VM will be using. You can create new or use existing one.
Lustre Subnets - this is the subnet that VM will be using. A new one will be created in case of new virtual network.
Public IP address name for MGS - This is the resource name for the public IP address. Not the hostname of the MGS which is what you will ssh into.
You can let it create a new one or use existing one. If you choose "None", you might not be able to ssh in your cluster.
DNS Prefix for public IP Address - This is the hostname of the public MGS node. Note that underneath the box is the rest of the public address.
When done with "Lustre Subnets", hit "OK" to go back to "Lustre Network Settings", hit "OK" again to move to the next screen.
in the "Management Server Setting", you can choose the type of VM for Management server (MGS)
click "Change size" if you want to choose a different size
When done, hit "OK" to move to the next screen
In the "Metadata Server Settings", you can choose the type of VM for Metadata server (MDS)
Click "Change size" if you want to choose a different size
Click the drop down menu for "Size of each MDT disk" if you want to change the Metadata Target size (MDT). default is set to 512GB.
Note: MDT is not the VM boot disk. It is Lustre Metadata disk
When done, hit "OK" to move to the next screen
In the "Object Storage Servers Settings",
First enter the number of Object Storage Servers (OSS) you would like to use. Default is 2
Then select the type of VM for Object Storage server (OSS)
Click "Change size" if you want to choose a different size
Also choose the size of each of the 3 OSTs that will be attached to your OSS
When done, hit "OK" to move to the next screen.
In the "Compute Client Settings", client lustre image is free of charge. You only pay for VM.
First enter number of compute client. Default is 2.
Click "Change size" if you want to choose a different size
When done, hit "OK" to move to the next screen.
In the "Summary" screen, a summary of what has been entered is shown in the summary.
Verify information you have entered and click "OK' when done
Read the terms of use and purchase your Lustre Filesystem.
Enter name, email adress and phone number
when done click "Create"
Next you will see the dashboard notification with deployment in progress...
Click on "Deployment in progress...", you will see the status of your deployment
Once the status changes to "Your deployment is complete"
Click on "Outputs" to view the "lustre Gred Report" (ie ganglia)
In the "Outputs" screen,
mgsNID - this is the private IP address of the MGS
mgsDomainName - this is public hostname for MGS
gangliaURL - this is the URL to view the grid report
mountCommand - this is the command to mount lustre. All compute clients created in this deployment should mount. if not run command 'mount -a' on the client
Copy and paste the "gangliaURL" on a different browser to view grid report
To access the Servers (MGS, MDS, OSS), we must go through MGS public IP
From the deployment complete screen "Overview", click on virtual machine name ending mgs. In this case "ldemomgs"
On MGS VM screen,
Click "Connect" to view info on the right
Choose "SSH", then copy the Login command and paste on to your terminal
If you need to access the other nodes in the cluster and you have used the key option, you have two options.
The best method is to run a ssh-agent and ssh with forwarding enabled so the MGS node can get the private key from your workstation.
The other less secure way as you make you are exposing your private key would be to copy your private key to the MGS so it can be used to access the other nodes.
Similar method can be used to access 'client0' (client0 acts as the gateway to access all compute clients)
Overview
Content Tools