How to deploy an EXAScaler Cloud Application

EXAScaler Cloud application can be launched on Microsofts Azure Portal by navigating to the link below and and clicking the Create button.

Step by step deployment video

Step by step deployment guide

The steps below will show how to create a EXAScaler Cloud Cluster on Microsoft Azure platform.

You will need an Azure account. Sign in.

Login to your account and go to the dashboard.

You can get to the dashboard from anywhere by clicking on the blue "Microsoft Azure" in the upper left corner of the GUI.

To get started click on "Create a resource" to get ready to start deployments of a new instance of EXAScaler Cloud.

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step01.png

In the search box that appears, type "EXAScaler Cloud" and hit return. For some reason the completions that start to show up do not work as expected.

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step02.png

Click "Create" button.

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step03.png

EXAScaler Cloud environment can be deployed as a standard Azure application using the Azure Marketplace. And the installation wizard is used to deploy it. Installation wizard contains few configuration steps.

➊ Basics step

Project details section

Subscription - all resources in an Azure subscription are billed together. So we need select one from list of available subscriptions.
Resource group - is a collection of resources that share the same lifecycle, permissions, and policies. And we can select an existing resource group or create a new one to deploy an EXAScaler Cloud environment.

and Instance details section:

Azure region for our deployment. For example, we can select the location closest to us. And we must have enough resources in this location for the deployment (number of available CPU cores).
Deployment type: EXAScaler Cloud Application on Microsoft Azure Marketplace to create a new deployment or EXAScaler Cloud Application on Microsoft Azure Marketplace to update an existing deployment.

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step04.png

➋ Deployment step

Deployment details section

Configuration Profile: we can choose predefined configuration profile or create a custom EXAScaler Cloud configuration. Two predefined profiles available:
- Small configuration profile that will configure Microsoft Azure resources to deliver 10TB capacity and 1.44 GB/s throughput performance.
- Medium configuration profile that will configure Microsoft Azure resources to deliver 50TB capacity and 6 GB/s throughput performance.
And one Custom configuration profile, that allows us to deploy any environment according to our requirements.

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step05.png
EXAScaler Cloud pricing is based on the number of CPU cores configured to meet the defined performance requirements. And DDN Premium Support is recommended for Custom configuration profile.

OS image: we can choose Red Hat Enterprise Linux or CentOS Linux as a base OS to deploy.

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step06.png

Filesystem name that will be mounted from compute clients as a distributed network resource.

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step06a.png

Availability options section

Azure offers a range of options for managing availability and resiliency. And we can choose:

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step08.png

None - no infrastructure redundancy required.
Availability set - logical grouping of virtual machines that allows Azure to provide redundancy and availability. Each virtual machine in an availability set is assigned an update domain and a fault domain by the underlying Azure platform. Fault domains define the group of virtual machines that share a common power source and network switch. And the virtual machines configured within an availability set are separated across 3 fault domains. Update domains indicate groups of virtual machines and underlying physical hardware that can be rebooted at the same time.
Availability zone - unique physical locations within an Azure region. Each zone is made up of one or more datacenters with independent power, cooling, and networking. To ensure resiliency, there is a minimum of three separate zones in all enabled regions. The physical separation of availability zones within a region protects applications and data from datacenter failures. An availability zone in an Azure region is a combination of a fault domain and an update domain.

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step10.png

➌ Security step

allows you to choose user name and authentication parameters to remote access to the management server. And Microsoft Azure provides two options for
authentication.

Password based authentication

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step12.png

and SSH public key-based authentication

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step13.png

➍ Network step

To configure Azure Virtual Network which enables resources to securely communicate with each other. And the subnet inside the virtual network where EXAScaler Cloud environment will be deployed to.

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step14.png

We can use an existing network or create a new virtual network and specify address range

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step15.png

Also we can assign external IP addresses:

Management server - public IP address will be assigned to the management server only
All servers - public IP addresses will be assigned to all servers
None - no public IP addresses sill be assigned

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step15a.png

And configure remote access for SSH and HTTP protocols

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step15b.png

➎ Management step

To configure EXAScaler Cloud management server.

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step16.png

For the Custom configuration profile we can configure:

Type of management server

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step18.png

Type and size of management target
Type and size of monitoring target

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step17.png

And all selected configurations are checked for compliance

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step19.png

For predefined Small and Medium configuration profiles all configuration options are read-only.

➏ Metadata step

To configure EXAScaler Cloud management server. For the Custom configuration profile we can change:

Type of metadata server
Type and size of metadata target

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step20.png

For predefined Small and Medium configuration profiles all configuration options are read-only.

➐ Storage step

To configure EXAScaler Cloud storage servers. For the Custom configuration profile we can change:

Type of storage server
Number of storage servers
Type, size and count of storage targets

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step21.png

For predefined Small and Medium configuration profiles all configuration options are read-only.

➑ Client step

Optionally we can deploy compute client instances and by default the size and number of compute client instances matches the size and number of object storage servers. As an option we can create and attach local disks to each EXAScaler Cloud compute client. We can change:

Size of storage client
Number of storage clients
Type, size and count of storage targets

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step24.png

And all compute client options can be changed for any type of the configuration profile.

➒ Review + create step

At this stage Microsoft Azure checks all selected options against the subscription quota. And we can review and check all the parameters we have chosen before starting the deployment:

Subscription
Resource group
Region
Deployment type
Configuration profile
OS image
Filesystem name
Availability type
Availability zone
User name and authentication parameters
Virtual network and subnet parameters
Public IP address options
Remote access parameters
Management server options
Metadata server options
Storage server(s) options
Compute client(s) options

And we can press a "Create" button and send the selected configuration for deployment.

Next we will see the dashboard notification with "Deployment is in progress":

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > step12.png

Once the status changes to "Your deployment is complete", click on "Outputs" to view the deployment output:

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step27.png

In the "Outputs" screen:

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step28.png

Dashboard URL to display the deployment details
Configuration profile name
HTTP console URL to display the grid report
SSH Console command to remote access to the management server
Command to mount EXAScaler Cloud distributed file system. All compute clients created in this deployment should mount this filesystem
Private IP addresses

Copy and paste the "EXAScaler Cloud HTTP Console" URL on a new browser tab to display detailed deployment statistics:

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step36.png

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step37.png

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step38.png

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step39.png

Some additional information is available on the Azure shared dashboard: Azure portal → Home → Resource groups → EXAScaler-Cloud:

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step29.png

Click the dashboard name and open the "Go to dashboard" link:

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step30.png

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step31.png

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step32.png

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step34.png

And you can customize the dashboard according to your requirements.

How to automate deployment with Terraform

Automated deployment is possible with EXAScaler Cloud Terraform Scripts.

How to access a deployment

To remote access the EXAScaler Cloud deployment (management, metadata, storage servers and compute clients), we must go through the management server public IP address, using the EXAScaler Cloud SSH Console command.

And then using the management server console as a jump host we can open a SSH sessions to all other servers.

localhost$ eval $(ssh-agent)
Agent pid 5095

localhost$ ssh-add 
Identity added: /Users/deiter/.ssh/id_rsa (/Users/deiter/.ssh/id_rsa)

localhost$ ssh -A stack@20.62.171.73
Last login: Tue Aug 24 19:11:18 2021 from 181.241.16.227

[stack@exascaler-cloud-65f1-mgs0 ~]$ loci hosts
10.0.0.7	exascaler-cloud-65f1-cls0
10.0.0.5	exascaler-cloud-65f1-cls1
10.0.0.8	exascaler-cloud-65f1-cls2
10.0.0.13	exascaler-cloud-65f1-cls3
10.0.0.12	exascaler-cloud-65f1-mds0
10.0.0.11	exascaler-cloud-65f1-mgs0
10.0.0.10	exascaler-cloud-65f1-oss0
10.0.0.4	exascaler-cloud-65f1-oss1
10.0.0.6	exascaler-cloud-65f1-oss2
10.0.0.9	exascaler-cloud-65f1-oss3

[stack@exascaler-cloud-65f1-mgs0 ~]$ ssh exascaler-cloud-65f1-cls0

[stack@exascaler-cloud-65f1-cls0 ~]$ lfs df
UUID                   1K-blocks        Used   Available Use% Mounted on
exacloud-MDT0000_UUID   315302464        5744   309928012   1% /mnt/exacloud[MDT:0]
exacloud-OST0000_UUID  3181466888   570662088  2578541648  19% /mnt/exacloud[OST:0]
exacloud-OST0001_UUID  3181466888   590910696  2558305972  19% /mnt/exacloud[OST:1]
exacloud-OST0002_UUID  3181466888   580374740  2568825852  19% /mnt/exacloud[OST:2]
exacloud-OST0003_UUID  3181466888   570645704  2578552816  19% /mnt/exacloud[OST:3]

filesystem_summary:  12725867552  2312593228 10284226288  19% /mnt/exacloud

How to run benchmarks

Steps to run IOR benchmark on the EXAScaler Cloud deployment:

Run ssh-agent
Add ssh private key
Open an SSH session to the EXAScaler Cloud management server
Run IOR benchmark using esc-ior tool

localhost$ eval $(ssh-agent)
Agent pid 5095

localhost$ ssh-add 
Identity added: /Users/deiter/.ssh/id_rsa (/Users/deiter/.ssh/id_rsa)

localhost$ ssh -A stack@20.62.171.73

[stack@exascaler-cloud-65f1-mgs0 ~]$ esc-ior
IOR-3.3.0: MPI Coordinated Test of Parallel I/O
Began               : Wed Aug 25 14:43:01 2021
Command line        : /usr/bin/ior -C -F -e -r -w -a POSIX -b 16777216 -t 1048576 -s 251 -o /mnt/exacloud/0b21199cf9682b9d/0b21199cf9682b9d -s 512
Machine             : Linux exascaler-cloud-65f1-cls0
TestID              : 0
StartTime           : Wed Aug 25 14:43:01 2021
Path                : /mnt/exacloud/0b21199cf9682b9d
FS                  : 11.9 TiB   Used FS: 0.0%   Inodes: 96.0 Mi   Used Inodes: 0.0%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : /mnt/exacloud/0b21199cf9682b9d/0b21199cf9682b9d
access              : file-per-process
type                : independent
segments            : 512
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 4
tasks               : 64
clients per node    : 16
repetitions         : 1
xfersize            : 1 MiB
blocksize           : 16 MiB
aggregate filesize  : 512 GiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
write     1482.70    1482.70    21.52       16384      1024.00    0.014559   353.60     19.92      353.60     0   
read      1480.89    1480.91    21.99       16384      1024.00    0.299514   354.03     43.40      354.04     0   
remove    -          -          -           -          -          -          -          -          3.36       0   
Max Write: 1482.70 MiB/sec (1554.72 MB/sec)
Max Read:  1480.89 MiB/sec (1552.83 MB/sec)

Finished            : Wed Aug 25 14:54:52 2021

Steps to run IO500 benchmark on the EXAScaler Cloud deployment:

Run ssh-agent
Add ssh private key
Open an SSH session to the EXAScaler Cloud management server
Get a list of EXAScaler Cloud compute hosts
Open an SSH session to the any EXAScaler Cloud compute host
Run IO500 benchmark using esc-io500 tool

localhost$ eval $(ssh-agent)
Agent pid 5095

localhost$ ssh-add 
Identity added: /Users/deiter/.ssh/id_rsa (/Users/deiter/.ssh/id_rsa)

localhost$ ssh -A stack@20.62.171.71

[stack@exascaler-cloud-2ed3-mgs0 ~]$ $ loci hosts -c
10.0.0.17   exascaler-cloud-2ed3-cls0
10.0.0.8    exascaler-cloud-2ed3-cls1
10.0.0.18   exascaler-cloud-2ed3-cls2
10.0.0.12   exascaler-cloud-2ed3-cls3
10.0.0.14   exascaler-cloud-2ed3-cls4
10.0.0.15   exascaler-cloud-2ed3-cls5
10.0.0.11   exascaler-cloud-2ed3-cls6
10.0.0.9    exascaler-cloud-2ed3-cls7
10.0.0.7    exascaler-cloud-2ed3-cls8
10.0.0.16   exascaler-cloud-2ed3-cls9
 
[stack@exascaler-cloud-2ed3-mgs0 ~]$ ssh -A exascaler-cloud-2ed3-cls0
 
[stack@exascaler-cloud-2ed3-cls0 ~]$ esc-io500
 
Start IO500 benchmark with options:
 
data directory:     /mnt/exacloud/071dfa36e6b20ca7/workload
hosts list:         10.0.0.17,10.0.0.9,10.0.0.14,10.0.0.8,10.0.0.12,10.0.0.5,10.0.0.11,10.0.0.19,10.0.0.18,10.0.0.7
processes per host: 16
files per process:  39637
number of tasks:    160
number of segments: 31500
block size:         4227858432
transfer size:      1048576
 
IO500 version io500-sc20_v3
[RESULT]       ior-easy-write        1.445976 GiB/s : time 364.894 seconds
[RESULT]    mdtest-easy-write       15.411987 kIOPS : time 304.382 seconds
[RESULT]       ior-hard-write        0.461174 GiB/s : time 410.219 seconds
[RESULT]    mdtest-hard-write        2.538281 kIOPS : time 449.131 seconds
[RESULT]                 find      583.795841 kIOPS : time 9.842 seconds
[RESULT]        ior-easy-read        1.450889 GiB/s : time 363.624 seconds
[RESULT]     mdtest-easy-stat       61.106840 kIOPS : time 75.517 seconds
[RESULT]        ior-hard-read        0.543306 GiB/s : time 348.233 seconds
[RESULT]     mdtest-hard-stat       20.753560 kIOPS : time 54.080 seconds
[RESULT]   mdtest-easy-delete        5.836530 kIOPS : time 789.832 seconds
[RESULT]     mdtest-hard-read       10.320768 kIOPS : time 108.658 seconds
[RESULT]   mdtest-hard-delete        4.647816 kIOPS : time 241.181 seconds
[SCORE] Bandwidth 0.851483 GiB/s : IOPS 17.322863 kiops : TOTAL 3.840589
 
/mnt/exacloud/b44731b2e4ac4fc2/sources/results
2021.06.17-20.26.19  io500-exascaler-cloud-2ed3-cls0-2021.06.17-20.26.19.tgz

How to collect inventory and support bundle

Steps to collect a support bundle on the EXAScaler Cloud deployment:

Run ssh-agent
Add ssh private key
Open SSH session to the EXAScaler Cloud management server
Collect a support bundle using esc-collector tool

localhost$ eval $(ssh-agent)
Agent pid 5095

localhost$ ssh-add 
Identity added: /Users/deiter/.ssh/id_rsa (/Users/deiter/.ssh/id_rsa)

localhost$ ssh -A stack@20.62.171.33

[stack@exascaler-cloud-55d2-mgs0 ~]$ esc-collector
 
The following is a list of nodes to collect from:
                              
    exascaler-cloud-55d2-cls0
    exascaler-cloud-55d2-cls1
    exascaler-cloud-55d2-cls2
    exascaler-cloud-55d2-cls3
    exascaler-cloud-55d2-cls4
    exascaler-cloud-55d2-cls5
    exascaler-cloud-55d2-cls6
    exascaler-cloud-55d2-cls7
    exascaler-cloud-55d2-cls8
    exascaler-cloud-55d2-cls9
    exascaler-cloud-55d2-mds0
    exascaler-cloud-55d2-mgs0
    exascaler-cloud-55d2-oss0
    exascaler-cloud-55d2-oss1
    exascaler-cloud-55d2-oss2
    exascaler-cloud-55d2-oss3
 
Connecting to nodes...
 
Beginning collection of sosreports from 16 nodes, collecting a maximum of 4 concurrently
 
Successfully captured 16 of 16 sosreports
Creating archive of sosreports...
 
The following archive has been created. Please provide it to your support team.
    /var/tmp/sos-collector-2021-06-18-nzsnm.tar.gz

Generate the inventory report by running the about_this_deployment command:

localhost$ $ ssh -A stack@13.87.189.176

[stack@exascaler-cloud-7819-mgs0 ~]$ about_this_deployment 
cloudName: AzurePublicCloud
subscriptionId: 9978cd1b-936a-4296-8061-67c9d963dd40
location: westus
resourceGroupName: CustomDeployment
deployment: exascaler-cloud-7819
filesystem: exacloud
capacityGB: 1024
configurationProfile: custom
instances:
- hostName: exascaler-cloud-7819-cls0
  userName: stack
  proximityPlacementGroup: exascaler-cloud-7819-proximity-placement-group
  passwordAuthentication: false
  instanceName: exascaler-cloud-7819-cls0
  instanceType: Standard_D16s_v3
  role: clt
  memoryGB: 64
  VCPUs: 16
  IOPS: 25600
  bandwidthMBps: 384
  network:
    interfaces:
    - name: exascaler-cloud-7819-cls0-network-interface
      acceleratedNetworking: true
      macAddress: 00-22-48-04-1F-A9
      ipAddresses:
      - privateIpAddress: 10.0.0.6
        subnet: exascaler-cloud-7819-subnetwork
  storage:
    image:
      offer: exascaler_cloud
      publisher: ddn-whamcloud-5345716
      sku: exascaler_cloud_523_redhat
      version: 5.2.3
    bootDisk:
      caching: ReadWrite
      sizeGB: 64
      name: exascaler-cloud-7819-cls0-boot-disk
      tier: StandardSSD_LRS
      type: E6
      IOPS: 500
      bandwidthMBps: 60
  publicKeys:
  - data: |
      ssh-rsa ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4haGlji/jE0B35HrBKXeIxos2YN8jMP2m1+cx1G7UYnqr7c1I4wv073ubEfOQM2Cww4+CKEAQT3ZC+EW79NQuwJPAzPr2y961INN1zTYEX84ejbmydXzdWJT5u8LKgwKFkkKPzOU7tWJ2Co6rhz796siOpH1bvRg8gFt1oEk9B2fd3O3OA62zq6La0IUYkMQzTlU2b0hNvrVuX+Ea38fi18R7op7tSulystSrvjW/8fzRU8r7c8oTD8kELRh9H3zm0S8mk+Nq1fO4liXtNldnnpl3qAV9NipGXb+L9De15ogtqJE1/2wyaEdbXjqpA2BkHeUY8B2K7K6afpndR+AT stack@host
    path: /home/stack/.ssh/authorized_keys
- hostName: exascaler-cloud-7819-mds0
  userName: stack
  proximityPlacementGroup: exascaler-cloud-7819-proximity-placement-group
  passwordAuthentication: false
  instanceName: exascaler-cloud-7819-mds0
  instanceType: Standard_E8s_v3
  role: mdt
  memoryGB: 64
  VCPUs: 8
  IOPS: 12800
  bandwidthMBps: 192
  network:
    interfaces:
    - name: exascaler-cloud-7819-mds0-network-interface
      acceleratedNetworking: true
      macAddress: 00-22-48-0A-05-EE
      ipAddresses:
      - privateIpAddress: 10.0.0.7
        subnet: exascaler-cloud-7819-subnetwork
  storage:
    image:
      offer: exascaler_cloud
      publisher: ddn-whamcloud-5345716
      sku: exascaler_cloud_523_redhat
      version: 5.2.3
    bootDisk:
      caching: ReadWrite
      sizeGB: 64
      name: exascaler-cloud-7819-mds0-boot-disk
      tier: StandardSSD_LRS
      type: E6
      IOPS: 500
      bandwidthMBps: 60
    dataDisks:
    - lun: 0
      caching: None
      sizeGB: 512
      name: exascaler-cloud-7819-mds0-mdt0-disk
      tier: Premium_LRS
      type: P20
      IOPS: 2300
      bandwidthMBps: 150
  publicKeys:
  - data: |
      ssh-rsa ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4haGlji/jE0B35HrBKXeIxos2YN8jMP2m1+cx1G7UYnqr7c1I4wv073ubEfOQM2Cww4+CKEAQT3ZC+EW79NQuwJPAzPr2y961INN1zTYEX84ejbmydXzdWJT5u8LKgwKFkkKPzOU7tWJ2Co6rhz796siOpH1bvRg8gFt1oEk9B2fd3O3OA62zq6La0IUYkMQzTlU2b0hNvrVuX+Ea38fi18R7op7tSulystSrvjW/8fzRU8r7c8oTD8kELRh9H3zm0S8mk+Nq1fO4liXtNldnnpl3qAV9NipGXb+L9De15ogtqJE1/2wyaEdbXjqpA2BkHeUY8B2K7K6afpndR+AT stack@host
    path: /home/stack/.ssh/authorized_keys
- hostName: exascaler-cloud-7819-mgs0
  userName: stack
  proximityPlacementGroup: exascaler-cloud-7819-proximity-placement-group
  passwordAuthentication: false
  instanceName: exascaler-cloud-7819-mgs0
  instanceType: Standard_F4s
  role: mgt
  memoryGB: 8
  VCPUs: 4
  IOPS: 12800
  bandwidthMBps: 192
  network:
    interfaces:
    - name: exascaler-cloud-7819-mgs0-network-interface
      acceleratedNetworking: true
      macAddress: 00-22-48-04-1B-91
      ipAddresses:
      - privateIpAddress: 10.0.0.5
        publicIpAddress: 13.87.189.176
        subnet: exascaler-cloud-7819-subnetwork
  storage:
    image:
      offer: exascaler_cloud
      publisher: ddn-whamcloud-5345716
      sku: exascaler_cloud_523_redhat
      version: 5.2.3
    bootDisk:
      caching: ReadWrite
      sizeGB: 64
      name: exascaler-cloud-7819-mgs0-boot-disk
      tier: StandardSSD_LRS
      type: E6
      IOPS: 500
      bandwidthMBps: 60
    dataDisks:
    - lun: 0
      caching: None
      sizeGB: 256
      name: exascaler-cloud-7819-mgs0-mgt0-disk
      tier: StandardSSD_LRS
      type: E15
      IOPS: 500
      bandwidthMBps: 60
    - lun: 1
      caching: None
      sizeGB: 128
      name: exascaler-cloud-7819-mgs0-mnt0-disk
      tier: StandardSSD_LRS
      type: E10
      IOPS: 500
      bandwidthMBps: 60
  publicKeys:
  - data: |
      ssh-rsa ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4haGlji/jE0B35HrBKXeIxos2YN8jMP2m1+cx1G7UYnqr7c1I4wv073ubEfOQM2Cww4+CKEAQT3ZC+EW79NQuwJPAzPr2y961INN1zTYEX84ejbmydXzdWJT5u8LKgwKFkkKPzOU7tWJ2Co6rhz796siOpH1bvRg8gFt1oEk9B2fd3O3OA62zq6La0IUYkMQzTlU2b0hNvrVuX+Ea38fi18R7op7tSulystSrvjW/8fzRU8r7c8oTD8kELRh9H3zm0S8mk+Nq1fO4liXtNldnnpl3qAV9NipGXb+L9De15ogtqJE1/2wyaEdbXjqpA2BkHeUY8B2K7K6afpndR+AT stack@host
    path: /home/stack/.ssh/authorized_keys
- hostName: exascaler-cloud-7819-oss0
  userName: stack
  proximityPlacementGroup: exascaler-cloud-7819-proximity-placement-group
  passwordAuthentication: false
  instanceName: exascaler-cloud-7819-oss0
  instanceType: Standard_D16s_v3
  role: ost
  memoryGB: 64
  VCPUs: 16
  IOPS: 25600
  bandwidthMBps: 384
  network:
    interfaces:
    - name: exascaler-cloud-7819-oss0-network-interface
      acceleratedNetworking: true
      macAddress: 00-22-48-0A-0D-54
      ipAddresses:
      - privateIpAddress: 10.0.0.4
        subnet: exascaler-cloud-7819-subnetwork
  storage:
    image:
      offer: exascaler_cloud
      publisher: ddn-whamcloud-5345716
      sku: exascaler_cloud_523_redhat
      version: 5.2.3
    bootDisk:
      caching: ReadWrite
      sizeGB: 64
      name: exascaler-cloud-7819-oss0-boot-disk
      tier: StandardSSD_LRS
      type: E6
      IOPS: 500
      bandwidthMBps: 60
    dataDisks:
    - lun: 0
      caching: None
      sizeGB: 512
      name: exascaler-cloud-7819-oss0-ost0-disk
      tier: Standard_LRS
      type: S20
      IOPS: 500
      bandwidthMBps: 60
    - lun: 1
      caching: None
      sizeGB: 512
      name: exascaler-cloud-7819-oss0-ost1-disk
      tier: Standard_LRS
      type: S20
      IOPS: 500
      bandwidthMBps: 60
  publicKeys:
  - data: |
      ssh-rsa ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4haGlji/jE0B35HrBKXeIxos2YN8jMP2m1+cx1G7UYnqr7c1I4wv073ubEfOQM2Cww4+CKEAQT3ZC+EW79NQuwJPAzPr2y961INN1zTYEX84ejbmydXzdWJT5u8LKgwKFkkKPzOU7tWJ2Co6rhz796siOpH1bvRg8gFt1oEk9B2fd3O3OA62zq6La0IUYkMQzTlU2b0hNvrVuX+Ea38fi18R7op7tSulystSrvjW/8fzRU8r7c8oTD8kELRh9H3zm0S8mk+Nq1fO4liXtNldnnpl3qAV9NipGXb+L9De15ogtqJE1/2wyaEdbXjqpA2BkHeUY8B2K7K6afpndR+AT stack@host
    path: /home/stack/.ssh/authorized_keys

How to stop and start a deployment

If you want to stop and start the servers in your EXAScaler Cloud deployment (for example, to save cost on weekends) - please use the esc-ctl script.

Prerequisites

You need a Microsoft account
Your system needs the Microsoft Azure CLI

Before use the Microsoft Azure CLI for Microsoft Azure, you will need to authenticate under the Microsoft account you used to log into the Microsoft Azure Portal. You will use a Microsoft account and its credentials to allow the shell script to start/stop the EXAScaler Cloud servers.

Steps to authenticate via Microsoft account

Obtains access credentials for your user account via a web-based authorization flow. When this command completes successfully, it sets the active account in the current configuration to the account specified. Learn more.

$ az login

To view the current Azure subscription ID, please use az account show command.

$ az account show
{
  "environmentName": "AzureCloud",
  "homeTenantId": "00000000-0000-0000-0000-000000000000",
  "id": "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",
  "isDefault": true,
  "managedByTenants": [],
  "name": "Pay-As-You-Go",
  "state": "Enabled",
  "tenantId": "00000000-0000-0000-0000-000000000000",
  "user": {
    "name": "user@domain.com",
    "type": "user"
  }
}

if you have more than one subscriptions - please set the default subscription. Learn more.

$ az account set --subscription XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX

List of existing resource groups

$ ./esc-ctl list
Name              Location    Status
----------------  ----------  ---------
EXAScaler-Cloud   eastus      Succeeded

List of existing deployments for the given resource group

$ ./esc-ctl EXAScaler-Cloud list
Name                            Created                    Status
------------------------------  -------------------------  ---------
exascaler-cloud-65f1            2021-08-24T18:56:27+00:00  Succeeded

List of EXAScaler Cloud servers for the given deployment

$ ./esc-ctl EXAScaler-Cloud exascaler-cloud-65f1 list
Name                       Size              Role    Version    PrivateIP    PublicIP      Status
-------------------------  ----------------  ------  ---------  -----------  ------------  ----------
exascaler-cloud-65f1-cls0  Standard_D16s_v3  clt     5.2.3      10.0.0.7                   VM running
exascaler-cloud-65f1-cls1  Standard_D16s_v3  clt     5.2.3      10.0.0.5                   VM running
exascaler-cloud-65f1-cls2  Standard_D16s_v3  clt     5.2.3      10.0.0.8                   VM running
exascaler-cloud-65f1-cls3  Standard_D16s_v3  clt     5.2.3      10.0.0.13                  VM running
exascaler-cloud-65f1-mds0  Standard_E8s_v3   mdt     5.2.3      10.0.0.12                  VM running
exascaler-cloud-65f1-mgs0  Standard_F4s      mgt     5.2.3      10.0.0.11    20.62.171.73  VM running
exascaler-cloud-65f1-oss0  Standard_D16s_v3  ost     5.2.3      10.0.0.10                  VM running
exascaler-cloud-65f1-oss1  Standard_D16s_v3  ost     5.2.3      10.0.0.4                   VM running
exascaler-cloud-65f1-oss2  Standard_D16s_v3  ost     5.2.3      10.0.0.6                   VM running
exascaler-cloud-65f1-oss3  Standard_D16s_v3  ost     5.2.3      10.0.0.9                   VM running

Stop the EXAScaler Cloud servers

$ ./esc-ctl EXAScaler-Cloud exascaler-cloud-65f1 stop
Stop compute client exascaler-cloud-65f1-cls0
Stop compute client exascaler-cloud-65f1-cls1
Stop compute client exascaler-cloud-65f1-cls2
Stop compute client exascaler-cloud-65f1-cls3
Stop storage server exascaler-cloud-65f1-oss0
Stop storage server exascaler-cloud-65f1-oss1
Stop storage server exascaler-cloud-65f1-oss2
Stop storage server exascaler-cloud-65f1-oss3
Stop metadata server exascaler-cloud-65f1-mds0
Stop management server exascaler-cloud-65f1-mgs0

Start the EXAScaler Cloud servers

$ ./esc-ctl EXAScaler-Cloud exascaler-cloud-65f1 start
Start management server exascaler-cloud-65f1-mgs0
Start metadata server exascaler-cloud-65f1-mds0
Start storage server exascaler-cloud-65f1-oss0
Start storage server exascaler-cloud-65f1-oss1
Start storage server exascaler-cloud-65f1-oss2
Start storage server exascaler-cloud-65f1-oss3
Start compute client exascaler-cloud-65f1-cls0
Start compute client exascaler-cloud-65f1-cls1
Start compute client exascaler-cloud-65f1-cls2
Start compute client exascaler-cloud-65f1-cls3

All required services will automatically start and the filesystem will be mounted and available on all compute clients.

How to upgrade an existing deployment

A software upgrade for an existing EXAScaler Cloud deployment is possible by creating a new deployment using a copy of the existing file system.

To upgrade the existing EXAScaler Cloud deployment you can use the standard EXAScaler Cloud Azure application by selecting the "Upgrade" value for the "Deployment type" option.

All running instances must be shut down for the existing EXAScaler Cloud deployment before performing the upgrade, this is required to ensure data consistency while creating a copy of existing data:

$ ./esc-ctl list                                                 
Name              Location    Status                                                                                              
----------------  ----------  ---------                                                                                           
EXAScaler-Cloud   eastus      Succeeded

$ ./esc-ctl EXAScaler-Cloud list
Name                            Created                    Status
------------------------------  -------------------------  ---------
exascaler-cloud-65f1            2021-08-24T18:56:27+00:00  Succeeded

$ ./esc-ctl EXAScaler-Cloud exascaler-cloud-65f1 stop
Stop compute client exascaler-cloud-65f1-cls0
Stop compute client exascaler-cloud-65f1-cls1
Stop compute client exascaler-cloud-65f1-cls2
Stop compute client exascaler-cloud-65f1-cls3
Stop storage server exascaler-cloud-65f1-oss0
Stop storage server exascaler-cloud-65f1-oss1
Stop storage server exascaler-cloud-65f1-oss2
Stop storage server exascaler-cloud-65f1-oss3
Stop metadata server exascaler-cloud-65f1-mds0
Stop management server exascaler-cloud-65f1-mgs0

Then launch the standard EXAScaler Cloud Azure application using the Azure Marketplace

➊ Basics step

specify your Azure subscription
new resource group
region
and select "Upgrade" value for the "Deployment type" option:

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > upgrade_step01.png

➋ Deployment step

Select an existing EXAScaler Cloud deployment:

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > upgrade_step02.png

Base OS image (you can upgrade Red Hat Enterprise Linux to CentOS Linux and vice versa) and availability options:

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > upgrade_step03.png

➌ Security step

Specify user name and authentication parameters (you can use user name and authentication parameters other than the existing environment):

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > deploy_step13.png

➍ Network step

Specify network and subnet (you can create a new one) and public IP addresses options:

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > upgrade_step03a.png

➎ Management step

You can change the type of management server and the type of management and monitoring targets, but the size of the targets must be the same as in the existing environment:

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > upgrade_step04.png

➏ Metadata step

You can change the type of metadata server and the type of metadata target, but the size of the target must be the same as in the existing environment:

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > upgrade_step05.png

➐ Storage step

You can change the type of storage servers and the type of storage targets, but the number of the storage servers and the number and size of the storage targets must be the same as in the existing environment:

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > upgrade_step06.png

➑ Client step

You can change the type of compute clients and the type of compute targets, but the number and size of the compute targets must be the same as in the existing environment:

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > upgrade_step07.png

➒ Review + create step

At this stage Microsoft Azure checks all selected options against the subscription quota. And we can review and check all the parameters we have chosen before starting the deployment:

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > upgrade_step08.png

Whamcloud Community Space > EXAScaler Cloud Application on Microsoft Azure > upgrade_step10.png

And then press a "Create" button to upgrade the existing environment. A new EXAScaler Cloud environment will be created in accordance with the selected parameters, and all new targets will be created as copies of targets in the existing EXAScaler Cloud environment.