CloudyCluster Quick Start Guides

The following instructions outline the various aspects of launching, configuring and running jobs in CloudyCluster.

Planning Overview 

CloudyCluster is launched in 3 phases or steps.  The first phase is to launch a control node, then an HPC environment, followed by autoscaling HPC compute resources based on the jobs that are submitted to CCQ.

 
CCDiagram.png
 

Step 1  To use CloudyCluster you must first launch the Control Instance, which provides the management web-ui and associated APIs, using one of two methods.  

The first option is through the AWS Marketplace and CloudFormation

The second option is to setup an IAM role manually, then launch your Control Instance using it.

Step 2 Configure the Control Instance.

Step 3 Launch the HPC Environment (VPC, login instance, scheduler Instance and any required storage)

Step 4 CloudyCluster is now ready for elastic HPC Job submission

Security Considerations

  • CloudyCluster follows the AWS security best practices as part of its automatic deployment.  The control node deploys a VPC and all computational and storage resources are created within the VPC.  The login instance performs the role of bastion host.   All permissions follow the least necessary privileges philosophy by assigning only the needed permissions to the IAM roles required by the instances to perform their functions.  There are additional security features that can be enabled to meet various security requirements including: encrypted EBS volumes for OrangeFS, Multi-Factor Authentication, and enforce s3 object encryption.  These features can all be enabled through the web user interface.

CloudyCluster Termination and Uninstall

Backups and Disaster Recovery

  • It is recommended that you backup data that is not easily reproducible to s3 and or glacier.
  • If you automate the automatic deployment and deletion of your environment with Automaton, make sure your critical data is saved outside the environment before you delete it.
  • If you need real time BC or DR capabilities, you can run multiple CloudyCluster environments in different regions.

Additional Documentation can be found at docs.cloudycluster.com