Architecture

CloudyCluster runs entirely within a single availability zone. The control node, which is first launched by the user, generally runs in the default VPC, but may run in any VPC which has outbound access to the Internet. When CloudyCluster is started, it creates a VPC for itself. This is split into a public subnet, which has Internet access, and a private subnet, which does not.

CloudyCluster consists of the aforementioned control node together with additional EC2 instances for the login, scheduler, and filesystem nodes. As jobs are submitted, compute nodes will be spun up. In addition, home directories may be placed on EFS if desired. An S3 bucket is created for the user as part of the cluster. CloudyCluster uses DynamoDB resources as its backend database.

CloudyCluster communicates with an Omnibond service (CloudyCluster Home) for provisioning of its domain name and Let’s Encrypt for HTTPS certificates. It may also communicate with OS repositories for software updates.

A user accesses the CloudyCluster web interface via the control node. The user may also access the login node, in the public subnet, directly with SSH or to Open OnDemand over the web. The login node also provides WebDAV access for file transfers. The scheduler, filesystem, and compute nodes are not directly accessible and are in the private subnet.