Setting up a cluster on Amazon AWS/Azure

Offtopic
1

Our university cluster is down and I have some time sensitive simulations I have to run. I looked into AWS and Azure, but never have really dealt with cloud computing… I need about 500 processors (simulation time is actually really low, maybe 5 minutes)…

Does anyone have any experience setting up Julia on a cloud host? I am thinking I need to spin up a virtual machine, install an OS on it, install Julia, and use addprocs(500) to add my 500 processors, but not sure if AWS/Azure will automatically allocate 500 processors?

2

Absolutely no experience with this, but maybe have a look at AWS ParallelCluster - Amazon Web Services?

To get something with less setup time, you could pick the largest EC2 instance which has like 100 cores and run it there perhaps?

3

I have run up a few pcluster clusters on Amazon.
Do you still need help here?

As @tbeason says if you can run on 64 cores, or around that number, allocating a single Intel or AMD Rome architecture server may do the job.

4

Sure, 64 cores sounds okay… the run time will just be a little bit longer. Is there a way to only pay for the actual compute times used? Or does it start billing as soon as you spin up the VM?

5

The billing is for as long as the machine is active – whenever you “possess” the resources. They don’t care if you use it or not :slight_smile: