Setting up a cluster on Amazon AWS/Azure

Our university cluster is down and I have some time sensitive simulations I have to run. I looked into AWS and Azure, but never have really dealt with cloud computing… I need about 500 processors (simulation time is actually really low, maybe 5 minutes)…

Does anyone have any experience setting up Julia on a cloud host? I am thinking I need to spin up a virtual machine, install an OS on it, install Julia, and use addprocs(500) to add my 500 processors, but not sure if AWS/Azure will automatically allocate 500 processors?

Absolutely no experience with this, but maybe have a look at https://aws.amazon.com/hpc/parallelcluster/?

To get something with less setup time, you could pick the largest EC2 instance which has like 100 cores and run it there perhaps?

I have run up a few pcluster clusters on Amazon.
Do you still need help here?

As @tbeason says if you can run on 64 cores, or around that number, allocating a single Intel or AMD Rome architecture server may do the job.

Sure, 64 cores sounds okay… the run time will just be a little bit longer. Is there a way to only pay for the actual compute times used? Or does it start billing as soon as you spin up the VM?

The billing is for as long as the machine is active – whenever you “possess” the resources. They don’t care if you use it or not :slight_smile: