Parallelize Julia on AWS

KissCluster is written in bash so it works in any Julia version (there is no direct Julia dependency and it works also with Python, Java and R).

KissCluster’s spot instance support is built on the fact that there is not central (master) node - instead there is just a DynamoDB table that holds the information about jobs and worker nodes.
Hence you can use KissCluster with in on-demand instances. An alternative for on-demand clusters is to use tools such as Slurm.

The difference is that KissCluster is very lightweight and is good for scenarios such as “run a cluster of 200 servers for 15 minutes and then terminate the cluster” while tools such as Slurm are much more advanced and offer functionality for maintaining and managing permanent long running clusters.

Hope that helps.