I will need to test some variable combinations in a month or so and wanted to use aws to give me a whole bunch of cores to run multiple instances.
I normally just run everything using addprocs and then sending each combination to a different process as they complete.
With an auto scaling group and spot instances this would obviously be quite different! The simple part of rerunning a job if the remoteref returns an error is not an issue of dealing with. How would one go about adding new processes when a new instance is available though?
A “simple” alternative would be to have a shellscript downloading a job and then posting the results as part of the startup script but I’m hoping there is a better way!
Thanks