I am in the process of scoping the budget for a project that I want to embark on which will involve large-scale 3D reconstruction from satellite images. I will need more than my desktop PC to handle the memory requirements and to the expedite computational time required to solve the large scale non-linear least squares “bundle adjustment” optimization problems associated with multiple-view image reconstruction.
I know that some of you (in particular Economists) regularly use Julia and cloud computing to conduct large-scale simulations for your experiments. I believe some of those experiments even run for a matter of days. I would be grateful if you could share some of your experiences with running Julia in the cloud, and in particular the ballpark costs associated with running your experiments.
For me, Hetzner offers a great value with a variety of options, especially as an American because I’m exempt from the VAT.
They have dedicated servers starting at 34 euros per month with a recent i7 or Ryzen and 64GB RAM. Their cloud instances are also good because they charge by the hour and have a simple API for spinning them up with up to 16 virtual cores and 32GB of RAM per instance.
If you can split the problems with many small task, ( and save the results )
then the key point is the “spot pricing” or similar service ( sometimes 70-80% savings compare to the on-demand request ) or “Buy unused compute capacity at deep discounts”
So you can start 100 parallel service … and you have the result in very short time …
see: ( AWS EC2 Spot Instances; Google Cloud Preemptible VMs; Azure Spot Virtual Machines )
Hetzner dedicated is OK ; but need an extra month payment for as a frontend … ( ZEN2 and Intel cores ; full machine) … Not ideal for high parallel ( 50 machine ) + short time experiment ( 3 days )
Hetzner Cloud - cheap ; but shared CPU … so not perfect for high CPU works ( my experience )
Now for personal testing I am using the extreme cheap Contabo ( Europe )
This is an old conversation, but I wanted to ask if people have new experience on Contabo? Looks cheap, but the 4.99€ version would be fine for a personal website, I think. It also includes 32TB transfer…
another (free) alternative - ideal for the ARM+Julia testing.
4x Neoverse-N1 core ; BogoMips=50.00 ; aarch64
Oracle Cloud Always Free Tier Ampere A1 Instance 4C 24GB RAM Details " In the Always Free tier, we get four cores and up to 24GB of memory so, in theory, we could make smaller instances. Still, we decided to go big and make one large instance."
I’m trying to get my feet wet on cloud computing. You mentioned last year that you pay 34 euros/mo for a dedicated server. Does that mean you can run all the threads on the i7 for 24 hours a day for that price?
However, the same is true for VPS. Usually you have a guaranteed availability of cores and ram. But, network, the physical processor and disk are shared.
As mentioned above, this is usually slower than a own dedicated machine.
Thank you for your response. I currently run my code 24/7 on 15 cores of an AMD 5950x machine that I built a few months ago - toward a computational goal that is still months away. Maybe I should sell it and rent 150 cores for a month… I’m not particular about when it runs or anything so maybe could get an interruptible service.