Standing up cluster on Azure

Hey All!

My program is trying to migrate a lot of our teaching and research towards using Azure instead of our on-site cluster (for which ClusterManager.jl has been amazing), and I was wondering if anyone had experience they could share.

Python (and dask in particular) has a great little package for starting up and configuring a cluster for dask – just a couple lines of code and the cluster gets stood up with proper images and vnet, and all you have to do is click a link to get a jupyterlab session on the head node. But I haven’t found anything similar for Azure.

Azure.jl provides access to the Azure API to turn things on and off, but I’m not clear how to do things like register workers with a head node once the cluster is up, or whether there are good images already out there to use.


(Something dynamically scalable, like clusters created in a Azure Machine Leaning workspace, would be ideal if possible…)