I teach a class on building ML APIs and deploying those APIs to K8s. Azure is kind enough to donate non-trivial amounts each semester to support the K8s workload, and I take advantage of the fact that all students have Azure AD credentials to ensure that we have MFA access for all users to the cluster.
Since students tend to have significantly different development environments (laptops include ARM based MacBooks) supporting their bespoke systems to deploy to the class cluster has become a significant burden.
I want to consolidate their development environment to a known state that I can reproduce while not spending significant additional dollars. I feel as if I have a few options:
Spin up a VM per student with 2 vCPU and 4 GB of memory and a 100 GB disk for 100 students that’s roughly 5k/month of spend.
Spin up a large shared VM with roughly the same specs, but consolidated. Still roughly 5k/month.
Configure development pods against the K8s cluster that automatically suspend after non-utilization. Turns out tooling here is rather nacent for streamlining the UX for students.
Purchase a similar large server/workstation and throw Proxmox or VMWare (free for our institution) on it, configure a VM per student and enable aad-auth with ssh on Ubuntu 23.04.
I rarely do non-cloud and the only workstations I purchase are more focused on GPU workloads rather than virtualization. I am looking for guidance on how we can build out a cost effective solution (option 4) while minimizing the operational burdens of supporting students. I will likely reimage the entire server each semester to wipe any student data. Used hardware is fine, and student use is extremely variable except the day prior to homework being due ;).
Hmm, I would try to avoid a graphical VM environment. We have those at work and they always lag, and is a high maintenance solution.
A better solution IMO is SSH driven remote debugging. Here is how to do it in VS Code, we have also got it to work for VS proper.
This allows for a remote toolchain but code is saved and edited locally. Kinda like best of both worlds. This can then be combined with a VM; and it pulls only the resources it needs.
I’m used to remote dev over SSH and that’s how I was going to enable the last option. They can actually generate SSH credentials from their Azure AD auth so I don’t have to manage the public keys myself.
This is basically what I planned but wanted help specing out a cost effective solution.
Yeah, the most cost effective would probably be to set up a spare 24 core server somewhere and let that be your toolchain, then mandate VS Code as official dev env. and ask them to git the code so you can have a look. All code is run on the server, so toolchain is on the server but editor on the local machine.
This can be done relatively cheaply and I remember one gig using this setup on an 8 core in 2015.
Another option is Docker or qemu VMs. Basic idea is you set up the toolchain once in a VM or Docker and then your users can run these however they wish - as long as it is the same toolchain the results should be reproducable regardless.
I think this is the way. If you distribute a Dockerfile or docker-compose.yml they can literally run the development environment anywhere, even a Chromebook.
Some of the enterprises started deploying VM for desktop users because they used to deploy physical machines. Moving to VM not only saved them money on hardware but came with lots of other benefit as well. VM is cheap but not that cheap. Also not efficient either but more efficient than individual physical desktop PCs.
Perhaps you should go back to re-think your strategy for an inexpensive and efficient shared environment. Containerisation is also in vogue but I would avoid if it can be avoided.
What’s the development environment for your students?