I have a Z87-Pro (g3220), ax1200, 32gb ssd, random mid-tower case, and a bunch of drives of various sizes from the mining days...
Already running a commercial 16TB NAS, which works very well indeed, but would like something with much more grunt and customization. Plus it would be a fun project and give me yet another layer of redundancy when it comes to backing up data.
Could something like this be done by running multiple instances of an OS under VMs? Or should it be a NAS first, then somehow run everything under that?
Hmm, interesting. I don't know if virtualization is your best option because it's generally better to let your router software access the interfaces at the host level. I'm already doing the NAS, VPN, media server, and 1/2 a router on my own home server, and as a linux junkie I could tell you it's all possible in one instance of debian.
I wouldn't put you router on the Sam machine as any of your servers since if it goes down it takes every thing with it and running the router as a VM is pretty tricky.
Every else can be ran on one box just use LXC containers in Linux so they share the same kernel and use less resources. Windows you can use KVM or Xen.