So I startet working at a local community.
Their IT equipment is outdated as hell. I mean EOL for all servers and network devices. Luckily the firewall still had service.
We are now tasked with updating the entire server and storage side to new systems.
We have a couple of Virtual Servers. Nothing fancy. Two AD servers, exchange. Couple File servers, two sql systems and that’s about it apart from some support hosts to manage the voip system and so on.
All in all let’s say about 10 TB of VMs and Data. Since we do not need enormous uptime and a system can be down in case of error for a few minutes or hours. Do we really need an expensive SAN to run our virtual machines from? ATM all our VMware hosts run on local storage so no vmotion in case of error or anything. If the VM hosts goes down that’s it. Please start from backup .
In that case would it be ok to migrate the virtual machines from local vmstorage to a NAS. License VMware essential plus to enable vmotion in the cluster and run the machines from the NAS with sufficient bandwidth.
Also we could enable replication between two NAS systems to enable redundancy even if the data is slightly behind on the passive system.
Not really since we a few older colleagues who really have problems with keeping up with new stuff and finding competent it personal is hard in our area.
10 TB is accurate growth would be about another 10 TB in the next 5 years since we outsource all hungry applications.
IOPs is another question. Any tips how to figure those out with all this spinning rust and local vm storage?
shared storage between VM hosts to enable things like live migration for vmware.
possibly: improved performance vs. local disks
possibly: improved resiliency vs. local disks
possibly: enterprise features like replication and snapshots
What it also causes you:
increased complexity: VM+SAN (or NAS) is more complex to configure and administer
other associated possible cost increases: for decent performance you’ll need either 10+ gig ethernet (preferably 40-100gig) or fiber channel. even if you don’t need the speed for network throughput storage needs high bandwidth for IO command latency as well as throughput (i.e., you can only send so many storage commands down a network connection due to framing overhead, etc. - it isn’t just bandwidth throughput for the data you’re transferring - the speed of your network will impact the command rate you can send.
Is it worth it ?
If you can afford it, it is nice. But it depends on the business appetite for cost vs. reliability, performance requirements, etc.
it aint cheap unless you DIY but then you’re adding all the complexity, points of failure, etc. without a decent level of support - its on you. You still need to be pretty aware of a lot of moving parts and how to diagnose/troubleshoot performance issues or faults as they occur.
Oh the other thing with SAN/NAS storage. Generally you’ll run out of throughput before capacity unless you’re talking archive style workloads. VMs? You’ll hit throughput bottlenecks most likely first.
Already all full. If we run out of storage in a host we have to options. One migrate the vm to another host with more storage the old way… two … move old data to archive partitions on other systems.
Yes that’s fucked up i know.
Well money is an issue so is adding complexity. I can document and configure a load of stuff and get them running at decent speed but what if i hit a tree. None of my colleagues are technical advanced enough to understand what that should do even if it’s documented well.
I setup netbox for system and network documentation…. The didn’t understand a the basics and how to use it.
it does NOT sound like a SAN is beneficial for this infrastructure. here is what i would build.
1x 4u or tower server with local storage. this would be you VM host.
1x TRUENAS NAS box. i would NOT use this for VM storage directly. i would connect this and use it for a disk to disk backup solution. you could also use it for shared drives from the VMs so that windows ‘previous versions’ of files are available with the benefit of ZFS.
this configuration is easy to manage, and way less than a SAN as far as ‘learning curve’ goes. plus the cost is less. (the TRUENas can be configured with SAN like features if you need them at some point)
I was going to suggest something similar but you beat me to it.
I wouldn’t say you’d get no benefit from a SAN for VM stuff directly, but you are a bit small to justify that sort of deployment. Simply from a cost vs. complexity vs. not so heavy workload perspective.
As above i’d focus on using some sort of NAS (i.e., file shares) level storage for more frequent disk to disk backup rather than over-spending on full VM level shared storage, etc. You’ll get better bang for buck i think.
Can you add a JBOD to one of the esxi server? or both to give you more disks on the server, probably your cheapest option (would allow you to have backup versions on the VM on the both if you added a jbod to both.
Also probably lowest technical knowledge needed for this expansion (Assuming you have the rack space)
Yeah more than 1 host would be prudent. Ideally you want say, 3 (or more ideally)… otherwise with 2 you inevitably end up tanking performance massively (i.e., 2x workload on 1 host). with 3 hosts, you’re only overloading the remaining two by 1.5x (assuming they’re all loaded - which will happen eventually) which is a little less… shit.
Without the heavy uptime requirement though vmotion might be overkill - do consider how long a restore will take though, copying a few TB around takes time.
also worth noting that if you do ever plan on doing windows clustering across hosts, there are some rules regarding how many hosts need to be up for the cluster to work.
The other way to skin this is to virtualise your storage across your hosts (hyperconvergence!) but the commercial solutions for this are either
expensive (e.g… nutanix, vmware vSAN)
somewhat flaky (vSan, last i heard, which was admittedly a while back)
combination of both
The non-commercial solutions? I’d run in lab but… i wouldn’t risk my career on experimenting with them at work in production.
and yet another way to skin this is just buy more internet and stick it in AWS/Azure/etc.