Just purchased a ton of DDR3-ECC for my vm servers in home lab. I have some sata2 ssds in raidz (ZFS) for the vm pool, but wondering how I could kick it up a notch. Any ideas how I could put some VMs completely in ram? I’m more of a devops engineer, so I was thinking serverless. Could I put containers, or boot disks in ram?
Or what sorts of services would speed up with ramdisks? and how could I effectively manage this, short of rsyncing and cronjobs. I’m generally using Proxmox for hypervisor, and Ubuntu for my web/development servers. I was thinking something like Varnish cache, and Mysql could benefit, but how could I speed up actual samba performance, or nextcloud/collaboration type workloads?
I’ve heard theres an opensource lamda project. This is another thing on my radar. Think ram and devops - GO!
If not… your network will be the bottleneck and taking the RAM off ZFS is probably a net lose. Because ZFS can cache at the block level whereas you can only really cache via your ram disk at the full VM level… even if only 5-10% of it is “hot” data.
If it is… again, you need to manually shuffle what you need in RAM at the cost of ZFS cache.
One of the big things you can do to speed up zfs is to turn off access time tracking, to avoid a write for every read
I hear what everyone is saying. ZFS block storage probably is helping, and it’s nice that I do have so much ram to let zfs breath.
I guess I am wondering what are some creative way that I could use ram more quantitatively? Ramdisk is one option, but I am wondering if there are ways that I could provision temporary services into ram, or if you’ve heard of people doing this. I wonder, for example, AWS general compute t instances have an interesting price/performance climb. AWS gives you more ram but not more cpu up until like $80. Maybe AWS wants you to use up the ram, because it’s backed by batteries and genies. Maybe AWS has some sort of ram disk failsafe. Just wondering if anyone has ever tried provisioning services into ram, and “reinventing” a ram disk with a buffer (on linux). If I had two minutes of battery backup, I could easily write a lot of ram to disk before the power left.
I think that inherently, trying to do ram-disk for “part” of your server is more complicated than it is worth unless you are doing it at epic scale with epic fault tolerance, and/or with data you do not care about (e.g., indexes generated from other data).
And even then…
You’re just trying to do what disk cache does with less granularity, less reliability, and more admin complexity. A filesystem cache will KNOW what data is hot for example, and adapt as your hot data changes. i.e., if your temporary services deserve to be in ram because they are hot, they will be in the cache anyway. And if they aren’t, then something else that is more deserving will be there. And your disk cache can do this at the block level. So if only 10 megs of a 4 gig file is hot, only 10 megs of ram needed to speed it up. Not the whole 4 gigs! The other 3990 MB can be used to cache something else that’s hot.
Plus… you need to copy all the stuff (even non-hot parts of the file(s)) from disk to RAM anyway. Every time you boot. If only 10% if your files are hot, you’re taking 10x the disk IO to move all the blocks there when they won’t even be hit…
Maybe a ram disk might make sense for a dedicated cache box or a dedicated database index, etc. But i still think that you’re wasting your time outside of extreme niche workloads.
edit:
i’m not devops, but i do wear a storage admin hat from time to time, and have done for past 10 years…
I’d take the effort you’re looking at putting into this, and throw it at tuning your storage… or your mysql instance (the ram cache it uses internally for example - many applications, especially io performance sensitive ones already have RAM caches that they can more effectively manage), etc.
RAM you dedicate to RAM disk is RAM that is unavailable to applications, even if said RAM DISK is only say 50% utilised (in that instance you have effectively cut 50% of your ram disk size out from the OS for nothing). Cache isn’t like that.
