So for the last few months I have been going on an adventure with storage and trying to create a shared space for data and config files.
Last night something hit me that I hadn’t thought of before. Since Ceph has its own network share thing I could just setup Ceph in Proxmox and then mount CephFS inside the VM. This would also have the advantage of allowing Ceph to run bare metal.
Is this as simple as I am thinking it is?
Is this as simple as I am thinking it is?
Depends, on your definition of simple :). IMO, nothing is simple, especially in the world of remote and/or distributed filesystems, especially if they aim to act as a POSIX fs. When I think back, Ceph stands out in my memory as a complex technology with many components that take time to fully understand. You could easily spend countless nights learning it, if that’s what you’re aiming for.
on an adventure with storage and trying to create a shared space for data and config files
Do you absolutely need or want a block storage for this? How about some object store? (disclaimer: my recent experience is mostly with the cloud stuff). Something like a Minio, which is quite simple to start with; that should get you covered for “shared data” with tons of various tooling existing around it. As for configs, maybe an etcd (or consul) glued with templating. (Again, this is just semi-devops/infra dude in me talking)
Also, maybe look in the direction of NVMe over Fabrics, in particular TCP
I did look into garage which is a s3 object store but the problem with s3 and object storage is that it is a very bad choice for rapidly changing data like a database. I want a filesystem so that it is easy to manage and it should be well optimized.
That sounds way expensive
There are ways to properly setup fileserver like things with Ceph(fs).
I’ve seen a bunch of talks on the matter
I’ve personally not tried it like this yet.
You mean the dedicated hardware? Doesn’t have to be necessarily implemented in hardware. Linux kernel supports both server and client parts of nvme-over-tcp
If you’re saying its for a database (i read it as postgres or mysql) why not just old NFS?
Though, i wouldn’t put the files backing a database onto a networked storage, because what’s the purpose? DB is by itself a networked service.