Not really a "build log" per se, but I am now running Ceph over 100 Gbps Infiniband

Depends on your requirements. Some people don’t need more than a file share.

No, your file will get locked by a user and another user can’t edit it, until the file lock gets released. That’s why I said it’s not for everyone and why I said nextcloud has its uses.

They’d literally run the same. Well, perceivably the same, i.e. an http 1.1 server loading a page in 0.01 seconds on x64 vs 0.002 seconds on aarch64. That’s twice as slow on arm, but you won’t notice it. If it can run on a tamagochi, the performance difference will be imperceivable. Of course those are number pulled out of my a**, just to illustrate the idea. You can’t do much with an http 1.1 server, let alone run things like nextcloud on it.

You’d be surprised.

On Fedora 33, the i5 9500T uses 1.5W of power at idle. The only SBC I have that beats that is the odroid n2+, at about 1W or less. Neither on AC power, on my UPS on a shared brick, nor directly on DC power from my power station, do the watt indicators go up by more than 1W. And this could literally be margin of error if another device I have, like my PC, might be loading something in the background. And the amlogic s922x is nowhere near the performance of a 9500T, not even in its dreams.

Nothing else goes really that low on idle. My pentium silver n6005, given a 5 to 10W power usage of my monitor only goes around 3 to 8W at idle (for the whole package). With the monitor, I see 13W power draw on my power station and around 20-ish on AC power ('cuz AC conversion is inefficient).

It depends what you’re running, though it is true. If I need the oomph, I have my threadripper 1950x and I also have a lower powered NAS hypervisor combo, a thinkpenguin 4-bay NAS with an i5 11500T and 4 sata SSDs (idles around 20W on proxmox, although I never tried to change the defaults to get it to a lower power state, I’m certain it can go way lower, but I power it on infrequently to test things, so I didn’t really care).

Everything else I need runs on a rockpro64, my PC and I’ll likely have some 24/7 services run on my odroid n2+ (once I finish my insane setup on it). But again, I don’t run services that require a lot of resources, you could probably run what I want to run on a pentium 2 (if it had the RAM).

While it’s cool to have a HA cluster at home (I used to have 1, until a power outage killed 2/3 servers at once - I had a UPS, but it didn’t matter), I don’t think people should run their homeprod on a cluster, unless they really want that HA for some reason. It’s very important to know what you want to run, plan out your system requirements and then plan out what you’re going to build.

Part of the planning is also budgeting. It might be more price conscious to have a smaller server hosting things like nextcloud, homeassistant and jellyfin, running 24/7 and have a larger one that’s powered on on-demand for tasks like, idk, AI image generation or something. Other scenarios might involve having a single beefier system being on 24/7 and not having split hardware.

And part of budgeting is also power bill costs (e.g. you can get a scrapyard server for cheap or even free, but that thing will raise your power bill so much, that it would have been worth spending a bit more money outright on a newer and more power efficient hardware, that between a couple of months and 2 years, it would’ve been amortized, compared to the server that just keeps accumulating operational costs).

At 1 of my previous workplaces, we used to have 7 racks of junk. We downsized to 2, but didn’t really reduce much of the power bill. Then we colocated the 2 racks. After a while I left the company, but they downsized from 2 racks to about half a rack (you still have to rent a whole rack) and they got the cost down significantly by going with newer, way more expensive servers, but that could pull the duty that 2 racks used to. I wish I was still there to play with those servers. Even Wendell, when epyc first came out, made a couple of videos of the basement rack, where he decommissioned the old yellow google servers, downscaling a rack to just 3 servers (IIRC).

Space is at a premium, but yes. I have a buddy who has a whole roof of solar and makes 12kW of power theoretical max (I think he said he has 30 panels - peak input he got was around 10.1kW). Some of his airbnb tenants used so much power after he installed the array, he said he would’ve been really pissed if he had to pay that much money for the power bill. For myself, I plan a tiny home, I might be able to make a separate adjacent solar array (although trees are pretty tall around where I live).

Out of curiosity, why not a zimablade? Not quite n100 level, but decent power in a small package (otherwise, the odroid h4 ultra is pretty hard to beat on size and different cases provided, for different scopes, unless you go a little bigger to mini-ITX).

I prefer doing the same, but I see why some people prefer the cash-flow approach. I don’t think it’s wrong, but it’s definitely more wasteful (unless you can recoup your money by quick upgrade and sell cycles, before the value of the products drops too much).

I prefer mapping my IP addresses in my DHCP server (MAC bound). That way, even if I have static IP addresses configured on servers (be them VMs or baremetal), I know I have a centralized location where I get the IP list. I back up the DHCP IP binding conf files. Bonus: if I ever boot a live USB on one of my PC and use DHCP, it gets the same IP address (although I have to either set up vlan trunk on the nic or put the switch port in access mode to that vlan).

Meet the universal USB type-C hack (from top 20v, 15v, 12v).

And when you need custom voltages, there’s always step-down converters, I’m using the same model as on my rockpro64 to power a phone directly with 4.2V (no battery).

1) that’s complicated and 2) it’s not portable (still requires you’re on debian). What if I want to run that on devuan?

Exactly 3 months ago, on proxmox 8.2.2.

RISC-V definitely has a path forward, but it’ll first be a replacement in mobile and low-powered stuff, before it becomes viable as a server platform. I’m pretty sure RISC-V will se performance uplifts faster than ARM saw. However, Linus Torvalds was kinda annoyed at the RISC-V development (and complaining that history’s repeating itself, as they fixed the same or similar bugs back in early intel days). I still have high hopes for RISC-V, but I’m not going to place money on it myself right now. I was interested in a BeagleV, but software support and hardware specs were hard to justify the purchase. The same applies to newer boards too, at least now probably bleaglev might have some decent support (a few years back, risc-v already had about 92% or so of debian’s repo ported, but that doesn’t mean it could run some of that software without choking).

That’s why I want to set up incus on my odroid n2+. I’ve looked into a bunch of other things, like firecracker, but none were easily deployable (documentation sucked). I have high hopes for QEMU microvms. Until then, incus has qemu VM options.

I kinda hate’s proxmox implementation of it. Last I tried to deploy a custom rootfs image, proxmox complained that the distro is unknown. That doesn’t happen with pure lxc or with incus (you can download any rootfs from anywhere and just import and launch it).

If you’re fine with what proxmox provides (gentoo, arch, alpine, fedora, opensuse, debian, ubuntu etc.), then what proxmox offers is fine.

lmao

Ditto.