powered LSRSV9500-4l connected to an unpowered host adapter/GPU?
A variation: c-payne’s CP-0550 or a PCI-e PLX88096 switch with two GPU-s on it plugged into LSRSV9500-4l. The whole cluster downstream of LSRV9500-4l is powered via a 2nd PSU.
The idea is to switch the GPU cluster on/off as needed. The 2nd PSU would be toggled on/off via the mechanical switch while the whole system is shut down, of course.
LLM-s get really confused when asked this question They oscillate between saying it is safe and that “parasitic currents” from “sidecar lanes” on MCIO cable will pump current into powered down device with unpredictable consequences.
My thoughts are that Nvidia GPUs are very good at low idle power draw, my 5060 Tis at work sit at 3-5W each when they aren’t doing anything. They don’t even spin the fans, so there’d be no noise concerns. This seems very much like a “juice not worth the squeeze” type of venture. It could have merit if you really gotta save power, and you’re using an Intel or AMD card (those like to sit at 20W idle no matter what I do).
I don’t see why a 48GB mod 4090 wouldn’t idle at the same low power as the stocker. It’s the same memory controller and firmware yeah? Don’t they just whack on 2x capacity chips?
… the origin of the BIOS the Chinese acquired for 4090 is a bit of mystery, it certainly came via non-official channels, can be anything. Being low on power draw was very low on their priority list I’d imagine when designing the boards. I also came across a statements that “bigger” cards have higher power draws - not measured personally though.
The manual for CP-0550 - based on Microchip Switchtec PM50052 - states that “typical” power draw is 30Wt. It’s bigger brother CP-0551 - which I don’t have need/money for - is documented to have a “typical” power draw of 50Wt.
In reality my PC is a lot more likely to have a PLX88096 because PCIe4 is sufficient for 4090-s and PLX88096 is much cheaper. PLX88096 card might have a power draw more similar to LSRV9500-4l which is documented to be about 12Wt at idle.
All in all I fully expect each GPU to idle at around 40Wt plus the switch at least at 12Wt.
That does not mean you cannot do it and have it likely sometimes “work” (until it all stops working)
The GPUS were not designed to have bus power from the PCIe slot constantly and then randomly have power applied or removed from the PCIe power cable supplied by your 2nd PSU. I also doubt the LSI Bifurcation controller will be happy about all this, it’s one step away from hot swapping (that controller says it supports NVME hot swap, not any other PCIe devices).
Not even sure the GPUs would start up properly on powering them up, and them becoming non functional when you turn them off will likely cause some kinds of issue with the OS even if you aren’t displaying anything off them.
Can you setup something to enable sleep mode on the PC and have it put the computer to sleep normally? A basic dry contact relay controlled off Home Assistant or similar would work, just parallel the case’s motherboard front io header for power, then setup your OS to sleep if the power button is pressed for a short duration. May need a small device to turn on & off your 2nd PSU with the 1st.
device adapter connected to it via MCIO cables with an PLX88096 in it not powered
GPU downstream of PLX88096 not powered
Then
power everything down
switch 2nd PSU on via mechanical switch
press power on button
everything switches on
2nd process is known to be relatively safe, ppl are running like that, with 2 PSU-s. Question is if 1st process is safe - powering on LSRV9500-4l while its downstream host adapter and everything beyond it are not powered up.
2nd PC is an obvious alternative. The cost is not that different from the cost of an LSRV9500-4I as well. It would prefer one PC if there was a way to arrange power.
If all the motherboards I have just get stuck on POST then it’s obvious the setup is not working.
It’s not obvious to me POST will get stuck though. That GPU requiring AUX power will likely not have any power via PCIe slot at all - because it’s host adapter will not be powered. So I’m not sure it will be able to complain.
The issue is simply what Agent9 laid out. GPUs are not designed for this. There’s likely to be some voltage running through some pins simply by having any PCIe device plugged into any other one, they wouldn’t be able to recognize each other if they didn’t do this. There’s issues enough if you run Nvidia GPUs in passthrough and try to switch them between VMs or even just restart the VM. In stock form Proxmox has a reset bug that doesn’t fully reset the power on the GPUs, so it fails to initialize them when the VM reboots or changes because the GPUs think they’re still working for something else. I forget exactly what the fix is as it was a one and done and I haven’t had issues since. I don’t think it’s anything that’ll help this use case though. Maybe if you set up a way to reset the GPUs the way a hypervisor does after booting them up or something.
Methinks if you’ve got a 2nd PSU and a frankenstein chain of PCIe switches and adapters you’re at… actually at a higher complication level than simply having a 2nd PC, because PCs are designed to work when put together normally, unlike this setup.
If you just want to attempt this for fun then all well and good. If you want a practical solution I just cannot see how this would be it.
For a single capable rig, can you run something like a single Silverstone Hela 2500Rz? It has a ton of PCIe 8 pin connectors (7 total) and 4ea of the 12v 2x6 600w connectors.
At 120v it will supply 1650W and at 240v it will give the max 2500W. So still usable on most North American outlets & common UPSs. I have one running my Threadripper 9970x rig with 4x R9700 GPUs. Idle the whole build is around 150W with some stuff loaded up which isn’t too bad.
Have you also considered selling the kinda cobbled together setup and building something more normal that will behave more predictably? Would should be able to sell pieces for a decent price and maybe transition over to a different platform. Maybe even just buy a slightly older DDR4 PCIe 4.0 Threadripper setup, typically 2 full 4.0 x16 slots and 2 4.0 x8 slots on a lot of the better TRX40 boards.
You can also get a remote KVM pretty cheap that has built in wire terminals to land wires that go to the motherboard power switch header, can remotely power it on, off, and control it remotely easily.
It’s the opposite, I’m designing the system and procuring hardware.
Nothing to sell ATM. Was considering a 4x V100 32Gb rig, but eventually gave up on the grounds that software support for LLM-s on 4x V100 is a bit shaky, even if individual ppl have been able to coax good performance out of particular models.
Need a daily driver and an LLM rig. Don’t need full LLM power all the time. Can have two computers, but space is limited. Can have one computer, but then it feels ethically wrong to be heating the atmosphere more than strictly necessary.
They oscillate between saying it is safe and that “parasitic currents” from “sidecar lanes” on MCIO cable will pump current into powered down device with unpredictable consequences.