Hi all,
been looking into possible solutions for moving my workstation (an Epyc 24 core) out of my studio and in the basement, distance would be about 10 meters.
I am running proxmox with multiple VMs, one of which is OSX with an USB controller passed trough and an AMD GPU, another is a win10 with a second usb controller and an NVIDIA GPU passed through
Both of them have audio interfaces connected to the USB controllers.
The video (assuming I can fit optical displayport connectors in the conduits) should be straightforward with a pair of optical 8k displayport cables.
For the USB side I am not sure ā¦ it would be either a pair of Icron units ($$$$) or one of these PCI Express over fiber Extenders plus remote USB adapter (Still $$$, but would need only one if lucky): https://adnacom.com/aus324/
The idea behind the Adnacon unit is intriguing, instead of extending the USB signal using a USB transmitter and receiver, they have a solution that extends the PCI Express bus over fiber, and the remote unit is a pair of standard Asmedia USB 3.2 controllers.
This means that if the host interface ābehavesā in terms of iommu grouping, I may be able to use only one remote unit for two VMs, and pass through to each one of the two ASMEDIA Controllers
That would still be a 1500EUR āifā between the units, the QSFP transceivers and the fiber
So, my question is, is anybody already doing it with one of these units?
Sounds pricey, 40Gbps thunderbolt 3 optical cables are ā500EUR ā use your monitor as a hub ā¦ Active optical display port 1.4 cables are ā100EUR a piece.
Itās not very clear to me what practical benefit youād be getting from the adnacom unit at a 1000EUR+ expense?
One thing to consider is do you really need USB 3.0? For my own use case, I donāt use anything that requires such a connection. Anytime I need to offload a ton of pictures or something, I can just plug it in in the garage. So for USB 1.1/2.0 access, I use old extron USB extender plus units from ebay which are compatible with your network as is, including through switches. There are often complete setups (transmitter/receiver/power bricks) that are about $70-90.
If you needed 5gbps speeds, then these USB3.0 units that use SFP+ connections are a possibility for under $300. Note that this one is USB 3.0 only, not 2.0/1.1, and requires itās own dedicated fiber line. Units that can do 3.0/2.0/1.1 double the price. Trying to go higher speed 10gbps 3.1 makes the price shoot to the moon.
With that out of the way, the idea of having PCIe 3.0 lanes over fiber is definitely an interesting one (if it werenāt silly expensive), I wasnāt really aware that it was a thing yet outside of wonky enterprise only āsolutionsā.
I donāt need bandwidth, the audio interfaces are latency sensitive though, so I have always been skeptical of solutions that convert the usb signal and send it over IP ā¦ the round trip latency usually is a killer, but I have never looked into the units you suggested that from the name at least look like have been specifically geared towards audio products ā¦ the power brick alone must weigh three kilos though
Iām with Log in suggesting you to go with USB over RJ45. Itās not really an IP conversion, the rignal is just converted to travel over a different cable with different specs.
If thereās even added latency I bet that good units keep it under 1ms since thereās not much processing done to the USB signal.
With Amazon returns, I think itās worth a shot to save thousands of euros. But I find PCIe over fiber as intriguing as you do, for sure! Maybe the cool factor makes it worth it, nothing wrong with that!
How do you measure latency? And I guess more importantly, what is latency in this case? I donāt do audio stuff, but I havenāt had any issues with a usb 2.0 (usb itself has pretty large latency) mic, over the network, and into a VM once I isolated/pinned cores and memory.
If my setup is applicable, I could runs tests that you recommend.
Audio latency: the audio signal not breaking down when coming out of the speakers ā¦ I have found that the Win10 VM, even with interrupts isolated, vfio usb controller, isolated and pinned cores, is extremely sensitive to audio driver related spikes. With my Zoom U-44 it soon becomes unbearable just listening to a youtube stream, if I measure the audio latency with a tool I have spikes of over 1000ms !
I am currently using a different audio interface, an old Roland UA-25 whose driver is way less picky, measured audio latency stays below 10ms with peaks at 20-30, so if the extender adds 1ms I would be golden as long as it doesnāt cause spikes.
The PCI-X extender, as everybody says is the cool sounding solution, but I am almost certain either iommu or the ASMEDIA chip would not let me off easily, especially in OSX, and I would still need to make two 20mm displayport fiber (so fragile) cables fit into a 25mm conduit that is already a quarter full ā¦
For now, I have found a good price on an HDBAseT extender from avaccess
200EUR for a unit that can definitely be reused for other purposed is way better than a 1500EUR (plus cables and tranceivers) shot in the dark ā¦ but still ā¦
As you surely know Windows has always been plagued by audio driver issues and latency. I think it has to do mostly with the kernel thatās not light enough on itās feet to process audio that quickly.
Back in the early 2000s Ableton Live had way lower latency running through Wine on Ubuntu Studio than natevely on Windows XP.
When things like virtualization pile up on top of the already unstable Windows audio stack things turn for the worse.
All of this to say that I donāt think a USB to RJ45 extender is gonna give you much troubles. Maybe the USB hub part in the thing could slight increase latency (since the KVM you listed is touting zero latency I bet itās in the order of 5 to 10ms tops) but it would never be anything like the 1000ms you experienced.
Iāll try and measure in a somewhat more scientific way than the usual āfinger in the airā I have used to validate if using an Epyc arch for my personal workstation would be a good thing
When having interruptions/dropouts in the audio stream latencymon will measure thousands or tens of thounsands microseconds for DPC latency, and depending on the app that is trying to output audio Iāll get dropouts, to a point where the audio interface just gives up and needs a reset or, if I try and play a game (say, apex legends) the game will hard lock and dependin on the state the game engine is in it will require a complete proxmox reboot to get the video card back in a state where itās usable ā¦
None of that happens with the current audio interface (and using pinning, interrupt affinity, and running other VMs on other EPYC CCDs than the one running the windows games ā¦)
This is my thing! I have a rack in my basement and have everything running over optical or CATx to my office!
Optical displayport is the way to go for video, USB is either cheap CATx for USB 2.0. USB 2.0 is fine for a keyboard, mouse, streamdeck, ectā¦ but not enough for a webcam, microphone, or external USB drive.
I just purchased and tested the Icron Spectra Icron USB 3.0 spectra 3022 (Two-port Multimode fiber 100m Extender) | eBay I have not yet run OM3 cable from my rack but I tested on a 20ft jumper and it worked as expected. I had a webcam, SSD, external HD, and USB 3.0 bluray player hooked up. I hope to have the fiber run in the next week (need to finish bending conduit).
I saw you posts (actually, they were what got me started in looking into it)
My use case would require 2x - Displayports, Spectras, fibers
Displayport aside, what the Adnacom would give me for the 1000CAD list price (plus transceivers, that in my case would be another 200-450EUR) would be the ability to drive two separate USB setups with only one controller and only one fiber ā¦ I donāt have a lot of space in the existing conduits, and I happen to just have promised the wife that we would be done with āadditional cablingā for the foreseeable future so anything that reduces the amount of masonry work and/or cables running around the house would definitely be a plus,
Initial update,
I just receivedmy used AV Access 4KEX100-KVM and have been testing various configurations.
First impressions: the unit is solid, connectivity is straightforward, the USB remoting works without any noticeable lag/delay, latencymonitor is happy, streamed music and games do not show any glitches.
The video side ā¦ all is well if I limit the output at 1080p, but when trying to force 4K out I think I hit the limitations of the device, that can output 4K@60Hz only using 4:2:0 chroma subsamplin, as opposd to the more universal (and better looking) 4:4:4 and 4:2:2
What this translates to in the wonderful world of EDID and video resolution negotiation is:
windows VM: no output when booting unless I force an EDID of 1080p, as the boot sequence cannot handle 4k@60Hz 4:2:0 ā¦, but I am using the windows vm at 1080p anyway, so all is good
OSX VM: no issues if setting 4k@30Hz4:4:4, couldnāt find a way to tell OSX to use 4:2:0 at 60Hz so no 4k@60hz with this unit ā¦
All in all I am very satisfied, so much so that I also bought the big brother that costs around 700EUR and can do 4K@60hz ā¦ will report how it works when it arrives
Video quality: I couldnāt notice any lag/quality difference when comparing similar resolutions and refresh rates. Also couldnāt notice a lot of difference on the WIn VM betweek 4K@60hz 4:4:4 on the displayport output and 4k@60hz 4:2:0 using HDMI, but I am color blind
One very cool feature of these devices is that they require power only on the transmitter unit (that is going to go in my attic), while the receiving unit is powered through the HDbaset connection, so one less cable/power brick/socket to worry about (two in my case) behind my desk ā¦
