Graphics tearing in looking-glass client window output

8-P · April 12, 2020, 2:29am

I’m seeing really bad screen tearing in the looking-glass output window even when playing fairly low-end games like half-life 2. This happens in looking-glass’s fullscreen mode and in windowed mode whether the game is windowed or fullscreen. There are no tearing or performance issues when I view the video card’s output directly, so I don’t think its a general performance problem.

My monitor resolution is 3440x1440. Guest vm windows and game resolution are this (native) resolution. When I enable the looking-glass win:showFPS option, I generally see ~ 60 updates / 100 fps. (game vsync on). The vm’s shm setting is:

<shmem name="looking-glass">
  <model type="ivshmem-plain"/>
  <size unit="M">64</size>
</shmem>

I’m using intel cpu (8700k) graphics for host primary display (hdmi + audio) and a gtx1080’s dp-out both hooked to the kvm built into my monitor.

Strangely, when I run CAD, unrealengine editor, etc everything runs beautifully.

Am I doing something wrong?

gnif · April 12, 2020, 2:47am

is it really? do you have a 100Hz monitor? If not then vsync is not working.

8-P · April 12, 2020, 3:19am

Hey @gnif, thanks for responding. And so quickly.

IMG_4368.zip (7.5 MB)

zip contains brief video of tearing

gnif · April 12, 2020, 3:25am

No worries :).

Yes I can see you have it enabled in game, but unless your monitor is a 100Hz monitor vsync is not working for you and is being overidden somewhere by windows/drivers.

That said, the LG client is what renders the frame to screen and this is where the tearing will be occuring. We disable vsync by defauilt as it decreases latency and most people are willing to put up with a bit of tearing, or can turn on TearFree in Xorg which gives the best of both worlds.

If you need to turn on client vsync you can do so with the following switch:

./looking-glass-client egl:vsync=yes

8-P · April 12, 2020, 4:47am

Thanks so much for the suggestions, @gnif. I’ll check them and report back if there’s anything interesting/useful to the community.

8-P · May 17, 2020, 9:38pm

Sorry about dropping the ball on this @gnif. I couldn’t make any progress on this issue, then family + work + sleep deprivation overrode things until today.

Take a look at the attached video clip:

IMG_4449.zip (4.4 MB)

This is cellphone-captured slow-motion video of my monitor in PiP mode. The lower-right quadrant is direct displayport output from the GTX 1080 card, everything else is looking-glass output via HDMI-out from motherboard UHD-630 (8700K CPU graphics). I.e. the same 1080 output from the same vm at the same time but screen-scraped and displayed by looking-glass. This is in looking-glass windowed mode (yeah ugly stretched-bitmap), but the same thing happens in full-screen mode.

Note how jumpy the Looking Glass output vs. the raw output. Interestingly, Looking Glass output seems to actually get ahead of raw output at times. Still, the overall effect is pretty unpleasant.

–> Is this the expected result or am I doing something wrong?

When I view the GTX 1080 DP output directly full-screen it’s smooth as butter and has no hitching, no tearing. So I don’t think this is a host or VM performance issue, it seems pretty specific to looking-glass or hopefully just my current configuration of looking-glass.
Currently my ‘gaming mode’ workaround is to just switch the monitor to the GTX 1080 output and use it that way. I recently realized that I can just use virt-viewer for windows from within ‘gaming mode’ to RDP back into the host and other VMs (ala UNRAID style). It’s a little bass-ackward but it works and is mostly an improvement for my use-case unless I can fix this problem.
I’m wondering if this issue is fixable? Or I should just abandon looking-glass, use input-swapping + RDP as mentioned. I’d prefer to stick with looking-glass, it lets me run the GPU-accelerated windows apps I need to run and it’s good enough for my day-to-day work.

Do you have any suggestions for things I should try, or am I just out of luck?

I look forward to using & supporting looking-glass. I see you just checked in some code so I’m going to give it a try! Keep up the hard work.

gnif · May 17, 2020, 9:44pm

It’s clearly still tearing, you have a vsync issue.

Edit: Actually it’s strange that you get an extra frame with it to the right and then it pulls back. Can you please test with USB/Mouse passthrough, I suspect this odd behaviour might be mouse jitter.

Edit2: I might have found it, give me a bit and I might have a fix.

8-P · May 17, 2020, 11:02pm

Excellent. I’ll keep pinging your git repo and when I see the commit I’ll get you feedback ASAP.

gnif · May 17, 2020, 11:06pm

Done, going to crash though and get some sleep so not sure when I will be reporting back again

8-P · May 17, 2020, 11:10pm

Thanks so much! Will try to report back results soon.

gnif · May 18, 2020, 6:51am

I’m back… has it had any improvement?

8-P · May 18, 2020, 8:18pm

Unfortunately no changes running the latest bits (commit hash f96f0fec)

A few notes that may or may not be relevant:

I notice the tearing has a diagonal effect where one side of the diagonal line displays one frame and the other diagonal displays another frame as generated by the offset of a window in motion as I drag it across the screen. Often these frames are many frames apart in synchronization.
I tried switching to a pass-through mouse device but got the same effect, so I don’t think this is spice mouse input refresh rate or lag related.
This lag / tearing also isn’t apparent in a standard headless vms accessed thru spice/vnc. I can watch youtube videos in a browser, drag windows around, etc in a spice vm without problem. So (again) I don’t think this is an OS-related problem.
The machine is an 8700K (6 cores, HT=on --> 12 threads) that’s bios-overclocked to run at 4600x100=4.6GHz, RAM is DDR4-3200 with a little lazy XMP default timing. I don’t recall the exact numbers but when Ive run various tests (ram bandwidth, GPU memory bandwidth, etc) the results have been quite high. For example right now Nvidia control panel’s system information for the GTX 1080 is reporting PCI x16 gen3 and memory data rate of 10GB/s. Currently I don’t think I’m overclocking the integrated graphics at all. I often mildly overclock the GTX1080 (~2Ghz) but this doesn’t seem to have any effect.
I get a fair amount of ‘timer drift detected’ in the client output. But there doesn’t seem to be a correlation with the effect. At least I don’t see messages spewing when I’m dragging windows around, etc.

Random theorizing:

I’m wondering about the relationship of the frame rate of the frame server running in the VM vs the frame rate of the display client. Is the lag on one or the other? Is there any way to isolate which it is? Is it possible to display both rates? That might be a useful feature… I haven’t done more than skim the source so far, but I can program in C. Would you be interested in accepting a pull request if I were to add the feature? (disclaimer, I’m pretty time & priority constrained)
I’ve got the gpu-passthru vm’s (I only run one at a time) set to sockets=1 cores=4 threads=2 (total allocation=8) which run great when viewing the gpu output directly as I mentioned. This should leave 2 cores / 4 threads for the OS and other vms, including the looking-glass display client. Is the display client somehow resource starved? I’ve tried pausing / stopping other running vms and other services but the effect remains. I’m not using any kind of core-pinning - would that be worth experimenting? My guess is no since the vm runs quite smoothly when viewing it’s output directly.

Thanks again for your efforts! I’d like to do everything I can to back up your work, so just let me know how I can help.

Edit: I ran a few tests with different vm socket/core/thread counts (12/1/1, 6/1/1, 2/1/1) and few different topologies and didn’t see any effect. Which kind of blows my resource starvation theory…

Edit2: well, this is interesting. Adding -k to the client command line seems to have improved the situation somewhat. Which is weird, I was under the impression the framerate display hurt performance. (?) Currently I’m getting UPS varying between 20-30. FPS varies between 90-120.

Edit3: Ok, I think I might have found something. Setting “egl:doubleBuffer=no” on the command line appears to reduce the stuttering effect dramatically. This is all subjective, I’m just dragging a window around, but it appears to be a big improvement. This is with -k. FYI complete command line is currently:

looking-glass-client -T egl:doubleBuffer=no win:title="GPU VM" -S -p 5903

Edit4: I notice “[I] egl.c:378 | egl_render_startup | use native: false” on the output. Is this nominal or a potential issue?

Edit5: (sorry this post going so long) running the client as nice --10 seems to hold up UPS a little. The vm refresh rate is set to 60Hz, shouldn’t UPS upper-bound at 60 not 30? VM & host resolution is 3440x1440. SHMEM size = 60M. Could this be an issue?

gnif · May 19, 2020, 2:20am

Yes, this is because for some reason it’s trying to use a texture before it’s ready or has been expired, this is what my last fix was trying to address…

Great, thanks for that, this eliminates a possible cause.

Well yeah, because it’s slow, high latency and works very very differently.

In this instance it has no bearing on this issue, but thanks anyway.

This wouldn’t cause the issue you have with the diagional tear but it is something you should address to reduce stuttering, etc. It’s due to the guest clock source getting out of sync with the host. You will need to experiment to find out what works best for your platform/hardware.

They are already completely decoupled for exactly this reason.

Even if it was, it won’t explain that diagonal tear.

Yup and this would also likely stop the diagonal tear, but it will hurt overall performance and needs to be properly fixed so you can use the double buffer.

gnif · May 19, 2020, 12:45pm

I have spent most of my day working on revamping the egl texture upload pipeline, it should help your issue, on my system it’s eliminated microstutters (where the mouse isn’t involved) provided I am running the client at 2-4x the guest refresh rate

8-P · May 21, 2020, 5:44am

Thanks for your patience, I couldn’t get back on this until tonight. And hey, congrats on the big improvement! That’s the part of programming I love the most.

So I’m now running on commit tag t2eea64 (May 21). In my initial tests I’m seeing an improvement, but not a huge one. Maybe 2x smoother in subjective terms, but I’m not seeing UPS really improve over earlier builds. Stutters are now less jumpy, more consistent, and tearing appears to be nonexistent even in doubleBuffer mode. The jumps are maybe 50 pixels when I drag a window compared to before.

Also, this may be subjective or irrelevant but I seem to be seeing more CPU load when dragging the window than before. (?) I notice the CPU fan throttles up and host CPU utilization during drag operation seems higher. Let me know if you want me to look into this issue further (whether load is coming from vm or host, etc).

Another probably irrelevant note: I’m testing with two VMs (one at a time), both running the updated looking-glass frame server and same viewer settings. For whatever reason my ‘daily driver’ cad/unrealengine/development vm only seems to get UPS up to about a maximum of 30. The second vm, which I pulled from a snapshot of the first vm where it was in a windows installed but otherwise clean state easily hits UPS of up to 40. Not sure why, this second vm has a lot less installed but only thing that should have an effect is a slightly newer nvidia driver.

Regardless, in both vm’s, when egl:doubleBuffer set to either ‘yes’ or ‘no’ I’m seeing your new ‘egl_warn_slow’ warning message in the display client output.

So it looks like I’m bottlenecking on the display client? If so, any suggestions for ways to speed up the client? My guess is better host graphics hardware would be better than the integrated graphics I’m currently running. Should I throw in a cheap video card and switch the host to that? I think I can free up an 8x slot although I was planning on using it for a 2nd HBA soon. Looks like I might be in the market for some new hardware; was hoping to hold off on that until the december hardware refresh.

I’m going to play around with core allocations and tweaking the igp BIOS settings a little, see if that helps. If you or anyone has suggestions, I’d like to hear 'em.

Tomorrow I’ll try to update my games vm and post results. Let me know if you want to see another video clip of the latest behavior and I’ll also get on that tomorrow. But I gots to hit the sack, I have an 8am meeting in the morning

gnif · May 21, 2020, 5:48am

Not sure which version this is… but if it was earlier today you need to and should update again as a ton more changes went in that improve things for everyone.

8-P · May 21, 2020, 5:49am

Wow, you’re on it. This is the latest commit from about two hours ago.

gnif · May 21, 2020, 5:50am

Latest is: https://github.com/gnif/LookingGlass/commit/ae2ffd0a28835822a10d489544c05fdadbdd7a90

8-P · May 21, 2020, 5:52am

Yup, looks like you committed after I pulled. I’ll try my best to carve out some time to get you new results ASAP. Probably lunchtime PST.

gnif · May 21, 2020, 5:52am

It looks more like you’re hitting the limits of the IGP bus transfer, this is a known issue on many of these systems.