Return to Level1Techs.com

Calling all Devs: Benchmarking and Diagnostic Tools for Linux Gaming

linux
gaming
tools
#1

In the recent Linux Gaming video from L1T, Wendell made a request to see if the Level1 community could lead a project in getting some diagnostic and benchmarking tools for Linux.

Pretty light specifications:

Measuring and benchmarking performance and frame times for games and APIs.

Tools like ocat, fcat, and fraps.

Additionally, these will need to be easy to use for gamers and non-engineer types. It will also need to be modern, to handle 60+ fps and high resolutions.

So, this is a tall task, which I think is why he asked for the community to frontier forward. It looks like ocat is C++*, and I’m willing to bet the others are as well. Definitely feel free to pitch some ideas.

Golang would be another option, as they have a plethora of tools in the standard library.

Anyone up for the challenge? Got any boilerplate or ideas to begin with?

* https://github.com/GPUOpen-Tools/OCAT

9 Likes

#2

for basic suites, it’s be good to point people to open benchmarking and Phoronix Test Suite as a starting point too.

For basic monitoring there’s also nvidia-smi, for OC there’s setting coolbits=28 and power state settings, and there’s also the very detailed intel-gpu-tools for igpus, and gmonitor has some low level generic monitoring as well, moreso than the smi anyway, and works with bbswitch.

psensors and freon are decent for generic sensor logging,

you can also see hidden processes by using fuser via sudo fuser -v /dev/nvidia*

it also seems that the newer versions of PTS have at least some frametime reporting capability:

https://www.phoronix.com/scan.php?page=news_item&px=MTQxNDI

and then there’s LGD:

0 Likes

Linux Gaming: Natively (part 1 of 4) | Level One Techs
#3

I’d like to help, but can you be more specific about what are you looking for?

What feature(s) do you feel like are most critical in the short term?

Is there actually a need for a new tool? Or would it be acceptable to simply extend/fix some members of the GPUOpen family?

2 Likes

#4

What currently exists for linux?

https://glxosd.nickguletskii.com/
But it is EOL and needs a logging facility too.

Vulkan:
https://gpuopen.com/archive/gpu-perfstudio/gpuperfstudio-vulkan-tools/

(needs guide? updated for new amdgpu?)

there are a few moreI should prolly add

0 Likes

#5

The phoronix-test-suite is open and broadly tests many things.

Is the idea to compare with other operating systems? If yes then maybe Superposition?


(Blistering result I know)

1 Like

#6

There’s already a ton of existing data from Phoronix, as well as the OpenBenchmarking.

It would be really awesome if we built on that, rather than making Yet Another Half Baked tool.

Wendell’s concerns were dead on though; this needs to be something that can be used broadly, especially by YouTubers and tech reviewers. At the very least, that’ll require tightening up the user experience.

0 Likes

#7

If what we should build is a nice GUI for the PTS, then that’s very doable.

Maybe even a web UI, so you could trigger tests and view results remotely?

0 Likes

#8

I’m not sure of the condition of Phoromatic these days, but a web UI once existed. It was aimed less at end users, and more at administrators managing test clusters, but the foundation already exists.

1 Like

#9

I didn’t know that portion existed.

I can confirm it’s still there. I installed it just now, running the client and server on the same host caused my system to reboot.
Not sure what exactly it was trying to do at the time, but I’m inclined to leave them both stopped for the moment.

0 Likes

#10

there was a start made on a debugger/capture/overlay for steam games by valve, that never got off the ground:

1 Like

#11

LGD seems pretty fully featured already:

this may also be of interest:

https://apitrace.github.io/

3 Likes

#12

So one problem we ran into with pts doing all this work with Linus’ team is that it does some super bizzare things sometimes. For example, some of the published results on phoronix show pts runs of bioshock on linux that outperform windows… “with the same settings” but when I was able to replicate those results I found the graphical settings between windows and linux different.

A lot of that benchmarking requires the programs themselves to report data, too, which is not quite the goal we have here.

We have seen on windows that reported data from the game doesn’t match reality with actual frame times/frame rates. That’s why ocat/fcat is so important because it allows analysis from raw captured video during both canned and “normal” gameplay.

tldr: can confirm bioshock infinite’s built-in benchmark routine is not foolproof.

2 Likes

#13

I can suggest s-tui as an alternative to AIDA64 stress test / Intel power gadget.
This is mainly for CPU stress bench-marking.

(Disclaimer: I am one of the creators)

4 Likes

#14

I’ve experienced the same variances in my own testing, and it’s been incredibly frustrating, building a testing methodology for linux that’s application/hardware agnostic and accurate. I’d love to be looped in if you find something that works well for OGL as well as vulkan

0 Likes

#15

does it log to csv

0 Likes

#16

It can

2 Likes

#17

it’s discontinued, but Bugle has framerate/frametime logging functionality:

https://www.opengl.org/sdk/tools/BuGLe/

0 Likes

#18

I need this in my life. Have to fish some kittens out of a toilet then maybe I’ll have time to poke at it tmrw

1 Like

#19

hmm

bugle still compiles fine with glew/SCons 3, but the resulting installation can’t find any of its shared objects

glxgears: error while loading shared libraries: libbugleutils.so.8: cannot open shared object file: No such file or directory

Anyone with experience with SCons want to chime in on how to configure it properly? I’d assume it has something with the install path or env but I don’t have any experience with this build system or tweaking it

edit: just me being dumb and tired, had to add /usr/local/lib to my LD_LIBRARY_PATH, got bugle up and running, tested on glxgears with logging enabled, and got some output

[INFO] logstats.ms per frame: 16.62 ms/frame
[INFO] logstats.frames per second: 58.3 fps
[INFO] logstats.ms per frame: 17.16 ms/frame
[INFO] logstats.frames per second: 55.6 fps
[INFO] logstats.ms per frame: 17.98 ms/frame
[INFO] logstats.frames per second: 49.6 fps
[INFO] logstats.ms per frame: 20.14 ms/frame
[INFO] logstats.frames per second: 83.7 fps
[INFO] logstats.ms per frame: 11.95 ms/frame
[INFO] logstats.frames per second: 60.8 fps
[INFO] logstats.ms per frame: 16.45 ms/frame
[INFO] logstats.frames per second: 59.4 fps
[INFO] logstats.ms per frame: 16.85 ms/frame
[INFO] logstats.frames per second: 60.1 fps
[INFO] logstats.ms per frame: 16.64 ms/frame
[INFO] logstats.frames per second: 60.0 fps
[INFO] logstats.ms per frame: 16.68 ms/frame
[INFO] logstats.frames per second: 60.2 fps
[INFO] logstats.ms per frame: 16.62 ms/frame
[INFO] logstats.frames per second: 59.9 fps
[INFO] logstats.ms per frame: 16.68 ms/frame
[INFO] logstats.frames per second: 60.0 fps
[INFO] logstats.ms per frame: 16.67 ms/frame
[INFO] logstats.frames per second: 60.0 fps
[INFO] logstats.ms per frame: 16.65 ms/frame
[INFO] logstats.frames per second: 60.6 fps
[INFO] logstats.ms per frame: 16.51 ms/frame
[INFO] logstats.frames per second: 60.2 fps
[INFO] logstats.ms per frame: 16.60 ms/frame
[INFO] logstats.frames per second: 59.4 fps
[INFO] logstats.ms per frame: 16.84 ms/frame
[INFO] logstats.frames per second: 59.8 fps
[INFO] logstats.ms per frame: 16.73 ms/frame

with a little profile config, this can easily spit out opengl data in csv format

hell, even the gui still works

I’ll be doing a writeup soon on this, but all you have to do is install, set the path, and run

BUGLE_CHAIN=<profile> LD_PRELOAD=/usr/local/lib/libbugle.so <application>

and you get whatever data you set up in the profile in a log or as output

Update: also works with the game transistor (GOG), yet to test on steam titles, but it should be as simple as adding the string above as a launch option

0 Likes

#20

I wouldn’t know where to start building something like this but if someone has a plan and needs help, I would love to help with this.

0 Likes