So I’ve seen a few tech publications mention compiling firefox (or chrome) as a benchmark. But I can’t seem to find any details about how this is done and how to reproduce it. In fact Wendell mentioned it in the threadripper review but it’s not part of any of the graphs in the video/article, and doesn’t seem to be a part of the open benchmarking suite (aside from some ancient thing no one uses). I think it’s mainly something LTT does, and to my knowledge, they don’t have the details of their benchmark stuff software wise documented anywhere.
Does anyone know of a standard, reproducible way to benchmark a firefox compile? I went off and followed the guides for doing the compile but it’s (understandably) set up in a way to be quick and easy to make and test code changes, which is to say a lot of stuff gets compiled once and then left in place, making re-running a bit of a chore.
I’ll go ahead and state I’m not really familiar with compiling code, I just think it’s an interesting test. So I might be missing obvious things.
yes categories are messy and tags are… well we have them. Count your blessings.
Anyways. Theres a terminal app out there sobewhere that is supposed to watch a build and notate what happened when, reports errors in a text file, and I believe has a timer built in. Howuver, I’ve only seen brief mentions of it but fuck me if I can remember what its called.
Basically what this test is for is to see how long it takes. Is it going to be all day? Is it going to be 20 minutes? IDK how big thexfirefox source is, nor chrome, xut I know viwaldi has a 1gb source code blob. So probably its just a bruteforce stress test. I would run multiple builds with different combinations of cores just to test the chip.
Just timing how long a command runs is no big deal, in linux you can just use “time” followed by a command.
I’m more worried about having a reliable state for the compile to be in to make sure the same things get compiled each time, as well as making sure the compile is run with the same flags between different systems (not sure if LVM changes those on it’s own).
Firefox makes a good benchmark as it is written in Rust (Mozzila’s in house language) nowadays. The way rusts integrated build system works can guarantee reproducible builds.
Tho you are correct these media outlets should document which release / git commit they are building and for which platform. But that is the only information needed for Firefox.
How would you re-run the compile after you’ve done it once without re-downloading all of the code? Whatever it’s version of “make dist clean” would be, or something like that.
Hmm, the guide I followed (from the firefox site) was using ./mach build for compiles. cargo clean doesn’t seem to clean things for that to do a full run again (takes >1m to run vs ~7m first run) and cargo build fails with a dependency error that the googles say should be fixable by deleting cargo’s registry folder. Alas, this does not work.
not having a thread-ripper I have not actually built Firefox. if they are using mach for the build step that might be pulling in external dependencies.
So back to the original point a bit, it’d be nice if this was standardized. However LTT is doing their firefox compile, it’s completing a good 5 min slower than what I get on my stock 3950X according to time ./mach build. The only difference I can think of is that I’m using Linux but… it shouldn’t be THAT much slower? Looking at the windows guide it’s a huge pain to compile on windows so I can’t be bothered to test it out.