Which shows exactly how much you know about our humour.
Is this a pissing contest about who's the expert in comedy? You're not gonna make me list my favorite standup comics are ya?
Contest and no. I live here, this is normal. You did not get it and reacted like he was attacking a guy.
Not to completely derailed this thread with who's the expert jokes.
AdoredTV vs Hardware Unboxed Rd.3
1 Like
i don't know, i like specific numbers/FPS. because i know if i get this thing, i'm going to get these numbers unless there's some variable like overclocking etc. minimum/maximum frames are a bit useless, but at least the minimum frames show the variance a little. but that's just me.
I like AdoredTV I listen to the guy and he makes science sense. He looks at the past a lot and that make me sometimes yawn but the past does tell us a lot of what is coming. Im just in the moment mode a lot.
Tech Youtubers are guys with cameras. Some of them are going to fuck up. Who hasn't fucked up.
1 Like
It's a coordination problem. Assuming all reviewers can perform tests equaly well, their combined forces could cover a broad range of cases. In a cooperative effort, each reviewer could contribute a thoughtful piece to the bigger puzzle.
In the competitive framework they face, however, the incentives are different. Some of those pieces are going to attract more attention than others, and everyone is going to be after those views and ad revenue. The market incentives they face are aligned with duplicating efforts and reaching simplified conclusions.
remember when teksyndicate had better fx8350 benchmarking numbers than others?
I don't see a difference.
1 Like
Look, you really don't want to start using subjective judgement when trying to compare things that can be measured objectively. That's the path audiophiles have taken, and it get's pretty silly honestly. Before you know it we will have gold plated cables for modular PSUs because "it makes the colors prettier!"
You can certainly use your own subjective experience to give you hints about what you should look into. But unless you use objective measurements then it's useless. This is a big problem when you have "benchmarks" where someone is actually playing the game.
2 Likes
If there's anything I've learned from so called "benchmarks" it's that subjective judgement is better than cherry-picked objective measurements. Even science is riddled with cherry-picking, obfuscating results that don't suit your agenda (if the scientific research is funded by companies), p-hacking, and a truckload of other maneuvers to achieve a certain result.
People are probably expecting too much from guys that make a living on Youtube and receive free products from companies, not to mention that we don't even know what contracts they are under.
Sure, you can "hack" objective benchmarks as well. Particularly if you are doing manual runs through the game and don't actually measure the same run for all cases (and don't do multiple runs to minimize any differences).
The point is that when it's all subjective then it's all garbage. You may as well not do it. And I don't say that because I don't trust the people doing the benchmarks. It's because it's impossible for them to do a fair "benchmark" like that. That's the reason I compared with audiophiles.
When you only rely on your judgement and senses to compare you need to actually take that into account when you are "measuring". That means running ABX tests, double blind. If you think it's a lot to ask YouTubers now then it's nothing compared to that effort. Placebo is a very real effect and you can absolutely not trust your subjective opinions. Or well, you can trust your subjective opinion that you "think" it's better one way. But it has nothing to do with the actual differences between the things you are comparing.
OK, first of all if "scientists" are paid to find a specific result, it ain't science.
Secondly, audiophile means literally that you like sound. Nothing more.
The idiots you are talking about are called ... idiots. They want to be elitests and look down on others. That is why they buy BS that doesn't do anything and that is why I don't have a problem with companies ripping them off.
Gaming benchmarks have nothing to do with either one of those. They are way to short, way to specific, in other words they are to small of a sample size to even be objective. And because they are automated, they aren't real world examples either. Like I said before, if you are playing games and notice a stutter happening here but not there, that is real world and provable in a video. That is why I like Jays video and why I liked TeamPGPs 4K Doom stream. You can simply watch the performance happening. With fps counter in the corner for your sweet sweet numbers.
Well you're right about most of that. I guess it's not necessarily "better" as I put it. It's better for me and most of the people I know.
It would force the reviewers to use descriptive words that would tell you what to expect in general. Only a small % of PC gamers give value to all those hard numbers and benchmarks. And if you ask me, a term like "smooth" can be applied to both 100 and 120 FPS, which would save a lot of people so much money, because they would mostly be picking AMD stuff which is cheaper.
In contrast, here you have hard numbers with precise percentages of difference and as a buyer you still, even with hard numbers, get the wrong impression of performance because everybody is setting up their tests in the same way which favors one side - Intel and Nvidia. So as a buyer, I would be tricked into thinking that Intel is better in all cases because all benchmarkers got that result.
Also, I live in Eastern Europe. Almost all merchandise (everything from electronics, to food, to laudry detergents) is different (lower) quality than the things they sell in USA, Canada, Western Europe etc. Some things are cheaper here, but electronics are the same price or more expensive, yet it's always C-grade, because most manufacturers have the world segmented into different-quality-standards markets. So the benchmarks I see by western authors don't really apply to my country either way.
edit:
what I was trying to say is that, for me, a descriptive judgement would at least (ironically) seem more genuine and less "serious", that is - you know that you should take it with a grain of salt. And in contrast, benchmarking and hard numbers are often presented as objective and absolutely true, when in fact they can be set up to favor one side over the other just as easily as personal descriptive judgement. As a result it can sway buyers one way or the other. And that's what I have a problem with, the fact that these benchmarks are presented as "benchmarks". By definition a benchmark is a reference point, a standard. There is nothing standard about these tests. It's marketing lingo paired with sponsored content. That's what I have a problem with.
1 Like
So, he tested with AMD's current highend GPU, FuryX, and R9 295x2 in place of x2 RX 480s, both against 1070 and Titan XP. He found that Titan XP wasn't performing as well on DX12, and even slower, under Ryzen in the most CPU demanding section of Rise of Tomb Raider, at best it only managed to pull a 12% lead against a 295x2 on occasions. A hypothesis, AdoredTV, proposes is a bottleneck in the Nvidia drivers, but I would go as far as architecture bottleneck since it isn't a secret Pascal doesn't have a true async architecture, most of the leg work is being done by the ridiculous clock speeds that AMD GPUs can never reach.
However, the most sensible conclusion I would go for is the same as Steve from Hardware Unboxed said, "Rise of Tomb Raider is an outlier...I don't think this one game tested using one level is enough to say that Nvidia GPUs are handicapping Ryzen CPUs". It's one of the few DX12 games that works with crossfire/sli in DX12, but it's also one of the most inconsistent games to bench with, and to top if off benching this game on a newer platform that has optimization issues on older games. It is very telling of how much work there is to be done with the down-to-the-metal-coding on PC, even, Frosbite, one of the most well regarded game egines today has problems with DX12.
1 Like
Let's not be obtuse here, I'm sure you understood what I was talking about.
The prevalence of snake-oil in audio gear is waaaaay higher than any other technical area that I can think of. It's even gotten so far as to banning the mere discussion of proper testing methodology (like double-blind and ABX).
Well, I fundamentally disagree with your premise (that scientific methods are useless on not applicable to graphics benchmarking) but if you replace the FPS counter with a frame timer I pretty much agree with your conclusion. Data combined with an actual video is really the only way to properly document the tests. For that reason I quite like DigitalFoundry and their tests.
And to reiterate. The reason you can't simply play the game and judge how it feels is because that won't actually tell you the facts. If you just got the latest Nvidia card and you like it then you WILL notice stutters less. You can't trust your senses, it's like the Renee Descartes of benchmarking.
Whatever upsets you about whatever you think is described by the term audiophile is not the topic right here, right now. So let's keep away from that.
That is not what I meant. But when I think about it, with so many variables like changing UEFI and drivers, boost clocks, windows ... being windows and the small sample size because 50 reviewers testing a five minute benchmark still only makes it five minutes of a ten to 50 or more hour game ... well, that isn't scientific at all. And it shows. Science can predict things. As far as I can see, benchmarks are software generated guesses at best, simply wrong at worst and very likely irrelevant tomorrow. That is not science.
I know what a placebo effect is. But again, just because stuff is presented in numbers doesn't mean it is absolute. And as @Helium_3 also pointed out very often those gaming benchmarks are indeed presented as absolutes and they simply aren't that.
TeamPGPs Doom stream was buttery smooth and looked amazing. They played at 4K on a 1080Ti while streaming downscaled 1080p game footage, video from the camera, audio from another source, al of that on only six cores of that ryzen chip. That is a real world example. That is what I want.
Here, look. Close to the end they up the bitrate and it looks awesome.
I see everyone talking about how techtubers are going towards more entertainment than actual knowledge impartment. This is what one should do if they want to attract the masses. It is a fact that is supported by veritasium (will link if you want) that when you make educational videos the average viewer doesn't understand much of it and only gets a feeling that they have learnt something when they actually haven't. To make them really learn something you have to force them to think and most don't like thinking as it is uncomfortable unless you REALLY like something. So in depth content is generally avoided. So their goal is to make you feel that you have learned something new irrespective of whether you have actually. So the techtubers have to add the element of fun more as compared to knowledge. What is the main difference between J2C and LTT? Although LTT has less in depth content it has more views but there is more fun element. One reason why this issue appears more on a forum is because there are more people who actually know technology in depth and hence feel the absence.
Wow I wrote this really long!
Same goes for a lot of those review sites and I am not questioning their numbers or integrity here, I do believe that they all do the best they can. But most of them simply don't play games anymore, they benchmark, fully automated if they can. They don't know if that thing in that level of the game is still making a little stutter or not because that isn't part of the benchmark.
So very true. Another way I sometimes notice this factor is how testers in these "CPU tests" lowers not only game options that makes it easier on the GPU, but also options that can have a significant effect on the CPU load. So without knowing how the game works outside of benchmarks their tests end up being performed in a limited situation with settings that don't always make sense for what they're trying to do. As long as the numbers in a test fall in line with expectations, the bad numericals gets incorporated with the rest.
We can't demand tech reviewers to be spend all their free time playing games since they probably have a life on the side as well, but before adding a game to a test suite mostly because it has a built in benchmark, it's on them to try to assess whether the game and its test run is relevant enough to be used at all. That's probably hard to do in most cases without at least getting their bearings on each title.
There are a lot of things that adoredtv says that make sense.. however I think he is too much of an idealist.
Things are still a bit of a clusterfuck when it comes to gaming / performance and until the whole gaming eco system sorts itself out (game engine / middleware / api's), gaming is always going to be single thread / ipc dominated.
I knew years back when mantle first reared its head that the industry crossover period between api's was going to be ugly.. with consumers pretty much looking at benchmarks for dx11 vs dx12 and scratching their heads wondering why we were even bothering for a lot of games.. real reasons being obvious, game company tried bolting it on to their creaky old dx9 engine or lack of common middleware for basic render functions meaning they had to write it themselves (often their first attempt at doing so).
In fact I may have old threads where I wrote that out.. will hunt for them now as that would make me seem like a PSYCHIC!! (in reality its me being a pessimist :D)
For a while I had high hopes for the industry as Intel's race to mobile / tablet and the wintel alliance meant that Intel / Microsoft would be forced to sort shit out, they have the muscle to make it happen.
However when intel dropped the atom bomb (see what I did there :D) it pretty much meant that there was no financial imperative for intel anymore to adopt the lower level api's in windows...
I worried that with the mobile / tablet path gone from intels roadmap that we would be back on the ipc race of diminishing return.
As much as I want things to get better I just dont see the financial imperative there anymore now that intel has dropped atom.
short answer: as long as people keep recomending dual cores - 4 threads or quad cores builds for gaming then developers will never start writing games properly for 6 cores / 8 cores as that is their customer base.
It's chicken / egg.