Hacker News Reader

Comment by drewg123 | Hacker News Reader

Towards the end of the article, they talk about high performance modes added by manufacturers and say "The phone essentially goes into a high-power mode throwing away any attempt to be efficient; it’s a nonsensical mode that is unusable in every-day use-cases beyond getting high benchmark scores"

What makes it "nonsensical" ? Are they exceeding the TDP of the phone? If so, then couldn't benchmarks just be made much longer, so that the phones would throttle or fail if they were cheating?

Or is it just about battery life? If so, why is it unreasonable to have a phone plugged in to get great gaming performance?

View on HN

The problem is basically a variant of Goodhart's law: "When a measure becomes a target, it ceases to be a good measure."

Publications want to use these benchmarks as a convenient proxy for real-world performance. Phones have "benchmark detection" and run their power management with different settings when they detect that a benchmark is running, defeating the purpose of running the benchmark.

BTW, I don't think this is about gaming vs. non-gaming workloads (there are different benchmarks for those), or even running plugged-in vs. on battery. My understanding is that some of these "benchmark modes" are so aggressive that the phones would become too hot to hold in your hands. No sane phone manufacturer could use settings like these for real-world usage.

Reviewers should simply say, "we've adjusted these numbers to account for benchmark manipulation", and give those phones worse reviews.

It sounds like it's likely to overheat if you're gaming with it. That's a pretty big problem.

And while making benchmarks longer helps, it doesn't tell you if that phone's new mode would have huge problems in a slightly warmer environment, or a case, and it only works on full-out gaming benchmarks. You could easily still get misleading results for something that tries to be a mixed user workload.

To your last point, I find it completely reasonable and something I configure for all mobile games I have on my phone. My Samsung Note 8 isn't able to handle smooth gameplay for many applications without tweaking, which I have configured through Samsung's Game Tuner app which does warn about potential overheating concerns. There are actually phone cases made specifically to help with phone cooling these days; tangent, but this reminds me of how CPU cooling gets more advanced as high performance CPUs generate more heat under heavy workloads.

>If so, then couldn't benchmarks just be made much longer, so that the phones would throttle or fail if they were cheating?

This. I think Geekbench needs an additional mode running it at 10min plus instead of the current 2min max.

> why is it unreasonable to have a phone plugged in to get great gaming performance

People don't always game near power outlets.

There's a difference between "don't always" and "unreasonable." It seems like if you can get a phone that performs as well as something that costs 2x as much, and the compromise is gaming near an outlet, or carrying an external battery, that's a choice some people would like to have.

And if it is the battery that's "unreasonable', then I wonder why the review sites don't just factor the cheating into their review by having a category like "battery life running PCMark" or something where they measure how long the phone will run the benchmark.

Customers are not informed, they just use the number - if you game the number you are unethical even if you pretend like its for consumer choice and enhanced information - no matter what that's not how consumers use it and you are juicing a number to get additional sales.

Reply on news.ycombinator.com