Hacker News Reader

Comment by ad404b8a372f2b9 | Hacker News Reader

I don't think that data gave them anything more than what testing on a few consumer computers in different price ranges would have.

(Edited, original comment read: "What more information does that give them than just buying a few computers at different price points?")

View on HN

A vastly larger variety of different computers and computer setups than they could have come up with on their own.

(Sorry I edited my comment before I saw your reply.)

That doesn't seem very useful for the metrics shown in that article. For hard to find bugs sure, for 95th percentile calculations and so on you can just buy a few computers at a retail store and get the same information.

New computers don't behave like old computers, and it's not worth trying to guess why that might be. Could be anything running in the background, old NAND, old battery, low disk space, satellite internet…

Once you do have a model of badness I agree it's better to try to set that up yourself.

That can get you 95th percentile calculations for brand new computers that you bought from the store in 2023 that are running Firefox alone, but that doesn't help you understand what your performance will look like when you're running on a 10-year-old machine running Windows 7 while the user is also running Microsoft Word, Excel, and Outlook at the same time. Your P95 numbers aren't especially meaningful if you've only tested ~10 different PC configurations.

Maybe you get the same result, but with the real user data, you can confidently say the performance has been improved without an disclaimer saying the data was collected in-house.

Reply on news.ycombinator.com