Hacker News Reader

Comment by k__ | Hacker News Reader

How long will it take to forget about this again?

The whole story reminds me of something a project manager, who worked at the Google homepage, once said.

They worked their asses off to get the page smaller and smaller so the UX was unbeatable. Then, after they got enough users, they focused on different things, and more and more stuff was included.

Somehow Firefox went the same way, right?

First they created it so it would be small, fast and modular, then it got bloated to the max and now they set it on a diet again.

Don't get me wrong, I prefer Mozilla to Google, and I like that they even explored experimental stuff like Rust and Servo to get on the right track again.

But for how long?

View on HN

Performance is usually death by a thousand cuts, and (imo) the best way of mitigating this is by standardizing some measurements and making that automated. Block changes that move performance past this bar, make sure the organization buys off on this approach. Any changes to a critical area that impact performance need to be zero-sum - if you add something, you need to take something away.

The cost of doing this in a large organization seem high though. Are there good ways to solve problems like this:

- A newbie developer adds small feature, triggering some performance test that was already close to failing. Now newbie has to learn all about the performance of this large project before they can land their first change.

- The performance test has a reasonable amount of noise. At first, 10% of test runs fail it, and people just rerun them. Eventually 90% of test runs fail, and people need to start dealing with it, but the effort required to get it all the way back down to 10% is unrealistic for any one person.

- The product is something like Microsoft Excel, that has a strong guarantee of backwards compatibility across a wide variety of hardware and OS versions that are difficult to test. Asking developers to delete code from parts of the product they're not experts in, is likely to break stuff.

> The performance test has a reasonable amount of noise. At first, 10% of test runs fail it, and people just rerun them. Eventually 90% of test runs fail, and people need to start dealing with it, but the effort required to get it all the way back down to 10% is unrealistic for any one person.

This is a tricky thing to deal with - (imo, again) how you deal with this is twofold:

1.) The automated test that measures performance needs to be rock solid - whatever you measure needs to be deterministic. Wall-clock time isn't always the way to go (although it's the easiest), maybe you profile and measure cpu cycles consumed by your application.

2.) If you capture performance over time then hopefully you can capture a trend-line that shows performance degrading from other changes that push the app past the performance bar. If your dev loop is fast enough you can isolate performance degradations to a small set of changes. Ideally if you handle #1 above this won't be much of an issue. In my experience changes that tank performance usually do so beyond the threshold of noise.

Regarding "newbie developer needs to learn all about performance" and "asking developers to delete code", that's why you need organizational buy-off. If your changes are regressing performance in a core area that is complex with a lot of legacy, maybe you shouldn't be making changes there, or the org needs to be aware of and sign off on letting your change go through. It doesn't always need to be a hard and fast "reject all changes", but there needs to be a discussion.

I'm not saying they can't do this.

They are all real smart people.

I just have the fear they will forget about it in a few years again.

The article mentions they're adding automatic tracking of performance to their tests.

> we’re adding lots of new tracking, including improvements to CI automation running performance tests, telemetry to track what users experience, and regression management inside of bugs.

I'm not saying this is a technical problem.

I'm saying, what prohibits them in the future from stopping to care again?

It's not all them -- I think the saying that "Moore given and Gates taketh away probably translates across to browsers and web pages, too. Browser speed increases to render HTML and run JS faster, people worry less about performance and write slower pages because browsers are faster now, rinse and repeat. This applies even more so to served ad code, where there are no incentives to perform well.

Reply on news.ycombinator.com