Hacker News Reader

Comment by simonsarris | Hacker News Reader

This frustrates me too. I've run into a laundry list of Browser (mostly canvas) bugs over the years, 95% in Chrome (my main browser) and 5% in Firefox.

Typically they'd be ignored, fixed several months later if I'm lucky[1], and still ignored as they get closed by an automated bot[2]. Chrome specifically created an "icebox" to automatically close old bugs.

I'm convinced that the Chrome team at least has an internal bug-tracker they use and the external one (crbug.com) serves the same purpose as fake thermostats on the walls of office buildings.

(To Chrome's credit, they took the only security bug I filed very seriously and fixed it in a couple days.)

[1] Unlucky: Still an issue with broken translating gradients in non-hardware-accelerated canvas: https://bugzilla.mozilla.org/show_bug.cgi?id=774387

[2] This bug was ignored, fixed, and for a year I asked for it to be closed. It was finally closed by a street-sweeping bot: https://code.google.com/p/chromium/issues/detail?id=62373

View on HN

I worked on Chrome for most of its life. The internal bug tracker was closed when the product went public (we manually migrated some bugs we were actively working on, and the rest got comments of the form "if you really want this fixed, we're sorry to say that you need to refile it on the public tracker"). The real truth is sadder: many bugs are lost, much like yours.

In the old days I (and a few friends) personally reviewed every bug marked "Linux" and we attempted to triage them, but at one point I realized bugs were filed faster than we could classify or fix them. That isn't saying new bugs were being introduced rapidly, but rather that there's a lot of random spam ("my site doesn't work" / "does it work in other browsers?" / "no") and not-yet-implemented feature requests that appear to users as bugs.

The latest bug on the public tracker is bug number 174163. Chrome was released in late 2008, so let's round up and say the tracker's been up for five years. That's averaging nearly 35k bugs a year, or 95 a day, and in reality the rate was slower in the early years so it's much faster than that now. Even just reading each bug and understanding whether it's reporting an actual issue takes hours.

I have come to believe that a publicly writable bug tracker is a bad idea. Something like what SQLite does (where once users must first establish via some other mechanism they know how to file useful bugs -- yours look good at a glance, but I'd recommend attaching the demo html to the tracker so the link doesn't eventually 404) likely makes a lot of people mad but at least they're not shouting into a vacuum.

Recently (like in the last few months) I heard they've allocated more resources to looking at the tracker, so maybe things will improve. It's honestly just a hard problem.

> It's honestly just a hard problem.

We really need a way to reset those bug report counters to zero in one fell swoop.

The solution, of course, is planned obsolescence.

Bonus: http://xkcd.com/844/

This makes me think there's a market for a better bug tracker - maybe one where people submitting bugs have karma, and you can sort bugs by that.

It's a tough problem. You really want to be able to take advantage of the tail of contributors -- the people who use one particular product, use it a lot, but have never filed bug reports for other products. I do think a stackoverflow-type solution would be worth a shot -- if you post a good bug report, it will be hopefully easy for people to understand or replicate and will get voted up, without requiring previous posts on your part. That's if you have a "crowd" actually willing to look at these things (might work for Chrome, not sure about small projects).

Maybe the starring system can be this?

You often feel that bug reporting just goes to dev/null

I like how Ubuntu handles this - each bug has a "Bug heat" number, which is determined by how many people say the bug affects them and by other criteria.

> The latest bug on the public tracker is bug number 174163. Chrome was released in late 2008, so let's round up and say the tracker's been up for five years. That's averaging nearly 35k bugs a year, or 95 a day, and in reality the rate was slower in the early years so it's much faster than that now.

That honestly doesn't sound too scary. I did QA for Opera for a few years, and triaging (for specific platforms, at least) is remarkably quick after a while. A glance and a quick test on two platforms should let you toss a bug in either of four bins:

a) WFM/invalid/bullshit

b) Duplicate (it's scary how many bug id's you remember after a while)

c) Likely a core/cross-platform problem (somebody else's problem)

d) Worthy of further investigation

Provided that you have some sort of bucket in which to toss the cross-platform/core bugs, only the bugs in category d should take longer than a couple of minutes each to triage. Some would argue that bugs in category d are already triaged at that point and should be put in some ANALYZE bucket.

I think the real problem is that any talented ambitious employee that spends enough 40-hour weeks to get good at bug triaging is going to want to get the hell out of there as soon as possible. The work is close to data entry in its rewardishness.

That's definitely true. It's important to spread the workload over an optimal number of employees so they do enough triaging to stay in the loop, but not enough to get sick and tired of it. A delicate balance. It's an important chore, though.

I love doing triage. It's one of the best times to learn more about how your customers actually use the product.

Apply at Google for the Chrome team :-) We'd be happy to have you do triage

The reason it takes hours to verify whether an issue is even an issue is because people that consume tech describe problems like this "My shit just broke, help!" while people that build tech need something like "I was logged in as an admin user on the dashboard interface using Firefox v.18 and I clicked on the submit button and validation didn’t work".

Most bug trackers worry about how best to show you a list of issues, the thing is if most of those issues aren’t reported correctly they’re useless.

We’re currently trying to resolve this problem for web devs specifically by creating an embedded tracker, where you integrate the bug reporting directly into a site or web app. Meaning people can report issues in the way that’s natural for them e.g. "this is broken" and the tracker handles capturing all the other data a dev needs to verify and fix the problem. Our app can be found at http://www.bugherd.com if it’s of any interest to anyone.

A big part of the problem is social, not technical. A bug report is essentially a set of instructions for a person who knows how to operate the software, but not necessarily anything else about the user's situation to follow. Most people are not used to writing instructions without conversational feedback and have a hard time figuring out what information in relevant and what to leave out.

For a high-volume end-user project like Chrome or Firefox, a wizard/flowchart type UI probably ought to be the default, with a more streamlined UI available to people who have been approved by admins or met some criteria programmed in to the system. An example of the process a user might go through is:

What's the problem? -> A web page doesn't look right in Chrome -> Have you tried it in another browser? -> No -> What other browsers do you have? -> None -> Ok. Here's a list of browsers that run on your OS and where to get them -> Firefox -> Ok, does it look right in Firefox? -> Yes -> Do you know how to take a screenshot?

And so on until the user has likely provided enough information for a good bug report. There could be short-circuits in the process - for example, when asking if the user has tried another browser, a small link a bit outside the main UI could say "I've tried another browser and I have screenshots from both".

I'm not sure that encouraging users to download another browser to try the page in would (a) encourage many bug reports (b) be a great idea for user retention.

"Does it work in other browsers" is a standard question I've seen in templates in bug databases for open source browsers for rendering issues. Encouraging many bug reports probably shouldn't be a goal for a project with as large a userbase as Firefox or Chrome; encouraging actionable bug reports should be.

I imagine by now someone at Google would have looked at the problem and written a basic bug clustering mechanism to group reports on the same issue. There's usually a very clear Pareto pattern when it comes to bugs, and fixing issues for 80% of users would be huge.

Sometimes adding humans is the best option? How about just hire some student workers to sweep through bug reports remotely? Even some Mechanical Turk approach might work, if you can create some checklist of what a useful bug report must have (and if you got this, it could also be integrated into the bug tracker workflow).

That's hard to do because you end up with situations where the bug submitter figures out s/he knows more about the product than the person on the other end of the line, and stops responding out of frustration.

So? Just close those reports then. I thought the issue here is to filter out anything spammy while not loosing any potentially valuable reports. A human should be able to filter out at least 90% of the spammy stuff while not loosing any good ones, thus significantly reducing the workload of engineers in higher pay grades.

Google sometimes give me the impression they want to absolutely only hire the best engineers, but I'd argue that many non automatable tasks can be done more efficiently by having some reasonably trained lower rank staff. See also all the complaints about missing service when trying to use Google products commercially - if they do want to seriously enter that market, this stuff needs to change significantly.

Other things that come to mind: Automated Youtube takedowns and Gmail account blocks. Do you really want to save those few million bucks a year for not hiring support staff, then have lots of disgruntled users blogging about how evil you are and then pouring those millions into PR campaigns?

" I thought the issue here is to filter out anything spammy while not loosing any potentially valuable reports. "

Yes! But, in my comment I pointed out a specific example of where you lose a potentially valuable report.

I think for such projects it would make sense to seek help from volunteers. People who don't necessarily contribute to the code base, but who could help organize the many bugs.

After all it's just a question of man power to remove duplicates, prioritize by commonness and make sure long-term bugs really get fixed. That latter has the advantage that the code base really gets improved.

In fact the Ubuntu project seems to do something similar eventhough it is IMHO not well enough organized: there is ubuntuforums.org, when I recently asked a question there, within minutes I got a response. The response was not helpful at all but still. The moderators were really doing a lot to keep the forum organized, and for instanced pulled me into a separate thread to not spam another bug. (Which was in their opinion a different issue.) Taking this a step further and bridging it with the actual Ubuntu bug tracker on launchpad would be awesome. Maybe with launchpad having only limited write permissions for the public.

This frustrates me too. I've run into a laundry list of Browser (mostly canvas) bugs over the years, 95% in Chrome (my main browser) and 5% in Firefox.

I think the thing that frustrates me the most isn't the bugs themselves. After all, in general for OSS I'm not paying anyone to make or fix things, and they owe me nothing.

What irritates me is when high-profile projects like Chrome and Firefox (coincidentally both browsers, but that's not my point) then promote themselves as serious, professional-standard projects that people can rely on. They and their evangelists take great pride in stealing market share from established closed source/commercial alternatives (IE in the browser case). Then normal people start using them, expecting a better experience. But then every time there's an update, we play the "will my software even still work?" lottery. (And this applies not just to the browsers, in that example, but also to all the web apps running on them, for which users might be paying real money.)

Unfortunately, on the kinds of geek-friendly forums where these movements tend to start before they expand to the general population, it's often sacrilege to criticise a favoured OSS project. You may get hostile replies, or simply voted into obscurity by whatever mechanism is in place, even if like jwz you take the time to cite specific problems or have provided detailed bug reports and test cases. (Doing the latter is absurdly difficult for many of the large projects anyway, but it gives them a convenient excuse to ignore problems.)

Again, it's completely up to these projects whether to promote helpful contributions or to be hostile. They're giving their stuff away, and owe the rest of us nothing. But it would be nice if there was a little truth in advertising. If an OSS project isn't going to take bug fixing or regression testing seriously, it shouldn't promote itself as professional standard software. (And if a project is created/maintained entirely by one amazingly generous person running the whole show but who has limited time to devote to the project and can't keep up with everything, it would be helpful if there was a prominent warning to that effect somewhere people considering that project would find it before they started relying on it unwisely. Perhaps that would even prompt more concrete support and/or contributions to help the project out.)

> What irritates me is when high-profile projects like Chrome and Firefox (...) then promote themselves as serious, professional-standard projects that people can rely on. They and their evangelists take great pride in stealing market share from established closed source/commercial alternatives (IE in the browser case).

I don't think that "closed source / commercial alternatives" (IE in the browser case) have done any better on the bug side than the open source ones.

One of the reason people use OSS is that -despite all the flaws that JWZ points out about the bugfixing process-, bugs have historically been fixed faster and the community support is (again, on the average and historically) more responsive.

And you can always fix the bug yourself.

Granted, web browsers are a special case, since you can't patch everyone's Chrome and Firefox and Opera, but for every other type of software it's another strong incentive for OSS.

I don't think that "closed source / commercial alternatives" (IE in the browser case) have done any better on the bug side than the open source ones.

So the received wisdom would have us believe. The thing is, my bug trackers objectively say different.

In the past 12 months, I have seen exactly one serious bug in IE (where I define "serious" to mean that user-visible functionality in at least one web app I've worked on does not work properly, even though the same functionality worked in earlier versions of the same browser and/or works in multiple other browsers). That one may have been IE's fault or it may have been a problem with a Java update, since the bad behaviour was at the interface between JS and Java and both had recently had updates when we started to see the problem.

In contrast, I have seen serious bugs in Chrome in every web app I work on, ranging from another Java integration issue through to messing up HTML5 video. Several of those bugs still haven't been fixed several update cycles later. Statistically, across all the projects I'm working on -- which since I'm doing various freelance work and running multiple businesses have little in common except for all being web apps -- it is clear that Chrome is by far the flakiest browser at the moment.

Firefox is actually faring slightly better lately, though as a Firefox user rather than a developer they still keep making usability screw-ups. I get that there have been a couple of critical security issues with Java recently, but since Firefox itself patches at least a handful of critical issues every release cycle I think the "Don't use this, run away, it's scary!" treatment is frankly rather childish. And as of version 18, Firefox frequently seems to corrupt the display of text areas like the one I'm typing this in.

I don't like to be negative about software since nothing's really perfect in this business, but I do point the finger squarely at the advocates of six-weekly releases on this one. The quality of Chrome has never been that great and it's always suffered from "UI rearranging itself at random" syndrome because of the silent updates. However, the quality of Firefox has nose-dived since the move to a rapid release cycle; I'd estimate we're seeing 4-5x as many minor issues in Firefox across all the projects I work on since that happened, plus a couple of howlers like the plug-in screw-up in the first so-called long term support release.

Could you point me to an example of Chrome "UI rearranging itself at random"?

The only time I've had issues with chrome, is when I switched to the dev stream... Stable has always been, well, stable.

I was speaking figuratively, referring to the way Chrome alters its UI during updates. Those updates happen in the background, so you can literally close your browser one day, and then open it the next to find something has changed.

In Chrome's case, the most obvious changes lately always seem to be adding junk like apps or a web store or logging in (logging in... to my browser... yes, really). I have no need of or interest in any of these things, yet they now clutter up the UI where once they did not.

> I was speaking figuratively, referring to the way Chrome alters its UI during updates. Those updates happen in the background, so you can literally close your browser one day, and then open it the next to find something has changed.

If you can't back up your argument with something substantial, then don't mention it. Automatic updates are an actual feature of Chrome and for a good reason. I wouldn't call updates "rearranging the UI at random". And the rest is also complete useless, because your "bug" essentially is that Chrome changes. Well duh!

You might not like logging in to a browser, and I can see the point. But that is not a bug in Chrome.

But that is not a bug in Chrome.

No, and I didn't say it was.

The bugs I was talking about are things like stopping playing a short HTML5 video before reaching the end, or failing to send basic event notifications to JavaScript before handing control off to the Java plug-in because the user interacted with an applet. Both of these are regressions in things that used to work, and they fundamentally break the behaviour of web apps using those features.

And of course, there have also been numerous rendering problems with everyday CSS in Chrome: banding in linear-gradients, inability to use fractional letter-spacing, I've lost count of how many attempts to get border-radius working properly, etc. These are what I would class as cosmetic issues rather than "serious" ones that actually break functionality like the ones I mentioned before, but they're still things where Chrome is coming in a fairly poor third place in quality of implementation today.

None of these is exactly a big secret or somehow overlooked in the Chrome and/or upstream bug trackers, as a quick Google search for any of the symptoms I've just mentioned will rapidly confirm. In fact, the letter-spacing one was first reported in WebKit Bugzilla more than four years ago and remains unfixed today.

Chromium has more bugs than devs to fix them all. There have been heated debates on chromium-dev about what to do about old bugs. Some devs think they should stay open till they are fixed, but given Chromium's development pace that doesn't necessarily make sense. So they close them, assuming if the bug is still relevant it will be reopened.

One way they prioritize bugs (I believe) is by the number of times the bug has been starred.

I do not believe they have separate bug databases. They do have the ability to mark bugs private, which afaik is only used for security issues.

This is a slightly worse version of the behavior that prompted JWZ's first rant about bugtrackers in open source software (cf. http://www.jwz.org/doc/cadt.html ).

Here's the most recent discussion on chromium-dev I can find about it - http://thread.gmane.org/gmane.comp.web.chromium.devel/34134

I've had quite good luck getting responses to my bug reports, but it requires active lobbying. You need to figure out whether it's a Chrome bug or a WebKit bug. If it's a WebKit bug, file a link in WK's Bugzilla and link to it in a crbug issue. Then, go on IRC and pester the person responsible for fixing that component. It's helpful if you can go through the source and propose where the bug might be.

Examples:

https://bugs.webkit.org/show_bug.cgi?id=32303 https://bugs.webkit.org/show_bug.cgi?id=35472

I hate to pester people who are already working on other things, but some bugs just need to be fixed at the browser level.

This is a good approach! Doing a little work to narrow down the scope of the bug and where it belongs can go a long way towards bringing it to the right developer's attention.

It also helps to attach a reduced test case and as much information as you can glean about the bug so that it can be reproduced.

And as you say, there is a point as well to go on IRC and ask. :D

Yes, reductions definitely help.

> serves the same purpose as fake thermostats on the walls of office buildings.

It's worth noting that many of those thermostats aren't "false", but just have their control disabled. That means they still provide current temperature information to determine if heating needs to be activated.

I've actually experienced the same problem with some government funded open source projects. Months go by without so much as a peep about the problem.

Reply on news.ycombinator.com