Comment by Groxx | Hacker News Reader

Comment by Groxx | Hacker News Reader

Parent Full thread

Editing this to make it clear: the article proposes local collection that could later be submitted.

This is the right path! Lots of manual submission processes can use this to improve data and streamline stuff, and keeping it local solves nearly all privacy concerns.

But nothing manual will ever get good submission rates unless it's zero effort, because that effort stands directly in the path of people's actual goals when they hit an error. It's the worst possible time to ask for something.

---

>People saw ICEs on nightly and didn’t report them. Maybe they thought they were already fixed, maybe they didn’t know how to report them. Whatever the reason, nightly and beta failed to be an effective canary.

Unless you make it trivial, you're asking for a lot from people.

I can't speak to Rust in particular[1], but it's VERY common for "please report this issue! [link]" to be met with a login wall, long list of requirements and metadata requested, asks for detailed lists of what you did and how it's relevant and what you expected and...

yeah, of course people don't usually report these issues.

They rightfully realize it's asking for a major time investment both right now (presumably they were trying to do a thing and would prefer to continue doing the thing rather than learning your reporting system) and in the future (questions for further info on bug reports are common).

Make it as easy as the telemetry you want to add, and it sometimes actually happens. i.e. no login, as near to zero effort as is possible, gather all information for them, anonymize it for them, etc. If you can do it automatically, you have trivially proven that you can also do it manually, so do that.

Give people a pre-filled link with a one-click submit and an optional "contact me at [email]" field.

---

[1]: https://github.com/rust-lang/rust/issues/93701 implies they do this too. They made or already had a github account, have run multiple commands to gather info, filled in a templated text field by hand, and then they're asked to make a minimal reproduction and check if it still exists in nightly? Is it any surprise at all that people do not usually do this?

I do want to make this as easy as possible. As part of that effort I am experimenting[1] with making filing an ICE report be as easy as "open ticket and attach this file". We do require a GitHub login because that is where we keep our issue tracker, and given the user base size I would have to expect that any alternative service would require some kind of login to cut down on spam.

[1]: https://github.com/rust-lang/rust/pull/114128

Frankly I think GitHub Issues is possibly the biggest barrier, and not just in UX terms, and it astonishes me that it's often the only (visible) option for so many major projects. I've watched multiple coworkers run across issues in other languages and systems (we don't use Rust, sadly), and when they get to the GitHub report they stop cold.

Submitting it there means making it literally public, and relatively highly visible. Can they do that with [internal software X]? Do they need to hide their internal username, which is all over the report? Are they publicly acting in the company's name by sticking it in a world-visible-and-google-indexed location with their work GitHub account? A good number don't even have a separate personal account - GitHub is big, but extremely far from ubiquitous.

These are not hypothetical examples either, I've had newer engs ask me about ^ multiple of those. I've seen more-senior-than-me people asking our legal department similar questions before clicking submit, and mostly getting non-answers two days later. I see almost every case including manual scrubbing of dozens or hundreds of lines of output to try to avoid these questions. With the increasing public pile-ons in GitHub lately, this is all likely to worsen.

All of that is coming from highly motivated people trying to improve things for themselves and possibly dozens or hundreds of others (else they just try to work around the crash). And they frequently stop.

Meanwhile everyone just clicks "report and clear" in Intellij error reports.

I wonder if splitting the telemetry upload into a separate command could make sense. So rustc could panic and say “Run 'rustup report' to report this issue”, and the user could send the report at their option. (Or ‘rustup report 12345' where 12345 identifies this panic.)

This would be the ultimate opt-in - the user has to specifically request each report.

Honestly this sounds pretty good to me. Unlikely to be accidentally triggered, easy to explain and run, lets you do it later rather than having to save things before continuing, and provides a few ways to preview the data before submitting (e.g. temporarily run a webserver to locally view and possibly clean up the report, which can easily include large files, and just click submit there to send it with CORS... or bundle it into a file that can be dropped somewhere if that doesn't actually work).

Why require them to open a ticket, though? Just allow an easy way for people to upload the file and be done with it. You should take care of whatever bookkeeping your process requires yourself.

I have no github account, for instance, and there's no chance that I'd create one just to provide this data.

< "We do require a GitHub login because that is where we keep our issue tracker"

The problem with this is that you are de-anonymizing the user when you ask for this and the user may decide that proceeding is not worth it. Also, they may find that logging in is a pain. Think of those companies who ask you to login before you can unsubscribe from their news letter. It is unpleasant for the user who is already upset.

I think that the GP made some really good points and may well have adequately explained the following point you made. Re-quoting: < "Whatever the reason, nightly and beta failed to be an effective canary." I think its worth the effort to figure out the reason why nightly is not an effective canary and if it's as simple as the user requiring a github login and not wanting to go that far then that should be fairly simple to solve.

I have a suggestion that may or may not be feasible. Logins are one way to cut down on spam but not the only way. You don't have to have users directly posting to your issue tracker, you could use a website. The concerns you have about telemetry are absolutely on point so this service could simply be an anonymous dumping ground for copy-paste ICE compiler output that anyone, including bots could use. No automation from the user's machine, no telemetry, no metrics. The service would be responsible for despamming / deduplicating the submissions and posting to the github issue tracker using an account not associated with the original poster.

Anyway, you guys do superb work and really think that articles like this are important. Thanks for posting it.

‘Whatever the reason, nightly and beta failed to be an effective canary’ - Im also curious about this? It is typical to try to move quality ‘left’ - which would imply understanding this issue and fixing it. But the focus seems to be on adding more logging - which will by definition have less impact on the quality of public releases than improving the quality of beta/canary releases will.

The Element Web Matrix client does that. I'm convinced that we're losing 99% of bug reports by making it so complicated.

That being said, the Rust crowd is very different, so it's definitely worth a try!

I've never had to log in to send a Firefox or Thunderbird crash report to Mozilla. A look at what they're doing might be fruitful.

> as easy as "open ticket and attach this file"

Does the "open a ticket" step expand into "create an issue and then fill out an elaborate questionnaire"? For instance, https://github.com/rust-lang/rust/blob/master/.github/ISSUE_... isn't... bad for what it is, but it's definitely asking for enough work that it falls back into "yeah no maybe someday when I'm not trying to get my code to work" territory. (Asking to clarify; I think you might be suggesting to reduce to the form just being "got a ICE? paste your error here and hit submit" which would be good)

The only mandatory fields in that form are the rustc version (that's printed as part of the error) and the backtrace. The file being generated includes both of those things.

> Asking to clarify; I think you might be suggesting to reduce to the form just being "got a ICE? paste your error here and hit submit" which would be good

I would do that once the new ICE storage mechanism is exercised enough for everyone to be confident in its behavior and enabled in the stable channel.

> it's VERY common for "please report this issue! [link]" to be met with a login wall

Which unintentionally, but strongly, communicates that the devs don't actually want the telemetry data.

What privacy concerns? What the hell are they putting in the telemetry that the user should be afraid of being sent over an encrypted connection? Is my bank account number being included in the compiler metrics?

At this point I consider any advocate of "privacy" in a tech context to be an anti-intellectual cargo-cultist. Just look at the linked article. Absolutely not one single example of a privacy violation or its implications. They just waive their hands in the air and say "it's contentious" and "risk is too great". Let's completely avoid an actual discussion of privacy, and just assume that any information about a user (like their IP address) will immediately result in them being sent to a gulag, despite no evidence that this has ever happened, much less how often, how likely, ways to avoid it, or the benefits that are lost by not allowing for said insanely-unlikely-risk. It's people afraid of their own shadow that would rather live in a prepper shelter than deal with the real world.

Of course we should completely ignore the fact that this is developer software, and they, being technical people, have many means to disable specific outgoing network connections if they so choose. But better to be paranoid than rational, because "trust is key".

>What privacy concerns?

The core thing to understand here is that what constitutes their concerns is not your decision. It's theirs.

You don't know what they put in their code. Maybe they did put their bank account in a function name. The tool won't tell them they shouldn't have done that, and there's currently no reason why they shouldn't if they're just playing around.

Similarly, you don't know what they're doing or how doing that might expose them in ways they might care about. If my country decides to outlaw something and starts looking for mentions of it and jailing people, your decision yesterday could endanger me. This happens with some regularity, and you can't reliably predict the topics that get targeted.

I generally approve telemetry for tools because I know how useful it is. But I see no reason why that should be your choice instead of mine.

> Maybe they did put their bank account in a function name.

Thank you for proving my point - again, no rational debate here. Obviously nobody would ever make a function name their bank account number. And why would metrics include function names? Metrics are numbers, not logs.

> Similarly, you don't know what they're doing or how doing that might expose them in ways they might care about. If my country decides to outlaw something and starts looking for mentions of it and jailing people, your decision yesterday could endanger me. This happens with some regularity, and you can't reliably predict the topics that get targeted.

Yet another contrived example that has nothing to do with the subject at hand and has no rational basis for happening. A random country creates a random law that happens to outlaw a compiler metric? You think a country's gonna make software development illegal soon?

> I generally approve telemetry for tools because I know how useful it is. But I see no reason why that should be your choice instead of mine.

Nobody's taking away your choices! You don't want telemetry? Block the outgoing connections! Disable it in a config file! Use a friggin' VPN like all the other privacy-paranoid do!

"Telemetry" frequently includes more than low-cardinality metrics. This article (and my root comment) mentions crash information, for example, which could easily include function names and other user information (username is often part of file paths, which are often part of crash information).

And the example is really not contrived. For example, homosexuality has waffled between "get married, love is love" and "your government is explicitly trying to kill you" repeatedly around the world. Congratulations! You worked on Grindr two years ago when they started their internet dragnet, as they can see by information coming from your compiler, on your machine, at your home! You now have to flee the country.

They're not going to be looking for your compiler crash logs, they're looking at everything they can possibly collect for keywords, and oopsie, there's one.

You (you personally and the "royal" you) cannot predict what will matter in the future, and your legitimate-threats do not match everyone else's legitimate-threats. I agree privacy-extremists exist, but so do other kinds of extremists and extreme situations. Ignorance of those doesn't mean they don't exist.

Crash reports are not telemetry, they are debugging/error reporting tools. Every application that has ever had a crash reporting function, did so by prompting the user to submit it first. The reason is pretty obvious: crash reports contain more data than a user might like to submit, and so the user gets to choose to submit it or not. Often they do choose to submit it, but again, they get the choice, and don't need to go through an awkward process to manually copy their offline data into an online process. Nobody has stopped using operating systems and browsers just because they have a crash reporter that can optionally submit the crash report to an internet site.

Telemetry is not sensitive data. Telemetry is "signals". For desktop software, telemetry is often what operating system, cpu architecture, and software version you're running. Your OS reports that, your browser reports that, every self-updating application you use reports that. This is not controversial nor sensitive data and has no privacy implication.

For crash reports the only elements included are coming from the compiler (the symbols of the stack trace do not leak user data), but the panic description can leak symbols from the user's code. For automated mechanisms you could make it so that nothing user generated is sent, but a lot of consideration is needed to not make a mistake.

Some apps collect full memory dumps - on Windows at least there is nothing stopping them. That will indeed contain your bank details.

We have had decades of companies demanding, sneaking, and exfiltrating data from us regardless of our wishes. Companies that put a great deal of effort even into bypassing various methods users have made to put a stop to this.

Collection of any data about me, my machines, or my use of my machines without my informed consent is spying. How sensitive the data actually is is beside the point.

In any case, so much good will has been burned now that it's not unreasonable to be very sensitive about any data collection whatsoever. People being reluctant to share isn't a case of people being unreasonable, it's an understandable response to so many years of abuse and violation of trust.

> What privacy concerns?

Most HN people are anti-telemetry and tracking. I've never actually seen these privacy concerns being fleshed out with specific details of how logs could be abused but it really doesn't matter if a real mechanism for abuse exists. What matters is people fear that it may exist and stop using the tool as a result.

I suggest taking a look at the thread suggesting telemetry in the Go toolchain (https://news.ycombinator.com/item?id=34707583). You can judge for yourself if there's legitimate concerns being surfaced, but there's no doubt that there's significant opposition to such the proposal.

Rust is getting ahead of this discussion by making it clear that their log collection won't phone home and it won't be automated.

Sometimes the subject of a person's concerns exists beyond their own ego. This is perfectly normal and surprisingly common behaviour.

Reply on news.ycombinator.com