Comment by imauld | Hacker News Reader

Comment by imauld | Hacker News Reader

Parent Full thread

> If you try to compete with leading webapps you quickly realize the extent to which collecting all that data actually helps them build better products.

I have an extremely difficult time believing this.

Having worked on platforms that use data from millions of users and probably billions of interactions to create features that no one uses. It would seem that even with all of the data that Amazon has been collecting from all of us, the best they can do is show you ads for the exact thing you just bought.

Windows was running in nearly every home long before they were collecting data about everything you did (some people would probably argue Windows has gotten worse since then). Most of my adult life online has been collected and cataloged and machine learned and statistically regressed by now and wielding all this data ads companies have managed to show me ads I've clicked on almost dozens of times.

Sure having more data can be helpful but I don't think it's a requirement. I also think that people can sometimes fool themselves in to thinking that without collecting every scrap from each user and without millions of users to collect from that we can't build great software.

I used to work on Google Search. I now work on my own startup, which uses ElasticSearch. In many cases, they use the exact same algorithms and data structures, or can easily be fine-tuned to be equivalent. Nevertheless, stock ElasticSearch results are much, much worse, because you start with zero data. To say nothing of not having Suggest, Refinements, universal results, good spell-correction, disambiguation, etc.

I also use DDG when I'm in Tor Browser, and it always strikes me how much worse the results are when I try to do technical searches there.

I've still got my music collection in MP3 form on my hard disk, and it's only an iTunes click away. Nevertheless, when I listen to music, it's all on YouTube now, because YouTube can actually recommend stuff that a.) I like b.) is of a similar genre and c.) I haven't heard before.

I rely on GMail's spam filtering for E-mail to be useful; when I've tried to go back to Yahoo Mail or (heaven forbid) running my own server, the result is just unusable.

You're welcome to try founding a startup that operates as a desktop app and collects zero data from users. With improvements in developer tools and payment systems, most of the ideas from the late 1990s that required millions in venture capital financing could be done by a single motivated dev in a few months. Good luck getting users to adopt your software, though, regardless of how privacy-conscious they say they are.

Search and recommendation services are definitely an example of things that do actually benefit greatly from more data. If you're building something like that then you will almost definitely benefit form more data. However, it might still be possible to generate music recommendations for a user based only on their usage patterns in their local music player. I'm not an expert in that area but I imagine it could be done.

> Good luck getting users to adopt your software, though, regardless of how privacy-conscious they say they are.

It would probably be difficult mostly because we have trained users that an app isn't useful unless you can sync it to all your devices. We are always going on about how much better things are once you can connect them to everything else on the planet. We've sold users on the idea that they need to be connected all them time and we need to know everything that they do so we can "improve their experience" which for most things means "show you ads."

I'm not knocking webapps in general. I make them all the time. Some of my favorite things are webapps. I'm writing this comment on a webapp. I just also believe that not everything needs to be a webapp and have a constant connection to some server telling it about everything I'm doing.

Reply on news.ycombinator.com