Hacker News Reader

> Now sure, you can complain that the algo isn’t showing you what you personally want - its trained on the entire viewer population, not on your personal tastes. So most algos let you personalize the feed. But without an algo…there’s just no hope.

I think that the common complain is not "it isn't showing what I personally want" VS "it isn't showing what people generally want" but rather that the algorithm is often not tuned to "what X want" at all, but tuned to "what brings company most money". For example, YouTube recommendations are biased towards channels that have monetization activated, as YouTube earns more money that way, while ignoring channels the viewer (at large, or personally) would enjoy more.

Most of the comments here are incredibly dystopian - automation is bad, rehire the editor you've replaced at 75% of the salary, stop pushing engagement, what brings the company most money, megacorporation pushing slushy mass....

Jesus!! Man, didn't know recsys invokes such vitriol.

There are dozens of news apps on smart TVs these days. Under the covers, it's pretty much the same deal - in go the raw feeds. Editors always do a first pass. Recsys algos do the 2nd pass. Ranking algos come in next. Personalization is usually last. Out comes the news. Viewers then have their say with thumbs up & downvotes. Those get fed back into personalization. On & on it goes...

Its not magic. Its not perfect. But its how pretty much every smart tv news app works under the hood. There's dozens of them. Pick any one of them that works well on your TV for your temperament, I guess. Recsys is just a standard comp science discipline. We have our textbooks, our journals, our papers. We all read the same stuff, pretty much roll the same code with minor frills.

Nobody's out to get you.

I don't have to think someone is "out to get me" to believe that the current state of affairs is deeply broken and that you can't fix it with a better mouse trap.

This is just a really easy way for you to dismiss any criticism and avoid cognitive dissonance over your work. I believe you have good intentions. I also believe that those "dozens of news apps" all burn a whole bunch of watt hours on producing nearly identical results and are largely just facades over a handful of media companies.

And that's not speculation or a conspiracy theory, it's plainly true. Media conglomeration has happened at an incredibly fast pace over the last 40 years and local media has been decimated. No one has to be a cackling evildoer with a fancy mustache and a british accent to make this happen, they just have to keep grinding harder and harder at inventing new ways to 'fix it' with more conglomeration.

“…what brings the company most money, megacorporation pushing slushy mass.... Jesus!! Man, didn't know recsys invokes such vitriol.”

yes, corporations do whatever makes the most money. pointing that out has nothing to do with emotions. welcome to the real world.

It's also "showing me stuff I may want when I least want to be aware of it". Stop the distractions already.

If anything algorithms that drive engagement are mutually exclusive with what people want to see. Most people don’t usually engage with anything but the odds are a lot higher for someone to reply etc out of anger or disgust than anything else.

Honestly, there's nothing inherently wrong with providing a limited and selective set of information to people. The problem is the tendency towards broader and broader aggregation.

Once upon a time, people got their news from approximately 3 "kinds" of places: - newspapers, locally managed, locally printed, locally distributed - radio, somewhere between locally and nationally managed, recorded, broadly distributed - television, probably regionally or nationally managed, maybe locally recorded, also broadly distributed.

And for these things they had between several and tens of choices to select in to their preferences, interests, and alignments. They got to self-select the 'algorithm' they viewed information through. There was obviously an algorithm here, and in spite of golden-age glasses it was a racist, sexist, deeply problematic algorithm, but it was at least chosen by people in your community.

Now though? Everything gets sucked up into some megacorporation's orbit through a rolling katamari of companies. We have facebook and twitter and things that will eventually be bought by facebook or twitter and approximately 2-3 companies that own every newspaper, radio station, and television station on the planet.

There's no choice here anymore. Your algorithm is chosen by someone else, and if by some chance an upstart comes along to break out of that monotony it's inevitable that they'll be sucked up into the katamari eventually for the sin of being successful at bucking the trend, and then they will be just like everything else within a year.

So you can do your best to provide a fresh, reasonable algorithm that does some good and helps people find something. But in the end, no matter how hard you try, it's gonna end up just like everything else and be a slushy mass of stuff that makes people angry to get money out of them.

> How else would you do it ? There are literally 100s of 1000s of newsclips per day!

No offence, but honestly, that scale doesn't require automation, and probably suffers from it. I'd rehire the full time editor (singular?) that you replaced, probably at 50-75% your salary.

I’m not sure what kind of news app you work on, so maybe this doesn’t fit, but what this makes me think is, you’ve optimized for putting the “best” stuff in limited real estate, but why does there need to be news headlines algorithmically ranked and dumped in front of me in a place that I’m not looking for them (e.g. the “home” screen of my smart TV)? If it’s an app where you are consciously seeking news headlines, maybe there’s a good case for collapsing the raw deluge of news into a recommended front page, but I still feel like we’re losing something.

Recommendations are never good, and always corrupted by what the business wants to push on me, but even if they were good, the hyper personalized world is just weird. I’ll never be able to ask someone, “did you see the paper today?” Instead I wonder if they’ve been pushed the same stuff that I have and if we live in the same universe.

> So most algos let you personalize the feed. But without an algo…there’s just no hope.

but there's an algo which is simple to implement and has very little biases: just show the things in temporal order ? Anything else is inferior to me. I don't want to see what's best for others, I don't want to see what some algorithm thinks is best for me, I just want to see what's last, with zero additional "reflection" on this.

"Temporal order" is another way of saying "prioritize accounts that post most frequently, while suppressing less-frequent posters." That obviously rewards e.g. unnaturally breaking up content into multiple parts, content-light posting, and other consumer-hostile behaviors. I'm not sure why so many seem to think this is obviously optimal for consumers.

"I'll unfollow the accounts that behave that way" doesn't work because a temporally-ordered system incentivizes all accounts to behave this way. An account that posts highly valuable content, but only posts once weekly, will not thrive in a temporal-only medium.

This might be true for social media but for a news app, they control their own pace.

This viewpoint underestimates a single person's ability to consume the output of a news organization. There are orders of magnitude difference between the number of stories that can appear in one's newsfeed and the number of stories produced bye a news organization in a day.

One could roughly estimate that by the time a news organization grows to need a full floor of office space, its pace of production will outstrip most readers' ability to keep up and not miss important stories. A temporal feed makes "keeping up" important, as relevant stories will quickly drop off the feed due to newer stories having been published more recently. The reverse is also true: a curated feed allows readers to step away and not worry that they will miss something important.

The pacing of news stories is dictated by a combination of the number of salient events occurring as well as the number of staff.

To use a concrete example, the (already-shortened) Yahoo! Finance News for Apple has 11 stories published today. There are thousands of public companies, and relevant news is published every day about many of them. A temporal feed of company news on the Yahoo! Finance homepage would therefore not be very useful. The pacing is essentially outside the control of Yahoo!, since it is driven by events at thousands of companies.

> just show the things in temporal order ? Anything else is inferior to me.

heh heh. In the last 5 minutes, since you typed in your comment, there’s been 100 news clips. Literally. So just show the top 5 after reverse sorting by time….newest to oldest. Temporal order. Now what would be the problem with that ? Almost always, your newsfeed will end up dominated by the latest weather clip where the guy says “and now the temperature is a sunny 82 degrees”…meanwhile 100s of people die in ukraine but you didn’t see that because it was like an hour ago, and 1 minute ago >>> 1 hour ago.

If you also allow the user to filter results with a search query then this shouldn't be a problem. If he wants to see Ukraine news he will search for that. If there are too many results, the query can always be refined by the user.

> what you personally want - its trained on the entire viewer population, not on your personal tastes. So most algos let you personalize the feed. But without an algo…there’s just no hope.

This is a problem, but from a different point of view. The algorithm is trained not to give me news but to keep me engaged. That should not be the purpose of a news app.

> So most algos let you personalize the feed.

This is where the failure is. Most algos I've experienced show me more of what I already know. Or they show me what I was interested in the past (and I'm done with that).

I want an algo that does the opposite - show me only things completely different - see if maybe I like them.

I'm bored, I don't want more of the same, and I want something new. And there is where every one I've tried fails.