Ask HN: The WSJ paywall workaround no longer works. Why are their links allowed? | Hacker News Reader

Ask HN: The WSJ paywall workaround no longer works. Why are their links allowed?

208 pointsddevault96 comments

https://news.ycombinator.com/newsfaq.html

The official guidelines only permit paywalls with a viable workaround and the WSJ workarounds have all stopped working for many people, myself included. I'm getting pretty tired of clicking links and getting a headline + 12 words.

Before someone asks, I don't think the WSJ has earned a subscription from me. But that's beside the point - isn't it now in violation of HN's rules?

View on HN

A few points: 1) The workaround works for me if I open an incognito chrome window and copy/paste the "web" link from HN. It doesn't work w/o incognito.

2) If the article gets upvotes, then enough people have subscriptions (or workarounds) to warrant a discussion. Let the votes do their job.

If you really don't want to see WSJ articles, install/make a browser extension that does it for you.

> Let the votes do their job.

Voting doesn't work if people are disenfranchised. 20% of HN discussing an article the other 80% can't read will be the end of HN. Maybe if this was a sub-reddit "community" where access to the WSJ was assumed, but HN is a general and open discussion board.

(Also, when politics or people like Taleb Nissam or Paul Krugman are discussed, the voting isn't very informative.)

Using "disenfranchised" in this case with made up statistics of 20% and 80% is rather odd.

There are _plenty_ of discussions to participate in here. WSJ make up maybe 1-2 articles on the front page per day? Can someone explain why this matters enough to even consider a banning of content?

For those who don't want to see it: upvote other things!

I'd much rather see a surge in voting non-mainstream media content then for HN to enforce banning. Come on people, we can be better!

[[As a personal aside, I typically don't upvote and rarely read discussions about mainstream press (WSJ, NY times, WashPost,Economist) here. ]]

>>There are _plenty_ of discussions to participate in here. WSJ make up maybe 1-2 articles on the front page per day? Can someone explain why this matters enough to even consider a banning of content?

1-2 articles on the front page per day is a lot.

To answer your main question though, while I think banning wsj.com would be heavy-handed, I'd be in favor of penalizing the domain so its articles are less likely to reach the front page and stay on it for so long.

I think the reason people are having issue with it is that the topics themselves tend to be interesting, but without access to the article, participating in the discussion becomes difficult. This can be rather frustrating for people.

I know this first-hand because the incognito "trick" used to work for me, but no longer does for some reason. So I find myself effectively blocked out of the conversations.

I don't think this is the kind of comment that should be downvoted. I'm in the same boat as enraged_camel.

At least make links to paywalled stories identifiable via text/colour, or allow logged in users to not even let them appear on the page on the first place. Otherwise you're simply frustrating users unnecessarily.

> WSJ make up maybe 1-2 articles on the front page per day

Per day!

Since when has reading the article ever been a prerequisite for discussing it on HN? ;) </snark>

That said, iirc, someone often helpfully posts the text of the article in such cases. Why go through the effort of banning something when either a) someone will continue to do that, or b) interest/upvotes in paywalled stories will die naturally?

Not having a subscription to the WSJ (I do not) does not make you "disenfranchised."

Incognito doesn't work for me. Like I said in the OP, all of the workarounds I've heard of have not been working for some months now. Neither does Google search or anything else. I don't even use a mainstream browser with an incognito mode anyway, I'd have to open up a seperate browser just to read WSJ articles even if it did work. This is ridiculous.

I'm beginning to believe that the WSJ and others are starting to monitor the number of referrals directly from google and if it hits some threshold they turn on the paywall for the article for some period of time as the direct link from google in incognito doesn't always work for me either.

"Oh look! Direct hits from google have gone up 300% (or have been hit XX times) in the last 2 minutes... turn on the sliding 5 minute paywall!"

Or they check by IP Address. They know the Google ones as they regularly crawl all of their site whereas the IP addresses of random HN users don't.

The workaround only exists because Google requires sites to serve users the same page as the crawlers, so if they do that Google should de-rank them or throw them out of the index.

I agree. But if they are in fact using some "temporary" paywall "technology" then it would be [more] difficult for google to know and/or catch them in the act.

As someone else stated in these comments: "If you start an incognito window with a Google News search for the headline and can't access it, use the "Send Feedback" link at the bottom of the search page to let Google know."

You should also take into account that by the time an article makes it here there are usually multiple copies of it from other news providers. WSJ rarely has exclusive articles so it makes no sense to provide a link to WSJ when the exact same article is on some other site.

The only unique pieces are usually op-eds which usually have similar articles on other popular publications as well.

This is in no way accurate. I've done this type of search almost every day in the last couple of weeks and my success rate is less than 50%. For example the Snapchat article on the IPO and voting rights did find any articles that were more than the WSJ headline.

To counter this point, when the WSJ does actually have an exclusive story or is the first to break it, we ought to recognize that. Rewarding blogspam copycats is a very bad policy.

Using a mainstream browser with incognito mode is a workaround. Although, as some others have pointed out, that may no longer work either.

That trick used to work for me, but as of last week it doesn't.

While I know it is unlikely, I wonder if WSJ has figured out a counter-trick.

If you had read my comment properly, you would have seen that the "workaround" doesn't work for me, and I was saying even if it did work, it's fucking stupid to have to go to such lengths to read an article here.

Upvotes != access

Many people may see the headline and upvote it so it attracts some discussion in the comments.

Agreed; I've also seen more than a few users mention opening many links at once, upvoting all of them, then going in and reading them.

Right click on the "web" link and click "Open Link In Incognito Window".

Hasn't worked for some time. All the articles on workarounds I've looked at recently have comments with no or low success rates.

"If you really don't want to see WSJ articles, install/make a browser extension that does it for you."

I stared something like this a few months ago and forgot about it. The general idea was to apply custom styling to regex matched page content. I got as far as hard coding everything I wanted in to JS file so there's no UI for it. Currently, it applies a black bar to links matching the patterns in the list but it could just as easily style it however you like. If anyone is interested, here's the JavaScript [0] and a basic manifest [1] to load it as a FF web extension (about:debugging -> Load Temporary Add-on). Edit: What it looks like with a match on HN [2]

[0] http://pastebin.com/raw/tPJXJcM5 [1] http://pastebin.com/raw/zi6UV9ZL [2] http://i.imgur.com/HQVopbQ.png

Problem is, sometimes people just upvote based on the title.

There's something very meta about the ideology of the WSJ as conveyed to us by a WSJ subscriber.

Let the vote market do its job, regulations are oppressive, don't fixate on inherent asymmetries to this decision making mechanism, etc.

If you start an incognito window with a Google News search for the headline and can't access it, use the "Send Feedback" link at the bottom of the search page to let Google know.

Why should we discourage articles behind a paywall (with or without a workaround) in the first place? What's so bad about a website that relies on subscriptions instead of ads?

The problem with paywalled links for a forum that is all about discussing the linked content is simple: If most users aren't able to read the content, the discussion either can't happen or is worthless.

Workarounds help, but if they aren't available (or not working any longer), the content isn't accessible for a lot of the international HN community. So, in this case I'd personally prefer another (non-paywalled) source for the topic or just no HN entry at all if no alternative is available.

It's not the case that most users aren't able to read the content. Some may find the content blocked by their government, or some similar entity, but many users simply refuse to read it, and insist that no one else be afforded the opportunity to do so.

It makes little sense to ban discussions among the set of users who can and are willing to read the content.

The quality of the discussion ends up a lot lower because most people don't read the article?

This. It's bad enough when people comment after reading no more than the headline (guilty!) but it's much worse when most people are commenting without getting the full picture.

Seems the problem is then with user behavior, not the link.

There is no rule that people have to comment on articles they can't read. Seems backwards to forbid interested people from linking to and discussing an article in earnest just to reduce posting from bullshitters.

> Seems the problem is then with user behavior, not the link

Perhaps in a sense, but many things that are discouraged by the rules (including paywalled articles without effective workarounds) are discouraged because they are observed to contribute to undesirable user behavior to a degree that responsibility g tomthe individual undesirable behaviors reactively is an inadequate remedy.

I never said anything should be forbidden, it's just a problem that happens from time to time that becomes somewhat larger when the subject of the discussion is largely inaccessible.

The discussion ends up focusing on if HN should allow paywall links.

I don't want to imply that journalists getting paid for their work is a bad thing. However, WSJ is doing it wrong. I recently paid the Guardian instead. They have a noticable but unobtrusive (and dismissable) banner asking you to pay, rather than a paywall. I've never seen them complain about my ad blocker (if they even have ads, I don't know), and they let you pay any amount you wish.

It's just paywalls as a particular implementation of monetizing news that I'm opposed to.

Still, most people won't pay if it's optional. I wonder what percentage of readers pay for the Guardian's news. In the old days, you could pickup a newspaper and set it back down, but if you wanted any sort of convenience, you had to pay for it. It's tough for a newspaper these days...

>Still, most people won't pay if it's optional.

I'd say the same applies if it isn't optional. They just go somewhere else, mostly.

There is nothing wrong with them trying to monetize. However, we should not link to content that a majority of the users cannot read.

because they use HN as their ad space. Not posting news, just teasers which imho are ads; almost as annoying as the cloudflare blog posts.

Besides the excellent reasons already posted here, I would also point out that many, perhaps most, Wall Street Journal articles posted here tend to be the same story that other outlets are covering, often better, and usually the WSJ articles has nothing to recommend it over these other, free outlets. So by blocking WSJ submissions, it would encourage submitters to use an alternate source for the topic.

Because deep down, when we talk about freedom of the press, we really meant free as in beer. No one likes paying for stuff. We will happily pretend that journalists don't need to eat, as long as we never have to see another paywall again.

Do subscribers get an ad-free experience?

Back in the day, Google didn't reference links that showed a different page to the bot from what it does to users. the WSJ presence in their search has substantially reduced the utility of Google's results. I do not understand why they allow it.

It's likely that the page is not different, just "updated", like if you fixed a spelling error on your blog post after it was indexed. There will be a lag where the index is different from the content but google can re-crawl and get the same content.

I believe the paywall is turned on and off consistently for all clients, probably not violating terms.

It's a bad user experience if a majority of readers can't actually RTFA.

I agree, but I don't think banning links to the WSJ is necessarily the right solution. They're presumably interesting articles that warrant discussion here if people are upvoting them, even if those users are in the minority. When a better (e.g. more accessible) link is found the HN admin frequently replace the original URL.

People are upvoting based on the article header, not content (because it's inaccessible).

Or they're WSJ subscribers who can read the article.

For good or ill, however, the WSJ still has a lot of articles and original reporting that isn't available anywhere else. It's unique, which makes it unusually valuable.

Just take the original url and put it behind http://facebook.com/l.php?u=<insert WSJ URL here>

Confirm the facebook warning about the redirect.

EDIT: Check your local law first. I've just noticed during research using facebook that i don't get the paywall coming from there. This is only a proof of concept, if you want to read WSJ, than you should consider paying for a subscription.

Why not just do that for all of the links by default?

Just confirmed: doesn't work with Firefox private browsing. Web -> right click -> New Private Window takes me to Google, and clicking the link there takes me to a pay-walled page.

I agree with OP.

If anyone on wants to know why paid content isn't welcome on HN just pay me $5 and I'll explain why.

Just forward me your bank account number, and I'll directly deposit the $5. Let me know when you've received it, so we can continue discussion. I eagerly await your response.

I also have similar information, my bitcoin address: 18ZcPit6NYHqj5rZ6hU2m3rAq8DeWHoEDq

Sorry, not familiar with bitcoin.

That's not true, you're probably top 20% most sophisticated users:

> I was doing this on the now defunct Cryptsy with the popular alt coins. Using relatively basic exponential moving average calculations. There just really wasn't enough volume to be doing it constantly. I was lucky to get a few trades in a day that made sense. I made money, and lost money. Basically breaking even. Mostly just a learning exercise. At this point though, all of the big players are getting their hands in. Not really worth it anymore.

(from "High-Speed Traders Are Taking Over Bitcoin")

pay up :-D

Quiet you!

Send him a cheque or EMT.

If I think only of the WSJ in isolation, the OP's position might seem extreme, but when I consider that the HN front page may be dominated by links to various paywalled sites (because we allow them, right?) I can see how the user experience will quickly become atrocious. As a site centered around voting and discussion, it's important to make access to the content being voted upon and discussed readily accessible to a large fraction of the readers. While it may come to pass that most HN readers will have a WSJ subscription, it will likely never be the case that most HN readers have subscriptions to most submitted paywalled sites. I would support a policy that forbids submissions that require payment to read for this reason.

I'm with Sir_Cmpwn. No sites with paywalls of any kind should be allowed here. I shouldn't have to go out of my way to get around barriers to get content for free. I'm also realizing I'm having a hard time saying what I mean effectively. If they want a paywall then it should just be a wall. It's awkward and unethical to bypass the wall to get the content for free anyway, yet that is what is expected when those links are allowed on HN. So those links should just not be allowed at all then.

Did you email mods about this? What did they say?

https://news.ycombinator.com/newsguidelines.html

> Please don't post on HN to ask or tell us something (e.g. to ask us questions about Y Combinator, or to ask or complain about moderation). If you want to say something to us, please send it to hn@ycombinator.com.

I did, they ignored my email.

I don't mind seeing paywalled articles on the front page but I feel I must be warned that it is a paywalled link. The same way we are warned when content is a PDF. That way I won't waste my time opening the link. I don't even want to use workarounds. If it is paywalled and/or requires me to disable AdBlock, I don't want to waste my time on it.

The WSJ "workaround" has never consistently worked for me.

It works sometimes, but other times it won't work even with a Google Search referrer in an incognito tab.

My friend works on WSJ for Dow Jones. Recently, he stumbled upon a query param that you can add to any article link to remove the paywall. It's ridiculous how many things have come up with WSJ to get around what I assume is one of their main sources of revenue besides ads.

You can't just say you have a workaround that someone stumbled on that addresses a problem for a huge number of people and then drop the mike and walk off stage. :-)

Replace WSJ with Netflix, would you suggest he share the link then? Many of us live of SaaS but complain about it when newspapers do it.

Yeh riiight, and my friend's second cousin's neighbor works for a company where the cleaning contractor also cleans the building where your friend works, and he overheard someone who said your friend was talking bullshit.

Can you please share that param?

The Google Search workaround does a perfectly fine job for me.

It has been working sporadically for me; sometimes it works, but then strangely other times regardless of using incognito mode or not, the Google search result still lands on the same paywall.

A couple of things I wonder: A) maybe is a rolling change, and not yet complete; or B) perhaps the goal is to disrupt it enough to make it not reliable, yet functional enough for plausible deniability.

The end result though, is that I read less of the WSJ. I'm sympathetic to the need for paying subscribers, but I've already subb'ed to the NYT, so if I run into unavoidable paywalls at other sources, the end result is that I'll read less of those sources.

It doesn't work for me any longer (maybe since a few days ago).

Seconded. Even articles that pop-up directly in my Google News feed don't display the full article.

It worked for me when I switched from https to http, but it may have had to do with the referrer. I do know that it doesn't work with encrypted.google.com, only with vanilla Google.com.

Being on mobile I agree. Add-ons aren't a viable fix

It's stopped working for me on Firefox using Private windows.

There are some days for no good reason I can't get past some paywalls using the HN workaround, when that happens I fall back on http://archive.is/

Yes, archive.is hides the paywall layer above the content. This is the workaround which still works. But how long will it work? WSJ could change the paywall logic at any moment.

Really baffling that HN management doesn't understand that paywalled links are poison to a discussion site. They're broken links.

Yeah, and also get rid of links to software that costs money, because that's free advertising, too!

The equivalent in this case would be if you couldn't even know what their software DID without paying for it.

Hacker news is a forum for discussing topics. How can you possible discuss an article you can't read? You can, however, discuss a piece of software based on it's intended purpose.

Eh - no problem with free advertising. A good chunk of what is posted on hacker news has some kind of profit motive or self-promotion attached it it.

My frustration with hard-core paywall sites is that most of the community is blocked (or has to jump through hoops) to read the content.

confirmed - I can't access via incognito or google search. I did the other week, as if by magic - but it's gotten closer to impossible now for whatever reason.

I don't understand the issue. If you don't like sites from a particular publisher, don't visit them.

HN lists the root domain right next to the link.

The issue is whether it makes sense to post a paywall link on HN, since only a (probably small) subset of users can access it, if there is no workaround available.

EDIT: It could be argued that the upvote system would take care of that naturally (if only few users are able to access it, few would upvote it), but I can understand why it could be considered an issue.

I too am disappointed that the Google redirect workaround no longer works. I recently cancelled my WSJ membership as part of an effort to minimize my monthly expenses, so now I am effectively cut off from all their news stories.

IMO it's not a big deal though, as I've turned to Bloomberg for most of my market news. My health has also improved considering that I no longer subject myself to skimming through the WSJ comments section.

I flag WSJ links wenever I see them but seems to have no effect.

Hopefully by now your flag carries little weight

The only thing that works for me now is to Google the first paragraph they let you see (not the headline). Usually the article is available elsewhere. None of the other techniques like incognito mode, copying link, Google referral, blocking Javascript or any combination thereof doesn't work. (Facebook referral still works but it's not easy to find a FB link to a specific article.)

Reply on news.ycombinator.com