Ok. How they can both be omnipresent and yet take hours to find, I guess we can leave as an exercise for the reader. Here's a simpler point:
> the omnipresence of abusive trolls is obvious to anyone who uses HN regularly
If that were the case, I'd be hearing about nothing else—the inbox would be dominated by it. But you're the only person saying this that I know of.
> If you were actually banning them, then they wouldn't be able to keep posting
You're changing the meaning of the word 'banned' to something other than the one it has had on HN for 18 years. That's fine in a Humpty Dumpty way (using the word to mean whatever you choose it to mean), but it makes your comments on this confusing, and I'm going to keep using the word to mean what it has always meant here.
By that standard definition, all the accounts you're talking about are already banned. The vast majority of readers see none of those comments—you see them only because you turned on the 'showdead' setting in your profile. You dislike many comments that show up when you do that, and want us to change HN so you can turn on 'showdead' but not see as many bad comments.
I explained why HN works this way (https://news.ycombinator.com/item?id=43143611), but you don't like that design and want us to change it. Specifically, you want us to (1) block banned accounts from posting at all, and (2) make it harder to create a new account on HN, so trolls find it harder to create new accounts to troll with.
That's my understanding of your argument, and you're welcome to correct me if I misread it.
Since you said you wanted my response, here it is: we're not going to do that, for several reasons:
(1) I don't want to make HN less accessible to new users. Older users often want us to add barriers for newcomers. I believe that would be a mistake. The risk to HN of failing to attract new legit users is higher than any benefit of making it harder to join.
This cost would be highest in the case of legit new users—who are inclined to bail when they encounter friction—and lowest in the case of serial trolls, who know better than anyone how to get around restrictions. Such a restriction would be least effective on the worst commenters and most effective at blocking good ones, like a chemotherapy drug that goes easier on cancer cells and is most toxic on healthy cells.
It's particularly important that new users (or existing ones who want to comment anonymously) who have unique expertise or experience about a topic be able to sign up and comment immediately. Some of HN's best-ever comments have been of that kind.
(2) The existing design—allowing banned accounts to keep posting but making their posts default-invisible, and then allowing any user to turn on a setting to read them—works remarkably well. (I get that you disagree with this, but you asked for my response, so bear with me.) If it didn't, we'd be flooded with complaints about it, and we aren't.
Not that it's perfect. Users sometimes forget that they turned on 'showdead' in their profile, and thus that they signed up to see all comments by banned accounts. In such cases we get emails saying "I can't believe that you condone accounts which post trash like <link>. How can they not be banned?" Then I have to explain that no, we don't condone it, yes the account is banned, and they (the emailer) must have 'showdead' turned on in their profile, since that's the only way they could be seeing those posts.
This is bad, but it doesn't happen often enough to fundamentally change the design—especially since these users typically respond by saying "well that's a relief! I forgot I had turned on that setting. Thanks!"
Your complaint is different: you haven't forgotten that you turned 'showdead' on, and you understand it fine—you just don't like having to see so much garbage when you do turn it on. And I agree—who would? A lot of what's in there is sewage, the worst that the internet has to offer.
But you signed up for this when you turned on 'showdead'—that's the 'contract', so to speak. You don't like this and want us to change it, but you're the only person asking for this. Everyone else who turns on 'showdead' understands that that's what they're signing up for, save for the few who (as I just described) forgot that they did it and need a refresher.
(3) The 'showdead' system is critical to community trust on HN. Many HN users worry about censorship and want to know what's going on behind the curtain. 'Showdead' is our way of saying to those users: yes, many posts (about 3%) get removed from the default public view, but there's a way for you to see them all and decide for yourself what you think—simply turn on 'showdead' in your profile. This makes most of those users happy because they get to see everything. I realize that you feel differently—you don't want to see everything—but this is definitely not how most users who turn on 'showdead' feel.
(4) The problem of serial trolls is not one we can ever solve, exactly, but it's a known, stable factor in the system. Trolls show up, users flag them or email us, we ban them, and then they either keep posting with their banned account (off in limbo where they're invisible to most, but accessible to those with strong enough stomachs to turn on 'showdead')—or they create new accounts and start the cycle all over again. This sucks, but it's a fixed cost: it's not growing, relative to other problems we face, and it doesn't threaten the survival of the community. (Side note: it may not feel that way at the moment because of current macro trends, but that's a separate issue which I've written a lot about elsewhere.)
They are pathogens that place a load on the 'body', but at a fairly fixed rate. The load is high, but not so high that it overwhelms the immune system.
They're also a known commodity to readers. People understand that some commenters will post garbage on the internet, and that as long as we deal with it within reasonable time and space, it's more cost-of-doing-business than existential threat. It is inevitable on the public-anonymous internet, and the majority of this community understands this.
In that sense, serial trolls are like spammers: both suck, both have always and will ever be with us, neither can be 'solved', but as long as the immune system is stronger than the toxic load, the community won't die because of it. When either of them upticks, so do the complaints we get, and then we work on strengthening the immune system. What we don't do is try to replace the immune system with a different one, because the risks of doing that would be higher—the cure could be worse than the disease.
There are other risks that actually do threaten the health and survival of this community. Those are the ones we need to focus on and put resources into addressing.
(5) There have been times in the past when we have added barriers, and the result was scandal and fiasco. I'm thinking, for example, of pg's old "pending comments" design, which led to outrage, accusations of elitism, and so on. I'm not saying this is the same as what you're proposing, but it's in the same ballpark, and it's a ballpark where we've had bad results in the past, leaving me inclined to avoid it.