Comment by the_pwner224 | Hacker News Reader

Comment by the_pwner224 | Hacker News Reader

Parent Full thread

> Google's ML mail spam models are that way because there's too many bad actors. Email, even in 2019, still extremely valuable.

Google's spam models freaking suck donkey dick. I use Fastmail but my university uses Google for email. A service we use (Piazza, class Q&A) sends notifications via email. Over the years I have gotten probably about 1k email notifications from them, and replied to a few of them too. Google has recently started marking about 5% of their notifications as spam. 95% of the notifications come through; 5% get rejected. There's no discernible logic to this.

There are organizations and people within the university which have me on their mailing lists; with them too 90% of the emails get through but 5% get randomly marked as spam. And they're perfectly normal emails that Google just decides to move to spam. Even with people that I have had email conversations with, it just spams random stuff.

Every couple of months I check my Fastmail spam folder just for the lolz of seeing what's in there. With Google I have to check spam every day or I'll miss important stuff.

There are tons of people who have posted very similar experiences on HN. On this thread I just saw another person comment that Google's own emails go to spam - I've observed this too on my personal Gmail account which I occasionally check.

The spam filters have a tough job, but other companies are succeeding at keeping all the spam out without a gigantic false positive rate. Google's filter is hilariously bad.

I'm not defending gmail, but I will say that some users simply suck (unintentionally). I bet some of your peers are just hitting the spam button instead of simply deleting the email, generating a bad reputation for the sender.

I've seen this with my SaaS. We ONLY send transactional emails like purchase receipts and login links (using postmark). We're very intentional about not sending anything remotely resembling marketing or spam, and sending as little as possible. Yet I still get the occasional user who will flag their purchase receipt as spam when all they really mean to do is delete it. They don't get the difference and don't realize that it matters which button they click.

I've noticed that in the past year or so gmail has made hitting the spam button more explicit and now have a confirmation message. Obviously they did this because some people were simply mashing that spam button instead of the delete or archive button.

I don't even blame the users. Most don't know the impact it has. But bad actors sending tons of spam have trained them to be cynical of any generic form email.

>Yet I still get the occasional user who will flag their purchase receipt as spam when all they really mean to do is delete it.

Maybe gmail shouldn't have the spam button right next to delete :)

At least that's better than web Outlook's invisible, until you click on the sender, microscopic disposition buttons. I can't count the number of times I've deleted an email instead of opening it.

I don't think it's always the user. The gmail and outlook UI makes report as spam more prominent then delete. In addition it's easy for accidental inclusion in a users batch spam report because of poor multi select UX.

It might be because the service sends both spam emails and legitimate emails. Many (maybe even most) sites do this. If this person is getting a thousand emails over the course of a few years, how on Earth are all of those legitimate, especially when they've only replied to a handful of them, implying the majority of emails can oe safely ignored.

What would be nice is a useful mail ML filter which would scan mail flagged spam to trigger a "are you sure this is spam, do you want to delete instead" question.

> I'm not defending gmail, but I will say that some users simply suck (unintentionally). I bet some of your peers are just hitting the spam button instead of simply deleting the email, generating a bad reputation for the sender.

Google does put some senders often into spam for me, but not always. I'm absolutely sure I've never reported them as spam, on the contrary: I've marked their emails as "Not Spam" when I caught them in the spam folder.

That had, so far, no influence on the filter. Those aren't "weird" emails either, they are in my language and individual communication, not mass mails.

> I'm not defending gmail, but I will say that some users simply suck (unintentionally). I bet some of your peers are just hitting the spam button instead of simply deleting the email, generating a bad reputation for the sender.

And yet other companies with spam reporting features are able to maintain a lower false positive rate with their spam filters.

I'm not sure that other companies near the scale of Gmail have a lower false positive rate. Sure something niche like fastmail which is going to have, on average, much more tech-savy users, will have more accurate user-reported spam filters. Yahoo / Microsoft don't seem to do any better than Google here though, in my experience they're both actually worse.

You should be able to set a filter to catch these before they get spammed out of things you want to receive aren't getting your inbox.

Also, a lot of users do perceive receipts as spam.

Yet I still get the occasional user who will flag their purchase receipt as spam when all they really mean to do is delete it. They don't get the difference and don't realize that it matters which button they click.

I have a relatively high profile email address (first initial and last name) and I almost daily get at least one form of "receipt" from a transaction someone made where they entered my email by mistake.

Sometimes I'm in a bad mood and I'll mark anything I see like that as spam partly because its frustrating and partly because I suspect a good number of these emails actually are spam masquerading as receipt/login/etc emails.

And I just don't have the energy to figure out which are legit or not.

I wonder how much bad spam marking happens maliciously. Sign up lots of accounts to your competitor's mailing lists, mark everything they send as spam so that hopefully their email will not make it to their customers giving you more of a chance.

May be you should not send purchase receipt by default? I don't know why everyone does that as I don't care about it at all. If it's not required by law, add button to your UI which will mail receipt (or better just let user download it) and that's about it. While I'm not marking those receipts as spam, I'm sure that some users would do that as it's unwanted mail for them.

I would find it odd not to receive a receipt. A receipt email should not be classified as spam. Anything else I would say would be spam.

Please keep sending email receipts by default.

The purchase history on website disappears, logins get forgotten and so on. Only receipts in the mailbox stay forever, as I want them to.

We actually have users contact our help desk asking for the receipt not realizing it's already in their inbox or spam folder.

I've considered adding an opt-out checkbox during checkout but I honestly doubt it would ever be unchecked and if it was opt-in we'd have users contacting support asking for it because they didn't realize they had to check the box. Either way it's one more toggle on the UI which I'm not a fan of unless absolutely necessary.

I'm curious if you've run a business and experimented with not sending receipts, and if so how it played out?

Yeah, gmail is really not great. I host my own email but use gmail for work. I hate everything in gmail, from the UI to the lack of really simple way to filter thing (like sieve) and to the totally random behaviour like you described. In the end, I don't really read emails on gmail anymore because it's just a huge pain.

My own server has a good reputation, but it meant setting up SPF, DKIM, DMARC etc.... Where it was the most painful was with microsoft email (I don't know the name of their service now, still live maybe). By default mails goes to spam, but really quickly they goes in limbo if people don't respond (on the smtp side the email is accepted but the recipient never see it, not even in spam). On new microsoft accounts I was always able to send a few emails in spam before it went into limbo. I ended up creating ~20 different accounts on their system (it was before they enforced giving a phone number) and had random discussions with myself for a while which was enough to build a reputation and not being put in spam anymore. I don't know if this behaviour was intended or not, I try to get in touch with someone from microsoft but couldn't find a way. Since then I do strongly advise not to use their system because that makes it really unreliable.

I have exactly the same problem on my personal email server, ms silently drops my email! Their server replies with OK, but the email is nowhere to be found in the recipient’s inbox. Even if the conversation was initiated by the microsoft account and I merely reply to it, the email still disappears.

I’ve got everything setup properly, all test websites come up green, and google receives my email just fine. SPF, DMARK, DKIM, PTR, HELO are all setup correctly. This isn’t an isolated problem either, the ms forums are filled with people reporting similar problems.

Same expirience here, but I must say it is becoming better since I have reverse resolve. Previously every mail went into spam. Regardless of SPF and DKIM.

"I have exactly the same problem on my personal email server, ms silently drops my email! Their server replies with OK, but the email is nowhere to be found in the recipient’s inbox."

Slightly tangential ... I heard recently that "proper" in-line responses (ie, not top-posting) is somehow penalized by Microsoft (or Exchange online or Outlook online or ... ?) and was advised to stop my very proper 1990s style in-line email quoting/responding and just top-post to avoid either non-delivery or something like the body of the email not being shown ...

Or something ?

Does anyone know what I am referring to and can shed some light here ?

Yeah, I only bottom-reply when I know (or can reasonably expect, e.g. on a mailing list) that the recipient is using a mail client that's not entirely brain damaged. Otherwise, I'll either top-reply or (if I really do want to respond to things in-line) actually say "See below for responses in-line" to clue in the recipient that yes, there's actually content in this email.

But yeah, that's usually more an issue with the mail client (or perhaps the mail server if it's being exceptionally stupid and modifying emails before final delivery). Top-replies became the norm for some god-forsaken reason, and now clients treat the quote and everything below it as entirely optional to display.

One thing I've noticed too, make sure that your time is synchronized with an NTP server. Even with a good reputation , MS drops emails with no error if you're a bit off.

MS is really bad at this, a few web forums I am involved with are unable to sign people up at all.

I've run my own personal email for around 20 years at this point using the same IP address, and have set up all the bells and whistles (SPF, DKIM) over the years as they became in vogue. To my knowledge, I haven't had any issues with getting caught up in Google's or MS's spam filters, but I have to wonder if that's simply because I don't send a lot of messages and for the most part reply to emails rather than being the initiator of a conversation.

Absolutely agree with you. Many people on HN suffers from Stockholm syndrome when it comes to Google. If Google wanted to be a customer focused company instead if being an ad revenue focused one they could expose the knobs to you to fine tune the spam filter. They do not care about users, Gmail does not even have a customer support line that you can call when you ended up in the wrong side of their automation code. I am in the middle of moving out every email address from Google asap. I just realized how disgusting their way of operating is and I do not want to continue contribute to this problem by using their service.

Just for fun I checked my spam folder as I read this.

I found an important email (from 3 days ago) from a recruiter asking for my CV in it.

Yeah, Google's spam has erred too far towards making false positive errors.

It's curious that people have such diverging experiences with the spam filter. I think the claim that Google's filter is objectively worse should at least be supported by some evidence if you're going to make a bold statement like that.

As another data point, for me and friends I it beats competitors by a large margin.

As a normal gmail user you won't notice this, all you see is that you get mail from your friends that you always mailed with. If you register on popular sites, their mail will also work. If however you are sitting on a new domain, sending mail to an old friend on gmail, it will very likely end up in the spam folder unless you are using a big provider that has reputation with google already.

I guess it's another case of survival bias. If it works you won't notice that it does not work in all cases.

If you never look in your spam folder you’ll never notice how utterly bad it is. Which is apparently the case for most people.

A lot of people don't even know there is a spam folder

Even a lot of “popular” site stuff ends up in spam. I basically have to regularly check my spam box, which previously I would look at a few times a year when a password reset or something didn’t show up.

Anti-trust regulators should specifically look at situations where Google began directly competing with other companies and those other companies gmail messages are being put in spam while Google’s messages are not. Whether or not it is intentional, it is a problem if Google is whitelisting their own stuff but not competitors.

Ultimately what is going on here is Google is using more deep learning for spam detection. Your friend’s email ending up in spam may be a false positive. It may also be a test, to see if you notice it.

So I own a very short, easy to type domain, and maintain a wildcard. By volume, I probably get 60% spam to 40% content. I apologize to all the hard working vendors being fed into my spam list, but there's a secret: double opt in.

As a bonus, if you require opt in, I won't be able to cancel your car reservations with budget ROBERT.

Your comment prompted me to go through my GMail spam folder. I noticed 4 emails (out of 100) that weren't spam, including one from Blue Origin (!), 2 from a mailing list that's usually delivered to my Inbox, and 1 from an online shop (confirming my purchase).

Did the same thing on work email and found only messages from gitlab. But only 10% of those messages were marked as spam the others were in inbox so I did not notice. This convinced me not to trust gmail antispam.

I didn't even know there was a spam folder hiding in the collapse ui.

Piazza absolutely sends spam about the recruiting side of their platform, whether you like it or not.

I absolutely click spam on every single one of those pieces of mail, and I'm glad that they are being penalized for it.

I've noticed this too. I've stopped telling people that their emails are getting spammed because I've determined it's not their fault, that google just randomly spams emails coming from lists. It's the main reason I'm working on moving away from google back to my own mail server.

Reply on news.ycombinator.com