SPF, DKIM, reverse DNS, no blacklists, no open relay, longtime ownership of IPs, etc etc. Using various mail testers returns a 10/10 deliverability score.
And yet, messages sent to Gmail always go into the spam folder, or are never delivered at all. These are everyday regular messages, I have never used mailing lists or sent bulk automated messages.
The issue is, there is no recourse, no fix, no acknowledgement of the problem with false positives. There is no tool available to me to understand or correct the "problem". Hint: this comes across as Gmail not giving a shit.
Gmail has a responsibility to be more accountable, even if these problems are unintentional, because Gmail is such an enormous node in a federated network.
> If we lose an open, healthy ecosystem with many providers, we'll destroy the base we stand on.
Correct. Gmail is contributing to the erosion of email reliability. Please course correct.
- Legitimate class action notices related to Amazon purchases.
- Email coming from addresses to which I had already sent email. (!)
- Email from my landlord.
- Email coming from Google itself.
Based on the contents of my spam folder, which I have to check fairly often because of the extreme overaggressiveness, I would be vastly better off if nothing ever got filtered at all. [1]
>> There are going to be false positives, we will make mistakes, but we certainly care a lot about fixing issues like this when we hear about them.
This doesn't sound honest, or at least not complete. People have been complaining about this for years. I have personally been complaining about this for years. The loss of obviously legitimate email is completely outrageous.
It doesn't look intentional (look at that fourth category!), but it certainly doesn't look like anyone is trying to address the problem.
[1] Yes, if spam filtering was disabled, more spam might get sent.
That didn’t stop me from repeatedly getting junk from every other record label my email was sold to. It was an endless procession of shit I never subscribed to.
When either provider decides your small email server is sending spam (eg: sending an email with an attachment, or any kind of form email like a daily report) you won't get through to user inboxes, and instead you'll be routed to spam, or for Outlook.com hosted addresses they will accept mail from your server and send it to /dev/null. Gmail's process is bad, but Microsoft has decided to accept emails and throw them away (which is ridiculous).
Yep. I missed an invitation to a Google-hosted event at a conference I attended because the email (from an @google.com address, no less) got caught in Gmail's spam filter.
On the other hand, once you train it a bit, it is mostly remarkable good. For me, switching from fastmail.fm (which was pretty good itself) to Gmail gave me a big improvement in spam control.
It doesn't make any sense since they are emails from Google, they are emails I even have a filter applied to so that a label is applied to them. Yes I can adjust the filter and choose "never send to spam" but the messages will still show a warning on them saying "This message was not sent to spam because of a filter you have applied".
Sure false positives makes sense, but I don't get how the majority of what is in my spam folder would be emails sent by Google.
It makes a lot of sense... people use the spam button as a lazy man's unsubscribe. Youtube adding the bell button, making mail opt in is probably a response to that.
I actually give Google a lot of cred for not simply white-listing its own domains. Though spammers would probably find ways to abuse it and make them look bad anyway.
Edit: Gah. Now I checked my work Gmail spam folder. There was an email from one of my users there. (and nothing else)
Now I'm considering migrating away from Gmail, at least for work related things.
If email coming from google itself got special treatment there would be masses with pitchforks complaining about that.
You've made a lot of good points, but I don't think that's one of them.
I disagree. I think in the absence of that point, it would have been hard to say this:
> It doesn't look intentional (look at that fourth category!)
But also, I think special treatment for trusted actors is a completely appropriate way to handle email delivery, and I also think it's appropriate for gmail to trust themselves to be sending legitimate mail. Blocking their own email makes them look totally incompetent. They absolutely should whitelist themselves. And they should have a way for you to be whitelisted too, if you want to send email.
That is understandable. It is hard to validate if an email is authentic. SMTP has no authentication built in. Gmail can't just blindly accept all emails from addresses that you have already sent an email to.
EDIT:
Look at this example: https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol#...
Anyone can connect to relay.example.com and pretend to be bob@example.com.
All those "I hacked your email and send you a message from you account" I don't get, because I have a DMARC policy that says if you don't pass SPF/DKIM then you get rejected. So try as the spammer might to connect to my mailserver and pretend to be me, they can't, because my mailserver sees they're not authenticated, and the mailserver they're sending from isn't in my SPF records, isn't signing the message with my DKIM key and therefore it gets rejected at the SMTP level.
The article discusses mail not being accepted by google/gmail in the first place.
Messages they think you won't want to receive are what the spam folder is for.
To be fair, sometimes Google does send spam.
You'd think Google had an incentive to get that delivered.
But it still beats no filtering.
There are layers of filtering beyond what appears in your spam folder, layers that block obvious spam long before it gets anywhere near your account. If every email ever sent to your address wound up in your spam folder you'd beg for filtering.
Even after running it for years, Spamassassin never marked a legitimate mail as spam, so i'm pretty sure that if i wasn't too lazy to configure it to move it to a spam folder, it'd work fine. Stuff did pass through it (at a ratio of one every four or so) but i was fine with deleting those.
What i'm trying to say is that from personal experience, i'd be fine with a spam filter that errs on the side of not marking stuff for spam and me deleting whatever goes through manually. Having to see a bit of spam mail is small cost for losing mail i'm actually interested in.
Thing is, most of my messages get delivered. Then suddenly they don't.
Google "giving a shit" means responding to hostmasters about delivery problems, and they just don't.
Please note this, Gmail PM.
I changed jobs as well, and now work is using a MSFT based hosted mail service, and I am getting delay messages.
Seriously, GOOG, MSFT, and others broke mail. This is not an improvement.
I've not looked into speaking with MSFT mail folks about their breakage yet. With GOOG, you have really no mechanism of reaching out to someone there and getting attention for the problem they are causing.
This is the much bigger problem with GOOG actually, in case any googley people are reading this. They just don't get customer service. At all. It is near impossible to be able to report a real problem across the spectrum of their services. Unless you are one of their bigger customers, you don't have access to even a telephone support number. Their online help is a crapshoot, with you getting useful information less than 50% of the time.
So where I am now is with locked down, long time existing domain mail servers, which send maybe 5-10 outbound messages per month, that suddenly and inexplicably, have a bad reputation. Well, no they don't have a bad reputation, they can send email just fine to other services.
what?
It is my belief that this is intentional and I would love to be corrected if it’s not.
https://public-inbox.org/git/CAHk-=whP1stFZNAaJiMi5eZ9rj0MRt...
Large providers tended to have a world view dividing all senders into two categories: 1) Bulk senders who are clearly mailing the same spam to a list of a billion addresses 2) Non-commercial individual hosts which should be sending five messages a day or less in total.
It felt like there was a huge missing third category for transactional emailers nobody wanted to acknowledge. They are probably difficult to score fairly. A hundred "Order details" emails are going to have the same level of randomness/templatedness as the old Viagra spam which had a random block of Project Gutenberg text pasted at the end to trip up filter math. You're not going to have a clear history of "this address bounced twice, let's stop sending newsletters" when most of your messages are to first time customers or once-every-few-years return ones. A lot of the messages will look generic because they use default shopping cart templates.
To the extent they provided sender guidance, it was focused around use case 1) -- sign up for feedback loops and deal with greylisting (because people really love waiting 18 hours for an acknowledgement)
Anyway, Gmail is getting some heat in this thread and rightfully so. We should however not forget that Microsoft and Yahoo are just as bad if not worse in this respect.
I had set up some commercial accounts on Zoho (my own e-mail is on Zoho) but they asked me to migrate to Google, even though it was more expensive.
I no longer use Gmail myself but half of my users relay some aliases to their main Gmail account. No problems with that, except my servers continuously get rate limited by Google:
Our system has detected an unusual rate of 421-4.7.0 unsolicited mail originating from your IP address. To protect our 421-4.7.0 users from spam, mail sent from your IP address has been temporarily 421-4.7.0 rate limited. Please visit 421-4.7.0 https://support.google.com/mail/?p=UnsolicitedRateLimitError to 421 4.7.0 review our Bulk Email Senders Guidelines.
This leads to family phoning me all the time to say my server is broken when it is just Google throttling everyone's emails. I frequently have to check Postfix queues and clear some obvious spam or just pointless Facebook update emails which seems to be the majority... (Yes your email admin can read your email...)
The only way to improve is to constantly remind my family to not sign up to all crap, use not so common aliases, and try to keep tightening my anti-spam configurations. But we really are not talking about a lot of emails. Probably less than a hundred per day spread across 5-ish end accounts of which 95% is probably legit. Yet Google is treating me as some totally open relay. (╯°□°)╯︵ ┻━┻
But it does feel like they are treating nearly all minor relays as spam relays. We can not block 100% of spam before we relay onwards to Gmail as that would mean too many false positives emails get blocked, but most try to block as much as possible.
In my case, my servers probably block 99% of spam, but some will get relayed, and most of those ends up being handled by Gmail's even better Bayes scoring and filtered to end users' spam folders.
I don't know what the automatic threshold limit is to be on their naughty list, but it must be very low, as in double digits per day.
https://toolbox.googleapps.com/apps/checkmx/check?domain=tab...
It is not to be set up correctly.
Now in your case, you failed to mention DMARC records. ALL of the big cloud email services have required that for years. Your SMTP server checklist is straight out of 2005.
This entire thread reminds me of the guy who pulled out of a desk drawer an Analog Startac cell phone and blamed AT&T for it not working. Email has moved on. It not Google's fault that people are configuring SMTP servers like it 2005.
> error SPF must allow Google servers to send mail on behalf of your domain.
So you must let Google spoof your domain? That seems crazy.
ProtonMail has the same error:
https://toolbox.googleapps.com/apps/checkmx/check?domain=pro...
For domains that are not using Google Apps, its results are meaningless.
I know because I haven't set up DMARC for my email domain, and my email still gets accepted by everyone.
I believe it helps that I accept on TLS with a valid certificate, and I used Google's postmaster site verification.
I _think_ this is because G doesn't want people gaming the system.
> or correct the "problem".
I regularly fish mail from my spam bin. From mailing lists, and other important stuff -- and indeed Google's own mail!
One thing you can do is to get people to add you to their contacts list.
It's a hard problem. There is one "solution" you probably do not want. Have Google (and other companies) give their imprimatur to certain mail senders.
9 out of 10 threads marked as spam were false negatives.
That's a 90% failure rate.
Since money was involved, I paid attention and followed up
Our finance person found many emails in Spam that were important and should not be there
On the other hand, my personal Spam folder is certainly full of crap I never want to see. But now I don’t trust the system, so I have to scan them anyway