- Performance. Unreasonably long amounts of time to send out e-mails. phpList degraded to the point of taking several days to process a campaign. listmonk can spawn N goroutines (~threads) and push e-mails to multiple SMTP servers. On a commodity ec2 instance, we're able to send 1.5mn+ e-mails in a couple hours.
- Subscriber imports were extremely slow. Direct integration to keep subscribers in sync with external CRMs was cumbersome. Direct DB inserts were complicated due to the complex table structures. listmonk imports 10k records/sec into a Postgres DB on a commidty ec2 instance.
- Segmentation. Often, we have to rapidly segment users by custom attributes and conditions and relay an update to them. listmonk supports SQL expressions to segment users on their attributes that are defined as arbitrary JSON maps (thanks to Postgres JSONB type).
- Unavailability of dynamic templates. listmonk templates support Go template expressions so it's possible to write logic in messages to make them dynamic.
I would like to add that listmonk is still work in progress and requires a number of essential features before it can come out of alpha.
Are you guys using returnpath' senderscore or a similar certification to send that amount of emails ?
So, is it connected to some kind of CRM ?
About attributes, every subscriber entry in listmonk can take a JSON map of attributes that can be queried.
Not sure what SMTP setup you had with phpList, but sending 1.5m messages on a VPS should take less than a day with the right Postfix tweaks.
We did try several things; Postfix tweaks, SES / Postal SMTPs, rate changes in the phpList config etc., but nothing seemed to work. This, along with the other factors (segmentation, dynamic templating), lead to listmonk.
This looks like the free/open-source version to Sendy based on more modern tech like Golang/Docker/etc I've been hoping for, congrats on your alpha release!
What are best practices for using/selecting an ESP if you were to use a project like this and want to ensure reasonable deliverability?
PS: Postal SMTP is awesome.
starting to sound hopelessly naive
you deliver email updates to fund LPs that aren't unilaterally marking you as spam for no reason
Using an ESP will generally simplify the amount of work necessary to achieve good delivery, but due to the complexities of domain reputation and fingerprinting, very occasionally they can also be the cause of delivery problems.
General guidelines:
- It's a bit harder to predict how deliverability will turn out at smaller ESPs, vs. well-established ones like Sendgrid, SES, Mailgun, and perhaps Mailchimp.
- DKIM, SPF, DMARC (p=none is fine to start; gmail has a guide for deploying dmarc https://support.google.com/a/answer/2466563)
- Use the same root domain everywhere within your email: From, Return-Path, DKIM signature headers; URLs for links and images in the message body. (Separate subdomains are fine, e.g., img.example.com, bounce.example.com)
- Having your sending domain show up in third party email can cause problems. Try to avoid it if possible; use a separate domain as a last resort.
- Send consistently. Some variation in sending patterns will be tolerated, but you'll probably have trouble if you send 500 messages a day most of the time, then once a month you send 100,000 messages.
- Consider a dedicated IP if you're sending over 50,000-ish messages a month.
- You may want to split up transactional email from marketing, sending these from separate IPs and/or subdomains.
- Finally, make sure you're sending email your recipients find valuable and are interested in receiving. Too much disinterest/disengagement can cause delivery issues.
I would recommend to NOT send different email types from different providers, like use SendGrid or SES for transactional emails and then some other platform for marketing emails, especially using the same sender domain. That's because email header signature and tracking links will be different and email box providers like Gmail can find it suspicious and send emails to Spam. And then there is the issue of processing and syncing bounces/unsubscribes/complaints data.
Keeping all email campaign types in a single platform allows for centralized processing of bounces/unsubscribes/complaints date and protects future campaign engagement + sender reputation.
Also, the quality and size of the starting list matters too because a large list (like the OPs) or unengaged list can have a high bounce and low engagement rate, which factors into future deliverability.
Now before you mention how this should be a solveable problem inn AWS, we’re already well underway testing with the client. The problem presented itself as a manifestation of inheriting the last guy’s infrastructure and working to...unfuck a lot of his questionable design choices.
I've been looking for a good solution for ad-hoc email campaigns (w/ templates) and for sending "systems" emails to an audience of customers.
I currently look after lists on both MailChimp and EmailOctopus and get frustrated by the flat pricing for the size of the list and not how you engage with the list (e.g. I'd love to see: store your list for $X/mo, or pay $X + (list size * $Y) in the months when you email the list).
The issue is that you need to put lots of effort into getting off of blacklists, or your email won't be delivered.
As a business that sends emails infrequently, it's probably saved me hundreds/thousands and it has good instructions to set it up with SES. One-time (ish) fee, until you need to upgrade to the next major version.
Besides, that an email is produced by an AGPL software does not make it covered by the AGPL. That should be exactly the same legal situation as with open source text editors.
So, who does need to be provided with a copy? Just the 'administrators', the people using it to send email?
What about if I modify it and repackage it as monklist or whatever, and charge people to use it. I must give them the source code for listmonk too, and, since another requirement is same licence, the source for my modified version? i.e. monklist couldn't be closed source, but myapp that happens to use listmonk unmodified for sending emails can be?
Have you considered using a hosted platform that uses Amazon SES rather then hosting Sendy or other tools yourself? I own BigMailer and our clients with established SES accounts (high daily rate limits > high limit per sec) see 1m emails go out in a little over 1 hour.
I always wonder why people choose to host email software in house and it seems like the requirement for direct integration with in-house system(s) is one of the biggest considerations when choosing in-house vs. off-the-shelf solution.
listmonk is a replacement for Sendy. You can connect your SES account to it using the SES SMTP interface.
Integration with in-house DBs was a top reason for us.
Try to implement bounce processing soon - an average list decays at 2% per month (3% for B2B lists) so in a few months, the bounce rate can be high enough for mail box providers to start blocking your emails due to attempts to send to large number of invalid email addresses. Especially sensitive issue with your list size and sending volume.
There might be a couple more useful nuggets in this article on email deliverability best practices https://www.bigmailer.io/blog/improve-email-deliverability-b...
I don't see anything in the docs about supplying an unsubscribe link. I don't see an endpoint in the API that could be used with an anchor tag in the footer of the email. Is this something that is supported?
And: my client is concerned about email/spam regulations, which I'm very unfamiliar with. Is there anything I'd need to look out for in this regard?
