Have you experienced EFS timeouts during bulk emailing?

[oytuntez]

Hi team and @heathdutton,

This is not about your repository, but we have quite a similar setup for Mautic. We are using file spool and we are experiencing not only a horrible performance, but also intermittent timeouts especially with a larger queue (20K, 30K messages in spool directory). EFS already takes at least 10 seconds to list the directory, which I believe is causing 451 timeout in the SMTP protocol.

I hate both file and memory spools, how are you managing this? I am going to experiment with some Redis hack to force Swiftmailer to use Redis as spool, but first, I have 30K emails stuck in the queue at the moment...

Any insights are welcome.

[heathdutton]

EFS for email spool is really just a stop-gap to get you rolling with an initial setup. I wouldn't recommend that at all for long-term use. We typically have very little in the spool because we're offloading as much as possible. We've got SES for mautic direct (forgotten password, user stuff), and Mailgun, Sendgrid and others for the campaign messaging using the "Contact Client" plugin. Anyway... I recommend using API based mail services as much as possible. With your volume, you need to avoid SMTP like the plague.

[oytuntez]

You have some amazing plugins under DMS which I am diving into soon!

We are using Mautic for transactional emails as well, and I want to keep using SES. How would you integrate this? Instead of using queue for emails, should I simply pick immediate?

I've been waiting for this (currently no bandwidth to undertake the PR myself): mautic/mautic#6977

Yesterday night, I was hard debugging and found 2 performance issues with both Mautic and Swiftmailer. They both used DirectoryIterator which caused bottleneck in NFS environments. Using opendir + readdir solves the issue to a big extend.

Now finalizing these findings and fixes on my local and staging envs.

[oytuntez]

Here is the issue I opened on Swiftmailer: swiftmailer/swiftmailer#1241