[Mailer] Memory leaking when sending many emails #45211
Description
Symfony version(s) affected
5.3, 5.4
Description
This problem I had before (https://stackoverflow.com/questions/67248353/symfony-5-memory-leaking-when-send-multiple-emails-through-mailer) but it fixed after upgrading Symfony.
Now I started to have again problems related mailer when sending many emails through command.
Code inside execute method of command:
// Total Users
$totalRecords = $this->getUserRepository()->count(['newsletter' => true]);
if ($totalRecords === 0) {
$output->writeln('No records found');
return Command::SUCCESS;
}
$offers = $this->fetchOffersByType($type);
$totalOffers = count($offers);
// Check if we have popular offers
if ($totalOffers === 0) {
$output->writeln('No Offers was found');
return Command::SUCCESS;
}
$totalPages = ceil($totalRecords / self::BUFFER_SIZE);
// Initializing one time and assign to users
$newsletter = (new Newsletter())
->setSentAt(new DateTime())
->setType(NewsletterType::MAP[$type]);
$totalSuccessSent = 0;
$total = 0;
for ($page = 1; $page <= $totalPages; $page++) {
// Get users to who we will send newsletters
$users = $this->getUserRepository()
->findBy(['newsletter' => true], null, self::BUFFER_SIZE, self::BUFFER_SIZE * ($page - 1));
foreach ($users as $user) {
$total++;
if (empty($user->getEmail())) {
continue;
}
if ($this->emailService->sendNewsletter($user, $type, $offers)) {
$user->addNewsletter($newsletter);
$this->em->persist($user);
$totalSuccessSent++;
}
}
$this->em->flush();
// Make clean up after specific number of users
if ($total % self::CLEAN_UP_AFTER === 0) {
$output->writeln('Clean Up');
$this->em->clear();
gc_collect_cycles();
}
}And here is the piece of method sendNewsletter:
try {
$email = (new TemplatedEmail())
->from(new Address($this->parameterBag->get('noReplayEmail'), $this->parameterBag->get('noReplayEmailName')))
->to($user->getEmail())
->priority(Email::PRIORITY_NORMAL)
->subject($subject)
->htmlTemplate($template)
->context([
'offers' => $offers,
]);
$this->mailer->send($email);
return true;
} catch (TransportExceptionInterface | RfcComplianceException | JsonException $e) {
return false;
}If to comment $this->mailer->send($email) no problem at all. For testing I'm using dsn: null://null
Upgrading / downgrading of symfony not helped me.
I'm using right now Symfony 5.4 and php 7.4
Note: Memory limit that is used for command is 512 MB.
How to reproduce
Just create a command send many emails and on each sent email track the memory peak and you will see how it's increasing.
Possible Solution
After investigation and debugging I found that so called MessageLoggerListener (vendor/symfony/mailer/EventListener/MessageLoggerListener.php) is collecting logs but don't reset the logs.
So I decorated this listener and just reset after ever 50 events. I don't like this solution so for this reason I wrote here, maybe there is better solution.
Here is the code:
<?php
declare(strict_types=1);
namespace App\EventSubscriber;
use Symfony\Component\EventDispatcher\EventSubscriberInterface;
use Symfony\Component\Mailer\Event\MessageEvent;
use Symfony\Component\Mailer\EventListener\MessageLoggerListener;
/**
* Decorating the listener that save logs related to message events but it not reset the data
* and this creates memory leaking.
*
* Temporary solution.
*/
class MessageLoggerListenerDecorator implements EventSubscriberInterface
{
private const BUFFER_LOG = 50;
private MessageLoggerListener $messageLoggerListener;
public function __construct(MessageLoggerListener $messageLoggerListener)
{
$this->messageLoggerListener = $messageLoggerListener;
}
/**
* @inheritDoc
*/
public static function getSubscribedEvents(): array
{
return [
MessageEvent::class => ['onMessage', -255],
];
}
public function onMessage(MessageEvent $event): void
{
if (count($this->messageLoggerListener->getEvents()->getEvents()) >= self::BUFFER_LOG) {
$this->messageLoggerListener->reset();
}
$this->messageLoggerListener->onMessage($event);
}
}And service.yml:
App\EventSubscriber\MessageLoggerListenerDecorator:
decorates: mailer.message_logger_listener
arguments: ['@.inner']Additional Context
No response