diff --git a/messenger.rst b/messenger.rst index db280b9cc4d..19572eb127e 100644 --- a/messenger.rst +++ b/messenger.rst @@ -623,6 +623,32 @@ config and start your workers: See the `Supervisor docs`_ for more details. + +.. tip:: + + To get a deterministic worker name like for e.g. the Redis transport, you may pass the process number as an + environment variable. + + .. code-block:: ini + + ;/etc/supervisor/conf.d/messenger-worker.conf + [program:messenger-consume] + […] + environment = REDIS_CONSUMER=consumer-%(process_num)d + + You can then use it in your messenger configuration: + + .. configuration-block:: + + .. code-block:: yaml + + # config/packages/messenger.yaml + framework: + messenger: + transports: + redis: 'redis://localhost:6379/messages/symfony/%env(REDIS_CONSUMER)%' + + .. _messenger-retries-failures: Retries & Failures @@ -900,6 +926,40 @@ serializer How to serialize the final payload ``Redis::SERIALIZER_PHP` ``Redis::OPT_SERIALIZER`` option) ================== =================================== ======= +.. tip:: + + At some point you'll likely want to scale the number of workers working on your queue. + Make sure you assign the correct ``consumer`` and ``group`` values in that case. + The more likely case is that you want every worker to work on the queue independently, reducing + the time needed to process the pending messages. In that case, every single worker + must have a different ``consumer`` option value so Redis can identify the different workers. + Your workers should thus be configured e.g. as follows: + + * Worker 1: redis://localhost:6379/messages/symfony/consumer-1 + * Worker 2: redis://localhost:6379/messages/symfony/consumer-2 + * Worker 3: redis://localhost:6379/messages/symfony/consumer-3 + * […] + + The less likely case would be if you wanted to have every single worker to process every single message. + That means messages would be processed multiple times. In that case, you must have different ``group`` + configurations: + + * Worker 1: redis://localhost:6379/messages/symfony-1/consumer + * Worker 2: redis://localhost:6379/messages/symfony-2/consumer + * Worker 3: redis://localhost:6379/messages/symfony-3/consumer + * […] + +.. caution:: + + Be careful about the ``consumer`` and ``group`` names you choose. It might be tempting to use e.g. the ``HOSTNAME`` + environment variable in orchestrated, containerized environments (Docker Swarm, Kubernetes, etc.) to get a unique + host name. However, they usually contain a random unique identifier which means if you destroy a container while it was + working on a message, this message will remain in pending state forever as it is very unlikely there's ever going + to be another worker with exactly the same ``HOSTNAME`` as the one you destroyed. In other words, you have to + make sure you're using deterministic values such as ``worker-1``, ``worker-2`` etc. + In case you are using Kubernetes to orchestrate your containers, consider using a ``StatefulSet`` rather than + a ``Deployment`` for example. + In Memory Transport ~~~~~~~~~~~~~~~~~~~
Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.
Alternative Proxies: