From cfd65b8d85cfd513344ff498c3468fb0a395a462 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Tue, 22 Jul 2025 18:25:23 -0600 Subject: [PATCH 01/14] Fix docs for Queue.shutdown --- Doc/library/queue.rst | 45 +++++++++++++++++++++++++++++-------------- 1 file changed, 31 insertions(+), 14 deletions(-) diff --git a/Doc/library/queue.rst b/Doc/library/queue.rst index fbbebcf4ed8f92..94fd36310382d1 100644 --- a/Doc/library/queue.rst +++ b/Doc/library/queue.rst @@ -187,9 +187,6 @@ fully processed by daemon consumer threads. processed (meaning that a :meth:`task_done` call was received for every item that had been :meth:`put` into the queue). - ``shutdown(immediate=True)`` calls :meth:`task_done` for each remaining item - in the queue. - Raises a :exc:`ValueError` if called more times than there were items placed in the queue. @@ -233,22 +230,42 @@ Example of how to wait for enqueued tasks to be completed:: Terminating queues ^^^^^^^^^^^^^^^^^^ -:class:`Queue` objects can be made to prevent further interaction by shutting -them down. +When no longer needed, :class:`Queue` objects can be wound down +or terminated immediately. .. method:: Queue.shutdown(immediate=False) - Shut down the queue, making :meth:`~Queue.get` and :meth:`~Queue.put` raise - :exc:`ShutDown`. + Put a :class:`Queue` instance into a shutdown mode. + + The queue can no longer grow. + Future calls to :meth:`~Queue.put` raise :exc:`ShutDown`. + Currently blocked callers of :meth:`~Queue.put` will be unblocked + and will raise :exc:`ShutDown` in the formerly blocked thread. + + If *immediate* is false (the default), the queue can be wound + down normally with calls :meth:`~Queue.get` to extract tasks + that have already been loaded. + + If the shutdown occurs during the brief window where the queue still + has data and callers to :meth:`~Queue.get` are blocked, those callers + will be unblocked. + + And if :meth:`~Queue.task_done` is called for each remaining task, a + pending :meth:`~Queue.join` will be unblocked normally. + + Once the queue is empty, future calls to :meth:`~Queue.get` will + raise :exc:`ShutDown`. - By default, :meth:`~Queue.get` on a shut down queue will only raise once the - queue is empty. Set *immediate* to true to make :meth:`~Queue.get` raise - immediately instead. + If *immediate* is true, the queue is terminated immediately. + The queue is drained to be completely empty. The count of + unfinished tasks is reduced to zero but without calling + :meth:`~Queue.task_done`. That then unblocks all callers of + :meth:`~Queue.join`. In addition, blocked callers of + :meth:`~Queue.get` are unblocked and will raise :exc:`ShutDown`. - All blocked callers of :meth:`~Queue.put` and :meth:`~Queue.get` will be - unblocked. If *immediate* is true, a task will be marked as done for each - remaining item in the queue, which may unblock callers of - :meth:`~Queue.join`. + Use caution when using :meth:`~Queue.join` with *immediate* set + to true. This unblocks the join even when no work has been done + on the tasks, violating the usual invariant for joining a queue. .. versionadded:: 3.13 From dc762d77e74fbece256432da875cc6ed86c3cc81 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Tue, 22 Jul 2025 18:37:14 -0600 Subject: [PATCH 02/14] Clarify adjustment to unfinished tasks --- Doc/library/queue.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/queue.rst b/Doc/library/queue.rst index 94fd36310382d1..04faa1b071d808 100644 --- a/Doc/library/queue.rst +++ b/Doc/library/queue.rst @@ -258,7 +258,7 @@ or terminated immediately. If *immediate* is true, the queue is terminated immediately. The queue is drained to be completely empty. The count of - unfinished tasks is reduced to zero but without calling + unfinished tasks is reduced by number drained but without calling :meth:`~Queue.task_done`. That then unblocks all callers of :meth:`~Queue.join`. In addition, blocked callers of :meth:`~Queue.get` are unblocked and will raise :exc:`ShutDown`. From 4e132c4695245ac55a2f72c61e685c9159ffe045 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Tue, 22 Jul 2025 18:41:07 -0600 Subject: [PATCH 03/14] Clarify that Queue.join is unblocked unconditionally --- Doc/library/queue.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/Doc/library/queue.rst b/Doc/library/queue.rst index 04faa1b071d808..be6668a41ce958 100644 --- a/Doc/library/queue.rst +++ b/Doc/library/queue.rst @@ -259,8 +259,9 @@ or terminated immediately. If *immediate* is true, the queue is terminated immediately. The queue is drained to be completely empty. The count of unfinished tasks is reduced by number drained but without calling - :meth:`~Queue.task_done`. That then unblocks all callers of - :meth:`~Queue.join`. In addition, blocked callers of + :meth:`~Queue.task_done`. All callers of :meth:`~Queue.join` + are unblocked even if the unfinished tasks is more than zero. + In addition, blocked callers of :meth:`~Queue.get` are unblocked and will raise :exc:`ShutDown`. Use caution when using :meth:`~Queue.join` with *immediate* set From 5a93c53f4be5ce4e903d569f3d1618f46dbf5388 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 24 Jul 2025 00:57:23 -0600 Subject: [PATCH 04/14] Clarify "wound down" --- Doc/library/queue.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/queue.rst b/Doc/library/queue.rst index be6668a41ce958..025feee61847f5 100644 --- a/Doc/library/queue.rst +++ b/Doc/library/queue.rst @@ -231,7 +231,7 @@ Terminating queues ^^^^^^^^^^^^^^^^^^ When no longer needed, :class:`Queue` objects can be wound down -or terminated immediately. +until empty or terminated immediately (a hard shutdown). .. method:: Queue.shutdown(immediate=False) @@ -258,7 +258,7 @@ or terminated immediately. If *immediate* is true, the queue is terminated immediately. The queue is drained to be completely empty. The count of - unfinished tasks is reduced by number drained but without calling + unfinished tasks is reduced by the number drained but without calling :meth:`~Queue.task_done`. All callers of :meth:`~Queue.join` are unblocked even if the unfinished tasks is more than zero. In addition, blocked callers of From 478cbab6be5db791187f7217eb773d3513745948 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 24 Jul 2025 01:09:01 -0600 Subject: [PATCH 05/14] Focus discussion on effects instead of process --- Doc/library/queue.rst | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/Doc/library/queue.rst b/Doc/library/queue.rst index 025feee61847f5..72e1e7791773a1 100644 --- a/Doc/library/queue.rst +++ b/Doc/library/queue.rst @@ -257,12 +257,11 @@ until empty or terminated immediately (a hard shutdown). raise :exc:`ShutDown`. If *immediate* is true, the queue is terminated immediately. - The queue is drained to be completely empty. The count of - unfinished tasks is reduced by the number drained but without calling - :meth:`~Queue.task_done`. All callers of :meth:`~Queue.join` - are unblocked even if the unfinished tasks is more than zero. - In addition, blocked callers of - :meth:`~Queue.get` are unblocked and will raise :exc:`ShutDown`. + The queue is drained to be completely empty. All callers of + :meth:`~Queue.join` are unblocked regardless of the number + of unfinished tasks. Blocked callers of :meth:`~Queue.get` + are unblocked and will raise :exc:`ShutDown` because the + queue is empty. Use caution when using :meth:`~Queue.join` with *immediate* set to true. This unblocks the join even when no work has been done From 2c20579edb8752d4ffd2ff462c8fdeb08e01e8bc Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 24 Jul 2025 01:10:42 -0600 Subject: [PATCH 06/14] Remove distraction about unblocking blocked gets for an empty queue --- Doc/library/queue.rst | 4 ---- 1 file changed, 4 deletions(-) diff --git a/Doc/library/queue.rst b/Doc/library/queue.rst index 72e1e7791773a1..597018b426ff73 100644 --- a/Doc/library/queue.rst +++ b/Doc/library/queue.rst @@ -246,10 +246,6 @@ until empty or terminated immediately (a hard shutdown). down normally with calls :meth:`~Queue.get` to extract tasks that have already been loaded. - If the shutdown occurs during the brief window where the queue still - has data and callers to :meth:`~Queue.get` are blocked, those callers - will be unblocked. - And if :meth:`~Queue.task_done` is called for each remaining task, a pending :meth:`~Queue.join` will be unblocked normally. From 4d14799b5bd6e1ba248d2819fadfb66e287171a0 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 24 Jul 2025 01:12:18 -0600 Subject: [PATCH 07/14] Update the docstring --- Lib/queue.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/queue.py b/Lib/queue.py index 25beb46e30d6bd..efb595fe03d08c 100644 --- a/Lib/queue.py +++ b/Lib/queue.py @@ -240,8 +240,8 @@ def shutdown(self, immediate=False): 'immediate' to True to make gets raise immediately instead. All blocked callers of put() and get() will be unblocked. If - 'immediate', a task is marked as done for each item remaining in - the queue, which may unblock callers of join(). + 'immediate', callers of join() are unblocked regardless of + the number of unfinished tasks. ''' with self.mutex: self.is_shutdown = True From 6740c3f808d89a49ac0cf51917637545c6fd8b9d Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 24 Jul 2025 01:18:17 -0600 Subject: [PATCH 08/14] Update asyncio docstring --- Lib/asyncio/queues.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/asyncio/queues.py b/Lib/asyncio/queues.py index 2f3865114a84f9..6b52bbdbc89de1 100644 --- a/Lib/asyncio/queues.py +++ b/Lib/asyncio/queues.py @@ -257,8 +257,8 @@ def shutdown(self, immediate=False): 'immediate' to True to make gets raise immediately instead. All blocked callers of put() and get() will be unblocked. If - 'immediate', a task is marked as done for each item remaining in - the queue, which may unblock callers of join(). + 'immediate', unblock callers of join() regardless of the + number of unfinished tasks. """ self._is_shutdown = True if immediate: From af1744e1956694dd8533909a67f037d5a0e88fbe Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 24 Jul 2025 01:22:09 -0600 Subject: [PATCH 09/14] Update asynico main docs --- Doc/library/asyncio-queue.rst | 37 +++++++++++++++++++++++------------ 1 file changed, 25 insertions(+), 12 deletions(-) diff --git a/Doc/library/asyncio-queue.rst b/Doc/library/asyncio-queue.rst index d99213aa81d53e..8954b7e655c7cd 100644 --- a/Doc/library/asyncio-queue.rst +++ b/Doc/library/asyncio-queue.rst @@ -102,17 +102,33 @@ Queue .. method:: shutdown(immediate=False) - Shut down the queue, making :meth:`~Queue.get` and :meth:`~Queue.put` - raise :exc:`QueueShutDown`. + Put a :class:`Queue` instance into a shutdown mode. - By default, :meth:`~Queue.get` on a shut down queue will only - raise once the queue is empty. Set *immediate* to true to make - :meth:`~Queue.get` raise immediately instead. + The queue can no longer grow. + Future calls to :meth:`~Queue.put` raise :exc:`ShutDown`. + Currently blocked callers of :meth:`~Queue.put` will be unblocked + and will raise :exc:`ShutDown` in the formerly blocked thread. - All blocked callers of :meth:`~Queue.put` and :meth:`~Queue.get` - will be unblocked. If *immediate* is true, a task will be marked - as done for each remaining item in the queue, which may unblock - callers of :meth:`~Queue.join`. + If *immediate* is false (the default), the queue can be wound + down normally with calls :meth:`~Queue.get` to extract tasks + that have already been loaded. + + And if :meth:`~Queue.task_done` is called for each remaining task, a + pending :meth:`~Queue.join` will be unblocked normally. + + Once the queue is empty, future calls to :meth:`~Queue.get` will + raise :exc:`ShutDown`. + + If *immediate* is true, the queue is terminated immediately. + The queue is drained to be completely empty. All callers of + :meth:`~Queue.join` are unblocked regardless of the number + of unfinished tasks. Blocked callers of :meth:`~Queue.get` + are unblocked and will raise :exc:`ShutDown` because the + queue is empty. + + Use caution when using :meth:`~Queue.join` with *immediate* set + to true. This unblocks the join even when no work has been done + on the tasks, violating the usual invariant for joining a queue. .. versionadded:: 3.13 @@ -129,9 +145,6 @@ Queue call was received for every item that had been :meth:`~Queue.put` into the queue). - ``shutdown(immediate=True)`` calls :meth:`task_done` for each - remaining item in the queue. - Raises :exc:`ValueError` if called more times than there were items placed in the queue. From 12d1ea96edf826373a5a759cc86160a58464e1c0 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 24 Jul 2025 03:19:27 -0600 Subject: [PATCH 10/14] Match the exception used by asyncio --- Doc/library/asyncio-queue.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/library/asyncio-queue.rst b/Doc/library/asyncio-queue.rst index 8954b7e655c7cd..dbeb1ad5778681 100644 --- a/Doc/library/asyncio-queue.rst +++ b/Doc/library/asyncio-queue.rst @@ -105,9 +105,9 @@ Queue Put a :class:`Queue` instance into a shutdown mode. The queue can no longer grow. - Future calls to :meth:`~Queue.put` raise :exc:`ShutDown`. + Future calls to :meth:`~Queue.put` raise :exc:`QueueShutDown`. Currently blocked callers of :meth:`~Queue.put` will be unblocked - and will raise :exc:`ShutDown` in the formerly blocked thread. + and will raise :exc:`QueueShutDown` in the formerly blocked thread. If *immediate* is false (the default), the queue can be wound down normally with calls :meth:`~Queue.get` to extract tasks @@ -117,13 +117,13 @@ Queue pending :meth:`~Queue.join` will be unblocked normally. Once the queue is empty, future calls to :meth:`~Queue.get` will - raise :exc:`ShutDown`. + raise :exc:`QueueShutDown`. If *immediate* is true, the queue is terminated immediately. The queue is drained to be completely empty. All callers of :meth:`~Queue.join` are unblocked regardless of the number of unfinished tasks. Blocked callers of :meth:`~Queue.get` - are unblocked and will raise :exc:`ShutDown` because the + are unblocked and will raise :exc:`QueueShutDown` because the queue is empty. Use caution when using :meth:`~Queue.join` with *immediate* set From 4620deb0e75d0088e87f7772e10bb2186a4c7b09 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 24 Jul 2025 03:31:12 -0600 Subject: [PATCH 11/14] Update task_done docstrings --- Lib/asyncio/queues.py | 3 --- Lib/queue.py | 3 --- 2 files changed, 6 deletions(-) diff --git a/Lib/asyncio/queues.py b/Lib/asyncio/queues.py index 6b52bbdbc89de1..e5d6f2e4b61e17 100644 --- a/Lib/asyncio/queues.py +++ b/Lib/asyncio/queues.py @@ -227,9 +227,6 @@ def task_done(self): been processed (meaning that a task_done() call was received for every item that had been put() into the queue). - shutdown(immediate=True) calls task_done() for each remaining item in - the queue. - Raises ValueError if called more times than there were items placed in the queue. """ diff --git a/Lib/queue.py b/Lib/queue.py index efb595fe03d08c..c90de8edc76c34 100644 --- a/Lib/queue.py +++ b/Lib/queue.py @@ -80,9 +80,6 @@ def task_done(self): have been processed (meaning that a task_done() call was received for every item that had been put() into the queue). - shutdown(immediate=True) calls task_done() for each remaining item in - the queue. - Raises a ValueError if called more times than there were items placed in the queue. ''' From 2ab910ed6711df65db68bc7d65fc730b2d1079e7 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 24 Jul 2025 10:17:02 -0600 Subject: [PATCH 12/14] Fix word order --- Doc/library/asyncio-queue.rst | 2 +- Doc/library/queue.rst | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/asyncio-queue.rst b/Doc/library/asyncio-queue.rst index dbeb1ad5778681..963bc1fb82c12f 100644 --- a/Doc/library/asyncio-queue.rst +++ b/Doc/library/asyncio-queue.rst @@ -110,7 +110,7 @@ Queue and will raise :exc:`QueueShutDown` in the formerly blocked thread. If *immediate* is false (the default), the queue can be wound - down normally with calls :meth:`~Queue.get` to extract tasks + down normally with :meth:`~Queue.get` calls to extract tasks that have already been loaded. And if :meth:`~Queue.task_done` is called for each remaining task, a diff --git a/Doc/library/queue.rst b/Doc/library/queue.rst index 597018b426ff73..9c32d95b44d07b 100644 --- a/Doc/library/queue.rst +++ b/Doc/library/queue.rst @@ -243,7 +243,7 @@ until empty or terminated immediately (a hard shutdown). and will raise :exc:`ShutDown` in the formerly blocked thread. If *immediate* is false (the default), the queue can be wound - down normally with calls :meth:`~Queue.get` to extract tasks + down normally with :meth:`~Queue.get` calls to extract tasks that have already been loaded. And if :meth:`~Queue.task_done` is called for each remaining task, a From 7ad150b7c244ffed322a0983382cfd9add327db0 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 24 Jul 2025 10:48:48 -0600 Subject: [PATCH 13/14] Add missing subsection header --- Doc/library/queue.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/Doc/library/queue.rst b/Doc/library/queue.rst index 9c32d95b44d07b..8202abdd3c1351 100644 --- a/Doc/library/queue.rst +++ b/Doc/library/queue.rst @@ -201,6 +201,9 @@ fully processed by daemon consumer threads. count of unfinished tasks drops to zero, :meth:`join` unblocks. +Waiting for task completion +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + Example of how to wait for enqueued tasks to be completed:: import threading From 7de4f73196c29581a9efc0181cf163c0a485d51b Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 24 Jul 2025 10:53:10 -0600 Subject: [PATCH 14/14] Replace parenthetical with a prepositional phrase. --- Doc/library/queue.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/queue.rst b/Doc/library/queue.rst index 8202abdd3c1351..6dcf06aab00295 100644 --- a/Doc/library/queue.rst +++ b/Doc/library/queue.rst @@ -234,7 +234,7 @@ Terminating queues ^^^^^^^^^^^^^^^^^^ When no longer needed, :class:`Queue` objects can be wound down -until empty or terminated immediately (a hard shutdown). +until empty or terminated immediately with a hard shutdown. .. method:: Queue.shutdown(immediate=False) pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy