Skip to content

Regression of 3.13.1 with iterator creation being duplicated #127682

@kayhayen

Description

@kayhayen

Bug report

Bug description:

For my Python compiler Nuitka, I use CPython as the oracle of what the correct behaviour is. I am running some tests that I used to clarify the behavior from decades ago, in this case I wanted to know when exactly the iterator creation is used. I striped this test a bunch, so the regression is still visible. I first noticed the issue on GitHub Actions, where 3.13.0 got replaced with 3.13.1 for Windows and Linux, but it applies to all OSes. See below for a diff, that the same iterator is created multiple times.

""" Generator expression tests

"""

from __future__ import print_function

import inspect

print("Generator expression that demonstrates the timing:")


def iteratorCreationTiming():
    def getIterable(x):
        print("Getting iterable", x)
        return Iterable(x)

    class Iterable:
        def __init__(self, x):
            self.x = x  # pylint: disable=invalid-name
            self.values = list(range(x))
            self.count = 0

        def __iter__(self):
            print("Giving iterator now", self.x)

            return self

        def __next__(self):
            print("Next of", self.x, "is", self.count)

            if len(self.values) > self.count:
                self.count += 1

                return self.values[self.count - 1]
            else:
                print("Raising StopIteration for", self.x)

                raise StopIteration

        # Python2/3 compatibility.
        next = __next__

        def __del__(self):
            print("Deleting", self.x)

    gen = ((y, z) for y in getIterable(3) for z in getIterable(2))

    print("next value is", next(gen))
    res = tuple(gen)
    print("remaining generator is", res)

    try:
        next(gen)
    except StopIteration:
        print("Usage past end gave StopIteration exception as expected.")

        try:
            print("Generator state then is", inspect.getgeneratorstate(gen))
        except AttributeError:
            pass

        print("Its frame is now", gen.gi_frame)

    print("Early aborting generator:")

    gen2 = ((y, z) for y in getIterable(3) for z in getIterable(2))
    del gen2

iteratorCreationTiming()

The unified diff between 3.13.0 output (and basically all Python versions before) and 3.13.1 output.


--- out-3.13.0.txt      2024-12-06 12:37:19.447115100 +0100
+++ out-3.13.1.txt      2024-12-06 12:37:23.452239500 +0100
@@ -1,9 +1,11 @@
 Generator expression that demonstrates the timing:
 Getting iterable 3
 Giving iterator now 3
+Giving iterator now 3
 Next of 3 is 0
 Getting iterable 2
 Giving iterator now 2
+Giving iterator now 2
 Next of 2 is 0
 next value is (0, 0)
 Next of 2 is 1
@@ -13,6 +15,7 @@
 Next of 3 is 1
 Getting iterable 2
 Giving iterator now 2
+Giving iterator now 2
 Next of 2 is 0
 Next of 2 is 1
 Next of 2 is 2
@@ -21,6 +24,7 @@
 Next of 3 is 2
 Getting iterable 2
 Giving iterator now 2
+Giving iterator now 2
 Next of 2 is 0

The duplicated prints out the iterator creation are new. This is not optimal and new. I don't know if the iterator being through a slot cause cause this or what it is. I checked if generator.c changed but I think it didn't at all.

My self compiled Python 3.13.1 for Linux and the official Windows download agree in behaviour.

CPython versions tested on:

3.13

Operating systems tested on:

Linux, Windows

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    interpreter-core(Objects, Python, Grammar, and Parser dirs)type-bugAn unexpected behavior, bug, or error

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      pFad - Phonifier reborn

      Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

      Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


      Alternative Proxies:

      Alternative Proxy

      pFad Proxy

      pFad v3 Proxy

      pFad v4 Proxy