Skip to content

Add addCleanup to unittest.subTest #134079

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
encukou opened this issue May 16, 2025 · 11 comments
Open

Add addCleanup to unittest.subTest #134079

encukou opened this issue May 16, 2025 · 11 comments
Labels
stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@encukou
Copy link
Member

encukou commented May 16, 2025

Feature or enhancement

Proposal:

The unittest.subTest context (i.e. the object you get using with's as clause, currently None) should get methods to manage cleanups:

  • addCleanup
  • enterContext
  • doCleanups

And similar ones for IsolatedAsyncIOTestCase.

They should do the same thing as the same methods on TestCase, but with subtest scope.
Example usage:

for param in 'a', 'b', 'c':
    with self.subTest() as sub:
        tempfile = make_tempfile()
        sub.addCleanup(os.unlink, tempfile)
        do_actual_test(tempfile, param)

See Discuss thread for motivation/discussion.

Has this already been discussed elsewhere?

I have already discussed this feature proposal on Discourse

Links to previous discussion of this feature:

https://discuss.python.org/t/unittest-add-addcleanup-to-subtest/91827

Linked PRs

@encukou encukou added type-feature A feature request or enhancement stdlib Python modules in the Lib dir labels May 16, 2025
@encukou
Copy link
Member Author

encukou commented May 16, 2025

(If anyone wants to take it, go ahead! Otherwise I'll get to this eventually.)

@StanFromIreland
Copy link
Contributor

I can get to this in the evening:-) It's suprising that this was not implemented before!

@ArunRawat404
Copy link

If it's alright with you guys, I'd love to take on this issue.

@StanFromIreland
Copy link
Contributor

My implementation is not idea since _SubTest is a subclass and would therefore also expose asserts. I'd be interested to see what you have planned.

@ArunRawat404
Copy link

I'm thinking of creating a small helper class that only handles cleanup methods — addCleanup, enterContext, and doCleanups — without inheriting from TestCase. That way, it avoids exposing any extra functionality like assert* methods and keeps things more contained. What are your thoughts on this?

@StanFromIreland
Copy link
Contributor

I thought about that, but IMO it is pretty messy with a lot of duplicated code, so I don’t know. It is probably the only option.

@ArunRawat404
Copy link

Yes, it will be messy with duplicated code but I guess it's better than exposing unnecessary functionality. I also not able to think of any other way.

@ArunRawat404
Copy link

ArunRawat404 commented May 18, 2025

@encukou What are your thoughts on the implementation for this issue?
I’m considering creating a separate helper class _SubTestCleanupHelper with functionality similar to TestCase, but without exposing unnecessary features like _subTest.
The downside is that it would introduce some duplicated code.

Would love to hear your thoughts or suggestions.

@picnixz
Copy link
Member

picnixz commented May 18, 2025

My 2c:

  • Leave the interface as is and expose _SubTest as an opaque object with a documented interface. Let's not care about the extra methods that are added, let's only care about documenting those that should be public.
  • Use a proxy object with an overridden __getattr__ that only returns a subset of allowed methods. The rest would raise RuntimeError if they are inherited from TestCase and not overloaded. We shouldn't expect _SubTest to be a public class so we shouldn't worry about users subclassing this class and deal with this in our custom __getattr__.

The first solution would at least make any (CPython) code that is currently using _SubTest internally easier to maintain.

@encukou
Copy link
Member Author

encukou commented May 19, 2025

I think it's best to leave _SubTest for reporting, and add a new object for the context.

Don't worry about duplicated code until the solution is working and tested. Then it's time to deduplicate.

@serhiy-storchaka
Copy link
Member

Three ways to solve this issue was mentioned on the Discuss thread. With examples:

  1. Add methods to subTest.
for param in 'a', 'b', 'c':
    with self.subTest() as sub:
        tempfile = make_tempfile()
        sub.addCleanup(os.unlink, tempfile)
        do_actual_test(tempfile, param)

But we need to add asynchronous variants to subTest in IsolatedAsyncIOTestCase. This means two implementations of subTest.
It's unique advantage is that you can add callbacks for the outer scope and break strict LILO order. But I don't know how large a demand for it. The implementation will be complicated.

  1. Add new methods with special infix to TestCase.
for param in 'a', 'b', 'c':
    with self.subTest():
        tempfile = make_tempfile()
        self.addSubTestCleanup(os.unlink, tempfile)
        do_actual_test(tempfile, param)

It may be easier to implement for IsolatedAsyncIOTestCase.

  1. Add a fencing mechanism independent from subtests.
for param in 'a', 'b', 'c':
    with self.subTest(), self.fence():
        tempfile = make_tempfile()
        self.addCleanup(os.unlink, tempfile)
        do_actual_test(tempfile, param)

addCleanup(), doCleanups(), etc will only work within a fence. All callbacks added in a fence will be called after leaving a fence. It may help to solve also other issues.

But we will be able to fully understand the advantages and disadvantages only when we try to implement all three options.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stdlib Python modules in the Lib dir type-feature A feature request or enhancement
Projects
Status: Todo
Development

No branches or pull requests

5 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy