Skip to content

DEV: stop monkeypatching and use of autouse=True fixtures by default in test suite #29090

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
rgommers opened this issue May 30, 2025 · 2 comments

Comments

@rgommers
Copy link
Member

rgommers commented May 30, 2025

tl;dr there is way too much messing around with tests, warnings, fixtures, environment variables, etc. going on in numpy/conftest.py, we should stop doing that.

I started looking at this because in the process of trying to use pytest-run-parallel, I found that every single test in our test suite uses the monkeypatch fixture. This is enabled here:

numpy/numpy/conftest.py

Lines 150 to 152 in 7d2e441

@pytest.fixture(autouse=True)
def env_setup(monkeypatch):
monkeypatch.setenv('PYTHONHASHSEED', '0')

The use of PYTHONHASHSEED=0 was originally introduced for gh-2939, which turned out to be a SciPy rather than an f2py bug. It seems like a bad idea to set such env vars to non-default values in general - even in that original issue, the hash randomization actually found a bug, so disabling it should have been a temporary workaround. There are multiple ways this can affect other tools and test scenarios, e.g.:

  • pytest-leaks doesn't work with it, and we document here that to use pytest-leaks effectively, this fixture and a doctest fixture should be disabled by hand in conftest.py`
  • pytest-run-paralllel is simply rendered ineffective; it disables running any test in parallel when the thread-unsafe monkeypatch fixture is used
  • hypothesis: it seems like turning off hash randomization is a bad default for it, at least that's what I deduce from TST: Add the first test using hypothesis #14440 (comment). The custom hypothesis profiles we have should be working around this already, but it still illustrates the problem.

There are other similar issues with the other fixtures (e.g., the warnings module is also thread-unsafe currently). We should clean this up and have saner defaults, then enable fixtures only when they are actually needed. E.g., all the mess we need for doctesting needs to only be turned on if doctests are actually requested.

The last thing to look at is the FPU checks. There are still two different ways of handling that, the comments on the oldest way say that it's to deal with tests that use yield. We still have a whole bunch of those, so that isn't easy to clean up - but it'd be nice to do that at some point. It looks like this is also a case where this check should be enabled in CI rather than disabled by default though, since there is only a single issues containing the string "FPU precision mode changed" (gh-23545), so this pretty invasive check catches very little.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy