DEV: stop monkeypatching and use of autouse=True
fixtures by default in test suite
#29090
Labels
autouse=True
fixtures by default in test suite
#29090
Uh oh!
There was an error while loading. Please reload this page.
tl;dr there is way too much messing around with tests, warnings, fixtures, environment variables, etc. going on in numpy/conftest.py, we should stop doing that.
I started looking at this because in the process of trying to use
pytest-run-parallel
, I found that every single test in our test suite uses themonkeypatch
fixture. This is enabled here:numpy/numpy/conftest.py
Lines 150 to 152 in 7d2e441
The use of
PYTHONHASHSEED=0
was originally introduced for gh-2939, which turned out to be a SciPy rather than anf2py
bug. It seems like a bad idea to set such env vars to non-default values in general - even in that original issue, the hash randomization actually found a bug, so disabling it should have been a temporary workaround. There are multiple ways this can affect other tools and test scenarios, e.g.:pytest-leaks
doesn't work with it, and we document here that to usepytest-leaks effectively, this fixture and a doctest fixture should be disabled by hand in
conftest.py`pytest-run-paralllel
is simply rendered ineffective; it disables running any test in parallel when the thread-unsafemonkeypatch
fixture is usedhypothesis
: it seems like turning off hash randomization is a bad default for it, at least that's what I deduce from TST: Add the first test using hypothesis #14440 (comment). The customhypothesis
profiles we have should be working around this already, but it still illustrates the problem.There are other similar issues with the other fixtures (e.g., the
warnings
module is also thread-unsafe currently). We should clean this up and have saner defaults, then enable fixtures only when they are actually needed. E.g., all the mess we need for doctesting needs to only be turned on if doctests are actually requested.The last thing to look at is the FPU checks. There are still two different ways of handling that, the comments on the oldest way say that it's to deal with tests that use
yield
. We still have a whole bunch of those, so that isn't easy to clean up - but it'd be nice to do that at some point. It looks like this is also a case where this check should be enabled in CI rather than disabled by default though, since there is only a single issues containing the string "FPU precision mode changed" (gh-23545), so this pretty invasive check catches very little.The text was updated successfully, but these errors were encountered: