Tox does not test the installed extension module #326
Description
When running tests with tox, this happens:
- tox creates an sdist of current state of python-ldap in the working directory
- tox install (dependencies and) that sdist into the virtualenv, buulding the extension module for that Python version
- tox calls
setup.py testand that builds the extension module again, testing the module from local build dir instead of the installed one
$ tox -e py38
GLOB sdist-make: .../python-ldap/setup.py
py38 inst-nodeps: .../python-ldap/.tox/.tmp/package/1/python-ldap-3.2.0.zip
py38 installed: coverage==5.0.4,pyasn1==0.4.8,pyasn1-modules==0.2.8,python-ldap==3.2.0
...
py38 run-test: commands[0] | .../python-ldap/.tox/py38/bin/python -bb -Werror '-Wignore:the imp module is deprecated:DeprecationWarning' '-Wignore:the imp module is deprecated:PendingDeprecationWarning' '-Wignore:Using or importing the ABCs from '"'"'collections'"'"' instead of from '"'"'collections.abc'"'"' is deprecated, and in 3.8 it will stop working:DeprecationWarning' -m coverage run --parallel setup.py clean --all test
running clean
'build/lib.linux-x86_64-3.8' does not exist -- can't clean it
'build/bdist.linux-x86_64' does not exist -- can't clean it
'build/scripts-3.8' does not exist -- can't clean it
running test
WARNING: Testing via this command is deprecated and will be removed in a future version. Users looking for a generic test entry point independent of test runner are encouraged to use tox.
running egg_info
writing Lib/python_ldap.egg-info/PKG-INFO
writing dependency_links to Lib/python_ldap.egg-info/dependency_links.txt
writing requirements to Lib/python_ldap.egg-info/requires.txt
writing top-level names to Lib/python_ldap.egg-info/top_level.txt
reading manifest file 'Lib/python_ldap.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
no previously-included directories found matching 'Doc/.build'
writing manifest file 'Lib/python_ldap.egg-info/SOURCES.txt'
running build_ext
building '_ldap' extension
creating build
creating build/temp.linux-x86_64-3.8
creating build/temp.linux-x86_64-3.8/Modules
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DHAVE_SASL -DHAVE_TLS -DHAVE_LIBLDAP_R -DHAVE_LIBLDAP_R -DLDAPMODULE_VERSION=3.2.0 -DLDAPMODULE_AUTHOR=python-ldap project -DLDAPMODULE_LICENSE=Python style -IModules -I/usr/include/python3.8 -c Modules/LDAPObject.c -o build/temp.linux-x86_64-3.8/Modules/LDAPObject.o
...
copying build/lib.linux-x86_64-3.8/_ldap.cpython-38-x86_64-linux-gnu.so -> Lib
test_cafile (Tests.t_ldap_options.TestGlobalOptions) ... ok
...
Adding this to a test file:
import _ldap
print(_ldap.__file__)yields .../python-ldap/Lib/_ldap.cpython-38-x86_64-linux-gnu.so instead of .../.tox/py38/lib/python3.8/site-packages/_ldap.cpython-38-x86_64-linux-gnu.so
This is problematic for at least 3 reasons:
- tox makes it simple to test the installed package instead of the stuff from PWD, while here we possibly completely bypass that feature - if the installation breaks, we'll never notice
- the code is built twice, which adds overhead
- when I attempt to run tox-current-env in Fedora during RPM package build to actually test the module that that has just been built, the module is built again, possibly with a different set of CFLAGS and we test stuff that we are not shipping
Note 1: Removing the clean call from setup.py clean --all test is not enough to solve this issue.
Note 2: The setup.py test generates this:
WARNING: Testing via this command is deprecated and will be removed in a future version. Users looking for a generic test entry point independent of test runner are encouraged to use tox.
Please consider calling the test runner directly from tox.ini to solve this issue.