Skip to content

BUG: pip install fails on ppc64le after VSX4 introducion #25436

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
cdeepali opened this issue Dec 20, 2023 · 6 comments
Closed

BUG: pip install fails on ppc64le after VSX4 introducion #25436

cdeepali opened this issue Dec 20, 2023 · 6 comments
Assignees
@seiko2plus
Labels
00 - Bug 32 - Installation Problems installing or compiling NumPy component: SIMD Issues in SIMD (fast instruction sets) code or machinery

Comments

@cdeepali
Copy link

cdeepali commented Dec 20, 2023 •
edited
Loading

Describe the issue:

Installation using pip fails on ppc64le.

Reproduce the code example:

conda create -y -n buildenvgxx11 gxx_linux-ppc64le
conda activate buildenvgxx11
pip install numpy==1.23.5

Error message:

INFO: compile options: '-DNPY_INTERNAL_BUILD=1 -DHAVE_NPY_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -Ibuild/src.linux-ppc64le-3.10/numpy/core/src/multiarray -Ibuild/src.linux-ppc64le-3.10/numpy/core/src/common -Ibuild/src.linux-ppc64le-3.10/numpy/core/src/umath -Inumpy/core/include -Ibuild/src.linux-ppc64le-3.10/numpy/core/include/numpy -Ibuild/src.linux-ppc64le-3.10/numpy/distutils/include -Ibuild/src.linux-ppc64le-3.10/numpy/core/src/npysort -Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/src/_simd -I/opt/conda/include/python3.10 -Ibuild/src.linux-ppc64le-3.10/numpy/core/src/common -Ibuild/src.linux-ppc64le-3.10/numpy/core/src/npymath -c'
      extra options: '-O3 -mcpu=power10 -mtune=power10'
      INFO: powerpc64le-conda-linux-gnu-cc: build/src.linux-ppc64le-3.10/numpy/core/src/umath/loops_arithmetic.dispatch.vsx4.c
      INFO: powerpc64le-conda-linux-gnu-cc: build/src.linux-ppc64le-3.10/numpy/core/src/umath/loops_trigonometric.dispatch.vsx4.c
      INFO: powerpc64le-conda-linux-gnu-cc: build/src.linux-ppc64le-3.10/numpy/core/src/umath/loops_hyperbolic.dispatch.vsx4.c
      during RTL pass: expand
      In file included from build/src.linux-ppc64le-3.10/numpy/core/src/umath/loops_hyperbolic.dispatch.vsx4.c:11:
      numpy/core/src/umath/loops_hyperbolic.dispatch.c.src: In function 'FLOAT_tanh_VSX4':
      numpy/core/src/umath/loops_hyperbolic.dispatch.c.src:374:9: internal compiler error: in rs6000_sibcall_aix, at config/rs6000/rs6000.c:25670
        374 |         npy_clear_floatstatus_barrier((char*)dimensions);
            |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      Please submit a full bug report,
      with preprocessed source if appropriate.
      See <https://gcc.gnu.org/bugs/> for instructions.
      during RTL pass: expand
      In file included from build/src.linux-ppc64le-3.10/numpy/core/src/umath/loops_arithmetic.dispatch.vsx4.c:11:
      numpy/core/src/umath/loops_arithmetic.dispatch.c.src: In function 'BYTE_divide_VSX4':
      numpy/core/src/umath/loops_arithmetic.dispatch.c.src:76:13: internal compiler error: in rs6000_sibcall_aix, at config/rs6000/rs6000.c:25670
         76 |             npy_set_floatstatus_overflow();
            |             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      Please submit a full bug report,
      with preprocessed source if appropriate.
      See <https://gcc.gnu.org/bugs/> for instructions.
      error: Command "<myenv>buildenvgxx11/bin/powerpc64le-conda-linux-gnu-cc -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O3 -Wall -mcpu=power8 -fPIC -O3 -isystem /opt/conda/include -mcpu=power8 -fPIC -O3 -isystem /opt/conda/include -mcpu=power8 -mtune=power8 -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O3 -pipe -isystem <myenv>buildenvgxx11/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem <myenv>buildenvgxx11/include -fPIC -DNPY_INTERNAL_BUILD=1 -DHAVE_NPY_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -Ibuild/src.linux-ppc64le-3.10/numpy/core/src/multiarray -Ibuild/src.linux-ppc64le-3.10/numpy/core/src/common -Ibuild/src.linux-ppc64le-3.10/numpy/core/src/umath -Inumpy/core/include -Ibuild/src.linux-ppc64le-3.10/numpy/core/include/numpy -Ibuild/src.linux-ppc64le-3.10/numpy/distutils/include -Ibuild/src.linux-ppc64le-3.10/numpy/core/src/npysort -Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/src/_simd -I/opt/conda/include/python3.10 -Ibuild/src.linux-ppc64le-3.10/numpy/core/src/common -Ibuild/src.linux-ppc64le-3.10/numpy/core/src/npymath -c build/src.linux-ppc64le-3.10/numpy/core/src/umath/loops_arithmetic.dispatch.vsx4.c -o build/temp.linux-ppc64le-3.10/build/src.linux-ppc64le-3.10/numpy/core/src/umath/loops_arithmetic.dispatch.vsx4.o -MMD -MF build/temp.linux-ppc64le-3.10/build/src.linux-ppc64le-3.10/numpy/core/src/umath/loops_arithmetic.dispatch.vsx4.o.d -O3 -mcpu=power10 -mtune=power10" failed with exit status 1
      
      

      ########### EXT COMPILER OPTIMIZATION ###########
      INFO: Platform      :
        Architecture: ppc64le
        Compiler    : unix-like

      CPU baseline  :
        Requested   : 'min'
        Enabled     : VSX VSX2
        Flags       : -mcpu=power8
        Extra checks: VSX_ASM

      CPU dispatch  :
        Requested   : 'max -xop -fma4'
        Enabled     : VSX3 VSX4
        Generated   :
                    :
        VSX3        : VSX VSX2
        Flags       : -mcpu=power9 -mtune=power9
        Extra checks: none
        Detect      : VSX3
                    : build/src.linux-ppc64le-3.10/numpy/core/src/umath/loops_trigonometric.dispatch.c
                    :
        VSX4        : VSX VSX2 VSX3
        Flags       : -mcpu=power10 -mtune=power10
        Extra checks: VSX4_MMA
        Detect      : VSX4
                    : build/src.linux-ppc64le-3.10/numpy/core/src/umath/loops_arithmetic.dispatch.c
                    : build/src.linux-ppc64le-3.10/numpy/core/src/umath/loops_trigonometric.dispatch.c
                    : build/src.linux-ppc64le-3.10/numpy/core/src/umath/loops_hyperbolic.dispatch.c
                    : build/src.linux-ppc64le-3.10/numpy/core/src/umath/loops_modulo.dispatch.c
      INFO: CCompilerOpt.cache_flush[857] : write cache to path -> /tmp/pip-install-kw88t5sf/numpy_228008dbdfce4094a9475c5197e2ff88/build/temp.linux-ppc64le-3.10/ccompiler_opt_cache_ext.py
      INFO:
      ########### CLIB COMPILER OPTIMIZATION ###########
      INFO: Platform      :
        Architecture: ppc64le
        Compiler    : unix-like

      CPU baseline  :
        Requested   : 'min'
        Enabled     : VSX VSX2
        Flags       : -mcpu=power8
        Extra checks: VSX_ASM

      CPU dispatch  :
        Requested   : 'max -xop -fma4'
        Enabled     : VSX3 VSX4
        Generated   : none
      INFO: CCompilerOpt.cache_flush[857] : write cache to path -> /tmp/pip-install-kw88t5sf/numpy_228008dbdfce4094a9475c5197e2ff88/build/temp.linux-ppc64le-3.10/ccompiler_opt_cache_clib.py
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for numpy

Python and NumPy Versions:

Numpy: 1.23.5, 1.26.2
GCC: Anaconda GCC 11.2

Runtime Environment:

No response

Context for the issue:

Unable to build Tensorflow on ppc64le - tensorflow/tensorflow#62659
@rgommers
Copy link
Member

Thanks for the report @cdeepali. Could you try a newer version to see if it's fixed? The last release which used numpy.distutils as the build system is 1.25.2, and the latest release (which uses Meson as the build system) is 1.26.2. Those are the two versions worth trying.

I realize that that doesn't immediately solves your issue if TensorFlow pins to 1.23.5, but things should work in general on ppc64le for us, and if the most recent versions build you can exclude an issue in your environment being the root cause.

@rgommers rgommers added the 32 - Installation Problems installing or compiling NumPy label Dec 20, 2023
@cdeepali
Copy link
Author

I see this issue with versions 1.25.2 and 1.26.2 as well. Seems to be caused by the addition of VSX4 (Power10) support. - #20821

@rgommers rgommers changed the title BUG: pip install fails on ppc64le BUG: pip install fails on ppc64le after VSX4 introducion Dec 21, 2023
@rgommers
Copy link
Member

Thanks for identifying the root cause @cdeepali. I think @seiko2plus may be able to help here.

@seiko2plus seiko2plus self-assigned this Dec 23, 2023
@seiko2plus seiko2plus added the component: SIMD Issues in SIMD (fast instruction sets) code or machinery label Dec 23, 2023
@cdeepali
Copy link
Author

cdeepali commented Jan 2, 2024

@seiko2plus, any suggestions on the above issue.

@seiko2plus
Copy link
Member

seiko2plus commented Jan 8, 2024 •
edited
Loading

It's a bit ambiguous build error, I'm trying to create a minimal producer to report upstream, however, the issue seems related to the gcc flag "no-plt".
Anaconda exports the following value of CFLAGS:

export CFLAGS="-mcpu=power8 -mtune=power8 -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O3 -pipe -isystem /root/anaconda3/include"

Try to remove -fno-plt and rebuild e.g.:

export CFLAGS="-mcpu=power8 -mtune=power8 -ftree-vectorize -fPIC -fstack-protector-strong -O3 -pipe -isystem /root/anaconda3/include"

@rgommers
Copy link
Member

Conda compiler activation strikes again. The only necessary flag is -isystem /root/anaconda3/include, the rest is aimed for conda-build usage during building of conda packages, and would ideally not be present.

I think we can close this as explained/resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@seiko2plus seiko2plus

Labels
00 - Bug 32 - Installation Problems installing or compiling NumPy component: SIMD Issues in SIMD (fast instruction sets) code or machinery
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@rgommers @seiko2plus @cdeepali
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy