Skip to content

Support for Python 3.11? #1452

New issue
New issue
Closed
Closed
Support for Python 3.11?#1452
Labels
investigate furtherIt's unclear what the issue is at this time but there is enough interest to look into it
@thibaultbetremieux

Description

@thibaultbetremieux
thibaultbetremieux
opened on Oct 25, 2022

Description

I tried installing confluent-kafka inside a conda environment with python=3.11. It did not work. It was working previously with python=3.10.6 and I'd like to use some features from 3.11 🙂 .

How to reproduce

  1. Create conda environment with Python 3.11 (enter the commands one by one and accept prompts, this is not a script):
conda create -n test_confluent_kafka
conda activate test_confluent_kafka
conda install python=3.11 -c conda-forge
  1. Attempt to install confluent-kafka inside the enviromnent
pip install confluent-kafka
Output Collecting confluent-kafka Using cached confluent-kafka-1.9.2.tar.gz (109 kB) Preparing metadata (setup.py) ... done Building wheels for collected packages: confluent-kafka Building wheel for confluent-kafka (setup.py) ... error error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [53 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-311
creating build/lib.linux-x86_64-cpython-311/confluent_kafka
copying src/confluent_kafka/error.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka
copying src/confluent_kafka/serializing_producer.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka
copying src/confluent_kafka/deserializing_consumer.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka
copying src/confluent_kafka/init.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka
creating build/lib.linux-x86_64-cpython-311/confluent_kafka/avro
copying src/confluent_kafka/avro/error.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/avro
copying src/confluent_kafka/avro/cached_schema_registry_client.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/avro
copying src/confluent_kafka/avro/load.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/avro
copying src/confluent_kafka/avro/init.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/avro
creating build/lib.linux-x86_64-cpython-311/confluent_kafka/admin
copying src/confluent_kafka/admin/_config.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/admin
copying src/confluent_kafka/admin/_acl.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/admin
copying src/confluent_kafka/admin/_resource.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/admin
copying src/confluent_kafka/admin/init.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/admin
creating build/lib.linux-x86_64-cpython-311/confluent_kafka/kafkatest
copying src/confluent_kafka/kafkatest/verifiable_client.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/kafkatest
copying src/confluent_kafka/kafkatest/verifiable_consumer.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/kafkatest
copying src/confluent_kafka/kafkatest/verifiable_producer.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/kafkatest
copying src/confluent_kafka/kafkatest/init.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/kafkatest
creating build/lib.linux-x86_64-cpython-311/confluent_kafka/schema_registry
copying src/confluent_kafka/schema_registry/error.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/schema_registry
copying src/confluent_kafka/schema_registry/schema_registry_client.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/schema_registry
copying src/confluent_kafka/schema_registry/json_schema.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/schema_registry
copying src/confluent_kafka/schema_registry/protobuf.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/schema_registry
copying src/confluent_kafka/schema_registry/avro.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/schema_registry
copying src/confluent_kafka/schema_registry/init.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/schema_registry
creating build/lib.linux-x86_64-cpython-311/confluent_kafka/serialization
copying src/confluent_kafka/serialization/init.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/serialization
creating build/lib.linux-x86_64-cpython-311/confluent_kafka/avro/serializer
copying src/confluent_kafka/avro/serializer/message_serializer.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/avro/serializer
copying src/confluent_kafka/avro/serializer/init.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/avro/serializer
running build_ext
building 'confluent_kafka.cimpl' extension
creating build/temp.linux-x86_64-cpython-311
creating build/temp.linux-x86_64-cpython-311/tmp
creating build/temp.linux-x86_64-cpython-311/tmp/pip-install-hrf7miay
creating build/temp.linux-x86_64-cpython-311/tmp/pip-install-hrf7miay/confluent-kafka_24487a6a36c04ac3bd52ab375f05fcd5
creating build/temp.linux-x86_64-cpython-311/tmp/pip-install-hrf7miay/confluent-kafka_24487a6a36c04ac3bd52ab375f05fcd5/src
creating build/temp.linux-x86_64-cpython-311/tmp/pip-install-hrf7miay/confluent-kafka_24487a6a36c04ac3bd52ab375f05fcd5/src/confluent_kafka
creating build/temp.linux-x86_64-cpython-311/tmp/pip-install-hrf7miay/confluent-kafka_24487a6a36c04ac3bd52ab375f05fcd5/src/confluent_kafka/src
gcc -pthread -B /home/thibaultbetremieux/anaconda3/envs/test_confluent_kafka/compiler_compat -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/thibaultbetremieux/anaconda3/envs/test_confluent_kafka/include -fPIC -O2 -isystem /home/thibaultbetremieux/anaconda3/envs/test_confluent_kafka/include -fPIC -I/home/thibaultbetremieux/anaconda3/envs/test_confluent_kafka/include/python3.11 -c /tmp/pip-install-hrf7miay/confluent-kafka_24487a6a36c04ac3bd52ab375f05fcd5/src/confluent_kafka/src/Admin.c -o build/temp.linux-x86_64-cpython-311/tmp/pip-install-hrf7miay/confluent-kafka_24487a6a36c04ac3bd52ab375f05fcd5/src/confluent_kafka/src/Admin.o
In file included from /tmp/pip-install-hrf7miay/confluent-kafka_24487a6a36c04ac3bd52ab375f05fcd5/src/confluent_kafka/src/Admin.c:17:
/tmp/pip-install-hrf7miay/confluent-kafka_24487a6a36c04ac3bd52ab375f05fcd5/src/confluent_kafka/src/confluent_kafka.h:23:10: fatal error: librdkafka/rdkafka.h: No such file or directory
23 | #include <librdkafka/rdkafka.h>
| ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
error: command '/usr/bin/gcc' failed with exit code 1
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for confluent-kafka
Running setup.py clean for confluent-kafka
Failed to build confluent-kafka
Installing collected packages: confluent-kafka
Running setup.py install for confluent-kafka ... error
error: subprocess-exited-with-error

× Running setup.py install for confluent-kafka did not run successfully.
│ exit code: 1
╰─> [55 lines of output]
running install
/home/thibaultbetremieux/anaconda3/envs/test_confluent_kafka/lib/python3.11/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-311
creating build/lib.linux-x86_64-cpython-311/confluent_kafka
copying src/confluent_kafka/error.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka
copying src/confluent_kafka/serializing_producer.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka
copying src/confluent_kafka/deserializing_consumer.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka
copying src/confluent_kafka/init.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka
creating build/lib.linux-x86_64-cpython-311/confluent_kafka/avro
copying src/confluent_kafka/avro/error.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/avro
copying src/confluent_kafka/avro/cached_schema_registry_client.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/avro
copying src/confluent_kafka/avro/load.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/avro
copying src/confluent_kafka/avro/init.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/avro
creating build/lib.linux-x86_64-cpython-311/confluent_kafka/admin
copying src/confluent_kafka/admin/_config.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/admin
copying src/confluent_kafka/admin/_acl.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/admin
copying src/confluent_kafka/admin/_resource.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/admin
copying src/confluent_kafka/admin/init.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/admin
creating build/lib.linux-x86_64-cpython-311/confluent_kafka/kafkatest
copying src/confluent_kafka/kafkatest/verifiable_client.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/kafkatest
copying src/confluent_kafka/kafkatest/verifiable_consumer.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/kafkatest
copying src/confluent_kafka/kafkatest/verifiable_producer.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/kafkatest
copying src/confluent_kafka/kafkatest/init.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/kafkatest
creating build/lib.linux-x86_64-cpython-311/confluent_kafka/schema_registry
copying src/confluent_kafka/schema_registry/error.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/schema_registry
copying src/confluent_kafka/schema_registry/schema_registry_client.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/schema_registry
copying src/confluent_kafka/schema_registry/json_schema.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/schema_registry
copying src/confluent_kafka/schema_registry/protobuf.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/schema_registry
copying src/confluent_kafka/schema_registry/avro.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/schema_registry
copying src/confluent_kafka/schema_registry/init.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/schema_registry
creating build/lib.linux-x86_64-cpython-311/confluent_kafka/serialization
copying src/confluent_kafka/serialization/init.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/serialization
creating build/lib.linux-x86_64-cpython-311/confluent_kafka/avro/serializer
copying src/confluent_kafka/avro/serializer/message_serializer.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/avro/serializer
copying src/confluent_kafka/avro/serializer/init.py -> build/lib.linux-x86_64-cpython-311/confluent_kafka/avro/serializer
running build_ext
building 'confluent_kafka.cimpl' extension
creating build/temp.linux-x86_64-cpython-311
creating build/temp.linux-x86_64-cpython-311/tmp
creating build/temp.linux-x86_64-cpython-311/tmp/pip-install-hrf7miay
creating build/temp.linux-x86_64-cpython-311/tmp/pip-install-hrf7miay/confluent-kafka_24487a6a36c04ac3bd52ab375f05fcd5
creating build/temp.linux-x86_64-cpython-311/tmp/pip-install-hrf7miay/confluent-kafka_24487a6a36c04ac3bd52ab375f05fcd5/src
creating build/temp.linux-x86_64-cpython-311/tmp/pip-install-hrf7miay/confluent-kafka_24487a6a36c04ac3bd52ab375f05fcd5/src/confluent_kafka
creating build/temp.linux-x86_64-cpython-311/tmp/pip-install-hrf7miay/confluent-kafka_24487a6a36c04ac3bd52ab375f05fcd5/src/confluent_kafka/src
gcc -pthread -B /home/thibaultbetremieux/anaconda3/envs/test_confluent_kafka/compiler_compat -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/thibaultbetremieux/anaconda3/envs/test_confluent_kafka/include -fPIC -O2 -isystem /home/thibaultbetremieux/anaconda3/envs/test_confluent_kafka/include -fPIC -I/home/thibaultbetremieux/anaconda3/envs/test_confluent_kafka/include/python3.11 -c /tmp/pip-install-hrf7miay/confluent-kafka_24487a6a36c04ac3bd52ab375f05fcd5/src/confluent_kafka/src/Admin.c -o build/temp.linux-x86_64-cpython-311/tmp/pip-install-hrf7miay/confluent-kafka_24487a6a36c04ac3bd52ab375f05fcd5/src/confluent_kafka/src/Admin.o
In file included from /tmp/pip-install-hrf7miay/confluent-kafka_24487a6a36c04ac3bd52ab375f05fcd5/src/confluent_kafka/src/Admin.c:17:
/tmp/pip-install-hrf7miay/confluent-kafka_24487a6a36c04ac3bd52ab375f05fcd5/src/confluent_kafka/src/confluent_kafka.h:23:10: fatal error: librdkafka/rdkafka.h: No such file or directory
23 | #include <librdkafka/rdkafka.h>
| ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
error: command '/usr/bin/gcc' failed with exit code 1
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> confluent-kafka

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

Checklist

Please provide the following information:

  • confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()): confluent-kafka-=1.9.2, librdkafka=?
  • Apache Kafka broker version: ?
  • Client configuration: {...}
  • Operating system: Pop!_OS 22.04 LTS
  • Provide client logs (with 'debug': '..' as necessary)
  • Provide broker log excerpts
  • Critical issue

Metadata

Metadata

Assignees

No one assigned

    Labels

    investigate furtherIt's unclear what the issue is at this time but there is enough interest to look into it

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions
      pFad - Phonifier reborn

      Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

      Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


      Alternative Proxies:

      Alternative Proxy

      pFad Proxy

      pFad v3 Proxy

      pFad v4 Proxy