Description

confluent_kafka python client does not use ssl.ca.location. I had to add to the certificate to system cacert as a workaround:

cat ca.cert >>/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem

The client was compiled using the following commands:

yum install librdkafka-devel python36-devel.x86_64 openssl-devel.x86_64 cyrus-sasl-devel.x86_64 krb5-devel.x86_64

pip3 install --no-binary :all: confluent-kafka

the errors are:

Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py", line 603, in urlopen
chunked=chunked)
File "/usr/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py", line 344, in _make_request
self._validate_conn(conn)
File "/usr/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py", line 843, in validate_conn
conn.connect()
File "/usr/lib/python3.6/site-packages/requests/packages/urllib3/connection.py", line 350, in connect
ssl_context=context)
File "/usr/lib/python3.6/site-packages/requests/packages/urllib3/util/ssl.py", line 356, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib64/python3.6/ssl.py", line 365, in wrap_socket
_context=self, _session=session)
File "/usr/lib64/python3.6/ssl.py", line 773, in init
self.do_handshake()
File "/usr/lib64/python3.6/ssl.py", line 1033, in do_handshake
self._sslobj.do_handshake()
File "/usr/lib64/python3.6/ssl.py", line 645, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:877)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 438, in send
timeout=timeout
File "/usr/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py", line 641, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/lib/python3.6/site-packages/requests/packages/urllib3/util/retry.py", line 399, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
requests.packages.urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='dp.ist.bns', port=1443): Max retries exceeded with url: /schemas/ids/4728 (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:877)'),))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/apps/algo4/alcr5_ist1/AnvilIntraday/system/python/lib/python3.6/site-packages/confluent_kafka/deserializing_consumer.py", line 137, in poll
value = self._value_deserializer(value, ctx)
File "/opt/apps/algo4/alcr5_ist1/AnvilIntraday/system/python/lib/python3.6/site-packages/confluent_kafka/schema_registry/avro.py", line 315, in call
schema = self._registry.get_schema(schema_id)
File "/opt/apps/algo4/alcr5_ist1/AnvilIntraday/system/python/lib/python3.6/site-packages/confluent_kafka/schema_registry/schema_registry_client.py", line 368, in get_schema
response = self._rest_client.get('schemas/ids/{}'.format(schema_id))
File "/opt/apps/algo4/alcr5_ist1/AnvilIntraday/system/python/lib/python3.6/site-packages/confluent_kafka/schema_registry/schema_registry_client.py", line 124, in get
return self.send_request(url, method='GET', query=query)
File "/opt/apps/algo4/alcr5_ist1/AnvilIntraday/system/python/lib/python3.6/site-packages/confluent_kafka/schema_registry/schema_registry_client.py", line 169, in send_request
headers=headers, data=body, params=query)
File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 651, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 502, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='dp.ist.bns', port=1443): Max retries exceeded with url: /schemas/ids/4728 (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:877)'),))

How to reproduce

Checklist

Please provide the following information:

• confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()):
• Apache Kafka broker version:
• Client configuration: {...}
• Operating system:
• Provide client logs (with 'debug': '..' as necessary)
• Provide broker log excerpts
• Critical issue