Description

I'm working on using confluent kafka python client to consume using ssl. I have no issue with setting the path of cert to ssl.ca.location. However, if I tried to use 'ssl.ca.pem', it will report error like 'SSL handshake failed: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed: broker certificate could not be verified, verify that ssl.ca.location is correctly configured or root CA certificates are installed (brew install openssl)'.

How to reproduce

There is a cert file in my project my_cert.crt with such structure:

-----BEGIN CERTIFICATE-----
first cert
-----END CERTIFICATE-----


-----BEGIN CERTIFICATE-----
another one
-----END CERTIFICATE-----

So, I'm trying to connect to Kafka with SASL_SSL. If I use ssl.ca.location way like:

conf["ssl.ca.location"] = os.path.join(cur_path, "my_cert.crt")
p = Producer(**conf)

everything goes without a doubt, but I can't find a way how to pass cert to ssl.ca.pem.
I've tried things like:

with open("my_cert.crt", "b") as f:
        b_cert = f.read()
        conf["ssl.ca.pem"] = b_cert 
        p = Producer(**conf)

####
#OR#
####

with open("my_cert.crt", "r", encoding='utf-8') as f:
        encoded_cert = base64.b64encode(f.read().encode("UTF-8"))
        decoded_cert = base64.b64decode(encoded_cert).decode('utf-8')
        conf["ssl.ca.pem"] = decoded_cert 
        p = Producer(**conf)

Feels like I'm missing something and need an assistance. Thank you!

Checklist

Please provide the following information:

• [confluent-kafka-python:('1.9.2', 17367552); librdkafka: ('1.9.2', 17367807) ]
• Apache Kafka broker version:
• Client configuration: {...}
• Operating system:
• Provide client logs (with 'debug': '..' as necessary):
• 1673960934.271|FAIL|rdkafka#producer-1| [thrd:sasl_ssl://url:port/bootstrap]: sasl_ssl://url:port/bootstrap: SSL handshake failed: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed: broker certificate could not be verified, verify that ssl.ca.location is correctly configured or root CA certificates are installed (install ca-certificates package) (after 3ms in state SSL_HANDSHAKE)
• Provide broker log excerpts
• Critical issue