Failing to get watermark offsets #294
Description
Description
I'm using confluent-kafka-python version 0.11.0 and seeing this issue on multiple OS's (namely OSX and Ubuntu 16.04). The broker version is 0.10.1.
Here's the error I'm seeing when starting a Kafka consumer:
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/amb-0.2.0-py3.6.egg/.../kafka/consumer.py", line 67, in listen
message = self._consumer.poll(timeout=1.0)
File "/opt/conda/lib/python3.6/site-packages/amb-0.2.0-py3.6.egg/.../kafka/consumer.py", line 51, in reset_offsets
current_offsets = consumer.get_watermark_offsets(p)
cimpl.KafkaException: KafkaError{code=_WAIT_COORD,val=-180,str="Failed to get watermark offsets: Local: Waiting for coordinator"}
How to reproduce
Basically, I'm calling this function:
current_offsets = consumer.get_watermark_offsets(p)
on assignment:
self._consumer.subscribe([self._topic], on_assign=reset_offsets)
Config info:
security_info = {
'security.protocol': security,
'ssl.key.location': settings.KAFKA_KEY,
'ssl.key.password': settings.SSL_PASSWORD,
'ssl.ca.location': settings.KAFKA_CA,
'ssl.certificate.location': settings.KAFKA_PEM
}
consumer_info = {
'session.timeout.ms': 6000,
'bootstrap.servers': settings.KAFKA_HOST,
'group.id': group_id,
'default.topic.config': {
'auto.offset.reset': 'smallest',
},
}
Does anyone know what might be causing this issue? It doesn't seem to break my code, but it just started happening seemingly out of nowhere.
Checklist
Please provide the following information:
- confluent-kafka-python and librdkafka version (
confluent_kafka.version()andconfluent_kafka.libversion()): - Apache Kafka broker version:
- Client configuration:
{...} - Operating system:
- Provide client logs (with
'debug': '..'as necessary) - Provide broker log excerpts
- Critical issue