|
1 | 1 | Confluent's Apache Kafka client for Python
|
2 | 2 | ==========================================
|
3 | 3 |
|
| 4 | +Confluent's Kafka client for Python wraps the librdkafka C library, providing |
| 5 | +full Kafka protocol support at great performance and reliability. |
4 | 6 |
|
5 |
| -Prerequisites |
6 |
| -=============== |
| 7 | +The Python bindings provides a high-level Producer and Consumer with support |
| 8 | +for the balanced consumer groups of Apache Kafka 0.9. |
7 | 9 |
|
8 |
| - librdkafka >=0.9.1 (or master>=2016-04-13) |
9 |
| - py.test (pip install pytest) |
| 10 | +See the [API documentation](http://docs.confluent.io/3.0.0/clients/confluent-kafka-python/index.html) for more info. |
10 | 11 |
|
11 | 12 |
|
12 |
| -Build |
| 13 | +Usage |
13 | 14 | =====
|
14 | 15 |
|
15 |
| - python setup.by build |
| 16 | +**Producer:** |
| 17 | + |
| 18 | + from confluent_kafka import Producer |
| 19 | + |
| 20 | + p = Producer({'bootstrap.servers': 'mybroker,mybroker2'}) |
| 21 | + for data in some_data_source: |
| 22 | + p.produce('mytopic', data.encode('utf-8')) |
| 23 | + p.flush() |
| 24 | + |
| 25 | + |
| 26 | +**High-level Consumer:** |
| 27 | + |
| 28 | + from confluent_kafka import Consumer |
| 29 | + |
| 30 | + c = Consumer({'bootstrap.servers': 'mybroker', 'group.id': 'mygroup', |
| 31 | + 'default.topic.config': {'auto.offset.reset': 'smallest'}}) |
| 32 | + c.subscribe(['mytopic']) |
| 33 | + while running: |
| 34 | + msg = c.poll() |
| 35 | + if not msg.error(): |
| 36 | + print('Received message: %s' % msg.value().decode('utf-8')) |
| 37 | + c.close() |
| 38 | + |
| 39 | + |
| 40 | + |
| 41 | +See [examples](examples) for more examples. |
| 42 | + |
| 43 | + |
| 44 | + |
| 45 | +Prerequisites |
| 46 | +============= |
| 47 | + |
| 48 | + * Python >= 2.7 or Python 3.x |
| 49 | + * [librdkafka](https://github.com/edenhill/librdkafka) >= 0.9.1 |
16 | 50 |
|
17 | 51 |
|
18 | 52 |
|
19 | 53 | Install
|
20 | 54 | =======
|
21 |
| -Preferably in a virtualenv: |
| 55 | + |
| 56 | +**Install from PyPi:** |
| 57 | + |
| 58 | + pip install confluent-kafka |
| 59 | + |
| 60 | + |
| 61 | +**Install from source / tarball:** |
22 | 62 |
|
23 | 63 | pip install .
|
24 | 64 |
|
25 | 65 |
|
26 |
| -Run unit-tests |
27 |
| -============== |
| 66 | +Build |
| 67 | +===== |
| 68 | + |
| 69 | + python setup.by build |
| 70 | + |
| 71 | + |
| 72 | + |
| 73 | + |
| 74 | +Tests |
| 75 | +===== |
| 76 | + |
| 77 | + |
| 78 | +**Run unit-tests:** |
28 | 79 |
|
29 | 80 | py.test
|
30 | 81 |
|
| 82 | +**NOTE**: Requires py.test, install by `pip install pytest` |
31 | 83 |
|
32 |
| -Run integration tests |
33 |
| -===================== |
34 |
| -**WARNING**: These tests require an active Kafka cluster and will make use of a topic named 'test'. |
| 84 | + |
| 85 | +**Run integration tests:** |
35 | 86 |
|
36 | 87 | examples/integration_test.py <kafka-broker>
|
37 | 88 |
|
| 89 | +**WARNING**: These tests require an active Kafka cluster and will make use of a topic named 'test'. |
| 90 | + |
| 91 | + |
38 | 92 |
|
39 | 93 |
|
40 | 94 | Generate documentation
|
|
51 | 105 | Documentation will be generated in `docs/_build/`
|
52 | 106 |
|
53 | 107 |
|
54 |
| -Examples |
55 |
| -======== |
56 |
| - |
57 |
| -See [examples](examples) |
|
0 commit comments