Merge pull request confluentinc#695 from confluentinc/docs1

mhowlett · web-flow · commit 2805c3fc2e24 · 2019-11-27T10:46:46.000-08:00
Iterating on the docs
diff --git a/DEVELOPER.md b/DEVELOPER.md
@@ -0,0 +1,36 @@
+# Developer Notes
+
+This document provides information useful to developers working on confluent-kafka-python.
+
+
+## Build
+
+    $ python setup.py build
+
+If librdkafka is installed in a non-standard location provide the include and library directories with:
+
+    $ C_INCLUDE_PATH=/path/to/include LIBRARY_PATH=/path/to/lib python setup.py ...
+
+
+## Generate Documentation
+
+Install sphinx and sphinx_rtd_theme packages:
+
+    $ pip install sphinx sphinx_rtd_theme
+
+Build HTML docs:
+
+    $ make docs
+
+or:
+
+    $ python setup.py build_sphinx
+
+Documentation will be generated in `docs/_build/`.
+
+
+## Tests
+
+
+See [tests/README.md](tests/README.md) for instructions on how to run tests.
+
diff --git a/README.md b/README.md
@@ -1,30 +1,19 @@
 Confluent's Python Client for Apache Kafka<sup>TM</sup>
 =======================================================
 
-**confluent-kafka-python** is Confluent's Python client for [Apache Kafka](http://kafka.apache.org/) and the
-[Confluent Platform](https://www.confluent.io/product/compare/).
+**confluent-kafka-python** provides a high-level Producer, Consumer and AdminClient compatible with all
+[Apache Kafka<sup>TM<sup>](http://kafka.apache.org/) brokers >= v0.8, [Confluent Cloud](https://www.confluent.io/confluent-cloud/)
+and the [Confluent Platform](https://www.confluent.io/product/compare/). The client is:
 
-Features:
+- **Reliable** - It's a wrapper around [librdkafka](https://github.com/edenhill/librdkafka) (provided automatically via binary wheels) which is widely deployed in a diverse set of production scenarios. It's tested using [the same set of system tests](https://github.com/confluentinc/confluent-kafka-python/tree/master/confluent_kafka/kafkatest) as the Java client [and more](https://github.com/confluentinc/confluent-kafka-python/tree/master/tests). It's supported by [Confluent](https://confluent.io). 
 
-- **High performance** - confluent-kafka-python is a lightweight wrapper around
-[librdkafka](https://github.com/edenhill/librdkafka), a finely tuned C
-client.
-
-- **Reliability** - There are a lot of details to get right when writing an Apache Kafka
-client. We get them right in one place (librdkafka) and leverage this work
-across all of our clients (also [confluent-kafka-go](https://github.com/confluentinc/confluent-kafka-go)
-and [confluent-kafka-dotnet](https://github.com/confluentinc/confluent-kafka-dotnet)).
-
-- **Supported** - Commercial support is offered by
-[Confluent](https://confluent.io/).
+- **Performant** - Performance is a key design consideration. Maximum throughput is on par with the Java client for larger message sizes (where the overhead of the Python interpreter has less impact). Latency is on par with the Java client.
 
 - **Future proof** - Confluent, founded by the
 creators of Kafka, is building a [streaming platform](https://www.confluent.io/product/compare/)
 with Apache Kafka at its core. It's high priority for us that client features keep
 pace with core Apache Kafka and components of the [Confluent Platform](https://www.confluent.io/product/compare/).
 
-The Python bindings provides a high-level Producer and Consumer with support
-for the balanced consumer groups of Apache Kafka &gt;= 0.9.
 
 See the [API documentation](http://docs.confluent.io/current/clients/confluent-kafka-python/index.html) for more info.
 
@@ -34,7 +23,10 @@ See the [API documentation](http://docs.confluent.io/current/clients/confluent-k
 Usage
 =====
 
-**Producer:**
+Below are some examples of typical usage. For more examples, see the [examples](examples) directory or the [confluentinc/examples](https://github.com/confluentinc/examples/tree/master/clients/cloud/python) github repo for a [Confluent Cloud](https://www.confluent.io/confluent-cloud/) example.
+
+
+**Producer**
 
 ```python
 from confluent_kafka import Producer
@@ -65,7 +57,7 @@ p.flush()
 ```
 
 
-**High-level Consumer:**
+**High-level Consumer**
 
 ```python
 from confluent_kafka import Consumer, KafkaError
@@ -177,28 +169,58 @@ while True:
 c.close()
 ```
 
-See the [examples](examples) directory for more examples, including [how to configure](examples/confluent_cloud.py) the python client for use with
-[Confluent Cloud](https://www.confluent.io/confluent-cloud/).
+**AdminClient**
+
+Create topics:
+
+```python
+from confluent_kafka.admin import AdminClient, NewTopic
+
+a = AdminClient({'bootstrap.servers': 'mybroker'})
+
+new_topics = [NewTopic(topic, num_partitions=3, replication_factor=1) for topic in ["topic1", "topic2"]]
+# Note: In a multi-cluster production scenario, it is more typical to use a replication_factor of 3 for durability.
+
+# Call create_topics to asynchronously create topics. A dict
+# of <topic,future> is returned.
+fs = a.create_topics(new_topics)
+
+# Wait for each operation to finish.
+for topic, f in fs.items():
+    try:
+        f.result()  # The result itself is None
+        print("Topic {} created".format(topic))
+    except Exception as e:
+        print("Failed to create topic {}: {}".format(topic, e))
+```
+
+
+
+Thread Safety
+-------------
+
+The `Producer`, `Consumer` and `AdminClient` are all thread safe.
 
 
 Install
 =======
 
+**Install self-contained binary wheels**
+
+    $ pip install confluent-kafka
+
 **NOTE:** The pre-built Linux wheels do NOT contain SASL Kerberos/GSSAPI support.
           If you need SASL Kerberos/GSSAPI support you must install librdkafka and
           its dependencies using the repositories below and then build
           confluent-kafka  using the command in the "Install from
           source from PyPi" section below.
 
-**Install self-contained binary wheels for OSX and Linux from PyPi:**
-
-    $ pip install confluent-kafka
-
-**Install AvroProducer and AvroConsumer:**
+**Install AvroProducer and AvroConsumer**
 
     $ pip install "confluent-kafka[avro]"
 
-**Install from source from PyPi** *(requires librdkafka + dependencies to be installed separately)*:
+**Install from source from PyPi**
+*(requires librdkafka + dependencies to be installed separately)*:
 
     $ pip install --no-binary :all: confluent-kafka
 
@@ -267,47 +289,7 @@ http://docs.confluent.io/current/installation.html#rpm-packages-via-yum
  * On **OSX**, use **homebrew** and do `brew install librdkafka`
 
 
-Build
-=====
-
-    $ python setup.py build
-
-If librdkafka is installed in a non-standard location provide the include and library directories with:
-
-    $ C_INCLUDE_PATH=/path/to/include LIBRARY_PATH=/path/to/lib python setup.py ...
-
-
-Tests
-=====
-
-
-**Run unit-tests:**
-
-In order to run full test suite, simply execute:
-
-    $ tox -r
-
-**NOTE**: Requires `tox` (please install with `pip install tox`), several supported versions of Python on your path, and `librdkafka` [installed](tools/bootstrap-librdkafka.sh) into `tmp-build`.
-
-
-**Integration tests:**
-
-See [tests/README.md](tests/README.md) for instructions on how to run integration tests.
-
-
-
-Generate Documentation
-======================
-Install sphinx and sphinx_rtd_theme packages:
-
-    $ pip install sphinx sphinx_rtd_theme
-
-Build HTML docs:
-
-    $ make docs
-
-or:
-
-    $ python setup.py build_sphinx
+Developer Notes
+===============
 
-Documentation will be generated in `docs/_build/`.
+Instructions on building and testing confluent-kafka-python can be found [here](DEVELOPER.md).
