Skip to content

Commit d69e310

Browse files
pranavrthpranavrth
authored
[KIP-430] Added ACL Operations to Describe Topics, Cluster and ConsumerGroup APIs (confluentinc#1635)
[KIP-430] Added ACL Operations to Describe Topics, Cluster and ConsumerGroup APIs
1 parent 6673afb commit d69e310

File tree

18 files changed

+1406
-49
lines changed

18 files changed

+1406
-49
lines changed

‎CHANGELOG.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,20 @@
11
# Confluent's Python client for Apache Kafka
22

3+
## v2.3.0
4+
5+
v2.3.0 is a feature release with the following features, fixes and enhancements:
6+
7+
* Add support for AdminAPI `DescribeCluster()` and `DescribeTopics()`. (@jainruchir, #1635)
8+
* [KIP-430](https://cwiki.apache.org/confluence/display/KAFKA/KIP-430+-+Return+Authorized+Operations+in+Describe+Responses):
9+
Return authorized operations in Describe Responses. (@jainruchir, #1635)
10+
* Add `Rack` to the `Node` type, so AdminAPI calls can expose racks for brokers
11+
(currently, all Describe Responses) (#1635, @jainruchir).
12+
13+
confluent-kafka-python is based on librdkafka v2.3.0, see the
14+
[librdkafka release notes](https://github.com/confluentinc/librdkafka/releases/tag/v2.3.0)
15+
for a complete list of changes, enhancements, fixes and upgrade considerations.
16+
17+
318
## v2.2.0
419

520
v2.2.0 is a feature release with the following features, fixes and enhancements:

‎docs/index.rst

Lines changed: 183 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,11 @@ Supporting classes
2525
- :ref:`Message <pythonclient_message>`
2626
- :ref:`TopicPartition <pythonclient_topicpartition>`
2727
- :ref:`ThrottleEvent <pythonclient_throttleevent>`
28+
- :ref:`TopicCollection <pythonclient_topic_collection>`
29+
- :ref:`TopicPartitionInfo <pythonclient_topic_partition_info>`
30+
- :ref:`Node <pythonclient_node>`
31+
- :ref:`ConsumerGroupTopicPartitions <pythonclient_consumer_group_topic_partition>`
32+
- :ref:`ConsumerGroupState <pythonclient_consumer_group_state>`
2833

2934
- Errors:
3035
- :ref:`KafkaError <pythonclient_kafkaerror>`
@@ -55,6 +60,19 @@ Supporting classes
5560
- :ref:`UserScramCredentialAlteration <pythonclient_user_scram_credential_alteration>`
5661
- :ref:`UserScramCredentialUpsertion <pythonclient_user_scram_credential_upsertion>`
5762
- :ref:`UserScramCredentialDeletion <pythonclient_user_scram_credential_deletion>`
63+
- :ref:`TopicDescription <pythonclient_topic_description>`
64+
- :ref:`DescribeClusterResult <pythonclient_describe_cluster_result>`
65+
- :ref:`BrokerMetadata <pythonclient_broker_metadata>`
66+
- :ref:`ClusterMetadata <pythonclient_cluster_metadata>`
67+
- :ref:`GroupMember <pythonclient_group_member>`
68+
- :ref:`GroupMetadata <pythonclient_group_metadata>`
69+
- :ref:`PartitionMetadata <pythonclient_partition_metadata>`
70+
- :ref:`TopicMetadata <pythonclient_topic_metadata>`
71+
- :ref:`ConsumerGroupListing <pythonclient_consumer_group_listing>`
72+
- :ref:`ListConsumerGroupsResult <pythonclient_list_consumer_group_result>`
73+
- :ref:`MemberAssignment <pythonclient_member_assignment>`
74+
- :ref:`MemberDescription <pythonclient_member_description>`
75+
- :ref:`ConsumerGroupDescription <pythonclient_consumer_group_description>`
5876

5977
Experimental
6078
These classes are experimental and are likely to be removed, or subject to incompatible
@@ -84,6 +102,7 @@ AdminClient
84102

85103
.. automodule:: confluent_kafka.admin
86104
:members:
105+
:noindex:
87106

88107
.. _pyclient_admin_newtopic:
89108

@@ -247,6 +266,123 @@ UserScramCredentialDeletion
247266
.. autoclass:: confluent_kafka.admin.UserScramCredentialDeletion
248267
:members:
249268

269+
.. _pythonclient_topic_description:
270+
271+
****************
272+
TopicDescription
273+
****************
274+
275+
.. autoclass:: confluent_kafka.admin.TopicDescription
276+
:members:
277+
278+
.. _pythonclient_describe_cluster_result:
279+
280+
*********************
281+
DescribeClusterResult
282+
*********************
283+
284+
.. autoclass:: confluent_kafka.admin.DescribeClusterResult
285+
:members:
286+
287+
.. _pythonclient_broker_metadata:
288+
289+
**************
290+
BrokerMetadata
291+
**************
292+
293+
.. autoclass:: confluent_kafka.admin.BrokerMetadata
294+
:members:
295+
296+
.. _pythonclient_cluster_metadata:
297+
298+
***************
299+
ClusterMetadata
300+
***************
301+
302+
.. autoclass:: confluent_kafka.admin.ClusterMetadata
303+
:members:
304+
305+
.. _pythonclient_group_member:
306+
307+
***********
308+
GroupMember
309+
***********
310+
311+
.. autoclass:: confluent_kafka.admin.GroupMember
312+
:members:
313+
314+
.. _pythonclient_group_metadata:
315+
316+
*************
317+
GroupMetadata
318+
*************
319+
320+
.. autoclass:: confluent_kafka.admin.GroupMetadata
321+
:members:
322+
323+
.. _pythonclient_partition_metadata:
324+
325+
*****************
326+
PartitionMetadata
327+
*****************
328+
329+
.. autoclass:: confluent_kafka.admin.PartitionMetadata
330+
:members:
331+
332+
.. _pythonclient_topic_metadata:
333+
334+
*************
335+
TopicMetadata
336+
*************
337+
338+
.. autoclass:: confluent_kafka.admin.TopicMetadata
339+
:members:
340+
341+
.. _pythonclient_consumer_group_listing:
342+
343+
********************
344+
ConsumerGroupListing
345+
********************
346+
347+
.. autoclass:: confluent_kafka.admin.ConsumerGroupListing
348+
:members:
349+
350+
.. _pythonclient_list_consumer_group_result:
351+
352+
************************
353+
ListConsumerGroupsResult
354+
************************
355+
356+
.. autoclass:: confluent_kafka.admin.ListConsumerGroupsResult
357+
:members:
358+
359+
.. _pythonclient_consumer_group_description:
360+
361+
************************
362+
ConsumerGroupDescription
363+
************************
364+
365+
.. autoclass:: confluent_kafka.admin.ConsumerGroupDescription
366+
:members:
367+
368+
.. _pythonclient_member_assignment:
369+
370+
****************
371+
MemberAssignment
372+
****************
373+
374+
.. autoclass:: confluent_kafka.admin.MemberAssignment
375+
:members:
376+
377+
.. _pythonclient_member_description:
378+
379+
*****************
380+
MemberDescription
381+
*****************
382+
383+
.. autoclass:: confluent_kafka.admin.MemberDescription
384+
:members:
385+
250386
.. _pythonclient_consumer:
251387

252388
********
@@ -255,6 +391,7 @@ Consumer
255391

256392
.. autoclass:: confluent_kafka.Consumer
257393
:members:
394+
:noindex:
258395

259396
.. _serde_consumer:
260397

@@ -275,6 +412,7 @@ Producer
275412

276413
.. autoclass:: confluent_kafka.Producer
277414
:members:
415+
:noindex:
278416

279417
.. _serde_producer:
280418

@@ -477,6 +615,51 @@ TopicPartition
477615
.. autoclass:: confluent_kafka.TopicPartition
478616
:members:
479617

618+
.. _pythonclient_topic_collection:
619+
620+
***************
621+
TopicCollection
622+
***************
623+
624+
.. autoclass:: confluent_kafka.TopicCollection
625+
:members:
626+
627+
.. _pythonclient_topic_partition_info:
628+
629+
******************
630+
TopicPartitionInfo
631+
******************
632+
633+
.. autoclass:: confluent_kafka.TopicPartitionInfo
634+
:members:
635+
636+
.. _pythonclient_node:
637+
638+
****
639+
Node
640+
****
641+
642+
.. autoclass:: confluent_kafka.Node
643+
:members:
644+
645+
.. _pythonclient_consumer_group_topic_partition:
646+
647+
****************************
648+
ConsumerGroupTopicPartitions
649+
****************************
650+
651+
.. autoclass:: confluent_kafka.ConsumerGroupTopicPartitions
652+
:members:
653+
654+
.. _pythonclient_consumer_group_state:
655+
656+
******************
657+
ConsumerGroupState
658+
******************
659+
660+
.. autoclass:: confluent_kafka.ConsumerGroupState
661+
:members:
662+
480663
.. _serde_field:
481664

482665
************

‎examples/adminapi.py

Lines changed: 92 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@
1818
# Example use of AdminClient operations.
1919

2020
from confluent_kafka import (KafkaException, ConsumerGroupTopicPartitions,
21-
TopicPartition, ConsumerGroupState)
21+
TopicPartition, ConsumerGroupState, TopicCollection)
2222
from confluent_kafka.admin import (AdminClient, NewTopic, NewPartitions, ConfigResource,
2323
ConfigEntry, ConfigSource, AclBinding,
2424
AclBindingFilter, ResourceType, ResourcePatternType,
@@ -492,8 +492,9 @@ def example_describe_consumer_groups(a, args):
492492
"""
493493
Describe Consumer Groups
494494
"""
495-
496-
futureMap = a.describe_consumer_groups(args, request_timeout=10)
495+
include_auth_ops = bool(int(args[0]))
496+
args = args[1:]
497+
futureMap = a.describe_consumer_groups(args, include_authorized_operations=include_auth_ops, request_timeout=10)
497498

498499
for group_id, future in futureMap.items():
499500
try:
@@ -502,7 +503,8 @@ def example_describe_consumer_groups(a, args):
502503
print(" Is Simple : {}".format(g.is_simple_consumer_group))
503504
print(" State : {}".format(g.state))
504505
print(" Partition Assignor : {}".format(g.partition_assignor))
505-
print(" Coordinator : ({}) {}:{}".format(g.coordinator.id, g.coordinator.host, g.coordinator.port))
506+
print(
507+
f" Coordinator : {g.coordinator}")
506508
print(" Members: ")
507509
for member in g.members:
508510
print(" Id : {}".format(member.member_id))
@@ -513,12 +515,93 @@ def example_describe_consumer_groups(a, args):
513515
print(" Assignments :")
514516
for toppar in member.assignment.topic_partitions:
515517
print(" {} [{}]".format(toppar.topic, toppar.partition))
518+
if (include_auth_ops):
519+
print(" Authorized operations: ")
520+
op_string = ""
521+
for acl_op in g.authorized_operations:
522+
op_string += acl_op.name + " "
523+
print(" {}".format(op_string))
516524
except KafkaException as e:
517525
print("Error while describing group id '{}': {}".format(group_id, e))
518526
except Exception:
519527
raise
520528

521529

530+
def example_describe_topics(a, args):
531+
"""
532+
Describe Topics
533+
"""
534+
include_auth_ops = bool(int(args[0]))
535+
args = args[1:]
536+
topics = TopicCollection(topic_names=args)
537+
futureMap = a.describe_topics(topics, request_timeout=10, include_authorized_operations=include_auth_ops)
538+
539+
for topic_name, future in futureMap.items():
540+
try:
541+
t = future.result()
542+
print("Topic name : {}".format(t.name))
543+
if (t.is_internal):
544+
print("Topic is Internal")
545+
546+
if (include_auth_ops):
547+
print("Authorized operations : ")
548+
op_string = ""
549+
for acl_op in t.authorized_operations:
550+
op_string += acl_op.name + " "
551+
print(" {}".format(op_string))
552+
553+
print("Partition Information")
554+
for partition in t.partitions:
555+
print(" Id : {}".format(partition.id))
556+
leader = partition.leader
557+
print(f" Leader : {leader}")
558+
print(" Replicas : {}".format(len(partition.replicas)))
559+
for replica in partition.replicas:
560+
print(f" Replica : {replica}")
561+
print(" In-Sync Replicas : {}".format(len(partition.isr)))
562+
for isr in partition.isr:
563+
print(f" In-Sync Replica : {isr}")
564+
print("")
565+
print("")
566+
567+
except KafkaException as e:
568+
print("Error while describing topic '{}': {}".format(topic_name, e))
569+
except Exception:
570+
raise
571+
572+
573+
def example_describe_cluster(a, args):
574+
"""
575+
Describe Cluster
576+
"""
577+
include_auth_ops = bool(int(args[0]))
578+
args = args[1:]
579+
future = a.describe_cluster(request_timeout=10, include_authorized_operations=include_auth_ops)
580+
try:
581+
c = future.result()
582+
print("Cluster_id : {}".format(c.cluster_id))
583+
584+
if (c.controller):
585+
print(f"Controller: {c.controller}")
586+
else:
587+
print("No Controller Information Available")
588+
589+
print("Nodes :")
590+
for node in c.nodes:
591+
print(f" Node: {node}")
592+
593+
if (include_auth_ops):
594+
print("Authorized operations: ")
595+
op_string = ""
596+
for acl_op in c.authorized_operations:
597+
op_string += acl_op.name + " "
598+
print(" {}".format(op_string))
599+
except KafkaException as e:
600+
print("Error while describing cluster: {}".format(e))
601+
except Exception:
602+
raise
603+
604+
522605
def example_delete_consumer_groups(a, args):
523606
"""
524607
Delete Consumer Groups
@@ -704,7 +787,9 @@ def example_alter_user_scram_credentials(a, args):
704787
'<principal1> <host1> <operation1> <permission_type1> ..\n')
705788
sys.stderr.write(' list [<all|topics|brokers|groups>]\n')
706789
sys.stderr.write(' list_consumer_groups [<state1> <state2> ..]\n')
707-
sys.stderr.write(' describe_consumer_groups <group1> <group2> ..\n')
790+
sys.stderr.write(' describe_consumer_groups <include_authorized_operations> <group1> <group2> ..\n')
791+
sys.stderr.write(' describe_topics <include_authorized_operations> <topic1> <topic2> ..\n')
792+
sys.stderr.write(' describe_cluster <include_authorized_operations>\n')
708793
sys.stderr.write(' delete_consumer_groups <group1> <group2> ..\n')
709794
sys.stderr.write(' list_consumer_group_offsets <group> [<topic1> <partition1> <topic2> <partition2> ..]\n')
710795
sys.stderr.write(
@@ -737,6 +822,8 @@ def example_alter_user_scram_credentials(a, args):
737822
'list': example_list,
738823
'list_consumer_groups': example_list_consumer_groups,
739824
'describe_consumer_groups': example_describe_consumer_groups,
825+
'describe_topics': example_describe_topics,
826+
'describe_cluster': example_describe_cluster,
740827
'delete_consumer_groups': example_delete_consumer_groups,
741828
'list_consumer_group_offsets': example_list_consumer_group_offsets,
742829
'alter_consumer_group_offsets': example_alter_consumer_group_offsets,

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy