MKC Examples - Connector Operations #13527

salmany · 2025-07-30T22:07:10Z

Adds Managed Kafka Connect (MKC) code examples for:

Getting, updating, deleting, and listing connectors
Starting, stopping, pausing, restarting and resuming connectors

Note: This builds upon #13522

Description

Fixes https://buganizer.corp.google.com/issues/430087669

Note: Before submitting a pull request, please open an issue for discussion if you are not associated with Google.

Checklist

I have followed Sample Guidelines from AUTHORING_GUIDE.MD
README is updated to include all relevant information
Tests pass: nox -s py-3.9 (see Test Environment Setup)
Lint pass: nox -s lint (see Test Environment Setup)
These samples need a new API enabled in testing projects to pass (let us know which ones)
These samples need a new/updated env vars in testing projects set to pass (let us know which ones)
This sample adds a new sample directory, and I updated the CODEOWNERS file with the codeowners for this sample
This sample adds a new Product API, and I updated the Blunderbuss issue/PR auto-assigner with the codeowners for this sample
Please merge this PR for me once it is approved

clusters * Adds code examples for creating, deleting, getting, listing and updating Managed Kafka Connect clusters

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

As per the [Authoring Guide] (https://github.com/GoogleCloudPlatform/python-docs-samples/blob/main/AUTHORING_GUIDE.md#dependencies), each sample is required to have a requirements.txt file that lists the dependencies needed to run the sample.

Connectors: * Creating BigQuery Sink Connector * Creating MirrorMaker Source Connector * Creating Pub/Sub Sink Connector * Creating Pub/Sub Source Connector * Creating Storage Sink Connector * Getting, updating, deleting, and listing connectors * Starting, stopping, pausing, restarting and resuming connectors

gemini-code-assist

Summary of Changes

Hello @salmany, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly expands the Python code examples for Google Cloud's Managed Kafka Connect (MKC) service. It introduces new snippets and corresponding tests that cover both the administrative tasks for MKC clusters and the end-to-end management of various Kafka Connectors, from creation to lifecycle operations.

Highlights

Managed Kafka Connect Cluster Management Examples: Added a comprehensive set of Python examples for managing Managed Kafka Connect (MKC) clusters, including operations for creation, retrieval, listing, updating, and deletion of clusters.
Managed Kafka Connect Connector Examples: Introduced new Python code examples for creating various types of MKC connectors, specifically BigQuery Sink, MirrorMaker Source, Pub/Sub Sink, Pub/Sub Source, and Cloud Storage Sink connectors.
Managed Kafka Connect Connector Lifecycle Operations: Provided Python examples demonstrating the full lifecycle management of MKC connectors, covering actions such as getting details, listing all connectors, updating configurations, deleting, pausing, resuming, stopping, and restarting connectors.
Unit Test Coverage: Included new unit tests for all added Managed Kafka Connect cluster and connector examples to ensure their correctness and reliability.
Dependency Update: Updated the google-cloud-managedkafka library version in the main requirements.txt file to 0.1.12.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments or fill out our survey to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request adds a comprehensive set of code examples for managing Managed Kafka Connect clusters and connectors. The new samples cover creation, listing, getting, updating, and deleting resources, as well as lifecycle operations for connectors.

The code is well-structured and the examples are clear. I've made a few suggestions to improve correctness, maintainability, and robustness:

Correcting the placement of a try-except block for error handling.
Removing a redundant API call.
Adding timeouts to long-running operations to prevent scripts from hanging.
Minor fixes in docstrings and code formatting for consistency and clarity.

Overall, this is a great addition. Once these points are addressed, the samples will be even better.

managedkafka/snippets/connect/clusters/list_connect_clusters.py

managedkafka/snippets/connect/clusters/update_connect_cluster.py

managedkafka/snippets/connect/connectors/connectors_test.py

managedkafka/snippets/connect/connectors/update_connector.py

managedkafka/snippets/connect/connectors/pause_connector.py

managedkafka/snippets/connect/connectors/restart_connector.py

managedkafka/snippets/connect/connectors/resume_connector.py

managedkafka/snippets/connect/connectors/stop_connector.py

managedkafka/snippets/connect/connectors/update_connector.py

snippet-bot · 2025-07-30T22:09:25Z

Here is the summary of changes.

You are about to add 18 region tags.

managedkafka/snippets/connect/clusters/create_connect_cluster.py:41, tag managedkafka_create_connect_cluster
managedkafka/snippets/connect/clusters/delete_connect_cluster.py:32, tag managedkafka_delete_connect_cluster
managedkafka/snippets/connect/clusters/get_connect_cluster.py:32, tag managedkafka_get_connect_cluster
managedkafka/snippets/connect/clusters/list_connect_clusters.py:27, tag managedkafka_list_connect_clusters
managedkafka/snippets/connect/clusters/update_connect_cluster.py:32, tag managedkafka_update_connect_cluster
managedkafka/snippets/connect/connectors/create_bigquery_sink_connector.py:38, tag managedkafka_create_bigquery_sink_connector
managedkafka/snippets/connect/connectors/create_mirrormaker_connector.py:40, tag managedkafka_create_mirrormaker_connector
managedkafka/snippets/connect/connectors/create_pubsub_sink_connector.py:41, tag managedkafka_create_pubsub_sink_connector
managedkafka/snippets/connect/connectors/create_pubsub_source_connector.py:41, tag managedkafka_create_pubsub_source_connector
managedkafka/snippets/connect/connectors/create_storage_sink_connector.py:38, tag managedkafka_create_storage_sink_connector
managedkafka/snippets/connect/connectors/delete_connector.py:34, tag managedkafka_delete_connector
managedkafka/snippets/connect/connectors/get_connector.py:34, tag managedkafka_get_connector
managedkafka/snippets/connect/connectors/list_connectors.py:29, tag managedkafka_list_connectors
managedkafka/snippets/connect/connectors/pause_connector.py:34, tag managedkafka_pause_connector
managedkafka/snippets/connect/connectors/restart_connector.py:34, tag managedkafka_restart_connector
managedkafka/snippets/connect/connectors/resume_connector.py:34, tag managedkafka_resume_connector
managedkafka/snippets/connect/connectors/stop_connector.py:34, tag managedkafka_stop_connector
managedkafka/snippets/connect/connectors/update_connector.py:36, tag managedkafka_update_connector

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://github.com/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

Refresh this comment

salmany and others added 12 commits July 24, 2025 00:36

Add Managed Kafka Connect code samples for

c928344

clusters * Adds code examples for creating, deleting, getting, listing and updating Managed Kafka Connect clusters

Update google-cloud-managedkafka version to 0.1.12

8f2224a

Update managedkafka/snippets/connect/clusters/create_connect_cluster.py

1feb354

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Update managedkafka/snippets/connect/clusters/create_connect_cluster.py

5840336

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Update managedkafka/snippets/connect/clusters/delete_connect_cluster.py

d0df10f

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Update managedkafka/snippets/connect/clusters/get_connect_cluster.py

505759a

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Update managedkafka/snippets/connect/clusters/list_connect_clusters.py

d3fba2c

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Update managedkafka/snippets/connect/clusters/update_connect_cluster.py

0a85a8b

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Add timeouts and improve error handling.

dfa07d2

Addressed PR comments.

ab697bf

Adds requirements.txt file for samples.

3339325

As per the [Authoring Guide] (https://github.com/GoogleCloudPlatform/python-docs-samples/blob/main/AUTHORING_GUIDE.md#dependencies), each sample is required to have a requirements.txt file that lists the dependencies needed to run the sample.

product-auto-label bot added the samples Issues that are directly related to samples. label Jul 30, 2025

gemini-code-assist bot reviewed Jul 30, 2025

View reviewed changes

salmany marked this pull request as ready for review July 30, 2025 22:09

salmany requested review from a team as code owners July 30, 2025 22:09

gemini-code-assist bot reviewed Jul 30, 2025

View reviewed changes

salmany changed the title ~~Adds MKC Connectors Examples~~ MKC Examples - Connector Operations Jul 31, 2025

salmany added 2 commits July 31, 2025 20:00

Minor fixes.

8084f5b

Remove redundant file.

328ab0c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

MKC Examples - Connector Operations #13527

MKC Examples - Connector Operations #13527

Uh oh!

salmany commented Jul 30, 2025 •

edited

Loading

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

snippet-bot bot commented Jul 30, 2025 •

edited

Loading

Uh oh!

Uh oh!

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

MKC Examples - Connector Operations #13527

Are you sure you want to change the base?

MKC Examples - Connector Operations #13527

Uh oh!

Conversation

salmany commented Jul 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

snippet-bot bot commented Jul 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

salmany commented Jul 30, 2025 •

edited

Loading

snippet-bot bot commented Jul 30, 2025 •

edited

Loading