Skip to content

Fix utf8 string conversion memory leak on Python 2 (#198) #239

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 9, 2017

Conversation

edenhill
Copy link
Contributor

@edenhill edenhill commented Sep 1, 2017

We need to keep and destroy the intermediary utf8 object on Python 2.

@edenhill edenhill requested a review from ewencp September 1, 2017 07:49
@@ -1245,6 +1260,7 @@ rd_kafka_conf_t *common_conf_setup (rd_kafka_type_t ktype,
"as a callable function");
rd_kafka_topic_conf_destroy(tconf);
rd_kafka_conf_destroy(conf);
Py_XDECREF(ks8);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know it becomes inconsistent with the way ks is handled, but couldn't we just Py_XDECREF immediately so we only have it once and don't have to scatter it on all of these paths? In fact, it looks like we could do that for ks in some of the cases that have been converted here.

Copy link
Contributor Author

@edenhill edenhill Sep 5, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Python 2.x ks8 will point to another object than ks, namely the utf8 representation of ks, and since k is a char pointer to ks's underlying memory we must retain the reference until all usage of k is over.

On Python 3 ks8 will always be NULL, thus causing XDECREF to be a no-op.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack, makes sense. Kinda sucks that the API naming doesn't make it really obvious when you're getting internal data vs a completely new object.

@@ -64,8 +64,15 @@

/**
* @returns Unicode Python object as char * in UTF-8 encoding
* @param uobjp might be set to NULL or a new object reference (depending
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Along the same lines as my other comment -- why not Py_XDECREF immediately inside these methods? Doesn't seem like there would ever be a case where we'd want to save this intermediate data, especially since it is only an issue in Py2..

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this comment is misleading, it is not a new object reference to the original object, but a new object reference to an utf8 reprsentation object of the original object, do you object?

Copy link
Contributor

@ewencp ewencp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@edenhill edenhill merged commit bbda10c into master Sep 9, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy