Skip to content

py/objtype: Add __dict__ attribute for class objects. #5324

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

andrewleech
Copy link
Contributor

@andrewleech andrewleech commented Nov 12, 2019

The behavior mirrors instance object dict attribute where a copy of the local attributes are provided.

The function is only enabled if MICROPY_CPYTHON_COMPAT is set, the same as the instance version of this function.

This can be useful to get access to the attributes on a class, eg. for #5322

@jimmo
Copy link
Member

jimmo commented Nov 12, 2019

Would it be worth making the returned dict readonly (setting is_fixed on the underlying map, just to make it really clear about thr behaviour?

@andrewleech
Copy link
Contributor Author

andrewleech commented Nov 12, 2019

That does sound sensible. The same change would make sense on the instance __dict__ function too!

@andrewleech
Copy link
Contributor Author

@jimmo at the definition of mp_map_t the is_fixed flash comes with a comment of

// a fixed array that can't be modified; must also be ordered

https://github.com/micropython/micropython/blob/master/py/obj.h#L355

I'm not sure how firm that rule of needing to be ordered is, perhaps that flag should also be set in this case?

@pi-anl pi-anl force-pushed the class__dict__ branch 2 times, most recently from c01b7b2 to 13a4ba7 Compare November 12, 2019 05:11
@andrewleech
Copy link
Contributor Author

@jimmo I've exposed dict_copy and used that, makes for much cleaner and safer implementation.

I've not touched the is_fixed flag, unless we can be sure it's safe for use on dynamic dict objects I'm not comfortable using it.

@dpgeorge dpgeorge added the py-core Relates to py/ directory in source label Nov 13, 2019
@andrewleech
Copy link
Contributor Author

@dpgeorge I believe this PR is ready for review, it behaves well for me at least. A couple of others on slack have asked about this functionality recently too!

@stinos
Copy link
Contributor

stinos commented Jan 21, 2020

I've not touched the is_fixed flag, unless we can be sure it's safe for use on dynamic dict objects I'm not comfortable using it.

In any case a short comment in the code stating that a copy is returned meaning the original cannot be modified would be good, likewise for adding a test to the CPython difference documentation? And while we're at it maybe same for __dict__ on instances since that has the same problem.

@andrewleech andrewleech force-pushed the class__dict__ branch 2 times, most recently from cb925cd to cdc47d8 Compare May 4, 2020 10:21
@dpgeorge
Copy link
Member

dpgeorge commented Jun 9, 2020

I think this is a good change, it mirrors the availability of this on instances.

But what I'd suggest to improve it is:

  • wait for py/obj.h: Clarify comments about mp_map_t is_fixed and is_ordered. #6129
  • make the existing instance code use the new mp_obj_dict_copy() function to reduce code size (this might have different memory usage to inserting each element, but I think it'd be very similar for dicts that have not had any deletions, which is most likely the case for class/instance dicts)
  • set is_fixed=1 on the dict before returning it
  • furthermore, as an optimisation, return locals_dict directly if it's already is_fixed

This should be a relatively small set of changes.

Also a test would eventually be needed, including a cpydiff test :)

@andrewleech
Copy link
Contributor Author

* make the existing instance code use the new `mp_obj_dict_copy()` function to reduce code size (this might have different memory usage to inserting each element, but I think it'd be very similar for dicts that have not had any deletions, which is most likely the case for class/instance dicts)

I had originally thought to do this, however class self->locals_dict is a dict, so mp_obj_dict_copy just works whereas instance self->members is a map so isn't conveniently compatible.
Turns out wrapping it in a dict obj to pass it into dict_copy does still save almost 100bytes from flash though so is likely worth it

    if (attr == MP_QSTR___dict__) {
        // Create a new dict with a copy of the instance's map items.
        // This creates, unlike CPython, a 'read-only' __dict__: modifying
        // it will not result in modifications to the actual instance members.
        mp_obj_t members_dict = mp_obj_new_dict(0);
        mp_obj_dict_t *dict = MP_OBJ_TO_PTR(members_dict);
        dict->map = self->members;
        dest[0] = mp_obj_dict_copy(MP_OBJ_FROM_PTR(dict));
        return;
    }

from

   text    data     bss     dec     hex filename
1841732     260  302044 2144036  20b724 firmware.elf

to

   text    data     bss     dec     hex filename
1841636     260  302044 2143940  20b6c4 firmware.elf

I also like your suggestions regarding the is_fixed flag too

@dpgeorge
Copy link
Member

dpgeorge commented Jun 9, 2020

mp_obj_t members_dict = mp_obj_new_dict(0);

No need to use heap memory, instead:

mp_obj_dict_t dict;
dict.map = self->members;
dest[0] = mp_obj_dict_copy(MP_OBJ_FROM_PTR(&dict));

@andrewleech
Copy link
Contributor Author

andrewleech commented Jun 9, 2020

'mp_obj_dict_copy' starts with mp_check_self(mp_obj_is_dict_type(self_in)); so the bare struct wasn't going to be enough. I should be able to point the base object to something that'll satisfy the check though

Also now with the next optimisation, in the case where the underlying instance map is_fixed, wont it need a real dict to return?

        // Returns a read-only dict of the instance members.
        // If the internal locals is not fixed, a copy will be created.
        // This creates, unlike CPython, a 'read-only' __dict__: modifying
        // it will not result in modifications to the actual instance members.
        mp_obj_t members_dict = mp_obj_new_dict(0);
        mp_obj_dict_t *dict = MP_OBJ_TO_PTR(members_dict);
        dict->map = self->members;
        if (dict->map.is_fixed) {
            dest[0] = dict;
        } else {
            dest[0] = mp_obj_dict_copy(MP_OBJ_FROM_PTR(dict));
            dict = MP_OBJ_TO_PTR(dest[0]);
            dict->map.is_fixed = 1;
        }
        return;

@dpgeorge
Copy link
Member

dpgeorge commented Jun 9, 2020

Also now with the next optimisation, in the case where the underlying instance map is_fixed, wont it need a real dict to return?

That optimisation won't work for instances, just classes.

@andrewleech
Copy link
Contributor Author

That optimisation won't work for instances, just classes.

Thanks for the suggestions and clarification, I've cleaned that up now and added a basic unit test.

dict.base.type = &mp_type_dict;
dest[0] = mp_obj_dict_copy(MP_OBJ_FROM_PTR(&dict));
mp_obj_dict_t *dest_dict = MP_OBJ_TO_PTR(dest[0]);
dest_dict->map.is_fixed = 1;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please put these changes to the instance __dict__ lookup into a separate commit?

@stinos as the original author of this code, what are your thoughts on this change? In particular do you think it's a good idea to set is_fixed=1 to prevent modification of the returned dict (and so prevent subtle errors when the user tries to change the dict and expects those changes to be mirrored in the instance, which they are not)?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be my preference to reduce chances of things being misused yes. And looking at the original discussion in #1757, I actually wanted it to be like that but that wasn't possible back then.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, so let's keep the is_fixed=1 part, to make the returned dict R/O. It will be a breaking change though, if users are relying on the dict being modifiable (eg because they know it's not reflected in the instance but they want anyway to reuse the dict for something else). But it'll fail pretty quickly now if someone is doing this and they can fix their code by simply doing d = dict(my_type.__dict__)


class Foo:
self.a = 1
self.b = "bar"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self shouldn't be needed here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whoops, fixed along with splitting the commits.

py/objtype.c Outdated
if (attr == MP_QSTR___dict__) {
// Returns a read-only dict of the class attributes.
// If the internal locals is not fixed, a copy will be created.
mp_obj_dict_t *dict = MP_OBJ_FROM_PTR(self->locals_dict);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doesn't need a cast

py/objtype.c Outdated
// If the internal locals is not fixed, a copy will be created.
mp_obj_dict_t *dict = MP_OBJ_FROM_PTR(self->locals_dict);
if (dict->map.is_fixed) {
dest[0] = dict;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needs MP_OBJ_FROM_PTR

py/objtype.c Outdated
if (dict->map.is_fixed) {
dest[0] = dict;
} else {
dest[0] = mp_obj_dict_copy(dict);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needs MP_OBJ_FROM_PTR

@dpgeorge
Copy link
Member

Rebased and merged in 28370c0 and 95cbe6b, with some minor edits to the comments, and fixed up the test so it worked correctly (the last print was printing False because other things are in the type dict) and tests the already-is-fixed path.

@dpgeorge dpgeorge closed this Jun 10, 2020
kamtom480 pushed a commit to kamtom480/micropython that referenced this pull request Sep 10, 2021
remove lingering reference to settings.py in docs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
py-core Relates to py/ directory in source
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy