Skip to content

Improve relation choices limiting #3329

@rubendura

Description

@rubendura

Building upon #3313 comments. This is a follow up for #3309.

There can be some memory issues with RelatedField.choices when dealing with really big querysets (~400k items in my case). This property fetches all models in the queryset and then builds a massive OrderedDict with them, which basically can clog the memory of many systems.

I suggest that we slice the queryset using html_cutoff before evaluating the queryset and building the returned OrderedDict. It seems to do the trick for me, but it'd be better if someone can confirm if this is a valid solution.

This is the code snippet that I used to do a quick (and so far successful) test. I've only added the bit slicing the queryset.

In relations.py:144 (RelatedField.choices)

@property
def choices(self):
    queryset = self.get_queryset()
    if queryset is None:
        # Ensure that field.choices returns something sensible
        # even when accessed with a read-only field.
        return {}

    return OrderedDict([
        (
            six.text_type(self.to_representation(item)),
            self.display_value(item)
        )
        for item in queryset[:self.html_cutoff]
    ])

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      pFad - Phonifier reborn

      Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

      Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


      Alternative Proxies:

      Alternative Proxy

      pFad Proxy

      pFad v3 Proxy

      pFad v4 Proxy