Checking for __iter__ not valid way to check is iterable #1569
Conversation
Some objects have an `__iter__` method even though they are not iterable - `SimpleLazyObject` for ex.
haystack/fields.py
Outdated
| @@ -153,7 +153,9 @@ def get_iterable_objects(cls, current_objects): | |||
| return current_objects.all() | |||
| return [] | |||
|
|
|||
| elif not hasattr(current_objects, '__iter__'): | |||
| try: | |||
| current_objects = [obj for obj in current_objects] | |||
There was a problem hiding this comment.
Is there any example of iterable without __iter__ method? All basic collections, generators and iterators have it.
I only noticed that 'string' object has no __iter__ in python2 (it is present in python3) but string is an atomic value in terms of fields. Also, checking for __iter__ is quite common practice to check for iteration ability.
>>> list().__iter__
<method-wrapper '__iter__' of list object at 0x7fb27115f908>
>>> set().__iter__
<method-wrapper '__iter__' of set object at 0x7fb271185d00>
>>> dict().__iter__
<method-wrapper '__iter__' of dict object at 0x7fb27116bb40>
>>> tuple().__iter__
<method-wrapper '__iter__' of tuple object at 0x7fb27126d050>
>>> (xrange(10)).__iter__
<method-wrapper '__iter__' of xrange object at 0x7fb271166648>
>>> (xrange(10)).__iter__().__iter__()
<rangeiterator object at 0x7fb271169480>
>>> def gen():
... for x in xrange(10): yield x
...
>>> gen().__iter__()
<generator object gen at 0x7fb271155820>
>>> gen().__iter__().__iter__()
<generator object gen at 0x7fb271155a50>
If there is any common case of such a iterable without __iter__, I think that should be shown in test.
There was a problem hiding this comment.
This issue was uncovered a lttle while ago - I can't remember the object that was being given here - if I recall it was a proxy object to a queryset or something to that effect - the proxy itself does not have an iter method, but when passed to a comprehension it behaves as one would expect.
In addition, the above change makes the logic here more pythonic as its a common idiom in python to ask for forgiveness not permission. https://docs.python.org/3.6/glossary.html#term-eafp
|
Failing test: |
|
Oh I wrote which object in that original comment - looks like it was a Not tested: Heres a link to where iter is defined on that object - but the object it's wrapping is not iterable. |
| @@ -153,7 +153,9 @@ def get_iterable_objects(cls, current_objects): | |||
| return current_objects.all() | |||
| return [] | |||
|
|
|||
| elif not hasattr(current_objects, '__iter__'): | |||
| try: | |||
| _ = [obj for obj in current_objects] | |||
There was a problem hiding this comment.
This will consume all the possible generators:
class Test(object):
@property
def test(self):
for x in '12345': yield x
current_objects = Test().test
_ = [c for c in current_objects]
print ', '.join(current_objects)
There was a problem hiding this comment.
Yes — I think the way to handle this would need to be something like this:
try:
iter(current_objects)
except TypeError:
current_objects = [current_objects]| @@ -153,7 +153,9 @@ def get_iterable_objects(cls, current_objects): | |||
| return current_objects.all() | |||
| return [] | |||
|
|
|||
| elif not hasattr(current_objects, '__iter__'): | |||
| try: | |||
| _ = [obj for obj in current_objects] | |||
There was a problem hiding this comment.
Yes — I think the way to handle this would need to be something like this:
try:
iter(current_objects)
except TypeError:
current_objects = [current_objects]
Some objects have an
__iter__method even though they are not iterable -SimpleLazyObjectfor ex.Hey, thanks for contributing to Haystack. Please review the contributor guidelines and confirm that the tests pass with at least one search engine.
Once your pull request has been submitted, the full test suite will be executed on https://travis-ci.org/django-haystack/django-haystack/pull_requests. Pull requests with passing tests are far more likely to be reviewed and merged.