32

I have the following code:

s = [1,2,3]
t = reversed(s)

for i in t:
    print(i)
# output: 3,2,1

If I pop one element from s (original), then the t (reversed) is emptied:

s = [1,2,3]
t = reversed(s)
s.pop()

for i in t:
    print(i)
# expected output: 2, 1
# actual output (nothing): 

Why does this happen?

5
  • can not reproduce your issue Commented Jan 4, 2021 at 2:41
  • 3
    @billz I have just run this using 3.8.7 in the command line and it does output nothing for me. Commented Jan 4, 2021 at 2:43
  • 2
    reversed(s) is an iterator, not a reversed list. Commented Jan 4, 2021 at 2:47
  • @blitz - reproducible with python 3.6.9. next(t) throws a StopIteration exception, which explains why nothing is printed. But I don't know why list(t) is []. I think this is a very good question. Commented Jan 4, 2021 at 2:50
  • @MichaelSzczesny: It's interesting, but not too surprising. t isn't a reversed list, it's basically just a wrapped iterator around s. Don't modify an iterator before you've consumed it, otherwise you get "surprising" results. Commented Jan 4, 2021 at 17:41

2 Answers 2

30

Taking a look at the cpython code on GitHub, we can get some intuition as to why it no longer works.

The iterator that is returned essentially requires knowing the position of the last index and the length of the array. If the size of the array is changed, the iterator will no longer work.

Test 1: Increasing the array length

This will not produce the correct results either, but the iterator does run:

s = [1,2,3]
t = reversed(s)
s.append(4)

for i in t:
    print(i)
# output: [3, 2, 1]

Test 2: Decreasing, then increasing the length

s = [1,2,3]
t = reversed(s)
s.pop()
s.append(4)

for i in t:
    print(i)
# output: [4, 2, 1]

It still works!

So there's an internal check to see whether or not the last index is still valid, and if it is, it's a simple for loop down to index 0.

If it doesn't work, the iterator returns empty.

Sign up to request clarification or add additional context in comments.

5 Comments

The check is in line 3339: if (index>=0 && index < PyList_GET_SIZE(seq)) where seq is a pointer to the original list object. If the PyList_GET_SIZE check fails the reference is removed and NULL is returned.
@MichaelSzczesny Feel free to add that to the answer if you want, instead of just a comment, since you were the one that pointed it out :D.
I updated the link to github to point in the current revision to the relevant line numbers.
@MZ I just created pure python implementation(for readability) of list reverse iterator. Would you mind adding on to this answer?
@AbdulNiyasPM: It looks good, and seems to show the same behaviour as the cpython code.
9

calling reversed return a iterator over that list, which a special object that allow you iterate in reverse order over the original list, is not a new list and is a one time use only

>>> s= [1,2,3]
>>> t = reversed(s)
>>> t
<list_reverseiterator object at 0x00000261BE8F0C40>
>>> list(t)
[3, 2, 1]
>>> list(t)
[]
>>> 

and because this iterator reference the original list, any change on it is reflected when you iterate over the iterator later.

Update

In particular and as MZ explain, if that change is such that the state of the list is different from when the iterator was created you get nothing if the size decreases or an incomplete version of the list if increased

>>> s= [1,2,3]
>>> t = reversed(s)
>>> s.insert(0,23)
>>> s
[23, 1, 2, 3]
>>> list(t)
[2, 1, 23]
>>> t = reversed(s)
>>> s.append(32)
>>> list(t)
[3, 2, 1, 23]
>>> s
[23, 1, 2, 3, 32]
>>> t = reversed(s)
>>> s.pop()
32
>>> list(t)
[]
>>> 

4 Comments

This does not answer the question. The item is only removed from the list and the iterator is untouched.
@Deadbeef The iterator is backed by the original list, though, so changing the state of the list affects the iterator in the sense that it is not necessarily valid after the alteration. In most languages, it's usually a Bad Idea to alter a collection while you're iterating over it using an iterator pattern.
@Deadbeef: It does answer the question. reversed(s) is basically just a wrapper around s, so it's fast to initialize and doesn't require much memory. But...
at first I get the full extend of the question, but by then MZ put his, I leave mine as a little complement to that

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.