19

I know I'm not supposed to modify the list inside a loop, but just out of curiosity, I would like to know why the number of iterations is different between the following two examples.

Example 1:

x = [1, 2, 3, 4, 5]
for i, s in enumerate(x):
    del x[0]
    print(i, s, x)

Example 2:

x = [1,2,3,4,5]
for i, s in enumerate(x):
    x = [1]
    print(i, s, x)

Example 1 runs only 3 times because when i==3, len(x)==2.

Example 2 runs 5 times even though len(x)==1.

So my question is, does enumerate generate a full list of (index, value) pairs at the beginning of the loop and iterate through it? Or are they generated on each iteration of the loop?

Improve this question
5
  • 5
    I can't pretend I know enough to answer your question but as to why it behaves differently here is my guess. In the first case you are deleting from the very same list so it makes sense the iteration stops before. In the second case however you are reassigning it. So python probably considers it as a different variable and keeps going with the original value of x. Commented Mar 7, 2017 at 17:35
  • The same happens without enumerate at all! The for loop does not re-evaluate the iterator, so even if you re-assign x inside the for loop the loop will still use the old value. Obviously if you remove elements from a list, the loop will complete with less iterations. Commented Mar 7, 2017 at 20:00
  • 1
    This has got to be a dupe of something. Commented Mar 8, 2017 at 1:32
  • enumerate needs to work for unbound sequences, so it wouldn't be possible to generate the pairs up front. If you want to, just use list(enumerate(...)) Commented Mar 8, 2017 at 3:52
  • @jpmc26 indeed there is stackoverflow.com/a/986145/3451198 which I did not find yesterday. It does explain the underlying issue very well, however not in the context of a loop. Should this be flagged? I'm rather new at SO and not sure about it. Commented Mar 8, 2017 at 8:41

6 Answers 6

Reset to default
22

In the first example, you're actually modifying the list you're iterating over.

On the other hand, in the second case, you're only assigning a new object to the name x. The object the loop iterates over does not change, though.

Have a look at http://foobarnbaz.com/2012/07/08/understanding-python-variables/ for a more detailed explanation about names and variables in Python.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

Image
Thanks for the answer! I chose Wasi's answer only because it was clearer to understand with the __next__() call.
Image
@dbdq I think it has more to do with how Python variables work, which is probably why this answer has more upvotes.
Image
@dbdq I think with my answer it's clear what is going on here
14

enumerate() returns an iterator, or some other object which supports iteration. The __next__() method of the iterator returned by enumerate() returns a tuple containing a count (from start which defaults to 0) and the values obtained from iterating over iterable.

__next__() returns the next item from the container. If there are no further items, raise the StopIteration exception.

Does enumerate() generate a full list of (index, value) pairs at the beginning of the loop and iterates through it? Or are they generated on each iteration of the loop?

So, enumerate() returns an iterator and at every iteration, __next__() checks if there are further items. enumerate() doesn't create a full list at the beginning of the loop.

As, @Wisperwind mentioned, in your second case, you're assigning a new object to the name x. The object, the loop iterates over does not change during the iteration.

Improve this answer

1 Comment

Image
So the reason that the original x's value [1, 2,..., 5] does not get garbage collected even after assigning x=[0] is because the iterator is still referring to this list.
8

Just a clarification to what Wasi Ahmad and Wisperwind have said. Both state that "you're only assigning a new object to the name x". This might be slightly confusing as it might be interpreted as saying "you're creating a new object ([1]) and storing it to the name x, to which you'd say "Well yah, so why isn't it changing?!" To see what's happening, print out the id of the object 

x = [1, 2, 3, 4, 5]
y = x  # To keep a reference to the original list
print id(x), id(y)
for i, v in enumerate(x):
    x = [1]
    print id(x), id(y)
print id(x), id(y)


# output (somewhat contrived as I don't have a python environment set up)
#    X ID            Y ID
10000000000001 10000000000001
10000000000002 10000000000001
10000000000003 10000000000001
10000000000004 10000000000001
10000000000005 10000000000001
10000000000006 10000000000001
10000000000006 10000000000001

You'll notice that the id of x is changing each time through the loop and when you're finished with the loop, x will point to the last modification made in the loop. When you're going through your loop, it is iterating over the original instance of x, regardless of whether you can still reference it.

As you can see, y points to the original x. As you make your iterations through the loop, even though x is changing, y is still pointing to the original x which is still being looped over.

Improve this answer

4 Comments

Image
Python is very much a reference based language. You don't assign a value to the name x, you assign a reference to the name x.
Image
When I run this code, the id of x inside the loop alternates between 139917134004304 and 139917134053248. This is because the new object is being created in the same location as the last-but-one object. However, that wrinkle would make your answer much more complicated!
Image
In addition, if the OP wants to reduce the original list that x referenced (and that he is looping over), he could write: del x[1:] : x[0] = 1
Image
@MartinBonner Or even: x[:] = [1]
2

Others have already pointed out that your second example only changes the value to which x points, but not the list over which you're iterating. This is a perfect example for the difference between ordinary assignment (x = [1]) and slice assignment (x[:] = [1]). The latter modifies the list x points to in-place:

x = [1, 2, 3, 4, 5]
for i, s in enumerate(x):
    x[:] = [1]
    print(i, s, x)

will print

(0, 1, [1])
Improve this answer

Comments

1

Indeed: Your first snippet modifies the iterated-over list in place; the second points the variable x to a new list, leaving unmodified the list transversed by enumerate(). You can see this in action by going to the following links on www.pythontutor.com, which allow you to single-step over your code and visualize the contents of your variables:

To better see what's going on, go here instead to step over the following expanded code:

x = [1,2,3,4,5]
view = enumerate(x)
for i, s in view:
    x = [1]
    print(i, s, x)
Improve this answer

Comments

0 
x = [1, 2, 3, 4, 5]

The list [1, 2, 3, 4, 5] is 'tagged' with x

for i, s in enumerate(x):

enumerate() attaches another tag, so [1, 2, 3, 4, 5] is now tagged x and y. enumerate() will continue using the y tag, not the x tag. 

del x[0]

The list stored in memory is modified, so x and y now both refer to [2, 3, 4, 5]

Alternatively, when you use

x = [1]

A new list [1] is created in memory, and the x tag now points to that. The y tag is still pointing to the original list.

How Python variable work:
http://foobarnbaz.com/2012/07/08/understanding-python-variables/

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.