Inverting floats in a list of lists [duplicate]

Question

I have the output of a Markov transition table, which is a list of 59 lists each with 59 floats. I want to invert each of the non-0 floats, and then normalise the output so that again I have a list of probabilities which add up to 1.

I have read the textbook on list comprehensions, and that seems relevant, but I can't for the life of me understand how to implement it.

The list of lists is m

for i in range(59):
    [1/item for item in m[i] if item > 0.]
    i += 1

This runs, but it doesn't change m. Am I wrong to be using item in this code? Should I be using some other reference?

Thanks Graipher. I'm not entirely sure what that means. Do you mean something like I need to change the values in the list? Could you show me how to do that? — Richard Hames
– Richard Hames, Commented Mar 15, 2018 at 10:19
Although this calculation in itself doesn't demonstrate a need, I recommend you use numpy arrays rather than Python lists for numerical analysis: here's why. — jpp
– jpp, Commented Mar 15, 2018 at 10:41
This is a (not-exact, but similar enough) duplicate of a question I answered earlier today. — coldspeed95
– coldspeed95, Commented Mar 15, 2018 at 13:15

timgeb · Accepted Answer · 2018-03-15 10:23:49Z

8

[1/item for item in m[i] if item > 0.] creates a new list. Then you do nothing with that list. What Graipher is telling you is that you need a reference to the newly created list:
m[i] = <your list comprehension> will assign back to row i of your list of lists.

Also, consider numpy for element-wise operations. Demo:

>>> import numpy as np
>>> m = np.array([[1,2], [3, 0]])
>>> m = 1.0/m
>>> m
array([[ 1.        ,  0.5       ],
       [ 0.33333333,         inf]])
>>> m[m == np.inf] = 0
>>> m
array([[ 1.        ,  0.5       ],
       [ 0.33333333,  0.        ]])

answered Mar 15, 2018 at 10:23

timgeb

79.3k20 gold badges129 silver badges150 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Graipher · Accepted Answer · 2018-03-15 10:32:25Z

5

You need to assign it back to m[i] in the loop:

for i in range(len(m)):
    m[i] = [1./item if item != 0. else item for item in m[i]]

And the i+=1 is not needed because the for loop does that for you.

In order to avoid the sublist being shorter than the original sublist, you need to move the if from the end of the list comprehension (which basically filters which items are processed) to the front (so the value itself is a ternary expression that evaluates to 1/item or item, depending on the value of item).

At this point you should probably watch the presentation Loop like a Native and rewrite it for example as:

for i, x in enumerate(m):
    m[i] = [1./item if item != 0. else item for item in x]

edited Mar 15, 2018 at 10:32

answered Mar 15, 2018 at 10:20

Graipher

7,25630 silver badges49 bronze badges

3 Comments

Richard Hames Over a year ago

Thanks so much. I now have a list of lists, but they are of different lengths. Is there a way to return a 0 value in the same place if item = 0? There is no 'else' possible after that 'if', as far as I can see.

Graipher Over a year ago

@RichardHames There is. I also changed > 0 to != 0 to allow negative values.

holdenweb Over a year ago

My bad. Apparently I'm having a bad reading day today and should stop answering questions. I just had to completely rewrite an answer to an XML question for similar reasons. Deleted.

Jean-François Fabre · Accepted Answer · 2018-03-15 12:50:20Z

5

[1/item for item in m[i] if item > 0.] alone on a line creates a throwaway object. Not really useful.

My suggestion: rebuild your list of lists using a double comprehension:

m = [[1./item for item in sm if item > 0.] for sm in m]

this solution is Python 2/3 compliant in regards to floating point division whatever the list contains (integers or floats)

EDIT: quoting the question: "I want to invert each of the non-0 floats", the formula is incorrect since it doesn't include the zeroes. So a more accurate version would be not to filter, but to test expression with a ternary so it keeps zeroes.

m = [[1./item if item else 0. for item in sm] for sm in m]

(if item is the shortcut to test item against 0)

edited Mar 15, 2018 at 12:50

answered Mar 15, 2018 at 10:21

Jean-François Fabre♦

141k24 gold badges180 silver badges247 bronze badges

1 Comment

Gsk Over a year ago

I'd go with this solution, but aren't you skipping items <= 0.? Could [[1./item if item > 0. else item for item in sm] for sm in m] improve the solution?

jpp · Accepted Answer · 2018-03-15 10:37:15Z

2

I recommend you use, if possible, tools specifically designed for this purpose.

In the below vectorised solution, inversion occurs in place via numpy, and sklearn.preprocessing.normalize is used to normalize.

import numpy as np
from sklearn.preprocessing import normalize

arr = np.array([1, 2, 0, 1, 1, 0, 4, 0], dtype=float)

def inv(A):
    m = (A != 0)
    A[m] = 1/A[m]
    return A

res = normalize(inv(arr))

# [[ 0.54944226  0.27472113  0.          0.54944226  0.54944226  0.
#    0.13736056  0.        ]]

edited Mar 15, 2018 at 10:37

answered Mar 15, 2018 at 10:31

jpp

166k37 gold badges303 silver badges365 bronze badges

Collectives™ on Stack Overflow

Inverting floats in a list of lists [duplicate]

4 Answers 4

Comments

3 Comments

1 Comment

Comments

Hot Network Questions