Incrementing add under condition in pandas

Question

For the following pandas dataframe

    servo_in_position   second_servo_in_position    Expected output
0   0   1   0
1   0   1   0
2   1   2   1
3   0   3   0
4   1   4   2
5   1   4   2
6   0   5   0
7   0   5   0
8   1   6   3
9   0   7   0
10  1   8   4
11  0   9   0
12  1   10  5
13  1   10  5
14  1   10  5
15  0   11  0
16  0   11  0
17  0   11  0
18  1   12  6
19  1   12  6
20  0   13  0
21  0   13  0
22  0   13  0

I want to increment the column "Expected output" only if "servo_in_position" changes from 0 to 1. I want also to assume "Expected output" to be 0 (null) if "servo_in_position" equals to 0.

I tried

input_data['second_servo_in_position']=(input_data.servo_in_position.diff()!=0).cumsum()

but it gives output as in "second_servo_in_position" column, which is not what I wanted.

After that I would like to group and calculate mean using:

print("Mean=\n\n",input_data.groupby('second_servo_in_position').mean())

user3483203 · Accepted Answer · 2019-07-29 13:36:57Z

11

Using cumsum and arithmetic.

u = df['servo_in_position']

(u.eq(1) & u.shift().ne(1)).cumsum() * u

0     0
1     0
2     1
3     0
4     2
5     2
6     0
7     0
8     3
9     0
10    4
11    0
12    5
13    5
14    5
15    0
16    0
17    0
18    6
19    6
20    0
21    0
22    0
Name: servo_in_position, dtype: int64

answered Jul 29, 2019 at 13:36

user3483203

51.3k10 gold badges72 silver badges104 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Scott Boston · Accepted Answer · 2019-08-06 14:47:52Z

10

Use cumsum and mask:

~~df['E_output'] = df['servo_in_position'].diff().eq(1).cumsum()\ .mask(df['servo_in_position'] == 0, 0)~~

df['servo_in_position'].diff().fillna(df['servo_in_position']).eq(1).cumsum()\
   .mask(df['servo_in_position'] == 0, 0)

Output:

    servo_in_position  second_servo_in_position  Expected output  E_output
0                   0                         1                0         0
1                   0                         1                0         0
2                   1                         2                1         1
3                   0                         3                0         0
4                   1                         4                2         2
5                   1                         4                2         2
6                   0                         5                0         0
7                   0                         5                0         0
8                   1                         6                3         3
9                   0                         7                0         0
10                  1                         8                4         4
11                  0                         9                0         0
12                  1                        10                5         5
13                  1                        10                5         5
14                  1                        10                5         5
15                  0                        11                0         0
16                  0                        11                0         0
17                  0                        11                0         0
18                  1                        12                6         6
19                  1                        12                6         6
20                  0                        13                0         0
21                  0                        13                0         0
22                  0                        13                0         0

Update for first position equal to 1.

df['servo_in_position'].diff().fillna(df['servo_in_position']).eq(1).cumsum()\
   .mask(df['servo_in_position'] == 0, 0)

edited Aug 6, 2019 at 14:47

answered Jul 29, 2019 at 13:36

Scott Boston

154k15 gold badges160 silver badges207 bronze badges

4 Comments

Tomasz Over a year ago

What to do if servo_in_position in 0th row is equal to 1? With your code E_output in 0th row equals to 0, but should be 1.

Scott Boston Over a year ago

@Tomasz Try that updated statement at the bottom. Let me know if that works or not. Thanks.

Scott Boston Over a year ago

@Tomasaz What I did is was to fill that first NaN caused by the diff for the first value, to fill with the first value in 'servo_in_position' hence, it will return the current state of the first value. I think this should work.

Tomasz Over a year ago

It worked, thank you! Here is the code:

import pandas as pd  import numpy as np    data = [1,1,1,0,0,0,1,1,1,1,1,0,0,0,0,1,1,1,1,1,1]  df = pd.DataFrame(data, columns = ['servo_in_position'])   print("Groupping data according to servo positions, please wait...") df['groupped_measurement']=df['servo_in_position'].diff().fillna(df['servo_in_position']).eq(1).cumsum().mask(df['servo_in_position'] == 0, 0) print("Data groupped successfully!") print("Input data:\n",df)

Quang Hoang · Accepted Answer · 2019-07-29 13:33:41Z

7

Try np.where:

df['Expected_output'] = np.where(df.servo_in_position.eq(1),
                                 df.servo_in_position.diff().eq(1).cumsum(),
                                 0)

answered Jul 29, 2019 at 13:33

Quang Hoang

151k11 gold badges64 silver badges86 bronze badges

1 Comment

Tomasz Over a year ago

What to do if servo_in_position in 0th row is equal to 1? With your code E_output in 0th row will equal to 0, but should be 1.

BENY · Accepted Answer · 2019-07-29 13:55:16Z

6

That is cumsum and mul

df.servo_in_position.diff().eq(1).cumsum().mul(df.servo_in_position.eq(1),axis=0)

answered Jul 29, 2019 at 13:55

BENY

324k22 gold badges176 silver badges250 bronze badges

Comments

piRSquared · Accepted Answer · 2019-07-29 15:21:47Z

4

Fast with Numba

from numba import njit

@njit
def f(u):
    out = np.zeros(len(u), np.int64)
    a = out[0] = u[0]
    for i in range(1, len(u)):
        if u[i] == 1:
            if u[i - 1] == 0:
                a += 1
            out[i] = a
    return out

f(df.servo_in_position.to_numpy())

array([0, 0, 1, 0, 2, 2, 0, 0, 3, 0, 4, 0, 5, 5, 5, 0, 0, 0, 6, 6, 0, 0, 0])

answered Jul 29, 2019 at 15:21

piRSquared

296k68 gold badges509 silver badges654 bronze badges

Collectives™ on Stack Overflow

Incrementing add under condition in pandas

5 Answers 5

Comments

Update for first position equal to 1.

4 Comments

1 Comment

Comments

Fast with Numba

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

Comments

Update for first position equal to 1.

4 Comments

1 Comment

Comments

Fast with Numba

Comments

Your Answer

Sign up or log in

Post as a guest

Related