The rolling method of fireducks.pandas.frame.DataFrame does not support window argument of type timedelta, str or offset

[ButteryPaws]

According to the Pandas documentation for pandas.DataFrame.rolling, the first argument rolling can be of type timedelta (e.g. pd.Timedelta(days=1)), str (e.g. 30min). These arguments are valid only when the index of the dataframe is of type pandas.DatetimeIndex and are immensely when working with time-series data.

Reproducible Code

import pandas as pd
import numpy as np

def get_rolling_sum(df: pd.DataFrame) -> pd.DataFrame:
    rolling_sum = df.rolling(window='15min', min_periods=5).sum()
    return rolling_sum

index = pd.date_range('2025-04-07 09:00:00', '2025-04-07 17:00:00', freq='min')
data = np.random.randn(len(index), 2)
df = pd.DataFrame(data=data, index=index)
rolling_sum_pandas = get_rolling_sum(df)

import fireducks.pandas as pd
df = pd.DataFrame(df)
rolling_sum_fireducks = get_rolling_sum(df)

The Error

Traceback (most recent call last):
  File "/home/user/src/temp.py", line 15, in <module>
    rolling_sum_fireducks = get_rolling_sum(df)
                            ^^^^^^^^^^^^^^^^^^^
  File "/home/user/src/temp.py", line 5, in get_rolling_sum
    rolling_sum = df.rolling('15min', min_periods=5).sum()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.venv/lib/python3.12/site-packages/fireducks/pandas/rolling.py", line 124, in sum
    return self.aggregate("sum")
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.venv/lib/python3.12/site-packages/fireducks/pandas/rolling.py", line 93, in aggregate
    if rolling_ns.min_periods > rolling_ns.window:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: '>' not supported between instances of 'int' and 'str'

On experimenting with other arguments to window which are not integers, the error persists. It seems like the support for non-integer arguments, specifically those involving DatetimeIndex of the dataframe and time-related operations has not been implemented yet.

System specifications

Python version: 3.12.3
Pandas version: 2.2.3
Fireducks version: 1.2.5
NumPy version: 2.1.2

[qsourav]

Hi @ButteryPaws,

Thank you very much for reporting the bug.

It seems like the support for non-integer arguments, specifically those involving DatetimeIndex of the dataframe and time-related operations has not been implemented yet.

As rightly mentioned, FireDucks backend kernel currently supports only numeric window (integer type) and for all other cases it should fallback to pandas (instead of throwing an error). Due to some if-else misses, the comparision of non-numeric window ("15mins") is taking place with the numeric min_periods (5), causing the error you are facing.

We will fix this bug in next release, but supporting non-numeric window (like your case) at FireDucks backend, might take some time.
For the time being, if your index is actually a DateTimeIndex with frequency as "min", can you try the following as an alternative (to use the optimized FireDucks kernel)?

# rolling_sum = df.rolling(window='15min', min_periods=5).sum()
rolling_sum = df.rolling(window=15, min_periods=5).sum()

Thanks and Regards,
Sourav Saha

[ButteryPaws]

Hi @qsourav ,
Thanks for your quick response. Really hoping to see this issue fixed in the next release.

For the time being, if your index is actually a DateTimeIndex with frequency as "min", can you try the following as an alternative (to use the optimized FireDucks kernel)?

# rolling_sum = df.rolling(window='15min', min_periods=5).sum()
rolling_sum = df.rolling(window=15, min_periods=5).sum()

I cannot use this because my index is not necessarily uniform or continuous. For example, I can have the index as

index
2025-04-08 09:00:00
2025-04-08 09:01:00
2025-04-08 09:02:00
2025-04-08 09:04:00
2025-04-08 09:04:20
2025-04-08 09:04:20
2025-04-08 09:04:21
2025-04-08 09:05:00
2025-04-08 09:06:00
2025-04-08 09:08:00
2025-04-08 09:09:00
2025-04-08 09:15:00
2025-04-08 09:15:30
2025-04-08 09:16:00
2025-04-08 09:16:01

While the example I mentioned in the code was quite simple, it does not reflect the complexity of the real data I am working with. Hence, having a time-based rolling window which efficiently checks for which rows can be aggregated (number can be a variable) is immensely useful.

Looking forward to having an algorithm for this being pushed as well. Thank you!

[qsourav]

Dear @ButteryPaws,

Thank you for sharing further details with us. I understood the use case.

We will surely work on this:

• 1st step: fix the bug: by correctly falling back to pandas without the runtime error: will be added in next release (by next week)
• 2nd step: support the algorithm at the fireducks backend: will take some time

Thanks and Regards,
Sourav Saha

[qsourav]

Hi @ButteryPaws,

(1) We have fixed the runtime error in FireDucks 1.2.7. It can just work fine without errors, but the performance will be somewhat similar to pandas (since the backend currently doesn't support rolling_aggregate based on DatetimeIndex).

(2) We will notify you when we implement the DatetimeIndex support at the backend for rolling_aggregate related operations. Can you please tell me what all the aggregate methods you are using? We will try to prioritize their implementation.

Thanks and Regards,
Sourav Saha

[ButteryPaws]

Hello @qsourav ,
Thank you very much for fixing the problem so quickly and responding!

1. That's all right. I hope you will implement the optimized Fireducks version soon. I hope this project is open-sourced in the future so that the community can take a chance at fixing these issues!
2. I mainly use rolling windows to compute aggregate statistics such as mean, median, std, count, quantile, etc. I avoid using apply because it is slow and not vectorised in Pandas, however, it is quite useful, so if you could implement an efficient version of that, it would be awesome.

Thanks again and best regards.