Skip to content

Conversation

@masenf
Copy link
Collaborator

@masenf masenf commented Nov 6, 2025

Try to make the test less flaky by making the iteration wait time equal to the exception_limit_window and reducing the exception_limit from 3 to 2. This should still catch the case in question, but be less timing dependent.

Try to make the test less flaky by making the iteration wait time equal to the
exception_limit_window and reducing the exception_limit from 3 to 2. This
should still catch the case in question, but be less timing dependent.
@codspeed-hq
Copy link

codspeed-hq bot commented Nov 6, 2025

CodSpeed Performance Report

Merging #5959 will not alter performance

Comparing masenf/flaky-ensure-task-limit-window-passed (e01a530) with main (3c3ddc2)

Summary

✅ 8 untouched

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greptile Overview

Greptile Summary

This PR reduces flakiness in test_ensure_task_limit_window_passed by aligning timing parameters. The test now uses exception_limit=2 with exception_limit_window=0.05s, matching the 0.05s sleep duration in the test coroutine. This makes the test more deterministic and less dependent on system timing variations.

Key changes:

  • Reduced exception_limit from 3 to 2
  • Reduced exception_limit_window from 0.1s to 0.05s (now matches the coroutine sleep duration)
  • Test logic remains valid: still verifies that exception counter resets after the window passes

Confidence Score: 5/5

  • This PR is safe to merge with minimal risk - it only modifies test parameters to reduce timing-dependent flakiness
  • The changes are isolated to a single test and correctly align the test timing parameters (sleep duration matches exception window). The test logic remains valid and will still verify the intended behavior while being more deterministic
  • No files require special attention

Important Files Changed

File Analysis

Filename Score Overview
tests/units/utils/test_tasks.py 5/5 Reduced timing dependency by aligning sleep duration with exception window, changed exception_limit from 3→2 and window from 0.1→0.05s

Sequence Diagram

sequenceDiagram
    participant Test as Test Runner
    participant ET as ensure_task
    participant RF as _run_forever
    participant FC as faulty_coro

    Test->>ET: ensure_task(exception_limit=2, window=0.05s)
    ET->>RF: create_task(_run_forever)
    
    Note over RF: Call 1 (T=0)
    RF->>RF: exception_count=0, set last_regular_loop_start=0
    RF->>FC: await faulty_coro()
    FC->>FC: call_count=1, sleep(0.05s)
    FC-->>RF: raise ValueError
    RF->>RF: exception_count=1 (< 2)
    
    Note over RF: Call 2 (T≈0.05s)
    RF->>RF: window not passed, exception_count=1
    RF->>FC: await faulty_coro()
    FC->>FC: call_count=2, sleep(0.05s)
    FC-->>RF: raise ValueError
    RF->>RF: exception_count=2 (>= 2, would raise)
    
    Note over RF: Call 3 (T≈0.10s)
    RF->>RF: window PASSED (0.05 < 0.10), reset exception_count=0
    RF->>FC: await faulty_coro()
    FC->>FC: call_count=3, sleep(0.05s)
    FC-->>RF: raise ValueError
    RF->>RF: exception_count=1 (< 2)
    
    Note over RF: Call 4 (T≈0.15s)
    RF->>RF: exception_count=1
    RF->>FC: await faulty_coro()
    FC->>FC: call_count=4, sleep(0.05s)
    FC-->>RF: raise RuntimeError("Test Passed")
    RF-->>Test: RuntimeError propagates
    Test->>Test: assert call_count == 4 ✓
Loading

1 file reviewed, no comments

Edit Code Review Agent Settings | Greptile

@masenf masenf merged commit 3c564ab into main Nov 12, 2025
58 checks passed
@masenf masenf deleted the masenf/flaky-ensure-task-limit-window-passed branch November 12, 2025 22:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants