ActualDropouts

Building resilient automation that survives real-world anti-bot defenses and unclear flows—not just ideal conditions but truly hostile web environments. demo : https://youtu.be/6kAAiUxhlhs

Comment

Inspiration

The Deck Dropout Challenge presented a unique opportunity to tackle one of automation's hardest problems: building resilient agents that can navigate real-world, hostile web environments. Traditional automation breaks down when faced with anti-bot mechanisms, dynamic UI changes, and unclear user flows. We were inspired by the challenge of creating an agent that doesn't just work in ideal conditions, but thrives in adversarial environments—much like the challenges faced in production-grade automation systems.

What it does

Our agent autonomously navigates a Mock University Portal to successfully withdraw a student from the university. From login to final confirmation, the system:

How we built it

Tech Stack:

  • Playwright: Primary automation framework for browser control
  • Python: Core logic and orchestration
  • OpenAI API (Claude/GPT-4): LLM-assisted reasoning for DOM interpretation and decision-making
  • Computer Vision: Screenshot analysis for dynamic element detection

Challenges we ran into

  1. Anti-Bot Detection: The portal employed sophisticated detection mechanisms including timing analysis, mouse movement tracking, and behavioral fingerprinting. We had to implement human-like interaction patterns and randomization.
  2. DOM Instability: Elements frequently changed IDs, classes, and structure. We built a multi-strategy selector system that could fall back to semantic understanding rather than relying on brittle selectors.
  3. Hidden Friction: Certain steps only appeared under automated conditions, requiring extensive testing and adaptive logic to discover and handle.
  4. Timing Issues: Balancing speed (under 3 minutes) with reliability required careful optimization of wait times and parallel processing where possible.

Accomplishments that we're proud of

  • 95%+ success rate across multiple test runs with varying starting conditions
  • Zero manual interventions required during successful runs
  • Robust error recovery that handles unexpected modal dialogs, redirects, and timeouts

What we learned

  • AI-assisted automation is powerful but requires guardrails: LLMs can make intelligent decisions, but need proper context and validation to avoid hallucination-based errors.
  • Modular architecture is essential: Separating concerns (authentication, navigation, action execution) made debugging and iteration significantly faster.

What's next for ActualDropouts

  • Performance optimization: Further reducing execution time while maintaining reliability
  • Generalization: Adapting the agent architecture to work across different web portals with minimal configuration

Built With

Share this project:

Updates