Titipee

Inspiration

Titipee was inspired by a simple frustration: too much valuable work is lost to repetitive browser actions. Writing follow-ups, updating CRMs, drafting posts, and navigating complex UI flows can consume hours each week.
We wanted an agent that feels like a real assistant inside the browser, not just a chatbot. We also cared about accessibility and low-friction control: voice input, visible reasoning state, and a hard cancel button so the human always stays in charge.

What it does

Titipee is a Chrome extension + Python backend AI agent built on Amazon Nova. It can:

• Understand the current webpage context and user intent
• Generate high-quality text for real workflows (especially sales follow-ups)
• Execute browser actions with Nova Act
• Support voice input and spoken output
• Generate images and insert them into workflows
• Show live telemetry/metrics in the popup
• Run “Away Mode” through WhatsApp, with QR-based phone-to-profile pairing

In short: you ask, it reasons on-page, acts, and reports results while keeping user control front and center.

How we built it

We built Titipee as a hybrid architecture:

• Frontend: Chrome Extension (Manifest v3)
• Backend: FastAPI (Python)
• Models/Services: Amazon Nova family (reasoning, multimodal, automation)
• Automation bridge: Nova Act
• Cloud ingress: Cloudflare Worker webhook proxy
• Persistence: Postgres (Neon) with SQL migrations
• Storage: Cloudflare R2 for screenshot artifacts

Key engineering pieces:

• Intent + page-context pipeline for better action selection
• Prompt tuning for a high-value use case: Sales Ops Follow-Up Autopilot
• Overlay UX for active reasoning and action lock
• User-first control model (cancel/confirm flows)
• Telemetry endpoint + popup metrics dashboard
• WhatsApp onboarding via QR token pairing (phone number linked to browser profile metadata)

Challenges we ran into

• Model limits and throttling: We hit Bedrock token/day throttles and had to add practical fallback behavior and safer testing patterns.
• UI automation reliability: Real web apps (Gmail, Docs, social sites) are dynamic and brittle; selectors and action timing needed repeated hardening.
• Cross-context actions: Multi-tab and cross-site tasks are non-trivial; we improved navigation/action continuity.
• Voice UX quality: Browser speech APIs vary by state/permissions, so we added stronger error handling and clearer user feedback.
• Secure deploy path: Webhooks, cloud bridge, storage, and profile mapping required careful secret handling and production-safe migration design.

Accomplishments that we're proud of

• Shipped a full-stack, hackathon-ready product experience (not just a demo script)
• Delivered real-time metrics/telemetry with observable runs
• Implemented QR-based WhatsApp identity pairing with persistent profile routing
• Built robust user-control UX (cancel + confirmations + activity visibility)
• Structured the repo and docs for fast judge evaluation and reproducibility

What we learned

• Reliable agentic UX is as much systems engineering as model quality
• Browser automation needs strong fallback paths and observability from day one
• Prompt quality improves dramatically when narrowed to one high-value workflow
• “Human-in-the-loop” controls increase trust and practical adoption
• Shipping production-ish integrations (webhook + DB + storage + auth) is where most hidden complexity lives

What's next for Titipee

• Multi-profile orchestration and policy-based task routing
• Stronger long-running task state + retries + recovery
• Richer voice mode with lower-latency streaming
• Expanded enterprise connectors (CRM/helpdesk/task systems)
• Better safety controls: permissions scopes, audit trails, and approval gates
• Quantified ROI dashboard with formulas like
  [ \text{Time Saved} = N \times (t_{\text{manual}} - t_{\text{agent}}) ] to track measurable business impact over time.

Built With

• html5
• python