User Flows And Pathways

My experience of working at Urban Company taught me one key lesson about Indian consumers: Convenience may be tempting, but control wins every single time! You see, I was a product guy at UC then. - And Abhiraj (the founder and CEO) had tasked me and my colleague Sripad (now heads product at Dezerv) to improve new user conversions - In our shoes, most people would have thought of shortening the flow by selecting a few options by default, so the user would have to make fewer decisions or taps - But, we went by the approach of adding more options that the user could choose. This was because we wanted to make the user feel way more in control and in charge - Driving the calls And this worked wonders for our conversion rates. Why? Because customer trust went up massively. Thus, when launching the feature to schedule weekly bookings for our Dubai business, we actually added a step, elongating the flow. And again, we saw conversions go up! This taught me: - You can cut a user journey by two clicks, nail a sleek UI, and still see drop-offs. Why? The user didn’t feel in charge - In a country where ration shops and bank queues have taught people to expect friction, control is power. Control is trust And anything that makes the user feel that they hold the decision making, the control - IT WINS! And this shouldn’t be surprising. I’ve seen users manually enter OTPs over auto-read because typing feels safer. They skip recommendations to re-search, ensuring they’re not tricked. That’s not inefficiency - it’s defence. Ignore this, and your retention tanks. A good example is that of a fintech (I won’t name) which launched an “auto-invest” feature - It ended up driving away 20% of users who felt sidelined. But apps like PhonePe thrived with the same product with “confirm payment” prompts. - It’s pretty simple and logical. Every flow should ask: “Where does the user say ‘I’m in charge’?” - A “you can change this later” label, a manual toggle, or a “review before submit” step builds comfort Zomato’s customisable delivery instructions are one more example. These trust signals scale because they align with India’s psyche, where almost every user prefers double-checking. Thus, I now always recommend to founders in my circles, if you are building for Indian audiences, audit for control points. Add confirmations, transparent labels like “No hidden fees.” Don’t force automation - offer manual options. Test retention, not just conversion. Study PhonePe or Paytm’s “over-communicative” designs. Those extra prompts aren’t accidents - they’re trust engines. They’re not hurdles - they’re well planned and well placed handshakes. What do you think? Do share below. Best, Shiv

🤖 How To Design Better AI Experiences. With practical guidelines on how to add AI when it can help users, and avoid it when it doesn’t ↓ Many articles discuss AI capabilities, yet most of the time the issue is that these capabilities either feel like a patch for a broken experience, or they don't meet user needs at all. Good AI experiences start like every good digital product by understanding user needs first. 🚫 AI isn’t helpful if it doesn’t match existing user needs. 🤔 AI chatbots are slow, often expose underlying UX debt. ✅ First, we revisit key user journeys for key user segments. ✅ We examine slowdowns, pain points, repetition, errors. ✅ We track accuracy, failure rates, frustrations, drop-offs. ✅ We also study critical success moments that users rely on. ✅ Next, we ideate how AI features can support these needs. ↳ e.g. Estimate, Compare, Discover, Identify, Generate, Act. ✅ Bring data scientists, engineers, PMs to review/prioritize. 🤔 High accuracy > 90% is hard to achieve and rarely viable. ✅ Design input UX, output UX, refinement UX, failure UX. ✅ Add prompt presets/templates to speed up interaction. ✅ Embed new AI features into existing workflows/journeys. ✅ Pre-test if customers understand and use new features. ✅ Test accuracy + success rates for users (before/after). As designers, we often set unrealistic expectations of what AI can deliver. AI can’t magically resolve accumulated UX debt or fix broken information architecture. If anything, it visibly amplifies existing inconsistencies, fragile user flows and poor metadata. Many AI features that we envision simply can’t be built as they require near-perfect AI performance to be useful in real-world scenarios. AI can’t be as reliable as software usually should be, so most AI products don’t make it to the market. They solve the wrong problem, and do so unreliably. As a result, AI features often feel like a crutch for an utterly broken product. AI chatbots impose the burden of properly articulating intent and refining queries to end customers. And we often focus so much on AI that we almost intentionally avoid much-needed human review out of the loop. Good AI-products start by understanding user needs, and sparkling a bit of AI where it helps people — recover from errors, reduce repetition, avoid mistakes, auto-correct imported files, auto-fill data, find insights. AI features shouldn’t feel disconnected from the actual user flow. Perhaps the best AI in 2025 is “quiet” — without any sparkles or chatbots. It just sits behind a humble button or runs in the background, doing the tedious job that users had to slowly do in the past. It shines when it fixes actual problems that it has, not when it screams for attention that it doesn’t deserve. Useful resources: AI Design Patterns, by Emily Campbell https://www.shapeof.ai AI Product-Market-Fit Gap, by Arvind Narayanan, Sayash Kapoor https://lnkd.in/duEja695 [continues in comments ↓]

Over the last year, I’ve seen many people fall into the same trap: They launch an AI-powered agent (chatbot, assistant, support tool, etc.)… But only track surface-level KPIs — like response time or number of users. That’s not enough. To create AI systems that actually deliver value, we need 𝗵𝗼𝗹𝗶𝘀𝘁𝗶𝗰, 𝗵𝘂𝗺𝗮𝗻-𝗰𝗲𝗻𝘁𝗿𝗶𝗰 𝗺𝗲𝘁𝗿𝗶𝗰𝘀 that reflect: • User trust • Task success • Business impact • Experience quality    This infographic highlights 15 𝘦𝘴𝘴𝘦𝘯𝘵𝘪𝘢𝘭 dimensions to consider: ↳ 𝗥𝗲𝘀𝗽𝗼𝗻𝘀𝗲 𝗔𝗰𝗰𝘂𝗿𝗮𝗰𝘆 — Are your AI answers actually useful and correct? ↳ 𝗧𝗮𝘀𝗸 𝗖𝗼𝗺𝗽𝗹𝗲𝘁𝗶𝗼𝗻 𝗥𝗮𝘁𝗲 — Can the agent complete full workflows, not just answer trivia? ↳ 𝗟𝗮𝘁𝗲𝗻𝗰𝘆 — Response speed still matters, especially in production. ↳ 𝗨𝘀𝗲𝗿 𝗘𝗻𝗴𝗮𝗴𝗲𝗺𝗲𝗻𝘁 — How often are users returning or interacting meaningfully? ↳ 𝗦𝘂𝗰𝗰𝗲𝘀𝘀 𝗥𝗮𝘁𝗲 — Did the user achieve their goal? This is your north star. ↳ 𝗘𝗿𝗿𝗼𝗿 𝗥𝗮𝘁𝗲 — Irrelevant or wrong responses? That’s friction. ↳ 𝗦𝗲𝘀𝘀𝗶𝗼𝗻 𝗗𝘂𝗿𝗮𝘁𝗶𝗼𝗻 — Longer isn’t always better — it depends on the goal. ↳ 𝗨𝘀𝗲𝗿 𝗥𝗲𝘁𝗲𝗻𝘁𝗶𝗼𝗻 — Are users coming back 𝘢𝘧𝘵𝘦𝘳 the first experience? ↳ 𝗖𝗼𝘀𝘁 𝗽𝗲𝗿 𝗜𝗻𝘁𝗲𝗿𝗮𝗰𝘁𝗶𝗼𝗻 — Especially critical at scale. Budget-wise agents win. ↳ 𝗖𝗼𝗻𝘃𝗲𝗿𝘀𝗮𝘁𝗶𝗼𝗻 𝗗𝗲𝗽𝘁𝗵 — Can the agent handle follow-ups and multi-turn dialogue? ↳ 𝗨𝘀𝗲𝗿 𝗦𝗮𝘁𝗶𝘀𝗳𝗮𝗰𝘁𝗶𝗼𝗻 𝗦𝗰𝗼𝗿𝗲 — Feedback from actual users is gold. ↳ 𝗖𝗼𝗻𝘁𝗲𝘅𝘁𝘂𝗮𝗹 𝗨𝗻𝗱𝗲𝗿𝘀𝘁𝗮𝗻𝗱𝗶𝗻𝗴 — Can your AI 𝘳𝘦𝘮𝘦𝘮𝘣𝘦𝘳 𝘢𝘯𝘥 𝘳𝘦𝘧𝘦𝘳 to earlier inputs? ↳ 𝗦𝗰𝗮𝗹𝗮𝗯𝗶𝗹𝗶𝘁𝘆 — Can it handle volume 𝘸𝘪𝘵𝘩𝘰𝘶𝘵 degrading performance? ↳ 𝗞𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 𝗘𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝗰𝘆 — This is key for RAG-based agents. ↳ 𝗔𝗱𝗮𝗽𝘁𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝗦𝗰𝗼𝗿𝗲 — Is your AI learning and improving over time? If you're building or managing AI agents — bookmark this. Whether it's a support bot, GenAI assistant, or a multi-agent system — these are the metrics that will shape real-world success. 𝗗𝗶𝗱 𝗜 𝗺𝗶𝘀𝘀 𝗮𝗻𝘆 𝗰𝗿𝗶𝘁𝗶𝗰𝗮𝗹 𝗼𝗻𝗲𝘀 𝘆𝗼𝘂 𝘂𝘀𝗲 𝗶𝗻 𝘆𝗼𝘂𝗿 𝗽𝗿𝗼𝗷𝗲𝗰𝘁𝘀? Let’s make this list even stronger — drop your thoughts 👇

LLMOps is about running LLMs like real products with feedback loops, monitoring, and continuous improvement baked in 💯 This visual breaks it down into 14 steps that make LLMs production-ready and future-proof. 🔹 Steps 1-2: Collect Data + Clean & Organize Where does any good model start? With data. You begin by collecting diverse, relevant sources: chats, documents, logs, anything your model needs to learn from. Then comes the cleanup. Remove noise, standardize formats, and structure it so the model doesn’t get confused by junk. 🔹 Steps 3-4: Add Metadata + Version Your Dataset Now that your data is clean, give it context. Metadata tells you the source, intent, and type of each data point: this is key for traceability. Once that’s done, store everything in a versioned repository. Why? Because every future change needs a reference point. No versioning = no reproducibility. 🔹 Steps 5-6: Select Base Model + Fine-Tune Here’s where the model work begins. You choose a base model like GPT, Claude, or an open-source LLM depending on your task and compute budget. Then, you fine-tune it on your versioned dataset to adapt it to your specific domain, whether that’s law, health, support, or finance. 🔹 Steps 7-8: Validate Output + Register the Model Fine-tuning done? Cool, and now test it thoroughly. Run edge cases, evaluate with test prompts, and check if it aligns with expectations. Once it passes, register the model so it’s tracked, documented, and ready for deployment. This becomes your source of truth. 🔹 Steps 9-10: Deploy API + Monitor Usage The model is ready! You expose it via an API for apps or users to interact with. Then you monitor everything: requests, latency, failure cases, prompt patterns. This is where real-world insights start pouring in. 🔹 Steps 11-12: Collect Feedback + Store in User DB You gather feedback from users: explicit complaints, implicit behavior, corrections, and even prompt rephrasing. All of that goes into a structured user database. Why? Because this becomes the compass for your next update. 🔹 Steps 13-14: Decide on Updates + Monitor Continuously Here’s the big question: Is your model still doing well? Based on usage and feedback, you decide: continue as is or loop back and improve. And even if things seem fine, you never stop monitoring. Model performance can drift fast. 📚 Research and Curation Effort: 4 hours If you've found it helpful, please like and repost it to uplift your network ♻️ Follow me, Bhavishya Pandit, to stay ahead in Generative AI! ❤️ #llm #opensource #rag #meta #google #ibm #openai #gpt4 #ml #machinelearning #ai #artificialintelligence #datascience #python #genai #generativeai #huggingface #openai #linkedin #computervision

Are you generating enough value for users net of the value to your company? Business value can only be created when you create so much value for users, that you can “tax” that value and take some for yourself as a business. If you don’t create any value for your users, then you can’t create value for your business. Ed Biden explains how to solve this in this week's guest post: Whilst there are many ways to understand what your users will value, two techniques in particular are incredibly valuable, especially if you’re working on a tight timeframe: 1. Jobs To Be Done 2. Customer Journey Mapping 𝟭. 𝗝𝗼𝗯𝘀 𝗧𝗼 𝗕𝗲 𝗗𝗼𝗻𝗲 (𝗝𝗧𝗕𝗗) “People don’t simply buy products or services, they ‘hire’ them to make progress in specific circumstances.”  – Clayton Christensen The core JTBD concept is that rather than buying a product for its features, customers “hire” a product to get a job done for them … and will ”fire” it for a better solution just as quickly. In practice, JTBD provides a series of lenses for understanding what your customers want, what progress looks like, and what they’ll pay for. This is a powerful way of understanding your users, because their needs are stable and it forces you to think from a user-centric point of view. This allows you to think about more radical solutions, and really focus on where you’re creating value. To use Jobs To Be Done to understand your customers, think through five key steps: 1. Use case – what is the outcome that people want? 2. Alternatives – what solutions are people using now? 3. Progress – where are people blocked? What does a better solution look like? 4. Value Proposition – why would they use your product over the alternatives? 5. Price – what would a customer pay for progress against this problem? 𝟮. 𝗖𝘂𝘀𝘁𝗼𝗺𝗲𝗿 𝗝𝗼𝘂𝗿𝗻𝗲𝘆 𝗠𝗮𝗽𝗽𝗶𝗻𝗴 Customer journey mapping is an effective way to visualize your customer’s experience as they try to reach one of their goals. In basic terms, a customer journey map breaks the user journey down into steps, and then for each step describes what touchpoints the customer has with your product, and how this makes them feel. The touch points are any interaction that the customer has with your company as they go through this flow: • Website and app screens • Notifications and emails • Customer service calls • Account management / sales touch points • Physically interacting with goods (e.g. Amazon), services (e.g. Airbnb) or hardware (e.g. Lime) Users’ feelings can be visualized by noting down: • What they like or feel good about at this step • What they dislike, find frustrating or confusing at this step • How they feel overall By mapping the customer’s subjective experience to the nuts and bolts of what’s going on, and then laying this out in a visual way, you can easily see where you can have the most impact, and align stakeholders on the critical problems to solve.

I redesigned my entire UX/UI process with AI. It’s not about “use ChatGPT to brainstorm.” I mean, I rebuilt the whole pipeline. From product idea to prototype. What used to take months? Now gets done in days. Here’s what it looks like step-by-step: 1. Instant User Flows I drop rough product ideas into ChatGPT. (It's not the public one; it's a custom GPT trained on how I think.) It gives me: - Sitemap - User journey - Logic flows All in less time than it takes to make coffee. 2. Wireframes Without Drawing I stopped sketching. I describe the layout in plain English, and Magician does the rest. "Hero. CTA. Testimonials." Boom. Wireframe. No more dragging boxes like it’s 2015. 3. AI-Built Design System Spacing? Typography? Button styles? I just describe the vibe. Tools like Relume and Uizard take that and build me a full design system. This used to take WEEKS. Now it’s done before lunch. 4. Smarter Figma Time Now everything moves to Figma. But I don’t waste time pixel-pushing. AI plugins handle: - spacing - responsiveness - and accessibility. I just make the ideas click. 5. Prototyping = Auto-On Final step? Auto-connect flows with Figma’s AI tools. Clickable. Shareable. Client-ready. Dev-approved. No extra buttons. No guesswork. Here’s the real punchline: AI didn’t replace my work. It replaced the boring parts, so I can focus on design thinking. It’s not about working faster. It’s about designing smarter. We’re not in 2015 anymore. Let’s build like it’s 2030. What part of your UX workflow do you still do manually? Curious to hear.

Every PM wants to measure the success of their product. But most struggle to do it correctly. As a product management hiring manager, leader, and coach, I've seen that many product managers struggle with defining the right success metrics They focus on generic metrics like acquisition, engagement,  retention These are insufficient. My recommendation is to ask concrete questions when thinking of metrics Here's a list of questions I ask: 𝗧𝗵𝗶𝗻𝗸 𝗮𝗯𝗼𝘂𝘁 𝘁𝗵𝗲 𝘂𝘀𝗲𝗿 𝗳𝗶𝗿𝘀𝘁 1. What is the user’s goal? 2. What human need do they want to fulfill? 3. What action signifies that their need is met? 4. Is that action enough to know user’s job is done? 5. How can I measure that action? 𝗧𝗵𝗶𝗻𝗸 𝗮𝗯𝗼𝘂𝘁 𝘂𝘀𝗮𝗴𝗲 𝗮𝗻𝗱 𝗮𝗱𝗼𝗽𝘁𝗶𝗼𝗻 1. How many users are using the product? 2. How many users should be using it? 3. Which users aren't using it but should be using it? 𝗧𝗵𝗶𝗻𝗸 𝗮𝗯𝗼𝘂𝘁 𝗵𝗼𝘄 𝗺𝘂𝗰𝗵 𝘂𝘀𝗲𝗿𝘀 𝗲𝗻𝗷𝗼𝘆 𝘆𝗼𝘂𝗿 𝗽𝗿𝗼𝗱𝘂𝗰𝘁 1. How many users like the product? 2. How much do they like it? 3. What action(s) show they “like” it? 4. How can I measure those actions 5. Do they like it enough to keep coming back? 6. If yes, how often should they come back? 𝗧𝗵𝗶𝗻𝗸 𝗮𝗯𝗼𝘂𝘁 𝘁𝗵𝗲 𝗾𝘂𝗮𝗹𝗶𝘁𝘆 𝗼𝗳 𝗲𝘅𝗽𝗲𝗿𝗶𝗲𝗻𝗰𝗲 𝘁𝗵𝗲𝘆 𝗮𝗿𝗲 𝗴𝗲𝘁𝘁𝗶𝗻𝗴 𝘄𝗵𝗶𝗹𝗲 𝘂𝘀𝗶𝗻𝗴 𝘁𝗵𝗲 𝗽𝗿𝗼𝗱𝘂𝗰𝘁 1. Are users finding it hard to complete certain actions? 2. Are there things that users dislike? 3. Are there enough options for users to choose from? 4. Are there things that users want to do, but the product doesn’t allow them to? 5. Can we measure all the above? 𝗧𝗵𝗶𝗻𝗸 𝗮𝗯𝗼𝘂𝘁 𝘁𝗵𝗲 𝗾𝘂𝗮𝗹𝗶𝘁𝘆 𝗼𝗳 𝗺𝗲𝘁𝗿𝗶𝗰𝘀 1. Can I cheat on any of the above metrics? 2. Do above metrics give the most accurate answer? 3. Are all metrics simple enough for everyone to understand? 𝗧𝗵𝗶𝗻𝗸 𝗮𝗯𝗼𝘂𝘁 𝘁𝗵𝗲 𝗻𝗲𝘁 𝗶𝗺𝗽𝗮𝗰𝘁 𝗼𝗻 𝘁𝗵𝗲 𝗼𝘃𝗲𝗿𝗮𝗹𝗹 𝗽𝗿𝗼𝗱𝘂𝗰𝘁/𝗰𝗼𝗺𝗽𝗮𝗻𝘆 1. Are  above metrics a true representation of success? 2. Any other parts of user journey I should measure? 3. Will a positive impact on above metrics lead to a negative impact on other critical metrics? 4. Is the tradeoff acceptable? -- How easy or tough do you find creating success metrics? What is your process?

LLMs are optimized for next turn response. This results in poor Human-AI collaboration, as it doesn't help users achieve their goals or clarify intent. A new model CollabLLM is optimized for long-term collaboration. The paper "CollabLLM: From Passive Responders to Active Collaborators" by Stanford University and Microsoft researchers tests this approach to improving outcomes from LLM interaction. (link in comments) 💡 CollabLLM transforms AI from passive responders to active collaborators. Traditional LLMs focus on single-turn responses, often missing user intent and leading to inefficient conversations. CollabLLM introduces a :"Multiturn-aware reward" system, apply reinforcement fine-tuning on these rewards. This enables AI to engage in deeper, more interactive exchanges by actively uncovering user intent and guiding users toward their goals. 🔄 Multiturn-aware rewards optimize long-term collaboration. Unlike standard reinforcement learning that prioritizes immediate responses, CollabLLM uses forward sampling - simulating potential conversations - to estimate the long-term value of interactions. This approach improves interactivity by 46.3% and enhances task performance by 18.5%, making conversations more productive and user-centered. 📊 CollabLLM outperforms traditional models in complex tasks. In document editing, coding assistance, and math problem-solving, CollabLLM increases user satisfaction by 17.6% and reduces time spent by 10.4%. It ensures that AI-generated content aligns with user expectations through dynamic feedback loops. 🤝 Proactive intent discovery leads to better responses. Unlike standard LLMs that assume user needs, CollabLLM asks clarifying questions before responding, leading to more accurate and relevant answers. This results in higher-quality output and a smoother user experience. 🚀 CollabLLM generalizes well across different domains. Tested on the Abg-CoQA conversational QA benchmark, CollabLLM proactively asked clarifying questions 52.8% of the time, compared to just 15.4% for GPT-4o. This demonstrates its ability to handle ambiguous queries effectively, making it more adaptable to real-world scenarios. 🔬 Real-world studies confirm efficiency and engagement gains. A 201-person user study showed that CollabLLM-generated documents received higher quality ratings (8.50/10) and sustained higher engagement over multiple turns, unlike baseline models, which saw declining satisfaction in longer conversations. It is time to move beyond the single-step LLM responses that we have been used to, to interactions that lead to where we want to go. This is a useful advance to better human-AI collaboration. It's a critical topic, I'll be sharing a lot more on how we can get there.

Your research findings are useless if they don't drive decisions. After watching countless brilliant insights disappear into the void, I developed 5 practical templates I use to transform research into action: 1. Decision-Driven Journey Map Standard journey maps look nice but often collect dust. My Decision-Driven Journey Map directly connects user pain points to specific product decisions with clear ownership. Key components: - User journey stages with actions - Pain points with severity ratings (1-5) - Required product decisions for each pain - Decision owner assignment - Implementation timeline This structure creates immediate accountability and turns abstract user problems into concrete action items. 2. Stakeholder Belief Audit Workshop Many product decisions happen based on untested assumptions. This workshop template helps you document and systematically test stakeholder beliefs about users. The four-step process: - Document stakeholder beliefs + confidence level - Prioritize which beliefs to test (impact vs. confidence) - Select appropriate testing methods - Create an action plan with owners and timelines When stakeholders participate in this process, they're far more likely to act on the results. 3. Insight-Action Workshop Guide Research without decisions is just expensive trivia. This workshop template provides a structured 90-minute framework to turn insights into product decisions. Workshop flow: - Research recap (15min) - Insight mapping (15min) - Decision matrix (15min) - Action planning (30min) - Wrap-up and commitments (15min) The decision matrix helps prioritize actions based on user value and implementation effort, ensuring resources are allocated effectively. 4. Five-Minute Video Insights Stakeholders rarely read full research reports. These bite-sized video templates drive decisions better than documents by making insights impossible to ignore. Video structure: - 30 sec: Key finding - 3 min: Supporting user clips - 1 min: Implications - 30 sec: Recommended next steps Pro tip: Create a library of these videos organized by product area for easy reference during planning sessions. 5. Progressive Disclosure Testing Protocol Standard usability testing tries to cover too much. This protocol focuses on how users process information over time to reveal deeper UX issues. Testing phases: - First 5-second impression - Initial scanning behavior - First meaningful action - Information discovery pattern - Task completion approach This approach reveals how users actually build mental models of your product, leading to more impactful interface decisions. Stop letting your hard-earned research insights collect dust. I’m dropping the first 3 templates below, & I’d love to hear which decision-making hurdle is currently blocking your research from making an impact! (The data in the templates is just an example, let me know in the comments or message me if you’d like the blank versions).

Everyone’s excited to launch AI agents. Almost no one knows how to measure if they’re actually working. Over the last year, we’ve seen brands launch everything from GenAI assistants to support bots to creative copilots but the post-launch metrics often look like this: • Number of chats • Average latency • Session duration • Daily active users Useful? Yes. But sufficient? Not even close. At ALTRD, we’ve worked on AI agents for enterprises and if there’s one lesson it’s this: Speed and usage mean nothing if the agent isn’t solving the actual problem. The real performance indicators are far more nuanced. Here’s what we’ve learned to track instead: 🔹 Task Completion Rate — Can the AI go beyond answering a question and actually complete a workflow? 🔹 User Trust — Do people come back? Do they feel confident relying on the agent again? 🔹 Conversation Depth — Is the agent handling complex, multi-turn exchanges with consistency? 🔹 Context Retention — Can it remember prior interactions and respond accordingly? 🔹 Cost per Successful Interaction — Not just cost per query, but cost per outcome. Massive difference. One of our clients initially celebrated their bot’s 1 million+ sessions - until we uncovered that less than 8% of users actually got what they came for. That 8% wasn’t a usage issue. It was a design and evaluation issue. They had optimized for traffic. Not trust. Not success. Not satisfaction. So we rebuilt the evaluation framework - adding feedback loops, success markers, and goal-completion metrics. The results? CSAT up by 34% Drop-off down by 40% Same infra cost, 3x more value delivered The takeaway: Don’t just measure what’s easy. Measure what matters. AI agents aren’t just tools - they’re touchpoints. They represent your brand, shape user experience, and influence business outcomes. P.S. What’s one underrated metric you’ve used to evaluate AI performance? Curious to learn what others are tracking.