Improving User Interaction Models

Exciting research from Snap Inc.'s engineering team! Just came across their paper on Universal User Modeling (UUM) that's revolutionizing how they handle cross-domain user representations. The team at Snap has developed a framework that learns general-purpose user representations by leveraging behaviors across multiple in-app surfaces simultaneously. Rather than building separate user models for each surface (Content, Ads, Lens, etc.) and combining them post-hoc, UUM directly captures collaborative filtering signals across domains. Their approach formulates this as a cross-domain sequential recommendation problem, processing user interaction sequences of up to 5,000 events and using sliding windows of 800-length subsequences to balance computational efficiency with capturing long-range dependencies. The architecture leverages transformer-based self-attention mechanisms to model these sequences, with a clever design that projects feature vectors from different domains into a shared latent space before applying multi-head attention layers. The results are impressive! After successful A/B testing, UUM has been deployed in production with significant gains: - 2.78% increase in Long-form Video Open Rate - 19.2% increase in Long-form Video View Time - 1.76% increase in Lens play time - 0.87% increase in Notification Open Rate They're also exploring advanced modeling techniques like domain-specific encoders and self-attention with information bottlenecks to address the challenges of imbalanced cross-domain data. This work demonstrates how sophisticated user modeling can drive substantial engagement improvements across multiple recommendation surfaces within a large-scale social platform.

LLMs are optimized for next turn response. This results in poor Human-AI collaboration, as it doesn't help users achieve their goals or clarify intent. A new model CollabLLM is optimized for long-term collaboration. The paper "CollabLLM: From Passive Responders to Active Collaborators" by Stanford University and Microsoft researchers tests this approach to improving outcomes from LLM interaction. (link in comments) 💡 CollabLLM transforms AI from passive responders to active collaborators. Traditional LLMs focus on single-turn responses, often missing user intent and leading to inefficient conversations. CollabLLM introduces a :"Multiturn-aware reward" system, apply reinforcement fine-tuning on these rewards. This enables AI to engage in deeper, more interactive exchanges by actively uncovering user intent and guiding users toward their goals. 🔄 Multiturn-aware rewards optimize long-term collaboration. Unlike standard reinforcement learning that prioritizes immediate responses, CollabLLM uses forward sampling - simulating potential conversations - to estimate the long-term value of interactions. This approach improves interactivity by 46.3% and enhances task performance by 18.5%, making conversations more productive and user-centered. 📊 CollabLLM outperforms traditional models in complex tasks. In document editing, coding assistance, and math problem-solving, CollabLLM increases user satisfaction by 17.6% and reduces time spent by 10.4%. It ensures that AI-generated content aligns with user expectations through dynamic feedback loops. 🤝 Proactive intent discovery leads to better responses. Unlike standard LLMs that assume user needs, CollabLLM asks clarifying questions before responding, leading to more accurate and relevant answers. This results in higher-quality output and a smoother user experience. 🚀 CollabLLM generalizes well across different domains. Tested on the Abg-CoQA conversational QA benchmark, CollabLLM proactively asked clarifying questions 52.8% of the time, compared to just 15.4% for GPT-4o. This demonstrates its ability to handle ambiguous queries effectively, making it more adaptable to real-world scenarios. 🔬 Real-world studies confirm efficiency and engagement gains. A 201-person user study showed that CollabLLM-generated documents received higher quality ratings (8.50/10) and sustained higher engagement over multiple turns, unlike baseline models, which saw declining satisfaction in longer conversations. It is time to move beyond the single-step LLM responses that we have been used to, to interactions that lead to where we want to go. This is a useful advance to better human-AI collaboration. It's a critical topic, I'll be sharing a lot more on how we can get there.

Traditional usability tests often treat user experience factors in isolation, as if different factors like usability, trust, and satisfaction are independent of each other. But in reality, they are deeply interconnected. By analyzing each factor separately, we miss the big picture - how these elements interact and shape user behavior. This is where Structural Equation Modeling (SEM) can be incredibly helpful. Instead of looking at single data points, SEM maps out the relationships between key UX variables, showing how they influence each other. It helps UX teams move beyond surface-level insights and truly understand what drives engagement. For example, usability might directly impact trust, which in turn boosts satisfaction and leads to higher engagement. Traditional methods might capture these factors separately, but SEM reveals the full story by quantifying their connections. SEM also enhances predictive modeling. By integrating techniques like Artificial Neural Networks (ANN), it helps forecast how users will react to design changes before they are implemented. Instead of relying on intuition, teams can test different scenarios and choose the most effective approach. Another advantage is mediation and moderation analysis. UX researchers often know that certain factors influence engagement, but SEM explains how and why. Does trust increase retention, or is it satisfaction that plays the bigger role? These insights help prioritize what really matters. Finally, SEM combined with Necessary Condition Analysis (NCA) identifies UX elements that are absolutely essential for engagement. This ensures that teams focus resources on factors that truly move the needle rather than making small, isolated tweaks with minimal impact.

Designing UX for autonomous multi-agent systems is a whole new game. These agents take initiative, make decisions, and collaborate, the old click and respond model no longer works. Users need control without micromanagement, clarity without overload, and trust in what’s happening behind the scenes. That’s why trust, transparency, and human-first design aren’t optional — they’re foundational. 1. Capability Discovery One of the first barriers to adoption is uncertainty. Users often don't know what an agent can do, especially when multiple agents collaborate across domains. Interfaces must provide dynamic affordances, contextual tooltips, and scenario-based walkthroughs that answer: “What can this agent do for me right now?” This ensures users onboard with confidence, reducing trial-and-error learning and surfacing hidden agent potential early. 2. Observability and Provenance In systems where agents learn, evolve, and interact autonomously, users must be able to trace not just what happened, but why. Observability goes beyond logs; it includes time-stamped decision trails, causal chains, and visualization of agent communication. Provenance gives users the power to challenge decisions, audit behaviors, and even retrain agents, which is critical in high-stakes domains like finance, healthcare, or DevOps. 3. Interruptibility Autonomy must not translate to irreversibility. Users should be able to pause, resume, or cancel agent actions with clear consequences. This empowers human oversight in dynamic contexts (e.g., pausing RCA during live production incidents), and reduces fear around automation. Temporal control over agent execution makes the system feel safe, adaptable, and co-operative. 4. Cost-Aware Delegation Many agent actions incur downstream costs, infrastructure, computation, or time. Interfaces must make the invisible cost visible before action. For example, spawning an AI model or triggering auto-remediation should expose an estimated impact window. Letting users define policies (e.g., “Only auto-remediate when risk score < 30 and impact < $100”) enables fine-grained trust calibration. 5. Persona-Aligned Feedback Loops Each user persona, from QA engineer to SRE will interact with agents differently. The system must offer feedback loops tailored to that persona’s context. For example, a test generator agent may ask a QA to verify coverage gaps, while an anomaly agent may provide confidence ranges and time-series correlations for SREs. This ensures the system evolves in alignment with real user goals, not just data. In multi-agent systems, agency without alignment is chaos. These principles help build systems that are not only intelligent but intelligible, reliable, and human-centered.

As AI reasoning models become more sophisticated, they're also becoming slower—deliberately taking time to process complex problems. This creates a UX challenge we haven't fully solved: How do we design interfaces that make AI thinking time productive rather than frustrating? One potential solution is to treat these windows like "supersets" in weightlifting. You do a push exercise, then immediately a pull exercise while your push muscles recover. You're always productive, just shifting focus. Applying this concept to AI interfaces: Imagine you're a lawyer using AI to review a complex 100-page contract: "Identify any unusual clauses, compliance risks, and compare terms to our standard agreements." While the AI works through this deep analysis, instead of watching a loading screen, the interface prompts you to begin preparing client-specific context notes or to outline negotiation strategy options based on different potential outcomes. The system intelligently guides you through complementary tasks matched to the processing time. When the AI completes its review, you've already completed valuable work that enhances your overall legal strategy. This "multitasking UX" approach seems better than the alternative of letting the user wait, sitting on their hands. Sure, over a long enough time horizon, this lag will eventually disappear. But in this emerging era, UX designers will increasingly need to solve for "reasoning model lag." Not by making users wait but by making waiting time productive.

Micro-interactions are no longer just a “nice-to-have” in UX— They’re a critical tool for guiding user behavior, building brand connection & improving retention. These small, purposeful elements like a progress bar, a loading animation, or a subtle vibration make a big difference when done right. How micro-interactions add value: 1. Clearer navigation: → Progress indicators or hover effects help users understand where they are— → And what’s happening— essential for reducing frustration. 2. User confidence: → Actions like a confirmation checkmark after a form submission reassure users that their actions are successful. 3. Brand differentiation: → Unique micro-interactions tailored to your brand’s identity make your app or website stand out in a crowded market. Here’s how to use them effectively: a. Prioritize user intent: → Focus on moments where users might feel uncertainty. → Such as waiting for a process to complete or interacting with a new feature. b. Keep it seamless: → Ensure micro-interactions don’t slow down or overwhelm the user experience. → They should complement, not complicate. c. Iterate & test: → Small doesn’t mean insignificant. → Test micro-interactions with real users to see what resonates. Let’s take a look at why they matter for retention: Memorable experiences aren’t always about big features— They’re often about how smooth & satisfying the small moments feel. By optimizing these “micro” details, you can create loyal users who notice the care & thought in your design. What are the overlooked moments in your user journey where micro-interactions could shine?

🧠 If you're building apps with LLMs, this paper is a must-read. Researchers at Microsoft and Salesforce recently released LLMs Get Lost in Multi-Turn Conversation — and the findings resonate with our experience at Vellum. They ran 200,000+ simulations across 15 top models, comparing performance on the same task in two modes: - Single-turn (user provides a well-specified prompt upfront) - Multi-turn (user reveals task requirements gradually — like real users do) The result? ✅ 90% avg accuracy in single-turn 💬 65% avg accuracy in multi-turn 🔻 -39% performance drop across the board 😬 Unreliability more than doubled Even the best models get lost when the task unfolds over multiple messages. They latch onto early assumptions, generate bloated answers, and fail to adapt when more info arrives. For application builders, this changes how we think about evaluation and reliability: - One-shot prompt benchmarks ≠ user reality - Multi-turn behavior needs to be a first-class test case - Agents and wrappers won’t fix everything — the underlying model still gets confused This paper validates something we've seen in the wild: the moment users interact conversationally, reliability tanks — unless you're deliberate about managing context, fallback strategies, and prompt structure. 📌 If you’re building on LLMs, read this. Test differently. Optimize for the real-world path, not the happy path.

Your model is ignoring or mixing up instructions on some queries... Why? Ambiguous separators let user content "leak" into your prompt structure, causing models to treat data as instructions and produce completely unexpected outputs. When you need to clearly separate different sections of your prompt—like instructions from examples, or user input from context—choosing the wrong delimiters can create parsing ambiguity. Models may struggle to determine where one section ends and another begins, especially with nested content or special characters. Effective AI Engineering #19: Use Unambiguous Separators in Your Prompts 👇 The Problem ❌ Many developers use markdown code fences or simple dashes for separation, which can become ambiguous when content itself contains similar patterns. Why this approach falls short: - Parsing Confusion: The model sees multiple `---` separators and nested ``` blocks, making it unclear which delimiters are structural vs. content - Content Conflicts: When user input contains the same separators you use (markdown, code fences), the model may misinterpret boundaries - Inconsistent Results: The ambiguity leads to unpredictable parsing, especially when content varies significantly The Solution: XML-Style Unambiguous Separators ✅ A better approach is to use XML-like tags that are inherently unambiguous and rarely appear in user content. XML provides clear opening and closing boundaries that models understand well. Why this approach works better: - Clear Boundaries: XML tags have unambiguous start/end markers (`<tag>` and `</tag>`) that don't conflict with common content formats - Hierarchical Structure: You can nest sections cleanly without confusion about which delimiter belongs where - Model-Friendly: Most modern language models are trained extensively on XML/HTML and parse these structures reliably - Content-Agnostic: User input can contain markdown, code, or other formats without breaking your prompt structure - Reduced Injection Risk: Clear boundaries make it harder for malicious user input to "escape" its designated section and interfere with your instructions The Takeaway ✈️ Stop using ambiguous delimiters that can conflict with your content. XML-style tags provide unambiguous structure that models parse reliably, regardless of what users input. This approach reduces parsing errors and makes your prompts more robust across different content types. Remember to also consult your model provider's documentation—most have specific recommendations for prompt formatting that can further improve reliability.

Foundation models have transformed natural language processing, but their impact goes beyond text. In a recent tech blog, Netflix’s machine learning team shared how they are building foundation models for recommendations, designed to learn from sequences of user interactions — much like how LLMs learn from sequences of words. At the center of this approach are three major components: - First, the data. Sequences of user interactions undergo tokenization. These tokens capture richer context than isolated signals and become the training ground for the foundation model. - Second, the prediction objective and architecture. Unlike standard LLMs, where every token is treated equally, in the recommendation context different user interactions carry different weights. For example, a full movie watch is more meaningful than a quick trailer view. The team also extends the training objective to predict multiple future items rather than just the immediate next one, aligning recommendations with long-term satisfaction instead of short-term clicks. - Finally, the team highlights unique recommendation problems such as the cold-start issue for new content and incorporates solutions like weighted representations from dual embeddings, as well as incremental training to help the system warm start and evolve smoothly. There’s much more technical depth in the blog, and I highly recommend checking it out. In short, foundation models for recommendations can’t simply copy LLMs. They must be carefully adapted — aligning data, objectives, and architecture to achieve meaningful personalization at scale. #DataScience #MachineLearning #Analytics #Recommendation #Personalization #AI #SnacksWeeklyonDataScience – – –  Check out the "Snacks Weekly on Data Science" podcast and subscribe, where I explain in more detail the concepts discussed in this and future posts:    -- Spotify: https://lnkd.in/gKgaMvbh   -- Apple Podcast: https://lnkd.in/gFYvfB8V    -- Youtube: https://lnkd.in/gcwPeBmR https://lnkd.in/g_33Tbfn

7 ways to seamlessly integrate AI into your users journey 1. The core purpose of AI directly shapes the user’s journey. 
 Conduct user research to identify key pain points or tasks users want AI to solve. ↳ if the startup’s AI helps automate content creation, what’s the user’s biggest friction in the current workflow? 2. Where will the AI interact with users within the product flow? Map out where AI should intervene in the user journey. For instance, ↳ does it act as an assistant (suggesting actions)
 ↳ a decision-maker (making recommendations)
 ↳ a tool (executing commands) 3. Simplify feedback loops help build trust and comprehension
 Focus on how users will receive AI feedback. ↳ What kind of feedback does the user need to understand why the AI made a recommendation? 4. Build a modular, responsive interface that scales with AI’s complexity. Visual elements should adapt easily to different screen sizes, user behaviors, and data volume. ↳ if the AI recommends personalized content, how will it handle hundreds or thousands of users while maintaining accuracy?
 
 5. Use layers of transparency At first glance, provide a simple explanation, and offer deeper insights for users who want more detailed information. Visual cues like "Why?" buttons can help. For more on how layered feedback can improve UX, check out my post here 
https://lnkd.in/eABK5XiT 6. Leverage Emotion Detection patterns that shift the tone of feedback or assistance. ↳ when the system detects confusion, the interface could shift to a more supportive tone, offering simpler explanations or encouraging the user to ask for help. For tips on emotion detection, check this https://lnkd.in/ekVC6-HN 7. Prototype different AI patterns ⤷ such as proactive learning prompts ⤷ goal-based suggestions ⤷ confidence estimation based on the business goals and user needs Run usability tests focusing on how users interact with AI features. ↳ Track metrics like user engagement, completion rates, and satisfaction with AI recommendations. Check out the visual breakdown below 👇 How are you integrating AI into your product flows? #aiux #scalability #designsystems #uxdesign #startups