Prototyping and Iterative Testing

I had a fascinating conversation with Steve Quinlan of NatWest Group recently, and it really highlighted a fundamental issue in how many product teams approach experimentation. Too often, "experimentation" is seen as something that happens after a feature is built. This is the cart-before-the-horse. You've already invested significant time and resources, and now you're hoping to validate if it was worth it. True experimentation should be about validating and developing ideas before they enter serious development and as they go through design. Steve sits with a 'prototyping' function at Natwest created with this purpose in mind. They focus on de-risking development by rigorously testing and iterating on ideas early in the process. This approach not only saves valuable resources but also ensures that the final product truly meets customer needs. Moreover, Steve's team's work disambiguates from the narrow view that experimentation is just about A/B testing. It's about a broader, more strategic approach to product research, discovery and validation. It begs the question: how many product teams are missing out on this critical early-stage validation? How often are we building features based on assumptions rather than solid evidence, even if they are 'tested' before release? Shifting our mindset to prioritize prototyping and early-stage experimentation can revolutionize how we build products and drive innovation. How does your team ensure that experimentation is integrated into the entire product development lifecycle, not just tacked on at the end? #experimentation #cro #productmanagement #growth #digitalexperience #experimentationledgrowth #elg  

What if the best solutions for your process started with cardboard? When testing new ideas or improvements, jumping straight to high-cost, permanent solutions can be risky—and expensive. That’s where cardboard engineering comes in. Cardboard is one of the simplest, most cost-effective tools for rapid prototyping and testing ideas. It’s lightweight, easy to shape, and lets you visualize, test, and refine your concepts before committing to more expensive materials. Why Cardboard Is Perfect for Prototyping: 1️⃣ Low-Cost Experimentation Testing with cardboard lets you try multiple iterations of a design without worrying about material costs. 2️⃣ Fast Feedback Loops You can build and modify a prototype in minutes, gathering instant feedback from your team or operators. 3️⃣ Hands-On Collaboration Cardboard prototypes allow teams to actively engage with ideas, making it easier to identify issues or opportunities for improvement. 4️⃣ Visual Validation Sometimes, seeing a physical model highlights challenges that wouldn’t be obvious in a drawing or plan. How to Use Cardboard for Lean Improvements: 🔍 Test Workstation Layouts Use cardboard cutouts to mock up layouts and placement of tools, parts, and equipment. Adjust until everything flows smoothly. 📦 Simulate Material Flow Prototype racks, bins, or carts to ensure materials are stored and moved efficiently before building them with more durable materials. 🛠️ Design Fixtures or Jigs Create cardboard versions of fixtures or jigs to test their functionality in the process. Refine the design before investing in the final version. 📐 Test Ergonomics Mock up equipment or workstation designs with cardboard to test ease of use, reach, and operator comfort. Example of Cardboard in Action: A manufacturing team wanted to redesign a workstation to reduce operator motion. Instead of committing to expensive reconfigurations, they used cardboard to prototype the layout. After several iterations, they found the optimal setup, reducing motion by 25% and saving hours of work. Cardboard isn’t just for packaging—it’s a powerful tool for testing and refining your ideas. By prototyping with low-cost materials, you can experiment, learn, and improve quickly without breaking the bank.

🕹 R&D Framework for Game Prototyping 🕹 Mykola and I have a combined total of over 100 prototypes developed. Below is the framework we often use when starting work on new projects: I. Pre-Production (2-4 weeks) 1. Idea Generation: ▶ Market analysis, gathering references; ▶ Brainstorming to pick the most promising ideas 2. Greybox Prototyping: ▶ Focus on a single core feature; ▶ Find the fun factor 3. Conceptualization: ▶ Preparing the GDD, user flow, and key shots; ▶ Selecting the right tools and creating a roadmap II. Production (4-10 weeks) 4. Features Development: ▶ Concentrate on key gameplay mechanics and speed in development; ▶ Minimal meta or progression 5. Content Creation: ▶ Craft key art, and fun levels, and use ready-made assets; ▶ Secondary art, vfx, sounds, etc. can be draft 6. Minimum Viable Product: ▶ Ensure the game offers 20-40 minutes of engaging gameplay with a distinctive style and polished game feel ▶ Better fewer, but better III. Product-Market Fit Validation (1-3 weeks) 7. QA & Playtesting: ▶ Conduct thorough testing, optimize performance, and balance difficulty 8. Marketing Preparation: ▶ Create captivating marketing materials; ▶ Focus on highlighting short but impactful gameplay moments 9. Validation Test: ▶ For Mobile: measure CPI, Day 1 Retention, and Playtime through the Facebook ads channel ▶ For PC: Collect feedback from players and publishers and assess virality using game expos, Steam Demo, and social networks like Reddit and Twitter 🔍 Decisions Based on Results: ✅ Great Results? Advance to full-scale production! 🔁 Borderline Results? Iterate and polish further. 🔴 No Results? Start over with a fresh idea. The whole point of this framework is to avoid wasting extra resources and time on something that will later change or be discarded due to the lack of product-market fit. ----------------------------------------------------------------------- DISCLAIMER: This framework is NOT a "one-size-fits-all" solution for all genres and business models, nor is it the only possible one. However, it is effective for small and medium teams without a huge budget who are working on new casual, hybrid-casual, or indie games.

"Getting real customer transcripts changed everything." 📝 Learn how I built my first AI product - an Interview Coach that helps product teams improve their customer interviewing skills. Follow my journey from initial prototype to production, including key lessons on: 🎯 Choosing the right problems for AI to solve: - Start with real customer needs and pain points - Look for opportunities that were previously hard to solve - Focus where you have domain expertise 🔬 Prototyping and testing approach: - Start simple with existing LLM tools - Test with a small, engaged user group - Get real user data early ⚙️ Architecture decisions: - Don't default to chat interfaces - Break complex tasks into smaller AI tasks - Start simple and add complexity as needed - Expect to outgrow your tools 🎯 Getting consistent results: - Build comprehensive evals - Ground evals in error analysis - Continuously monitor performance - Start simple with tools you know 💪 Continuous improvement: - Regular trace review and annotation - Ongoing eval updates - Continuous experimentation - Quick iteration cycles 🔒 Data responsibility: - Be transparent about data collection - Consider compliance requirements early - Partner rather than build infrastructure - Delete data you don't need Check out the comments for a link to the article. 💭 What's been your biggest challenge when building AI products? Share your thoughts in the comments below.

Systems design needs both short and long feedback loops. I work with teams that are often asked to grow or pivot a product to create more impact. Over the years, I’ve seen two main types of leaders trying to make progress in their products. The first type thinks very short term. They design page by page and try to keep the problem small. Sometimes this comes from not being able to sell a bigger vision, or from holding too tightly to release cycles. While this approach can create quick wins, it often comes at the cost of a bigger product vision. Great products can’t be built block by block, as great experiences are connected, and each part adds value to the whole (despite what some enterprise companies still think). I find this approach painful. And slow. The second type wants to solve the whole experience at once. They might copy a competitor or follow a logical sequence of steps. But most platforms don’t map one-to-one, so they end up backtracking to fix gaps. Iteration turns into rework, and learning is lost. Sometimes this works, and brute force is needed. You see this emerging with AI prototyping. Both approaches fall short. One misses how small decisions shape the system, and the other skips over details as it charges ahead. Modern software needs both views. And AI won’t fix these gaps… it will only make them worse if the right mindset isn’t there. That’s why I push for a third approach: progressive design. It tests assumptions at the screen level while shaping the bigger experience. Short, iterative cycles build the story. Adding signals from UX metrics and audiences makes it stronger. The beauty is that you can test incrementally with attitudinal metrics while also building a behavioral profile through actions. A recent customer used this approach with us using Helio on a complete homepage redesign and saw: +28% engagement on target KPIs +39% lift in positive impressions vs. baseline Curious what working styles have you seen work? #productdesign #uxmetrics #productdiscovery #uxresearch

From idea to prototype in hours, not weeks. That's been my recent experience experimenting with Lovable, and it's completely changed how I approach ideation and product thinking. Turning abstract ideas into clickable, interactive prototypes in no time means less talking about the concept, and more showing. In one recent build, the moment I shared the prototype, the conversation shifted from “What do you mean?” to “Is this how you see it?” That one shift sparked faster clarity, better feedback, and deeper alignment. No more endless meetings trying to describe what’s in everyone’s head. Here’s what I’ve learned along the way: 𝟭. 𝗦𝘁𝗮𝗿𝘁 𝘄𝗶𝘁𝗵 𝗮 𝗰𝗹𝗲𝗮𝗿 𝗼𝗯𝗷𝗲𝗰𝘁𝗶𝘃𝗲 𝗳𝗼𝗿 𝘆𝗼𝘂𝗿 𝗽𝗿𝗼𝗱𝘂𝗰𝘁. Even with powerful tools doing the heavy lifting, I start by organizing my thoughts on paper—with a clear outline, defined scope, and key user flows. The tool amplifies good product thinking, but it can't replace it. 𝟮. 𝗔𝗹𝗶𝗴𝗻 𝘆𝗼𝘂𝗿 𝘁𝗮𝘅𝗼𝗻𝗼𝗺𝘆 𝗮𝗻𝗱 𝗻𝗮𝘃𝗶𝗴𝗮𝘁𝗶𝗼𝗻 𝗲𝗮𝗿𝗹𝘆. This becomes incredibly clear when you're building a visual prototype. Getting your information architecture right from the start saves significant rework later. 𝟯. 𝗘𝗺𝗯𝗿𝗮𝗰𝗲 𝘁𝗵𝗲 𝗳𝗶𝗿𝘀𝘁 𝗱𝗿𝗮𝗳𝘁 𝗳𝗼𝗿 𝗰𝗹𝗮𝗿𝗶𝘁𝘆 𝗮𝗻𝗱 𝗳𝗲𝗲𝗱𝗯𝗮𝗰𝗸. Don't aim for perfection on the first build. Get something clickable in front of people quickly. The real insights come from watching others interact with your prototype, not from endless polishing. You can always go deeper and refine the prototype based on those initial insights. 𝟰. 𝗟𝗲𝘃𝗲𝗿𝗮𝗴𝗲 𝗹𝗼𝗰𝗮𝗹 𝗳𝗶𝗿𝘀𝘁. For initial builds, leverage local browser cache before connecting to databases or other external tools. It speeds things up considerably and keeps you agile. 𝟱. 𝗦𝗲𝗰𝘂𝗿𝗶𝘁𝘆 𝗯𝗮𝘀𝗶𝗰𝘀 𝘀𝘁𝗶𝗹𝗹 𝗺𝗮𝘁𝘁𝗲𝗿. A crucial reminder: never store your LLM API keys in plain text, especially if your project is public or remixable. Low-code tools like Lovable don’t just speed up the work—they unlock momentum, clarity, and collaboration. These change the way we think, not just what we build. Been experimenting with Lovable, Replit, v0 dev, or similar tools? I’d love to hear your best practices. ------------------------- P.S Curious about prototyping, product thinking, or AI workflows? I host Sunday brainstorming sessions — DM me if you'd like to join the next one!

For founders, building a successful product is often less about the idea and more about the process. Knowing when to use a Proof of Concept (PoC), Prototype, or Minimum Viable Product (MVP) can be the difference between scaling up or burning out. 🌱 Proof of Concept (PoC): Testing “Possible” vs. “Impossible” Insight: PoC is about finding limits, not solutions. It’s the stage to test if your concept is even technologically achievable with current resources. This stage isn’t about showing off or impressing; it’s about brutally honest assessments. It’s where you ask, “Are we chasing something we can’t feasibly build?” Use it when: You’re unsure if a novel tech component will work in practice. Example: Is the AI algorithm actually capable of processing data at this scale? Founder's takeaway: Don’t fall in love with the concept just because it’s new. PoC is where you might need to abandon the idea early, saving resources and learning key constraints. 🎨 Prototype: Bringing Ideas to Life, Not to Market Insight: The Prototype phase isn’t about building a working product; it’s about exploring user interactions and design flow. A good prototype reveals what’s broken in the user journey before you commit resources to coding and development. You’re here to answer, “Is this something people will want to use? Is the experience intuitive?” Use it when: You need a vision, not a finished product. Example: How will users navigate the app? Does the layout make sense? Founder's takeaway: Prototyping forces you to confront your assumptions about user behavior and design. A great idea with poor UX is doomed, so listen to feedback carefully and iterate. 🚀 Minimum Viable Product (MVP): Testing If People Actually Care Insight: MVPs are not meant to be “perfect”—they’re meant to be functional enough to test market need. The MVP is your experiment in real-world conditions. The goal isn’t to sell the product but to learn what will make it sell. This is where startups often discover whether they’re solving a problem worth paying for or just building something “cool.” Use it when: You have a clear problem you’re solving and need to validate that users care enough to engage (and hopefully, pay). Founder's takeaway: An MVP is not your “launch” but a learning opportunity. Don’t be afraid to fail or pivot based on real-world feedback—it’s cheaper and easier than overbuilding a product people don’t want. 🔍 Key Takeaway: Not Every Stage Is Necessary Founders often assume they need to go through all three stages, but the reality is that many products don’t need a PoC, and some products might skip a Prototype if the MVP serves that purpose. Final Thought: Think of PoC, Prototype, and MVP as tools for falsifying assumptions. Each stage should help you eliminate uncertainties before moving to the next. #startups #productdevelopment #founderinsights #innovation

THE POWER OF PROTOTYPING: A CRITICAL STEP IN YOUR AI JOURNEY In AI implementation, the organizations that thrive aren’t the ones that dive headfirst into full-scale deployment. They’re the ones that experiment, iterate, and refine—leveraging prototyping as a strategic advantage. ➤ Why Prototyping Matters in AI By embracing prototyping, organizations can: ✅ Test concepts with minimal resources – Start small, learn fast, and refine before heavy investment. ✅ Gather real-world feedback – Move beyond theory and understand actual user experiences. ✅ Identify unforeseen challenges – Discover technical and practical roadblocks early. ✅ Build institutional knowledge – Develop AI expertise that benefits future projects. ➤ Best Practices for AI Prototyping 🔹 Begin with conceptual modeling to set a solid foundation. 🔹 Test multiple versions to explore different approaches. 🔹 Involve end-users early to ensure human-centered design. 🔹 Define success metrics and measure specific outcomes. 🔹 Iterate—observe, refine, and evolve with each prototype. The difference between AI leaders and laggards often comes down to methodical experimentation rather than premature full-scale deployment. 💡 What prototyping strategies have worked for your AI initiatives? 👇 #Innovation #ArtificialIntelligence #DigitalTransformation #management

Show, Don’t Tell: Vibe Prototyping Is the New PM Superpower I've shipped hundreds of features—from tiny ones like tags to major launches like Rescale’s AI Physics—and one thing holds true: prototypes beat specs. Every time. Now, with AI, you can prototype at the speed of thought. I call it Vibe Prototyping—a way to build and validate product vibes before real investment. Using tools like ChatGPT and Replit, you can go from insight to working UI in hours. Here’s how I do it: (1) Extract needs (<1h): Use ChatGPT DeepResearch to synthesize user insights from Reddit, support tickets, research, etc. (2) Draft a spec (1h): Write your vision, constraints, and references, then turn it into a detailed PRD with ChatGPT. (3) Generate a working prototype (1h): Feed the spec into Replit and get a working prototype in minutes. (4) Validate the need (days): Share with users, design, and stakeholders. Iterate fast. Why this matters: - Speed > Slides: You validate in hours, not months. - AI is the new IDE: It turns your intent into working code instantly. - No prototype = no meeting: Talking in abstract is a waste. - This is the new PM stack: Ignore it and get left behind. Agile is starting to feel like waterfall. The future isn’t more process—it’s better intuition, faster loops, and showing instead of telling. Even companies like Shopify are shifting to this. PMs who build prototypes will ship 10x more, with 10x less friction. The rest will be stuck writing PRDs no one reads.

A great product isn’t built overnight; it’s optimized through continuous testing and improvement. Making product decisions based on assumptions is risky. Continuous experimentation is a proven way to provide clarity. Beyond metrics, A/B testing builds a culture of continuous learning and iteration, where we experiment, learn from failures, and refine until we get it right. Some direct benefits of A/B testing for Product Managers: - Data-driven decision-making: It eliminates guesswork by providing measurable results, ensuring that product decisions are backed by data rather than intuition. - Improved user experience: A/B testing identifies what resonates best with users, leading to enhancements that make the product more intuitive and enjoyable to use. - Increased conversion rates: By optimizing elements like CTAs, layouts, or messaging, A/B testing helps boost user actions, such as sign-ups, purchases, or clicks. - Risk mitigation: Testing changes on a small scale before a full rollout minimizes the risk of implementing features that could negatively impact users or business metrics. - Faster iteration cycles: It provides quick insights, enabling teams to iterate rapidly and focus on what works without spending excessive time on less impactful changes. - Enhanced collaboration across teams: The results of A/B tests often spark cross-functional discussions, aligning design, engineering, and marketing teams around user-centric outcomes. - Informed roadmap prioritization: Insights from A/B tests guide Product Managers in prioritizing features or improvements with the highest potential ROI. - User segmentation insights: A/B tests can reveal how different user segments respond to changes, enabling tailored strategies for diverse audiences. Incremental improvements driven by A/B testing compound over time, leading to substantial product and business growth. #productmanagement #product #pm #career #growth