{"version":1,"type":"rich","provider_name":"Libsyn","provider_url":"https:\/\/www.libsyn.com","height":90,"width":600,"title":"Ep. 261 | Andy Pierce: False Confidence at Machine Speed: How to Make Synthetic Customers Useful","description":"Episode 261: What if you could use AI-generated customer twins to test value propositions, pricing, and demand before you go to market? That is the idea my Bain colleague Andy Pierce is exploring. Synthetic customers can help teams move faster, pressure test offers, and simulate trade-offs much faster and at far lower cost than traditional research alone. Says Andy, \u201cYou can do things at half the time and a third the cost. You can be a hero internally by helping your business functions get to success better, faster, cheaper.\u201d But synthetic customers can also mislead you if you treat them like magic. Left ungrounded, large language models tend to be overly positive, can drift over time, and may reflect bad data rather than real human behavior.  To put theory into practice, we\u2019ll explore a 24-month telco case study that ran in parallel with a synthetic panel and hit ~85% overlap with human survey responses after confronting both bias and brown-nosing behavior. Guest: Andy Pierce, Partner, Bain & Company Host: Rob Markey, Partner, Bain & Company Give Us Feedback: Help us improve the podcast here: https:\/\/bit.ly\/CCPodcastFeedback Time-Stamped Topics: 00:00 \u2014 Value proposition basics and the design target question 00:05 \u2014 Pricing as the most natural lever and why finance often doubts research 00:09 \u2014 Early LLM experiments 00:10 \u2014 Telco case study 00:12 \u2014 Over-positivity and tuning to an ~85% survey overlap 00:14 \u2014 Quarter-by-quarter improvement 00:35 \u2014 Rational vs. experiential vs. emotional  Time-Stamped Quotes: [32:00] \u201cWe\u2019ve already started to see a world where I can train an LLM on a new idea, develop a new value proposition\u2014or an extension to an existing value proposition\u2014and instead of trying to predict the research outcome, I\u2019m trying to predict actual sales in the marketplace.\u201d [37:00] \u201cIt\u2019s not good enough to just create the persona \u2026 you still have to test against real humans.\u201d [38:00] \u201cSynthetic customers are here to stay. It\u2019s not a fad. And clients are already using them to build a competitive advantage.\u201d Resources Referenced on Today\u2019s Show: Generative Agent Simulations of 1,000 People \u2014 https:\/\/arxiv.org\/abs\/2411.10109 Simulating Human Behavior with AI Agents \u2014 https:\/\/hai.stanford.edu\/assets\/files\/hai-policy-brief-simulating-human-behavior-with-ai-agents.pdf How Synthetic Customers Bring Companies Closer to the Real Ones \u2014 https:\/\/www.bain.com\/insights\/how-synthetic-customers-bring-companies-closer-to-the-real-ones\/ UXAgent: A System for Simulating Usability Testing of Web Design with LLM Agents \u2014 https:\/\/arxiv.org\/html\/2504.09407v2 The Rise of Synthetic Respondents in Market Research \u2014 https:\/\/nielseniq.com\/global\/en\/insights\/education\/2024\/the-rise-of-synthetic-respondents Synthetic Users: If, When, and How to Use AI-Generated \u201cResearch\u201d \u2014 https:\/\/www.nngroup.com\/articles\/synthetic-users\/ A Tale of Two Identities: An Ethical Audit of Human and AI-Crafted Personas \u2014 https:\/\/arxiv.org\/html\/2505.07850v1 Luxury in Transition: Securing Future Growth \u2014 https:\/\/www.bain.com\/insights\/luxury-in-transition-securing-future-growth\/ ","author_name":"Customer Confidential: Untold Stories of Earned Growth","author_url":"http:\/\/baincompany.libsyn.com\/site","html":"