So you want to build an AI voice agent | Tom Alterman | ProductTank SF

At ProductTank San Francisco, Tom Alterman delivered a talk on the challenges of deploying voice AI in real-world environments, with a focus on healthcare. Drawing from failure and the lessons that followed, he reframes how product teams should think about AI—less as a magic trick, more as a systemic product discipline. Essential listening for product managers navigating the complexities of AI deployment.

Key Takeaways
— AI needs product thinking, not stagecraft. Voice AI may impress in a demo, but it only proves valuable when it solves real user problems—especially in messy, emotional, high-stakes scenarios.
— Your metrics may be lying to you. Success dashboards often fail to capture human context. Good product judgement still matters.
— AI should be onboarded like an employee. Don’t just prompt—coach. Provide documentation, feedback, and testing before unleashing agents in the wild.
— Edge cases are not optional—they are the product. Most failures occur not on the happy path but in real-world ambiguity. Design for that.
— Build your evaluation framework before your prompts. Success depends on clear definitions, systematic testing, and robust red-teaming.
— Leverage what AI does uniquely well. 24/7 availability, infinite patience, and perfect recall are assets—when paired with thoughtful system design.
— Don’t hide the AI. Transparency builds trust. Attempting to fool users creates more risk than reward.

Chapters
00:00 – Opening Skit: The Two Voice AIs
02:00 – Launch Failure: A Mother, A Hospital, and a Misguided Agent
04:00 – Why Most Voice AI Projects Miss the Mark
06:00 – The Myth of the Happy Path
09:00 – Flawed Thinking Around Human vs AI Capabilities
12:00 – The Limits of Prompting
13:00 – Recognising AI’s Superpowers
14:30 – Onboarding AI Like a Team Member
17:00 – Giving Agents the Right Tools
19:30 – Experience Design: Not Just Functionality
21:00 – How to Test Voice AI in the Real World
23:00 – Build Evaluation Before You Build Prompts
26:00 – Monday Morning Playbook
30:00 – Q&A: Transparency, Metrics, Adoption
45:00 – Deploying AI in Healthcare Call Centres
50:00 – Closing Thoughts on Risk, Trust, and Practicality