The Hidden UX of AI: What Every PM Needs to Know - Nina Olding (Weights&Biases, Gemini, Meta)

In this episode, Nina Olding, Staff Product Manager at Weights & Biases and formerly at Google DeepMind, working on trust and compliance for AI, joins Randy to explore the UX challenges of AI‑driven features. As AI becomes increasingly woven into digital products, the traditional UX cues and trust‑signals that users rely on are changing. Nina introduces her framework of the three “A’s” for AI UX: Awareness, Agency, and Assurance, and explains how product teams can build this into their AI‑enabled products without launching a massive transformation programme.

Key Takeaways
— As AI features proliferate, the UX challenge is less about the technology and more about how users perceive, understand and trust the interactions.
— Trust is based on three foundational dimensions for AI‑enabled products: Awareness, Agency, Assurance.
— Awareness: Make it clear when AI is involved (and when it isn’t). Invisible AI = risk of misunderstanding. Magical AI without context = disorientation.
— Agency: Give users control, or at least the option to opt‑out, define boundaries, choose defaults vs advanced settings.
— Assurance: Because AI can be non‑deterministic, you must design for confidence—indicators of reliability, transparency about limitations, ability to question or override outputs.
— Many organisations treat AI as a “feature” rather than a shift in UX paradigm. But the shift is major: the interactive cues and trust signals we’ve relied on may no longer apply in the same way.
— Simple implementations matter. You don’t need a massive replatform to start building trust. Even small design changes (badges, disclosures, data‐state transparency) can start to build the muscle.
— From a product‑management perspective: ask the three questions (above) early and often. If any are “no”, map out remediation.
— The business case: building trust is both retention/competitive advantage and forward‑looking (regulation will arrive). So investing in this now is strategic.
— For PMs working on AI features: collaborate closely with design, data, engineering and governance to ensure the UX signals for the “three A’s” are baked in, not an afterthought.

Chapters
00:00 – Intro: Why AI products are failing on trust
00:47 – Nina Old’s journey from Google DeepMind to Weights & Biases
03:20 – The UX of AI: It's not just a chat window
04:08 – Introducing the Three A’s framework: Awareness, Agency, Assurance
08:30 – Designing for Awareness: Visibility and user signals
14:40 – Agency: Giving users control and escape hatches
21:30 – Assurance: Transparency, confidence indicators, and humility
28:05 – Three key questions to assess AI UX
30:50 – The product case for trust: Compliance, loyalty, and retention
33:00 – Final thoughts: Building the trust muscle