Build a Deep Research Agent | Python, OpenAI, Temporal

Learn how to create resilient AI research agents using Python, OpenAI, and Temporal. This tutorial covers implementing durability, handling failures seamlessly, and maintaining workflow progress without re-running expensive calls.

🔗Resources & Links
Repo with solution: https://github.com/temporalio/edu-deep-research-tutorial-template/tree/video_tutorial
👉Text Tutorial: https://learn.temporal.io/tutorials/ai/deep-research/?utm_campaign=awareness-nikolay-advolodkin&utm_medium=video&utm_source=youtube
🤝Connect with Nikolay: https://www.linkedin.com/in/nikolayadvolodkin/  

🕐 Timestamps
 00:00 - Introduction to the problem of process disruptions in AI agents
 00:48 - Overview of the OpenAI deep research agent setup
 01:14 - Observability and workflow visibility with Temporal
 01:43 - Cloning the repo and environment setup instructions
 02:12 - Configuring API keys and running the research agent locally
 03:09 - Demonstration of research workflow and disruption scenarios
 03:39 - Common failure points and solution overview with Temporal
 04:08 - Deep dive into agent pipeline components and their functions
 04:34 - Visual map of research pipeline: triage, planning, search, and report
 05:03 - How OpenAI SDK simplifies agent orchestration
 05:44 - Chaining agents: plan, search, and write in sequence
 06:13 - Durability through Temporal's integration with OpenAI SDK
 06:43 - Introducing Temporal for state persistence and error recovery
 07:42 - Building the interactive research manager class
 08:06 - Methods for planning, searching, and report generation
 09:04 - Managing clarifications and user input with workflows
 09:33 - Implementing triage and clarification questions
 10:00 - Automating research pipeline execution based on user interaction
 10:30 - Limitations of in-memory state management, solution with Temporal workflow
 10:55 - Persisting session state and indefinite human waiting with Temporal
 11:24 - Defining data classes and workflow signals for communication
 11:52 - Instantiating manager inside the workflow for durability
 12:22 - How durable activities are executed within Temporal context
 12:50 - No manual retry or checkpointing needed with Temporal
 13:19 - Pattern for human-in-the-loop interactions: start, clarify, resume
 13:46 - Temporal's handling of paused workflows without resource usage
 14:15 - Workflow handlers for get status, provide clarification, start research
 14:45 - Pausing workflow until user input, no polling or timeouts required
 15:14 - Difference between queries, signals, and updates in Temporal workflows
 15:43 - Initiating research session and storing clarification questions
 16:10 - Handling user inputs and updating workflow state with signals
 17:05 - Main run method: waiting for user input and progress management
 17:33 - Workflow resumption after user input, intelligent waiting
 18:00 - Cleaning up old in-memory research manager
 18:27 - Creating the worker to run Temporal workflows
 18:57 - Configuring worker with OpenAI plugin and retry policies
 19:27 - Setting timeout and retry parameters for durability and fault tolerance
 19:55 - Connecting to Temporal client with OpenAI plugin support
 20:25 - Running the worker and managing server restarts seamlessly
 21:21 - How Workflow handles session IDs, persistence, and resuming work
 21:51 - Full demo of the agent executing with failures and restarts
 22:21 - Explaining the role of Temporal client for starting and managing workflows
 22:50 - Frontend API: starting research, querying status, submitting answers, fetching reports
 23:18 - Handling workflow handles and querying current state without disruption
 24:16 - Endpoint implementations: get status, submit clarification answers, fetch results
 25:33 - Pattern for session management with session IDs and workflow handles
 26:03 - Running server, worker, and UI setup: a step-by-step guide
 26:30 - Demonstration of workflow execution, clarification, and report generation
 27:29 - Troubleshooting and fixes for demo success
 28:28 - How to handle server restarts during research sessions
 29:27 - Workflow continues seamlessly after worker restart with Temporal
 30:26 - Showcasing retry attempts, handling failures, and retry policies
 31:26 - Final report review and insights on the demo success