SE Radio 681: Qian Li on DBOS Durable Execution/Serverless Computing Platform

This is Software Engineering Radio, the podcast for professional developers on the web at se-radio.net. SE Radio is brought to you by the IEEE Computer Society and the IEEE Software Magazine. Online at computer.org slash software. Hello, everyone. Welcome to this episode of Software Engineering Radio. Our guest today is Chon Lee. Chon Lee is a co-founder and chief architect at DBOS. Before founding DBOSS, Chan completed her PhD in Computer Science at Stanford in 2023. Her PhD research has focused on abstractions for efficient and reliable cloud computing. Chan is also the co-organizer of the South Blair Systems Club, which is an independent talk series focusing on systems programming. Before we actually talk a little bit more about DBOSS and its research origins, I'd like to point listeners to episode 596. which is Maxim Fatih on Durable Execution with Temporal. There are also a few other episodes we have done on related topics, and I'll put those in the show notes. These are episode 351, 223, and 198. So happy to have you here, Chon, to talk about DBOS and Durable Execution. Welcome to the show. Would you like to add something to your bio before we jump right in? Thanks for inviting me to the show. Yeah, so originally, DBOS started as a joint research project. It's a collaboration between Stanford and MIT. So we started the project since 2020. It was also led by the Postgres creator, Professor Mike Stonebreaker, and the Spark creator and Databricks co-founder, Matei Zaharia. So during the research project, we really tested the capability of databases and see how databases can help you. create reliable programs and how databases can help you make your programs more observable and debuggable. So during the research project, we built several prototypes and we wrote several papers. And when we presented it, people were really excited about the capability that Dboss can bring. So when we graduated in 2023, we decided to, based on the research project, co-found Dboss. And Dboss as a company right now, we focus on building durable software and we believe that all software should be reliable and observable and scalable by default so now the boss stands for durable backends observable and scalable so you mentioned the use of databases for basically running your applications but why is that a new concept everybody uses databases for running apps or most people too so what is the difference here what is the secret sauce that you're talking about Yeah, so it's true that people have been storing their business critical data in databases for like 30, 40 years. But the new concept is that we also want to persist programs execution state in a database. Like here's a program and it has multiple steps and we want to persist the steps output and the input into the database. so that if your program crashes or machine failed, we'll be able to resume from exactly where you left off. So the idea is to, in addition to your application data, we also store your program execution state in the database. Especially if you're having long-running and dynamic workflows in your programs, you really don't want to restart from scratch every time you hit a system error or you have a machine failure. So Chan, maybe you can explain to your listeners what exactly do you define a workflow as opposed to a service? Yeah, so to get started, I think we can talk about what is a workflow in this context. So traditionally, people think workflows as state machines, you have to define a DAG, stuff like that. But actually in Dboss, anything can be a workflow. So to give a concrete example, a workflow is a sequence of operations or function calls. A very typical example in a durable execution is, let's say, checkout service. If you are implementing a checkout service, usually you have to call, say, reserve inventory. You have to update a database to make sure that you have enough inventory. Then after that, if you successfully reserve the inventory, you will cut out to the payment process. for example, this could be an external service like Stripe or PayPal, you will say, I want to charge a user this much. And after that, you need to wait for the response from those services. And then based on the result, you will decide whether to fulfill the order and send a confirmation email to the user, or you have to undo your reservation for the inventory and then cancel the order and send a cancellation email. So this process can be abstracted as a workflow. And then, like, what guarantees do we want for this workflow? First, we want to make sure that once I click that checkout button, all steps will eventually succeed or complete, right? I don't want to say I pay for my process, but I never receive my item, or I reserve the inventory, but never charge a user. So that's a first guarantee. And the second guarantee is that I want to guarantee effectively exactly once. So if I charge a user, I want to charge them once. And if I reserve the inventory, I also want to only reserve it once. So you talked about workflow and you mentioned transaction. What's the relationship? So in this context, a transaction is essentially interaction to a local database. And then the workflow can be composed of transactions and also external services, or we just call it steps in general. So in DBOS, a workflow is a sequence of steps. And some steps could be transactions that talk to a database. And other normal steps, well, normal steps can be any functions, right? It can be talking to external APIs or can have some any non-deterministic factors, operations in that step. But overall, the workflow needs to be deterministic. And determinism and idempotency are two foundations for durable workflows. And essentially, what we build is a library. that you can just easily install it and then add annotations to your program. So I described the checkout workflow. To make it durable in DBOS, you essentially put annotation at dbos.workflow in front of the overall orchestration workflow function, and then you put at step at the function definition of every individual step. Then when we call that function, essentially we'll first We basically put wrappers around those functions to make them durable. So to make it more concrete, basically, when you start a workflow, when you call the workflow function, the wrapper would first checkpoint the inputs of the workflow in the database. And then for each step, we first check, have we executed this step before? If so, we directly read the recorded output from the database. and return the output instead of re-executing it. So this way we guarantee exactly once. So the thing is, for external systems, the way to guarantee exactly once is to guarantee at least once plus idempotency key. And Dboss basically automatically generates an idempotency key per workflow and per step, so we can guarantee that each step will execute effectively exactly once. And then after each step finishes, we'll checkpoint the output into the database, and so on and so forth until we reach the end of the workflow. And say if something crashes in the middle, when we recover, the DBOS library will look into the database and see which workflow is still pending. And then if the workflow is still pending, we'll just essentially replay the workflow from the start and walk through all the steps. And then for the finished steps, we'll already see the database record, and then we'll skip those steps, and then we'll eventually land to the last completed step, and then we'll continue, resume the workflow from where it left off. So that's basically a high-level idea of how it defaults. You mentioned the idempotency key. Blame that further. Yeah, so essentially, idempotency key is a way to guarantee that each step is executed once. and only once. So essentially, when I work with external API, say Stripe payment, you want to say, I only want to charge this user once. And then you can piggyback an idempotency key in your request to Stripe. So if Stripe receives the same key again, it will not issue another invoice to the user. Stripe will be able to look up its own database and say, hey, I already have an outstanding invoice for this checkout session. they will only charge a user once. So I think idempotency is a very important concept in terms of durable execution. And in DevOps, we automatically generate an idempotency key for a workflow if you don't already provide one. And then for each step, we'll basically append the step sequence number to it. So every step in the workflow will also be uniquely identified. So the reason we need idempotency key is that you can't really control external systems, right? Say, even if we only call Stripe API once, we don't know whether the network connection will fail, or maybe Stripe will fail, or there are some intermediate transient errors that may happen. So if we don't get the result back, we may think it failed. But in order to make sure that the checkout goes through, we may want to retry it. And when we try it, we can't generate a new session because maybe the Stripe payment already succeeded, but the network was cut. when Stripe tried to send the result back. So we have to retry it. And when we retry it, the way to tell Stripe that this is the same checkout request I sent before is to append the idempotency key. So idempotency key is the key to guarantee correctness and exactly once execution when you work with external systems. Could you talk more about the DBOS architecture? You did talk about the workflow and the step. Is there any other key elements of the architecture you'd like people to know about, and especially if they have origins in the research that you had done in the past? Yeah, so DBOS workflows and steps and transactions were implemented directly as SQL interactions in the library. And then beyond that, we also have several other primitives. The one is widely used, it's called DBOS queues. So we also implement queues on top of Postgres. With queues, basically, you will be able to group. So instead of directly invoking or executing a workflow synchronously, you can enqueue a workflow, and then other workers will be able to pick it up. So the queue is also just a database record, right? Which is enqueue by saying, here's the task, here's the queue name, and then for the worker, they can just pull from each queue. So the benefit of queue is that it makes it really easy to control concurrency and rate limiting. Say, for example, we can have different queues like OpenAI Queue, Cloud Queue, stuff like that, and we want to say, for the OpenAI Queue, we only want to send the request five times per minute. So this is a way to group your invocations and to control how many outstanding requests you send to external systems. And another thing is based on research is messaging system. So when workflows and steps, basically they are all within the DBOS territory. But when you need to interact with external systems, you need a way to communicate with the workflow. So give an example of this. So say, actually the checkout workflow, right? When you call Stripe, you don't block and wait until user to pay out. So what do you actually do is that Stripe will directly return a confirmation saying, okay, we are processing your payment. But when your payment is done, we'll send you a webhook invocation to say, this payment is done. And now how to do it in dboss is that, so in a workflow, we can have a dboss.receive on a specific topic, waiting for the specific Stripe payment session. And then we can implement a webhook that listens Stripe callback. When a Stripe callback, we can extract, because as we mentioned, we have the item policy key. we can uniquely identify which workflow we should call back to. And then we will be able to send, say, dboss.send with that IDMPotency key and then send Stripe payment results, either paid or payment rejected, stuff like that. And then the workflow, while using the receive, will be able to get that information. So this is really useful if you want to. I think almost all of the payment system, for example, is using this type of pattern. And also, if you want anything that goes to a human in the loop, say you want to send a human verification email and then you want to wait for the human confirmation to come back, you can't just block there and wait. You have to wait for a callback. And then dboss send and receive primitive allows you to do that. And this library is what you call a dboss transact. Anybody can use it. as long as they annotate their code. Yes, so DBOS Transact is an open source library. It's MIT licensed, so anyone will be able to use it. It's currently available in TypeScript and Python, so we are adding more language support. So for the audience, if you have any feedback on what languages you wish to see, please contact us and we'll add more. So for the Transact library, you can install it and run it anywhere. What is the DBOS Cloud? Right. So DBOS Cloud is a serverless hosting platform for Transact applications. So you can run your Transact applications anywhere, on your own laptop or on a Kubernetes cluster, but if you don't have any resources and if you don't want to manage any clusters, you can deploy your app to DBOS Cloud. So if I do use DBOS Transact library, I must get a Postgres? If I wanted to use another database system, could I? Currently, DBOS Transact support any Postgres-compatible databases, so you can use it with your own Postgres server. Many of our users just simply add DBOS Transact because they already have a Postgres server, so they just use the same server to store your execution state data. But you can also use other offerings, like each cloud provider has some Postgres offering. And we're also compatible with new serverless Postgres like Neon, or Superbase, or CockroachDB, or YugaBitDB. There are a lot of options here. So is that just because that's the database you tested with, or is there any specific features of Postgres, Postgres-compatible databases that you leverage? Yeah, I guess it goes back to the question of why did we choose Postgres? So there are a couple of reasons we chose Postgres. First is that the ecosystem is huge. As I said, there are a lot of providers in the cloud, or there are a lot of on-prem solutions as well. People know how to operate Postgres. It's a very mature and battle-tested technology, and people really trust Postgres. So that's one reason. And the second reason is that it's a relational database, so it has built-in transactions. It's very reliable, so we don't need to worry about, like, it has backups and it has replication if you want to. And finally, the extension ecosystem is great. Some of our users actually use Postgres as a vector store. So some people also use Postgres to store time series data. So really, with a single database, you can achieve a lot of things. You can store basically all data from transactional data to analytics data to vector data. So we really like this versatility of Postgres. So Chen, a lot of our listeners, I think, are really interested in the how, like peering behind the black box. So if they will have access to the Postgres database where the state is getting stored, if they peek in there, what will they see and where should they peek? Yeah, so DBOS stores all the information in the separate logical database inside the Postgres server. So you can think of a logical database as another namespace within the Postgres server to isolate it from your main application database. so that we don't interrupt or we don't disturb your normal tables or your normal queries. So in that database, under the DBOS schema, you will be able to find several tables that stores information. So there are several tables you want to look into. The first is workflow status table. That's the core table that stores workflow information. And then when you first execute a workflow, it will say workflow pending. And then as your workflow progress and eventually succeed or fail, will update the workflow status to error with the error, the actual error, or success with the actual output. So that table is the core that drives the DBOS, that stores the state changes of DBOS. And then the second table you want to look at is operation outputs table. In that table, that's where we store the serialized output of each step. And by looking into that table, you will see the workflow ID and then the step ID inside the workflow. And the result of every single step. So by looking at those two tables, you can piece together what workflow has executed when and what steps has executed at what timestamp. Besides that, we also have the workflow queues table. The queues table basically groups workflows into different queues. And then by saying different queues, it's just we assign a different name in the queue names table. And that will allow us to quickly look up what workflows are assigned to be executed in that queue. So I would say the real beauty is in SQL. You can use SQL to manage your workflows, and you can also use SQL to simply query them and observe what happened in your system. Do you provide out-of-the-box visualizations for what you see in these tables? Yes. So if you log into DBOS console, like console.dbos.dev, you'll be able to see we have table visualization. You can basically select or filter. based on the workflow names or based on the workflow status. And we are also actively developing a graph visualization of the workflow execution graph. So say what workflow has started and how many steps it has finished, and what's a workflow parent-child relationship between different workflows. Say one workflow can invoke sub-workflow that executes other tasks, so you'll be able to connect the dots by looking at that graph. Cian, do you have any real-world example where maybe talking to your customers where unreliable workflow executions cause them really problematic system failures or inefficiencies? Yeah, so let's see. There are several use cases. My favorite one is that one of our customers have to persist data across multiple systems. Say, Shopify sends some data through Kafka, and for each message, Each message contains some customer data. They want to persist in their local Postgres database. They want to persist in their CRM. They want to persist in their ERP system. So they want to make sure that the data is consistent across all systems. So they chose DBOS. And if you don't have correct customer data, you can probably lose customers. So that's what they really don't want to see. And DBOS guarantees that whenever you receive a message from Kafka, We guarantee that the message will appear across all systems. Maybe that answers one of my questions, which was, why would I not roll my own or use code generators to help me write the code for durable execution? I think one of what you say tells me that there's a lot of guarantees and compliance that using DBOS might help me with. Is that fair? Are you going through some kind of compliance process that people can leverage? Yeah, so I think we can talk a bit more about the observability angle of DBOS. So because everything is stored in Postgres, it's really easy to query and visualize what's going on. So my favorite quote from customers is that they said, DBOS is great because everything is in Postgres. I can just use SQL queries to see what workflows have run, what workflows have failed, and what happened at each step. And based on the information we store in Postgres, we're also developing more observability features like graph visualization of the workflow and the steps and the workflow maybe spawning multiple workflows there. So we are visualizing the parent-child relationship there. And more than that, what we can provide is actually management over your workflows. So because those are just database records, if you want to, say, resume a workflow, you can just re-enqueue it, put it back into a queue stable. If you want to cancel workflow, then you can just mark the workflow as canceled, and then the downstream process will just cancel the operation. And if you want to say, we can talk more about it later when we talk about operations with EVOS, but if you want to restart a workflow with our new versions of a code, you can just copy the workflow input information and assign it with a new idempotency key, and then just start execution on a new version. And because data is in Postgres, it's a traditional database, it's structured data. And structured data is really easy to analyze and to observe. A little bit of segue here, and we'd like to contrast Dboss with solutions already in the market. And I did mention our earlier episode, the latest one being with Maxime Fatih on Temporal. deboss contrasts with temporal, and is your definition of workflow consistent with temporal's definition of workflow and durable execution? Yeah, so that's a good question, and we got asked about that question a lot. So I think the workflow definition in deboss and temporal are essentially the same, but the core difference is how we perceive the implementation of this durable execution. So in temporal, we call the pattern, external orchestration pattern. With external orchestration, which means you have to start a temporal server. And then when you run your workflow, instead of directly executing each step, your workflow function will need to enqueue a step into a temporal worker. And then temporal worker, temporal server will instead push the notification or will instead notify a worker node to process that. They call it activity. We call it a step. And then after the worker node finishes a step or activity, they will send it back to Temporal Server. Temporal Server will persist the result and then send it back to the main workflow function. So basically for every invocation of your step, there are multiple network hops. And then by contrast in DBOS, we essentially embed double execution in your program. So when you call a workflow, it's just a function call. And when a workflow calls a step, Again, it's a function call, but the function call is intercepted by the DBOS library. So everything will happen just in your program. So we believe the benefit of DBOS is that it's really simple. All you need is your program and your Postgres database. You don't have to deploy an actual orchestration server, and you don't have to deploy distributed workers to process your steps. I get the part about not having to deploy an orchestration server, but... then how do you achieve orchestration? So DBOS implements durable execution as orchestration. So just to clarify that, we achieve durable execution by intercepting the function calls. So you can think of it as when you call a DBOS-decorated function, instead of directly executing the function, we wrap around the workflow function, for example, to say we first persist the input and then call the function. when execute the workflow function, the workflow function will call each step. And again, each step is also wrapped by dboss. So in the wrapper, dboss will first check if the step has finished before. If not, execute it, persist the result. If it has been executed before, directly return the result. So everything happens in the language layer. So that's why we don't need to send it over to a separate worker, to a separate message queue to achieve this. Is there any other areas where it's important to compare and contrast with temporal? I think the two main areas, one is simplicity. In order to add DBOS to your program, it's really simple. You install it as a library, add to your existing program by decorating your functions, and you're done. And then to implement something temporal, you have to basically restructure a program into like, you have to Think in a distributed system way. Every time when you cut out a step, it is essentially an RPC call to another worker and then to execute it and then wait for the result to come back. So we think simplicity is the number one differentiator. And then it's also very simple to operate. So say like to run Dbos in production, all you need is a Postgres server. And then basically people already know how to operate Postgres servers. so we don't add much of the operational overhead to it. By contrast, if you use any external orchestration services, you have to host their service or you have to rely on their cloud providers' cloud offerings. For example, every time you want to call a staff, you have to communicate to their cloud and say, I want to execute a staff, and their cloud will send a message to the worker to execute it, so on and so forth. So that will also give some like implication to performance right in dboss every step is will add basically add a database right which is like a few milliseconds well if you do everything over the network that will easily go a few hundred like a millisecond other than the simplicity of use do you have any thoughts about when a developer should when they need durable execution they should consider dboss versus temporal yeah i mean temporal is really great technology like it's used by a lot of companies. I think one benefit of the temporal model is that if you want to have a workflow that's consisted of different steps are written in different languages, it could be easier to use the temporal model. Say you have a workflow written in Python, but maybe some steps are written in Go, other in Java and Rust. If you have such heterogeneous workflow, it will currently be easier to do it in temporal. Well, on the other hand, if your program is just in Python or TypeScript, it would definitely be easier to do in DBOS. In fact, a current trend is we call full-stack applications. So when users write their front-end code in TypeScript, they actually also want to write their back-end in TypeScript. So that's why DBOS in this case makes more sense because it's very lightweight. You just add it as another TypeScript library and then use it in a program. How much of this comparison also applies to some of the other technologies like AWS step functions? Yeah, so the external orchestration part is the same. So we actually had some performance benchmarks against step functions, and we found that for DBOS, we are like each step is like a few milliseconds, whereas in step functions, every time you have to schedule a step, it will push to a queue and then you have to dequeue and wait for a result. So it will come back as like, 200 milliseconds. So the performance gap is pretty large. And another difference is that Step Functions requires you to use a late JSON description language to basically specify the DAC. And that could be another overhead, I would say. Well, though Step Functions also provide a graphical interface, you can drag and draw your workflows into a DAC. This is really nice. And this can be used by for example, developers or people without too much coding experience. However, the problem is that it's very easy to use when you have simple tasks, but when you have a more complicated task, it's hard to manage. Just share an anecdote from our conversations with users. Someone was switching from step functions to DevOps because they said eventually they gave up on code review because they have to review 3,000 lines of JSON code. And then... In that case, using DevOps is better because all your workflow logic is essentially just your code. So you write workflows as code, and then you can do code review, you can do your debugging normally as you would when you develop your functions. A follow-up question there. In some of the previous episodes on this topic, there was pros and cons of the graphical representation discussed. Certainly, there was a sentiment expressed that developers are not very fond of the graphical representation, but more business users find it useful. What is your experience there? And if I did want to communicate that with a business user and I was, as a developer, using DBOS, what are my options? Yeah, that's a great question. I think I have the same experience when talking to developers. They usually say they want code. They want to see code. But when we talk to more business-focused people, they want to see, like, what's going on. Like, say, they want to have a visual or a graph visualization of what happened, like how many steps happened, how many workflows happened, and if something failed, which step failed. So in DBOS, we basically provide, we're actually actively developing a graphical visualization based on the information stored in the database. The reason is that you can't define workflows based on graph, but we provide observability into what happened, what have executed before. I think that is a good balance, I'd say, because developers usually use code to develop their workflows, while business-focused people usually need to see what happened, like the execution of those workflows. I think that's a good combination. You had talked earlier about scheduling, how one would achieve that. Can you talk about rollback with DBOS? Because rollback was another big plus of a workflow system, being able to achieve that. How would I, as a developer, achieve that if I was using DBOS? Yeah, so if you use DBOS, you can just handle exceptions normally as you would when you develop your API or your service. So in the checkout example, so if the workflow says, sees the payment failed, then you will need to call the undo inventory reservation function. And you also need to call some functions to cancel the order. So in DevOps, you basically express them in code as error. Like if you see this error, call those sequence of functions. If you see other types of error, do something else. So we do ask users to explicitly specify the rollback actions. And the rollback actions are also steps. So the benefit of using DBOS is that we'll guarantee everything, the workflow, that all the steps will run to completion. How do I, as a developer, take an existing body of code and the annotations that DBOS wants? What process would I go through? Yeah, so we think it's pretty easy. So you first install the library. If you use Python, it's pip install. If you use TypeScript, it's npm install. And then after that, you can just import dboss. Then you say you have a function you want to be a workflow. You just decorate it as at dboss.workflow. And then within that workflow function, you will see what function calls it makes. And then you decorate those functions as at dboss.step. I think my question was more like, how do I figure out which functions I should now go and decorate? All right. So... I think you will essentially decorate any functions that will talk to external APIs or talk to the database, anything that will generate side effects outside of your program. Thanks, Chian. So I'd like to now go into a different section where we talk about maybe the more use cases that we see for durable execution, especially with AI agents. Before we go there, is there anything else you'd like to add? Yeah, so actually, I'd like to talk a bit more about workflow recovery. and failure recovery when you use in production. So as we talked earlier, it's true that developers can write checkpointing code manually, but you want to use a job execution system or a library like DBOS or others because you want to have automatic recovery. So to give you a concrete example, in production, you usually have to deploy your code in multiple... If you use Kubernetes, you will deploy to multiple containers or pods. And in a large production development, you could probably develop maybe 100 or 1,000 pods that each will serve, well, you'll load balance between those pods to serve requests. So the problem is that at any point in time or within any hour, some pods will definitely fail. So it's a probability to fail is high when you have a large deployment. And if you already use Kubernetes, you may think, yes, I can just restart those pods. But the problem is that, sure, when you restart a pod, you still need some application logic to decide how to resume the work you've done before, before you crash. And if you use Dboss, we provide a service called Conductor. So this service basically connects to your running applications, and then it will detect when some of the workers are failing. So if it detects some workers failed, it will try to redistribute. the pending workflows running on that worker to other healthy workers. And this way, we automatically recover from failures, and we guarantee that all workflows will eventually run to completion. So I think this is one reason that you really want to delegate this kind of recovery to some library or services like DBOS. How does that coexist with the recovery mechanisms in Kubernetes? Yeah, so in Kubernetes, basically when you restart, it will restart your application, but it doesn't have any application semantics. It doesn't know what workflows failed before. Essentially, DevOps as a library will have some background threads to listen for tasks, say, you have to recover this and that. So DevOps will dispatch certain recovery commands to your workers. And this works perfectly with other mechanisms in, like, say, Kubernetes or other deployments. So what you mean is that I develop my application with the Dbos Transact library, and then when I deploy it to production, I can leverage Dbos Conductor, whether or not I'm deploying to Kubernetes. Yes, exactly. Do you use Dbos Conductor automatically in the Dbos Cloud? Yes, we have the same capability in Dbos Cloud. So if you deploy to Dbos Cloud, you have all the automatic recovery plus auto-scaling. spend some time on ai agents that was one big use case on your website why is the problem surfacing now with ai agents and requiring durable executions specifically for that so i think this is a really new and emergent use case for dboss so actually besides ai agents we also have another ai use case which is ai data pipeline if we have time we can dive into that as well and for ai agents basically the program will be driven by AI or LLMs. So say instead of developers coding what functions to call, the function calling or tool calling will be decided based on the LLM responses. So I actually, I built a very interesting refund agent using LandGraph and DBOS. So basically what an agent does is that when they receive a refund request, it will talk to LLM to decide which tool to use, maybe based on the customer record, based on the order, it will call a refund workflow. So very simplistically, am I right in that the agent is essentially an LLM with tools at the most basic level? Yeah. Okay. Sorry. Continue, Chen. Yeah. So then the agent can say basically, oh, if the purchase was above a threshold, I will need to send an email to an admin. to verify whether you want to approve this refund or not. And then the refund workflow will have to wait for human input and then decide what to do next. So I would say with AI agents, the most challenging part is first, dynamic execution of workflows. You can no longer have a very static workflow, like workflow branches or function calling will be very dynamic based on the LLM output. And second, it's unreliable in many ways. Like from my point of view, we treat AI as an unreliable service in a stack. So your AI call may fail, and then every time you call it, it may give you a different result. And then third thing is that human-in-the-loop is really tricky. Like, how do we make AI work together with human? It's a kind of new and challenging topic these days. Can you talk more about the agent you developed and how you use DevOps there? Yeah. So for the agent I developed, basically, I decorate my tool function as a DevOps workflow so that it will guarantee basically if I refunded it before, I will only refund it once. So this guarantees that if the user asks, I want to refund it again, based on the record information, we'll say, you've already been refunded before, so we'll skip the refund. And they will guarantee that once we kick off the refund workflow, it will always finish. So say if something happened to the program, if it crashes in the middle, we will just resume the refund process from where it left off. So let's say if I already process that payment or kind of return the money back to users, we'll also restore the inventory and stuff like that. Basic question though. So if the agent is the LLM tool and the LLM is the one calling the tool, which in many cases, the tool is an API call, how does now DBaaS get involved in that part of the workflow? Yeah, exactly. So the cool thing is that, as I introduced before, DBOS is a library and annotations that you can wrap around your function with. So when LLM calls the function, it will call the wrapped function, and that wrapped function is a durable workflow. So that's why it's super easy to add DBOS with any AI frameworks. Like, instead of passing your bare function, you pass the annotated function into your AI tools. That's interesting. You want to talk about data pipelines as well? Is there an example there? Yeah, so data pipeline is one huge use case in Dbos, where basically the typical pipeline sometimes goes, the first, it's a workflow of first scrape websites or scrape other data sources and store the PDFs or images into some S3 bucket. And then the second step will in parallel. We'll use LLMs to analyze those images or PDFs. And then the third step will be to persist the return value from LLMs into multiple data stores, either vector database or Postgres or multiple data sources. And then maybe another step is to do some decision based on or analytics based on the results from previous steps. Like a concrete use case is, for example, stock marketing monitoring. So some of our use cases would be to scrape the website of stock markets and then put into LLM to do some analysis and then do some business decision based on like, do I want to invest in the stock? Do I want to sell the stocks as a final step? Why durable execution? Yeah, durable execution is essential because you don't want to lose data. And once you process, like usually there are two reasons. One, the data volume is huge. right so easily you can process 1000 documents and if something fell in the middle or not just 1000 maybe 10 000 documents if something fell in the middle you don't want to restart from the beginning like you don't want to say oh now something happened to my pipeline either ai failed or i i hit some ai rate limit i don't want to say i have to reprocess all 10 000 documents again i want to resume from where it left off so that's the first thing the second thing is you want everything to complete say if i process 10 000 documents i want all of them to complete otherwise my business decision for example which stock to trade may be based on incomplete data that can cause like financial loss and other sequence consequences so in this specific example if i did decorate the loading step with dboss and the error was on let's say the 100,000 document in my set of 500,000. What is the behavior? So basically, in this case, you want to use DevOps queues to parallelize those tasks. You don't want to process all those documents in one giant step. You want to make sure that you want to basically say each step is a queued task spawned by my workflow, like from documents. 1 to 10,000, I will enqueue a task. And then it will become a fan out pattern. And then we will wait until all tasks finish. So I will fan in by awaiting those results. So if any of the tasks failed, then when we recover the workflow, we'll just say, OK, now I have to check all those queued tasks. And if some tasks fail, then we'll restart execution of those tasks. But for those tasks that have already finished, We'll just get the results from the database and then return it. So that's how you can correctly recover from those failures. So you had earlier mentioned about using Conductor for DBOS in production. Besides that, are there other recommendations about being able to run DBOS at scale? Yeah, so the question is mostly about the best practices for using DBOS. Yeah, so I think to run DBOS at scale, you first really want to make sure that your workflows are deterministic. That's the core to guarantee your workflows are recoverable. Because if you have non-determinism in your workflow, when we recover it, we may go through another execution path that we don't really know how to recover. And then the second thing is use queues wisely. So for example, when you deal with LLMs, they typically have rate limiting. And they also have concurrency limits, so how many outstanding requests you can have to those LLMs. So in this case, use debug queues. So with queues, you can add a rate limiter, say I want at most five API calls within like 30 seconds, and I want at most 10 outstanding requests at a time. So with those preventives, you can build your apps with parallel tasks, but also within the limit of those LLMs. Is there anything else you'd like to add for folks that would like to get started with DevOps Transact? Yeah, so for DevOps Transact, download it, install it, run it locally on your laptop. The cool thing is that because it's a library, we can also leverage the debugger. So we actually developed a time travel debugger that you can install from VS Code Marketplace, where you will be able to replay traces happened in anything happened in the past. say, I want to investigate why the workflow executed that way, like executed yesterday. Then I would just pick the workflow and then tell our debugger extension to say, I want to re-execute that. And when we re-execute, instead of writing to the database, we'll just pull the information from the database, say, this is the workflow, this was the input, and those were the output of each step. So we can, instead of actually say, If your workflow contains sending email or calling out to Stripe, we'll not actually call those external APIs. We'll instead return the recorded information, and then you'll be able to step through your workflow as if it happened in the past. So that's very cool. Thanks so much for today, Chian. I think a lot of the goal of the conversation, at least what I was trying to achieve, was distinguish between workflow as a graphical representation aid for business users, versus grouping together the series of steps or services that need to be durable? And how does that play into the now where a lot of the cogeneration is happening with AI as well? And where exactly does Dboss fit into it? You've explained that you can definitely use the simplicity of DBOS and the performance of DBOS in grouping together steps that you need as a workflow. And I presume one would be able to use AI to then introspect that code and generate the graphical representation as required for business users. But then the question would be, why would you not use AI to even write all the framework that comes with DevOps? Could you just maybe spend a few minutes on your thinking about how AI fits into all this? Yeah, so it's a really interesting question. And I've been thinking a lot about it recently, especially with AI, we've seen a lot of AI generated code. It's true that someone may say, okay, AI may be able to generate those checkpointing code, but AI won't be able to generate automatic workflow recovery, it won't be able to generate the control plane that can automatically recover on failure. So we think DBOS and other workflow engines still play an essential role here in the AI era. Right. And then with AI-generated code, I think we really want to make sure they are reliable. And by reliable, I mean like, yes, those codes may be buggy. So you want to have a way to say, if I caught a bug, I want to be able to investigate a bug. I want to be able to say, restart my workflow from a specific step and then fix the bug. So those are the capabilities where those durable execution engines really shine. Because with those AI-generated unreliable code, you want a way to correctly store those information for debugging. And because I think the one advantage of DBOS is that because we store those like what step has executed and at one at a time stamp of it in a relational database, so we really have this structured data for AI to analyze. It's easy for human to analyze, and it will also be easy for AI to analyze what's going wrong. So I think it's a really exciting new capability that we are exploring. And I think simplicity and structured data logging are important. So simplicity is that we actually try to add a prompt for DBOS, and we give the prompt that will incorporate our latest version of the code, will basically give AI instructions on how to add DBOS to your code. With that, we can just tell AI, make this code durable. And then AI will correctly do that in one shot. And that's really interesting because if we put dual execution as a library, we will be able to make AI-generated code durable very easily. And then when you execute that code, we automatically checkpoint data in a database. So it will be very easy for a human or for AI to investigate what's going on. And then because everything is a database, it also allows users or AI to automatically fix those code. Because to fix those code, you just need several SQL statements to modify the table, to change the result, to change the output of the recorded outputs table, and then to continue from where it left off. So we are really exploring the synergy between your code, your database, and the way to modify your code to generate new traces, to modify the database, to observe databases. It's a really exciting area. How can listeners contact you if they have any feedback or questions? Yeah, so to contact me, you can visit my website, chanli.dev. You can follow me on LinkedIn, follow me on Blue Sky or Twitter. But also, don't forget to visit dboss.dev. This is our main website, and we're looking forward to your feedback. Thank you so much, Chan, for coming on. Thanks, Kanjuan, for inviting me. Thanks for listening to SE Radio, an educational program brought to you by IEEE Software Magazine. For more about the podcast, including other episodes, visit our website at se-radio.net. To provide feedback, you can comment on each episode on the website or reach us on LinkedIn, Facebook, Twitter, or through our Slack channel at seradio.slack.com. You can also email us at team at se-radio.net. This and all other episodes of SE Radio is licensed under Creative Commons License 2.5. Thanks for listening.

Qian Li of DBOS, a durable execution platform born from research by the creators of Postgres and Spark, speaks with host Kanchan Shringi about building durable, observable, and scalable software systems, and why that matters for modern applications. They discuss database-backed program state, workflow orchestration, real-world AI use cases, and comparisons with other workflow technologies.

 Li explains how DBOS persists not just application data but also program execution state in Postgres to enable automatic recovery and exactly-once execution. She outlines how DBOS uses workflow and step annotations to build deterministic, fault-tolerant flows for everything from e-commerce checkouts to LLM-powered agents. Observability features, including SQL-accessible state tables and a time-travel debugger, allow developers and business users to understand and troubleshoot system behavior. Finally, she compares DBOS with tools like Temporal and AWS Step Functions.

 Brought to you by IEEE Computer Society and IEEE Software magazine.