Restate 1.3 - Resilient Concurrency - Community Meeting May 2025

Hi everyone, and thanks for attending this community meeting. Today we're going to talk a little bit about what we did there, what are the new features, and also a little bit about the new internals, how the general works, something that you've already heard already, it's a project called shared core, and yeah, and then we can get through there. So, let me show my screen. All right. For Hoot as a nominee, I'm Francesco Vardiani. I'm working with State and I'm working there pretty much on the SDKs, developer experience, and APL viewings when you interact with your estate. And yeah, again, today we're going to talk about the new features and then we're going to go under the hood and see how things work. So, for Hoot has not seen the blockbuster, ASIN 1.3 has features. in three major categories one is that there are more support to what we call that the music resident concurrency which is this ability of executing concurrent operations and then i wrote the events in a way that you can either domestically await them always in the same way support for a founder in control over service to service communication which is a a set of features that have to build things like task orchestrators and then we also have improved our service capital quite considerably and now we have a good support for jesus schemas and um the ways to generate them out of your own service definition and so on and so forth so i will have to start just jumping straight to code and uh as usual this is a community meeting so be free to impart and ask more questions you don't have to wait the end for this so we're going to start first by looking a little bit about um at the features surrounding uh deterministic concurrency here we have an example where we have to do an email verification and to do online verification you need to send an email with a link and uh in the recent work there we can create uh a workbench using the recap box so we send an email with an email id and then you want to start a verification timer such that either the user clicks on the link first or you want to make a further verification time to expire and then we also want to add another timer which is a reminder that we say one in five seconds for example we send it to the user reminder asking to click on the link And the way we will do this with the new deterministic concurrency features is that we have, for example, and here I'm showing cotton, but you've got the same feature, same all the other SDKs. We use this feature called select that lets you essentially erase either on the available to complete, which means the user click on the link, or the reminder timer was fired, you need to send another email, or the verification timer. fire them meaning that the verification expired here we are essentially leading them through events that are increasingly concurrent and the person that fires completes this select and then if they're there if we have verified there we don't anymore if we need to verify loop again so and uh i'm going to show you actually this running and uh show you how this completes between state and is a resilient and is able to replace it terministically so let me go over here we go and we have our own cti and here are we registered the services just for simplicity now go here and verify email where you see also i get auto uh completed there with a an example of body shape but that's because now we propagate json schemas to the state server meaning that and we say server it wants uh whether it's the json object that our services is expected and let's say i'm going to send an email to you so i want a very farmed email and there was a new developer and he wants to request and we can see here in the console that link of the vote you send the email the first time and then we have here the sending reminder email that is after any other now and then that's because we are in this loop and every time we will select the branch where we have the longer time if now i kill this or do you expect from restate you expect that we state will be able to restart the location from the point where left and you can also see here in the journal that we got um we sent a different um we started executing the the invocation then we send a verification email then we create a first time note that is the verification timer that is 24 hours long and then we created this reminder timers and here we see that the invocation is stuck because we cannot switch the sdk so now we go back and we restart we see that we very much restart from the point where we left so we start from the reminder in this case was probably one and this wasn't probably the fifth or the sixth reminder here and uh just to complete it let's try to compute this signal here so we take the other cable id which is a decision that starts with the signal underscore and we go here and we complete the awaitable and this completes the verification process step for share close the other window but this completes the verification process so that's just a simple example of how you can await concurrent events and then maybe a question in the meantime someone in chat asking how do you recover in cases where the journal is corrupted missing or lost well i mean uh first of all every state um The restate cluster stores the journal, persists it, and replicates it across nodes. So if you have a cluster of three restate nodes, depending on the replication factor, of course, you will have more nodes in the same journal. So that's, of course, something important to say for people who don't know it. And generally, the idea is that when restate tries to replay, And when it loads the general aid, of course, does some sanity check to make sure that actually the general is saying, the general is not saying they won't try to apply or to re-execute it. But we also, right now, don't imply anything like checksums or anything like this in place to make sure that the general is actually correct. Does that answer the question? I think for now it does. I think you can continue in this app. what consensus algorithms do you use to maintain consistency across different journals across nodes can you manage multiple failures uh yes we can manage multiple failures there's actually uh there was a community meeting there i think was the previous one right although two committee meetings ago actually went pretty deep on this I can post a link to that community meeting here in the chat as well, together with the blog post, which explains a bit more about how this works. Yeah, I think it's better because it's a reasonably long topic. All right. Yeah, so this one is one of the examples about between EC concurrency. We've got another example. It is where I want to show you a classic example of fan out, fan in. a service that just needs to execute some tasks in concurrently so it gets a um the task definition splits it in sub tasks and then after that you can answer the tasks asynchronously and then it just wants to wait and then again it needs to wait for all the results So then just a little note for people that don't use the Kotlin API. The Kotlin API is just a little bit of a difference from the other APIs. Run block is just context run. And run async is run block, but it retails your future that you can await later on. So, again, what we do, we split the tasks, we then schedule the tasks to execute, then it is run async, and then we wait for all the tasks to complete. And I want to show you a little bit how it behaves. So, let's go here in fan-out worker. No, sorry, is this one? Yes, this fan-out worker. Okay, let me just give a list of comma-separated task lists. A, B, C, D, E, and G. All right. And now you can see here, all right, all the tasks that are completed. And this is already we're trying because some of the tasks... they have a boolean or random boolean fail or not fail and we can see here that we have a journal where some tasks completed and orders and other tasks did not complete and now we just read a little bit and actually this this is gonna look fine but another thing that i want to do here actually is that i want to cancel so oh i cancelled too late probably i probably cancelled too late let me try again uh okay let me send the record yeah i cancelled too late because everything was done afterward let me try this again and then all right because this one of them is failing and then let me try to cancel it now okay and then the reason i'm showing you this is that one of the updates that we say 1.3 is that now cancelling the context line actually works completely fine and another update obviously 1.3 is that now you will see the cancellation in the journal so here we see that we are um again we have this journal where we started by splitting the tasks then we have these tasks and then we'll try to run all of them concurrently but then when when awaiting them when awaiting all of them we got a cancellation and that and what we did after the cancellations is we just run asleep for it make sure that we can read the journal so nothing special but the point is that now you can run multiple side effects in parallel and wait all of them in parallel and at the same time you can wait for a cancellation to go and eventually run the compensations in case you are building a saga or something like this this is extremely helpful so we have a use case that uh previously unfortunately i don't have the code here but we have a use case here in the sagas I think it's somewhere examples. There's also a guide on this topic in the documentation guides. True, true, true. There is a guide on this topic. But the cool thing about Restate 1.3 is that now you can go here. Yeah, let's see. Show the code, right? And now the saga here can essentially wait in your parlor. all the different operations and also here it can fire all the operations and then await all of them in parlor and then eventually execute the compensations and all of this will just work fine essentially so and the other end the reason why this is the deterministic part of the deterministic concurrency it's so important is that because we state requires you code to be deterministic it requires that you can replay in the same order that you executed the code the first time, then with this deterministic concurrency, you are actually able to essentially await any events to happen at a given point in time and take on each replay the same decision of which event you take. I'm going to go away between the details of this in a few minutes. And, um, okay. So this is for the deterministic decision and concurrency. I want to. Just to touch a little bit on what features about service-to-service communication, because those are actually very highly requested from people. Let me go over this. We got another use case. This one is pretty much a task orchestrator's use case. We've got a task runner that just orchestrates the different tasks to run, and then a task service. The task service is just a mock that sleeps for a random amount of time. easy and then we got the task orchestrator now the task orchestrator and we see a request to start a task it starts a task and they remembers about it and then it has a get task result to retrieve the task of the result at a later point in time and then it also has uh cancel them all which just takes all the tasks and cancel them now to start a task with your business as usual we just send a request you use a send to start a task but then what we now can do is that we can add a reporting key and on top of this we also now propagate back to the caller service we now propagate back the invocation id so when i do a request i can also get the invocation id of the request that i just made so now when i start a task in this task render what i do is that i send the request to start the task i provided input in ck and then i just store it this is a virtual object so business as usual i just uh i store the map of the tasks that i started and then in the get task result what i can do now is that i get the task invocation id by reading my state and then after that i can use the functionality called attach now this functionality just lets me connect to an invocation that either just started that either is still running or it was already completed and get back to result when it will be available so in case the location is still running i will connect to it and wait and listen for its result or if the location completed already i just get back the result immediately and this attach is like all the other methods of the context meaning that it returns me back a doable future that then i can put inside for example select or an await or learn you know use deterministic concords essentially and the third method cancel them all what it does is something similar to the previous method so it takes all the running tasks that i started and then after that it will um it will use again the same api invocation handle then to cancel the task and this cancel behaves alike the cancellation that you trigger from the eye so it puts the he sends the signal uh to execute the cancellation which gets written to the journal and then this signal uh it gets processed by the colleague service so in 50 lines of code we wrote a very simple task orchestrator that is able to start invocations we can show you start them once keep track of the invocations that started, get back the results of these invocations, and then be able to essentially do the cleanup and cancel them when you don't need them anymore. So, I think this is it for what it belongs to the new features. The improved service catalog is, well, you saw it. uh so like now you see that this playground is something we're already shifting 1.2 but the 1.3 we expanded at it and now it's just much more filled with details and the schemas are just much more rich and not all the sdks now will propagate out of the box wireless the schemas so i mean doesn't matter which language you choose you will get to these schemas and depending on the sdk you can tune out the schema gets generated all right do you have any questions about this part all right i just want to conclude this section saying uh thanks for your feedback really um the features that you see here where that that we worked on in 1.2 where features that people uh very much requested for and um as usual as we like to stay in the state we also like to drive our roadmap to gain some community feedback so thank you so much for this it's very very helpful um all right so i want to go a little bit now in the section where i talk about how m thinking works under the hood so for all of you that knows restate already you know that at a very high level when you send requests to restate you send requests to the restate cluster the restate cluster will then um persist invocations replicate them and then start sending the request to your service code that ultimately hits your business logic when there is a cluster does request to the service code it does it doesn't buy the energy to be request and it transports back and forth what we call the journal the journal is where we store the invocation progress now in our receipt 1.2 we did a huge update to this journal and we we modified the data model to make sure we could represent new things inside it i'm going to get back to this in a second and the important thing and what i would like to stress out is that this journal data model is actually really the representation of the apis of restate so when you do something inside a context you write an entry to the journal and this means that effectively what how this journal works is fundamental to the semantics of the restate of the system now um this new journal is what we call the append on journal and i want to show you here a little bit of an animation of how this works so you see on the right you have code you can imagine this being taxed by there because it works the same with all the other languages uh we will contact sleep now which returns me a future then i do context one which returns me a future then i await the context run and then i wait the context sleep and then you see on the left what restate writes for any of these operations so when i even do context asleep it will write what we call a command for sleep then we will do context.1 we write in the journal a command for run then eventually the run completes and we write a completion for run and then eventually the sleep completes and we write a completion for sleep what we did with this new data model is that we now write separately commands and completions and we establish a total order between the command and completions now the commands are driven by your code of course this means that like it literally depends on whether you write in your code first the context sleep and or context run if you swap line one and two you will have in the journal in the reverse order the commands while completion order we is is not something that depends strictly on your code it's actually intrinsically non-deterministic and you can have first the completion for run and then the completion for sleep but you can also have the vice versa So you can have the completion for sleep first and then the completion for run first. And the reason why this appen-only journal really is helpful is that now we record which of the two completed first. So you can see how we essentially profit from knowing which of the two completed first to implement the deterministic concurrency. If you imagine line three and four rather being a promise dot any. I can deterministically always choose either one or the other, because I have this order recorded in my journal. So, yeah, what are the benefits here? Obviously, we can now do deterministic concurrency, thanks to this total order. But also, we have a good paper trend that is very useful for an inspection, in particular if you are in environments where you have... constraints that are about compliance, restrictions, and things like this, then this is really helpful in that context. And then, in general, it also adds debugging. I don't want to spoil your features too much, but we are working on something in UI that will actually profit from this fact that now we store completions, the completion order, and it will help you much more understand, essentially, like visualize. exactly how things went inside your invocation and then they will help you a lot more to actually debug effectively what happened in your invocation and one of the things that we did with this new app and only journal is that before we had a let's say a specialized design for context run where we had in the previous data model we had entries uh completions and then we had context running which was very much special thing on its own And now we changed this a bit with the append only journal and this new design actually allowed us to put context run inside the rest of the design. And this is very important because thanks to this context run now works fine when we interplace with all the other restate features. Before we had this issue where context run did not really work when you use cancellation, while now it works completely fine and we can expect it to work as any other function inside the context. And then the last thing that I showed you already in one of the demos is that now, actually, the cancellation is a message to receive explicit in the journal. So you can, like, when you open the journal view, you see cancellation happening here. That's very helpful. Now, the journal is, of course, a data model. It's a data model that restates server implements. But, of course, we will wait to transport this to the SDKs, because then the SDKs are ultimately... the library that you can bring in your code to develop pre-state services. Now, in order to do this, we have this protocol that we call the invocation service protocol. Shorten at the service protocol. And this protocol is essentially just sending protobuf messages on top of a HTTP to invocations thing. And if you are curious and want to know a little bit more about this, you can look at this link where and you can see that there is a pretty much a one-to-one mapping between everything he had in the context api and a message that it goes on the wire to give you an example of how it works a normal restating location is that when we say starts it sends a start message to the sdk and then every time you call a context method it creates a command message which gets sent to the server the server writes this command in the journal then the server of course registers the timer for the sleep the sleep at some point completes the timer fires and it writes a completion slip in the journal and then after that happens we send back a slip completion back to the sdk and the peculiarity of this protocol is that it is also able to manager in place of course so what is able to do is that if you start this uh like the first time space it just sends a start and the input and then the subsequent times it will resend again start the input and then the zip and the completion to replay that part of that you have already executed now as you can imagine this requires a little bit of a business logic in the sdk to employ this and uh when we started uh uh restate at the very beginning we had just two sdks we had just java and the tagscript and uh back then things were manageable but then when we introduced the old sdks we introduced something like three months python and uh rust started to become a little bit more complicated to make sure that all the sdks were in sync would support all the features so we had to find a solution to this and we came up with this project that we called the sdk shared core so on the on the right you see uh how more or less every sdk works at a glance is composed by three layers you've got the apis the context you interact with then you got the state machine which is the implementation of the service protocol and then at the bottom you have the network which is an http server essentially and all sdk's work like this even before we actually worked on the shared core and what we thought to do is actually to create a unique code base which is what we call the sdk shared part for the state machine and such that every sdk now works as follows it has an api that is the one that you use which then interacts with the state machine that is this is a sdk shared core and then at the bottom there is a network layer that is essentially just an hp server this one is still separate and only region for choose decay because for example in task we want to support node.js but we also want to support the dino we want to support cloud for workers done and i don't know if there is anything else i think it's this four uh in python we want to support ascii which uh has a new corner hypercorn and a few other hp servers so this network layer is a just a little thin layer that really depends on the platform where you end up deploying your service. Well, the API layer is essentially the experience that you get when you use the SDK. And the state machine is literally this implementation of this service protocol. And we've already looked at this. It's built in Rust, and we use it in the Python SDK using Pyotree, the disbinding generator between Rust and CPython. In a growing task feature using a WebAssembly, so we compile a shared core. WebAssembly, and then we ship it with the SDK. And then the last SDK, of course, use a link here. And why did we do this? Well, it's, I hope it's very clear to me, like, just update one codebase and all the SDKs profit from it. Just to give you a very simple example, yesterday I called my mind that there is a bug, actually, now we notify an error in the SDK. and i update the shared core and then i just do a patch release of all the sdks and then we just get essentially the bug fixed so that's pretty it's pretty useful and um just to give you a little bit of idea again about this actually works um like the shared core as an api that is uh that it's not too far from the context api again for each corresponding method in the context let's say context.sleep you got a corresponding sys underscore method that is inside the shared core and then when you await a feature you have something that we call take notification that essentially lets you pull a notification from the shared core and just essentially surface it back to the user code um there's a go here to this hour i looked at this with a passion you've probably noticed that The SLEEP does not take SLEEP duration as input, and it takes this wake-up time since UNIX epoch. That's because the shared core is designed to be a pure code base that does not execute any syscore, and this is what makes it easy then to compile it to WebAssembly and all of these platforms, where you cannot access essentially the work clock.

With Restate 1.3, you can now implement even complex, concurrent applications, and let Restate make them easy to implement and failure-proof.

​In this community meeting, Francesco Guardiani shows us what's possible with the new SDK features of Restate 1.3.

​This release also makes some changes to the protocol with which the SDK interacts with the Restate Server. Francesco gives an overview of how this communication works.