Next.js + Inngest: Unlocking Long-Running AI Workflow Automation

Back-end services like AI can be tough to handle in an application. AI jobs need to be able to retry after errors and they often need throttling and typically these jobs have a sequence of tasks that need coordination. And to handle all that complexity, you should consider something that can orchestrate those tasks like Ingest, the sponsor of today's video. So let's jump into building this really cool AI application and see just how much easier your life can be using a workflow engine. All right, so here's our application. I'm going to choose my webcam and then I'm going to start capture and you can see me and my background. And the idea of this app is that we are going to rate the background. It's going to take a look at the background and give it a review. What's good about it? What's bad about it? And then it's going to also in parallel find the theme of the background and generate a new background using Dali using that theme. So let's give it a try. All right, so there was the background that it took. Now it's going to stream out on the left hand side. It's review. Then on the left hand side, you can see the theme that it came up with. In this case, a creative home office with eclectic decorations. And in the background, it is asking Dali to generate an image, which you should see pretty soon here. And there you go. I guess this image has crosshairs on it. I run this demo a bunch of times and I've never seen that come out, but I don't know. I guess it looks pretty okay. So how is this actually working? So here's our background rating workflow. So we're going to start with the image and then we're going to send that off to two different AI jobs. One is going to review the background and the other is going to get the theme and we're going to run those in parallel. So those are going to go off and hit OpenAI with different prompts, but the same URL of the image. The review background prompt is going to give it a more broad prompt, tell us about the good things, the bad things, and we want to stream back that response because you know it's going to be pretty big. the get theme response is going to just ask it for a very short theme for the image and then it's going to use that once it gets that back to then generate a new background using dali so it's going to fire off a prompt to say generate an image of webcam background based on this theme and dali is going to come back with the image that we just saw with the crosshairs in it okay so where is this all going to get stored Well, the images again get stored in the Vercel blob store. The text parts are going to get stored in Vercel's Postgres database. The reason I chose Vercel was that you can deploy this application pretty easily on Vercel. Both those things are free and you can try it out for yourself. That's just a nice way to get this all set up. The resultant image from DALI is stored on OpenAI storage. To coordinate all this work with the LLM calls and DALI, we're going to use the sponsor of this week's video. ingest ingest is going to handle the retries of those flaky llm calls and it's going to handle the throttling that we need to do for dolly requests and we get to configure that all in code it's really cool let me show you all right so this is the finished version of the background raider i'm going to open up the unfinished version of the background raider and we're going to build it up from there you get both of these projects for free in a link in the description right down below All right, I've got my background radar starter up. So I'm just going to take a look and see where we're at. I'll go back and try another background. Now we have the same starter screen. I can hit capture and take a photo. And that's all it does. This is the detail page where we get redirected to. And right now, all it has for us is the image that we just took. Now we need to go and send all of that off to the AIs and get the review and also get the theme and also get the image. Let's take a look at it in code. Now, in the source of this application, if we go to app and then page, we can see the homepage. And in there, we have a server action called on upload image that on upload image is called by the camera. When you click take photo, it creates a form data object and then puts the image in the form data. We then extract the form data. We create a new random file name. We upload that to Vercel's blob store by using put. And then we add a record in Postgres using add background. And once we get back the background ID, then we can redirect to that page. Pretty simple. Now you might be tempted to do is simply just start dropping that AI code in here. Start calling out to open AI to go and get the review and stream that all back and to do another in parallel request to go and get the theme and then another request to go and get Dali. But there are retries that you'd want to do if the. LLM work didn't work and there's throttling to be done on DALI. So there's just a lot of complexity there. I think too much complexity to put into this little server action here. So we want to do is instead use ingest. So first things first, we need to start ingest. I'm going to create a new terminal and in there I'm going to run ingest locally. All right, let's bring that up. I'll bring that up in Arc. We can now see that we have our Runs tab. We're going to see all of these workflow jobs and I'm actually going to put that side by side with our Next.js application so we can see all of that happen in real-time and it is really, really cool. So this is the local version of Ingest. You can also use the hosted version of Ingest and you can deploy your own Ingest server. There's all kinds of ways to deploy Ingest. So let's get this party started by sending an event to ingest to tell it that we've uploaded this photo. So the first thing I want to do is simply add ingest to our app. To do that, I do add ingest with the npm. Let's bring back the server. And now I want to create an ingest client. So I'm going to create a new directory for all the ingest code, call it ingest inside a source, and then in there client.ts. And then I'll create the ingest client. All I need to do is pull the ingest class from ingest. I instantiate the ingest class that gives me ingest which I then export and I give it an id I can call that id whatever I want I'm just going to call it rate my background basically identifies the app all right so now we want to use this client over in our page so I'm going to import that client and then I'm going to send an event to ingest to tell it that I've uploaded an image so I'm going to give it the background id if it needs to update the database which it will as well as the url of the image from the versal blob store all right let's give it a try so I'll go back to try another background I'll start capture, take a photo. Now, we didn't start any runs, but we did send an event which shows up in the stream panel. So we get the radar image uploaded event. So now we know that we are sending an event back to ingest. Now we go back to the runs tab because that's much more informative when it comes to how things are playing out and how it's interpreting those events. So now let's talk about what's going to happen next. So we've got our Next.js app and we've got our ingest server. and we have that on upload image server action that we just edited. We sent out our rater image uploaded events to ingest, and that's great because now the Next.js app can actually go off and field the next request. What's going to happen next is that ingest is going to call back to our server on a specific endpoint, in this case, API ingest with the function invocation of get reaction because we're going to bind the event of rater image upload to the get reaction function. I'll show you how that's done. But that's how the communication between ingest and the Next.js app works. It's just calling an API endpoint. It's really simple. So let's first build out that API endpoint. To do that, I go over to the API directory. We already have an API directory because there is a background API that allows us to get the current state of the background that's called by React query on the client. I don't really need to deal with that all that much. Let's go and create a new ingest folder in there. And then within that route.ts. And then we'll create our endpoints for that route. To do that, we bring in the serve function from ingest next. Ingest has a really nice next JS integration, makes this really easy. You get this serve function that you can then give the client as well as any functions that you want to register. And it gives you back get post and put all in the next JS format for the route. So that's really easy. So the next thing we want to do is create that get reaction function because that's what's going to get called when we register against that. image uploaded event. So I like to put all my ingest functions inside of the ingest directory in a new directory called functions. And then within that, get reaction. So in that file, we're first going to bring in that client again. And now we're going to use the create function method on the ingest client to create that function, get reaction. And then we're going to export get reaction. Now we're going to name it using ID, so we call it get reaction, and then we bind it to an event. In this case, rater image uploaded, which is what we send out of the page. So that matches the send over here. Next thing we want to do is get the data from that event. So we're going to look at the event object, and then we're going to deconstruct out and get the URL and the background ID because those are both sent from the page as part of that data payload. You get to define. those events and those data payloads, whatever you need for your application. Now the next thing we want to do is call out to the AI and get the review. We're going to bring in add review from AI review. Over here in our code, we've got the review file. This is just a call out to open AI. We're going to bring in the open AI client and we're going to define our system context, your webcam, background reviewer, and so on and so forth. We're going to create a chat completion and give it that systems context so that it knows what it's trying to do. And then we ask it as a user to rate the composition that we're going to give it and we give it the image URL that we just got. We're going to set streaming to true because we want to stream back those tokens. And then we're going to go through each chunk of the stream coming back and call out to the database and set the review to the string that has been we're joining together as we get new chunks out of the stream. And then we're done. We set the review to complete it. And we also return the complete review text. So now we could do is we could. just call add review like this. But what we want to do is handle retries. So we're going to do is we're going to wrap it in a step. So we're going to use the step object that we got from ingest and we're going to use the dot run function on it. We're getting the name in this case, add review, and you can do anything inside of there. And if there's an error, it'll automatically retry. Super cool. All right. Now we need to get the review. So we await that promise. Now we can return anything we want out of here. I'm going to turn the background idea review so we can see what the output was. in our dev console. All right, now the last thing we need to do is connect it to the route, right? Because our functions are currently empty. So let's bring in that function, get reaction and then add it to our list of functions. All right, let's hit save and then try it again. So go back, try another background, start the capture, take photo. So we can see the run is started and it's now doing that step. Now we get the output of that function, you can see the completed review, we can see how long it was, we can see the step, we can get all the step information. We can even go and rerun it if we want to. You can see it coming out again. How cool is that? Well, while that's all going, let's talk systems architecture. Here's where we were. We fired off GetReaction and we've sent out our OpenAI API requests. Where are those OpenAI requests happening? Well, they're happening on the Next.js app. which is great because the app has all the environment variables to talk to the database to talk to the blob the store to talk to open ai it's got all the business logic to create this workflow like getting this reaction we're using ingest to do the coordination of all of that The ingest server doesn't have any of the business logic or application. We can define everything in our existing code base. We don't need to deploy code to another platform to have things get out of sync or be a security issue, which would be a nightmare. Now you just have the one app and the code base that's deployed to a single place. But you have this workflow engine that's just telling the app this is what you need to do and this is when you need to do it. And ingest can do things like run code in parallel and fan out and add sleeps and cron jobs and all kinds of powerful job management utilities. All right, let's go back into the code and do the theme stuff as well as Dali to show you how to cascade events and how to do throttling. It is super easy. All right, so to start implementing on theme, the first thing I want to do is bring in GetTheme from AI theme. That just makes the call out to OpenAI. Now that just gives us back some text. So we're in charge then of putting that in the database to do that. We're going to bring in set theme from the DB. And then down here, we're going to use the same kind of step run. And we're going to call this step get theme. We're going to call out to get theme with that URL. We can get that theme back. And then we're going to set the theme with the background ID and return the theme. Now we could, if we want to run these in series, we could just do another await on the theme promise. But this is, again, it's going to run the first one and it's going to block on that. And it's going to then. go to the second the theme promise that's what we want we want these both run in parallel so how do we do that well there's no magic here it's just javascript in fact that's one of the things that i really like about ingest is the lack of any kind of magic what you're seeing is what you're getting there's no extra setup or anything like that it's just native javascript and in this case what you want to do for that is promise.all because that's how you can resolve two promises in parallel and get back their data so let's try that So to do that, I'll replace the code below with this promise.all, which we're going to await. I give the review promise and the theme promise, and I get back the review and the theme in the same order. Let's add that to the output, and let's give it a try. Let's try another background. Start the capture, take a photo. Now we've got a new run. We can see the get theme is already done. That's really cool. Get all the information out of that. Really nice. The output of that step, we see it right there. So you get a really great breakdown by each step. The streaming is still continuing. So this is perfect now. All right. Now, going back in the code, what I could do is I could just add the Dali request right here, but I do want to get a throttling behavior. So what I'm going to do is I'm going to send another event out and we're going to use another function, a throttled function to handle that event. So the first thing we need to do is send out event saying that we have a theme. So let's go and send out an event. Now we've already got the ingest client. So we just use that same send function and we'll call that event rate or theme updated. And what data does it need? Well, it needs a background ID as well as the theme. So we'll send that out and now we'll create a new function to handle that. So go back over here to functions and we'll create a new function called make new background. Now, just like before we use create function and this time we're going to give it the idea of make new background and we're going to bind it. to that event of Raider theme updated. Now let's go and get the data from the event. So we're going to get back to that background ID and that theme. And then again, we need to get another AI function, in this case, generate image. That's one that's going to call out to Dali. We need to set the URL of that image in our database. So we're going to bring in the set new background function and I'm going to call it generate image. I'm going to call out that generate image with that theme that we just got and then set it in the database and return that. And then finally, I'm going to turn the background ID in the new background. The last thing we need to do is connect that to our API. So import make new background and then we'll add it to the list of functions. Let's hit save and we'll try it one more time. We got a new photo of me smirking up there and it's running. We got the theme back so fast that we've now initiated the make new background and that's calling off to Dali to get that image. And there you go. We got our image. You got the background over here. We can hit rerun if we want to rerun it again. But here's the thing about Dali. You want to make sure that you don't get rate limited because it can only handle 50 requests per minute. So you want to make sure that you're below that. To make sure that we don't get rate limited on the app side, we're going to use ingest to automatically handle throttling of that function. Go check out how easy that is. To do that, I simply add on to the definition of our make new background function, a throttle parameter. say that we have a limit of one every five seconds at the highest level, a burst of two. And that's going to say that we're only going to handle that 12 of these jobs per minute. And that's well below that 50 jobs per minute rate limiting of DALI. So as you can see, ingest is a fantastic way to manage complex workflows in your application, the ability to see the whole flow, handle retries, the throttling, the cron job support. It is awesomely powerful and Thank you so much to Ingest for sponsoring this video. In the meantime, if you have any questions or comments, be sure to put that in the comment section right down below. And I'd appreciate it if you hit that like, if you like the video, and if you really like the video, hit the subscribe button and click on that bell and you'll be notified the next time a new blue collar coder comes out.

Build durable AI workflows in JavaScript or TypeScript to work with your NextJS or React applications quickly and easily using Inngest.

Code: https://github.com/jherr/background-rater

👉 ProNextJS Course: https://pronextjs.dev
👉 Don't forget to subscribe to this channel for more updates: https://bit.ly/2E7drfJ
👉 Discord server signup: https://discord.gg/ddMZFtTDa5
👉 VS Code theme and font? Night Wolf [black] and Operator Mono
👉 Terminal Theme and font? oh-my-posh with powerlevel10k_rainbow and SpaceMono NF

00:00 Introduction
00:37 The Background Rater app
01:37 The AI workflow
03:16 Getting started with the code
04:37 Why not just call AI directly?
05:06 Installing Inngest
06:02 Creating the Inngest client
06:25 Sending your first event
07:09 The NextJS/Inngest event flow
07:50 Creating the Inngest endpoint
08:34 Adding your first function
10:59 Connecting the function
11:39 Systems architecture
12:50 Adding another AI call
13:41 There is no magic here
14:48 Calling Dall-e
16:50 Adding throttling

This video is sponsored by Inngest