← все видео

The Inbox Pattern in .NET (The Outbox's Missing Half)

Milan Jovanović · 2026-06-09 · 13м 3с · 7 745 просмотров · YouTube ↗

Топики: durable-execution

🎧 Аудио

📝 Summary

model=deepseek-v4-flash · prompt=summary-v7 · 4 795→2 002 tokens · 2026-07-20 15:09:32

🎯 Главная суть

Inbox-паттерн решает проблему повторной обработки сообщений на стороне потребителя (consumer) в распределённых системах. Он действует как буферная таблица в БД: полученные сообщения сохраняются с уникальным идентификатором, а дубликаты игнорируются. В паре с Outbox-паттерном (на стороне издателя) это даёт гарантию exactly-once side effects, компенсируя at-least-once доставку, которую предоставляют большинство брокеров.


Проблема повторной доставки сообщений

Первое правило распределённых систем — ничто не гарантировано. Обмен между брокером и потребителем может ломаться множеством способов: сетевые сбои, повторная отправка после успешной обработки (если подтверждение не дошло до брокера), двойная обработка одного события. Брокеры обычно работают в режиме at-least-once delivery — одно сообщение может быть доставлено более одного раза. Inbox-паттерн делает потребителя идемпотентным: дубликаты сообщений не приводят к повторному выполнению бизнес-логики.


Как работает Inbox: буферная таблица

Потребитель получает сообщение из брокера, открывает сессию с БД и сохраняет его во входящей таблице (Inbox table). Ключевой момент — уникальное ограничение (primary key или уникальный индекс) на идентификатор сообщения. Если при вставке встречается дубликат, запись игнорируется (ON CONFLICT DO NOTHING). Это позволяет не проверять существование сообщения заранее — достаточно одной операции вставки. Бизнес-логика при этом не выполнятся на этапе получения; сообщение лишь сохраняется. Дальше фоновый процессор забирает необработанные записи и передаёт их соответствующим обработчикам. Такой двухфазный подход снижает риск оставить систему в неконсистентном состоянии (когда вставка в Inbox прошла, а бизнес-логика упала, или наоборот — бизнес-логика выполнилась, но транзакция не зафиксировалась).


Отличие Inbox от Outbox

Outbox-паттерн работает на стороне издателя: исходящие сообщения сначала пишутся в таблицу Outbox в той же БД, где выполняются бизнес-операции, а затем фоновый процесс публикует их в брокер. Таким образом гарантируется, что событие будет отправлено хотя бы один раз. Inbox — зеркальное отражение: входящие сообщения буферизируются в БД, обеспечивая обработку тоже хотя бы один раз (но с защитой от дубликатов). Вместе они образуют сквозную гарантию надёжности для всей цепочки. В демонстрации автор использует приложение, где уже реализован Outbox (с таблицей OutboxMessages, 2 млн тестовых сообщений), и добавляет к нему Inbox.


Реализация Inbox потребителя на C# с MassTransit

Для интеграции с брокером используется MassTransit (open source версия). Создаётся класс InboxConsumer, который реализует IConsumer<T> — в примере OrderCreatedIntegrationEvent. В методе Consume через NpgsqlDataSource открывается соединение с Postgres. SQL-запрос: INSERT INTO inbox_messages (id, type, content, received_on) VALUES (@Id, @Type, @Content::jsonb, @ReceivedOn) ON CONFLICT (id) DO NOTHING. Параметры: Id — идентификатор сообщения из ConsumeContext.MessageId (или Guid.NewGuid()), Type — полное имя типа сообщения, Content — JSON-сериализованное тело сообщения, ReceivedOnDateTime.UtcNow. При конфликте дубликата запрос просто ничего не делает. Для удобства рекомендуется базовый абстрактный класс IntegrationEvent с полем MessageId, чтобы использовать один и тот же ID на стороне Outbox и Inbox.


Обработка дубликатов через ON CONFLICT

Вместо явной проверки существования записи (SELECT перед INSERT) или перехвата исключения уникальности, автор использует конструкцию ON CONFLICT (id) DO NOTHING. Это один запрос без лишних обращений к БД. Если сообщение уже есть в таблице — вставка игнорируется. Такой подход прост и эффективен, особенно при высокой нагрузке, где дубликаты могут приходить одновременно.


Фоновый процессор для обработки сообщений из Inbox

После того как сообщение сохранено в Inbox, оно не обрабатывается сразу. Отдельный фоновый сервис (BackgroundService) периодически (например, каждые 5 секунд) вызывает InboxProcessor, который выбирает необработанные сообщения из таблицы пачками, десериализует их и передаёт в соответствующий обработчик (в демо — просто лог). После успешной обработки запись помечается как обработанная (например, устанавливается флаг или очищается поле ошибок). Это позволяет обрабатывать сообщения даже в случае сбоя после вставки — повторный запуск процессора подхватит оставшиеся записи.


Использование готовых библиотек вместо hand-made реализации

Поддерживать собственные реализации Outbox и Inbox нетривиально: нужно аккуратно обрабатывать граничные случаи, транзакции, блокировки, тайм-ауты. Большинство популярных библиотек для .NET (MassTransit, Wolverine) предоставляют встроенную поддержку обоих паттернов. Достаточно нескольких строк конфигурации, чтобы получить готовую инфраструктуру с буферизацией сообщений, защитой от дубликатов и гарантией доставки. В видео упоминается, что MassTransit уже поддерживает Outbox и Inbox, и на канале есть отдельное видео о настройке через Wolverine.

📜 Transcript

en · 2 484 слов · 35 сегментов · clean

Показать текст транскрипта
If you want to improve the reliability of your distributed messaging system, you're probably going to introduce an outbox on the publisher side. But what about the consumer side? Well, this is where the inbox pattern comes in, and I'm going to show you how you can implement it in this video, as well as explain why you would want to use it and which problems it solves. So the first rule of distributed systems is that nothing is guaranteed and if you've worked long enough with distributed systems you probably know the fallacies of distributed computing by heart. Now why am I telling you this? Well this communication here between our message broker and our consumer is a small component of a larger distributed system but nonetheless we've got an external component the broker talking to our consumer running inside of our application and there are multiple ways this can fail for example we could run into network issues where the consumer ends up processing the same message twice maybe the consumer completes successfully but it doesn't succeed in notifying the broker about this so it may attempt a re-delivery so the inbox pattern tries to solve this problem by introducing an intermediate buffer table inside of a SQL database, for example, and we call this the inbox. It's basically the exact opposite of an outbox and what you do is you open up a database transaction which gives you atomic guarantees and after this you have a couple of options. You can process the message by taking the message id and the payload and storing them inside of our inbox table. If we run into a duplicate message id which means we got a re-delivery from the broker then we can just ignore this because we've got a unique constraint on our inbox table and if we do manage to store this, we can just commit the transaction. Now when you think about it, this is just one database insert, which is atomic on its own, so there's even no need to open up an explicit transaction. Now where our transaction comes into play is if you want to store the message in the inbox table, which is this type here, and also execute the business logic associated with a specific consumer. Now this introduces some problems, for example, what if the insert into the inbox succeeds, and our business logic. fails, then we would have to revert the whole thing. And it's even more problematic if the insert initially succeeds, we execute our business logic, which also produces some side effect, but committing our transaction fails. So there's also a possibility of ending up in an inconsistent state, which is why I like to treat my inbox as just storing the payload within the consumer. And then I've got a background processor that's going to look for any pending messages in the inbox and go ahead and process them one by one. by handing them off to the respective message handler. Also, when you think about it, most message brokers come with at least once delivery semantics, which means there's always the possibility that you may get the same message delivered more than once and the inbox pattern solved this problem by making sure that your consumers execute exactly once although in practice we are more concerned with exactly once side effects but either way the inbox is going to buffer your incoming messages so it also reduces the chance of losing any messages so that's the high level introduction but how do we actually implement this so i've got an application here that's already implementing the outbox pattern which means that when we want to publish some message we're going to store it inside of our database table and then we've got an outbox processor which is going to handle any unprocessed outbox messages and hand them off to a queue. For the messaging aspect I'm using mass transit the free and open source version to be specific and for my infrastructure I'm using Postgres as my database and then RabbitMQ as the message broker. So why am I starting from an outbox? Well there are two reasons. The first one is I've already got an example of the outbox and now I don't have to implement everything from scratch and it's also to serve as a nice parallel when we want to compare the inbox and the outbox pattern. So let me create a folder called the inbox and then the first thing we would want is a parallel to the outbox message. So I can just copy my outbox message into the inbox folder. I'm going to change the name, make sure the type name matches and I'll also update my namespace. Now instead of occurred on I want to call this colon receive on UTC. So this is going to represent what we're going to store inside of our inbox table. And that's the next thing I want to talk about. Inside of this database initializer, I've got some logic that's going to seed my database with around 2 million outbox messages that we're going to process in this demo. But it's also going to initialize my database tables. And in this case, I've got an outbox messages table and a useful filtered index is going to improve the performance. my queries. So I want to effectively have the same structure for my inbox tables and it could look something like this. I'm going to create a database table called inbox messages. It's going to have an ID, the type of the message that we are consuming, the contents in binary JSON format, when it was received or processed, and any potential errors that happened during processing. I also created a filtered index to help us with finding any unprocessed inbox messages. And this is going to be a one-for-one match of the contents of my inbox message type. Then the next thing I want to do, is to also truncate this table just to make this demo more reliable. And instead of the outbox here, I'll specify my inbox. The rest of the code here is going to populate my outbox table. So when the processor starts, it's going to start publishing these messages to the queue. This happens in the outbox processor class. And this is the specific piece of code is going to publish this to RabbitMQ. So now we want to implement our inbox consumer. and we're going to rely on mass transit for this. Let me create a type called inbox consumer. I want to use a file scope namespace. Let's make this internal and sealed and we'll implement the iConsumer interface. Now what is the message type that we are consuming? Well in this specific demo I'm publishing an auto-created integration event. So this is what mass transit is going to route to my consumer. So that's what I will also implement in my iConsumer interface. Now mass transit is interesting because it can make your consumers generic which means you could create an inbox consumer of t, make your iConsumer also generic and you probably need a generic constraint that t is is a class. And this may make sense because the logic for every consumer, at least in this implementation, is going to be to just take the message coming from the broker and store it in the inbox table. Just for the record, to show you how you could implement this, you can say add consumer, specify your inbox consumer and then you need to define the concrete type so in this case the order created integration event and mass transit is actually able to pick this up successfully and this is going to work at runtime now i do want to comment this as i want to fall back to the direct version where we're going to handle the order created integration event so let me update my consume method and delete my generic parameter here and to register this with mass transit we'll say add consumer and just inbox consumer so what do i want to do here well i'm going to inject my data source from mpg sql and i'm going to use it to open up a database connection so let's make this consume method async i'll say await using var connection and we're going to get this using our data source and i'll call the open connection async method we can also pass in the cancellation token from our consume context and then we want to persist this in the database and let me define my sql query that's going to represent this command so i want to do an insert into my inbox messages table and i'll be inserting the id the type the content and when this message was received then we're to specify the values and I'm going to add these as parameters. So I'm going to add an add sign in front of them and because I'm passing these through using Dapper, I'll just make sure that these are using Pascal case and because our content is binary JSON, we can also tell our query that to prevent any type mismatch issues. Now what happens when we encounter a violation of the unique constraint? We're using our message ID as the uniqueness constraint, which is why we made it our primary key. We have an elegant way to solve this in SQL where I can say onConflict and I can specify which columns I want to use for this detection. So I'll just use the ID column and I can instruct the database to do nothing and simply ignore this failure. So whenever we receive the same message multiple times, we're going to attempt to insert it into the database, essentially sending just one database query, but on any duplicate, we're going to fail silently. And when you think about it, this may make sense. Otherwise, you can explicitly check if this message exists or wait for the insert to blow up and then catch and handle the uniqueness violation exception finally i'll say connection execute async we're going to pass in our sql and we need to pass in our parameters for the id i'll use the message id coming from the consume context or if this is null we can create a version 7 message another thing i like to do is to have some sort of base class for my integration events. So let's say I've got a public abstract record, let's call it integration event. and on this i'm going to have a message id property so when i inherit from this i need to pass in the integration id value and by default we can pass in a random good or even a version 7 good now remember that we are serializing this to json so we don't really have to do anything special to get this working with our outbox and In the inbox, this lets us fall back to the actual message ID on our integration event. So now I can use this for my message ID and you probably want to use the same message ID in both your inbox and your outbox tables. Then for the message type, I'm going to say type of or the created integration event, and then full name. Of course, we could also get this from our message itself. So I can say context message get type, and then access the full name. Then for the content, we want to say JSON serializer, serialize, and then pass in the message. And then for the receive down parameter, I'm going to use UTC now, and let me just update my query to make sure this works. So let's go ahead and actually test this out. I'm going to update my Outbox processor to generate, for example, one message per batch that's going to be more than enough and I will update my background service to just non-run continuously and it's going to execute every five seconds this will just make it more predictable to test out our consumer. So I'm just going to start the application, let this scaffolding execute behind the scenes, and we're going to hit this breakpoint inside of our consumer in a couple of moments. Of course, this assumes that you've got a Polkris and RabbitMQ instance running locally, which I do, and I've got both of them running in Docker containers. So my application is going to connect to these. And if we take a look at our database table, you can see that we have an Outbox Messages table with a bunch of data inside, but we've also got our Inbox Messages table. which is currently empty. And as I was explaining this, we're going to hit a breakpoint behind the scenes and execute our SQL query to insert our incoming integration event into the inbox table. So I'm going to press continue and then jump into the database. So now if I go ahead and refresh my inbox messages table, you will see that I've got a couple of messages here and right now they are not processed. However, they contain the actual message in the content column, which we're going to deserialize. in our processor and hand off to the respective message handler. And just for completeness, I'm going to drop in the missing pieces, which are going to be the Inbox background service. So this is going to run continuously behind the scenes and is going to kick off our Inbox processor, which is very similar to the Outbox processor. It's going to select any unprocessed messages. in batches from the inbox table, hand them off to a consumer, which I'm mocking here with just a log statement, and then update the database to notify it that this batch of inbox messages was processed. So this is how the inbox and the outbox work together. Now if you want to see how you can make the consumer itself idempotent, I'm going to leave a useful article in the description of this video where I explain the problems that you have to solve, including a bunch of edge cases that aren't so obvious at first look. And if you think this is a lot of code to write, I do agree. It's not trivial to maintain your own Outbox and Inbox implementations. So here's a video where I explain how you can get all of this with a couple of lines of code if you're using Wolverine. Mass Transit also supports an Inbox and an Outbox, as do most popular messaging libraries. If you enjoyed this video, consider gently tapping the like button to let me know. Thanks a lot for watching and until next time, stay awesome.

⚙️ Pipeline jobs

StageStatusAtt.UpdatedError
download done 1/3 2026-07-20 15:08:45
transcribe done 1/3 2026-07-20 15:09:10
summarize done 1/3 2026-07-20 15:09:32
embed done 1/3 2026-07-20 15:09:33

📄 Описание YouTube

Показать
Want to master Clean Architecture? Go here: https://dub.sh/clean-architecture
Want to master Modular Monoliths? Go here: https://dub.sh/modular-monolith

Join the .NET Architects Club: https://www.skool.com/mj-tech-community-5418/about
Get the 2026 .NET Developer roadmap here → https://the-dotnet-weekly.ck.page/2026-roadmap

The first rule of distributed systems: nothing is guaranteed. Most message brokers give you at-least-once delivery, which means your consumers will eventually receive the same message twice — through network failures, missed acks, or redeliveries. If your handler isn't ready for that, you'll process duplicates and corrupt your side effects.

In this video, I implement the Inbox Pattern — the consumer-side counterpart to the Outbox Pattern — to make message processing reliable and duplicate-proof.

We cover:

- Why duplicate delivery happens and which problems the inbox solves
- How the inbox buffers incoming messages and guarantees exactly-once side effects
- Why I keep business logic OUT of the inbox transaction (and the inconsistent states you avoid)
- Using a unique constraint + ON CONFLICT DO NOTHING to silently ignore redeliveries
- Building the InboxConsumer with MassTransit
- A background processor that drains the inbox and dispatches to handlers
- How the inbox and outbox work together

Tech stack: .NET, MassTransit (free/OSS), PostgreSQL, RabbitMQ, Dapper.

This builds directly on the Outbox Pattern, so we get a clean side-by-side comparison of the two.

Check out my courses:
https://www.milanjovanovic.tech/courses

Read my Blog here:
https://www.milanjovanovic.tech/blog

Join my weekly .NET newsletter:
https://www.milanjovanovic.tech

Chapters