Harness Engineering: What Separates Top Agentic Engineers Right Now
Cole Medin · 2026-05-28 · 17м 9с · 69 307 просмотров · YouTube ↗
Топики: ai-loop-engineering
🎧 Аудио
📝 Summary
model=deepseek-v4-flash · prompt=summary-v7 · 6 103→2 190 tokens · 2026-07-20 11:47:04
🎯 Главная суть
Harness engineering — это создание надстройки (wrapper'а) вокруг языковой модели, которая определяет контекст, правила, процессы и инструменты для агента. В отличие от context engineering (управление контекстом внутри одной сессии), harness engineering включает ещё управление потоком работы между несколькими сессиями агентов, автоматизацию и системную эволюцию — превращение каждой ошибки агента в новое правило или улучшение обвязки.
Что такое harness engineering: два уровня
Любой кодинг-агент состоит из базовой LLM (GPT, Claude) и обвязки вокруг неё. Первый уровень обвязки — это сам инструмент (Claude Code, Codex, Cursor и т.д.), который даёт модели доступ к файловой системе, выполнению команд, системному промпту. Второй уровень — AI layer, который определяет пользователь: глобальные правила, MCP-серверы, навыки, хуки, код-поиск и саб-агенты. Это шесть компонентов, встроенных в современные кодинг-ассистенты.
Второй уровень — оркестрация нескольких кодинг-сессий: большая задача разбивается на мелкие, каждая обрабатывается отдельной сессией агента с передачей артефактов между ними. Это позволяет масштабировать работу и избежать перегрузки модели.
Компоненты AI layer
AI layer — это всё, что вы добавляете поверх выбранного кодинг-агента. Шесть ключевых элементов:
- Global rules — ограничения и конвенции, которым агент обязан следовать (стиль кода, запрет на определённые действия).
- Skills — предопределённые рабочие процессы для планирования, реализации, валидации (например, skill «plan» создаёт документ с планом, «implement» выполняет код, «validate» проверяет тесты).
- MCP servers — дополнительные инструменты (поиск в документации, доступ к базам данных).
- Code-based searching — LSP, графы знаний для навигации по коду.
- Hooks — триггеры до или после действий агента (предотвращение разрушительных команд, принудительный прогон тестов).
- Sub-agents — специализированные агенты внутри сессии (например, ревьюер кода).
Отличие от context engineering: контроль и системная эволюция
Harness engineering — эволюция context engineering. Основное различие — контроль: оркестрация сессий, RALF-циклы, автоматические цепочки агентов — это новое. Второе — skill issue reframe: типичная реакция инженера на ошибку агента — «подождём следующую версию модели». Harness engineering учит вместо этого добавлять правило, хук или навык, чтобы та же ошибка не повторилась. Каждый сбой — повод улучшить обвязку. Это системная эволюция: вы берёте ответственность за поведение агента и улучшаете его с каждым циклом.
Практика: правила, навыки и хуки
Правила задают конвенции (имена переменных, структура папок, запрет на чтение .env). Навыки лучше разделять: план, реализация, валидация — каждая в отдельной сессии, чтобы сессии оставались токено-эффективными. Хуки особенно полезны для безопасности: pre-tool use hook проверяет, что агент не выполнит опасную команду (например, rm -rf). Stop validation hook прогоняет юнит-тесты, линтинг и type checking после того, как агент заявил о завершении, и заставляет его чинить ошибки до зелёного статуса. Также полезен хук на быстрый линтинг после каждого редактирования файла — кодовая база остаётся чистой, что улучшает работу будущих агентов.
Оркестрация кодинг-агентов: RALF loop
Пик эволюции harness engineering — автоматизация цепочек сессий. Пример — RALF loop (создан Jeffrey Huntley). Скрипт (Python или bash) получает крупное описание работы (PRD), разбивает его на отдельные задачи и запускает для каждой новую сессию кодинг-агента (Claude Code, Codex и др.). Сессии выполняются последовательно, каждая получает сфокусированное подзадание. В конце каждого цикла агент решает, завершена ли вся работа, и при уверенности создаёт файл done.txt — условие выхода. Также можно запускать несколько ревью-агентов параллельно: один проверяет безопасность, другой — корректность, третий — простоту кода. Всё это автоматизировано: не нужно вручную открывать новые терминалы и передавать артефакты.
RALF loop и PIVLU: базовый подход к агентной разработке
Для небольших задач достаточно PIVLU — ручного разделения на Plan, Implement, Validate. План создаётся в одной сессии (skill «plan»), результат — markdown-документ. Этот документ подаётся в сессию с skill «implement», где агент пишет код. Затем выполняется валидация (например, прогон тестов). Каждая стадия — отдельная сессия, что сохраняет фокус и токены. RALF loop автоматизирует этот же процесс для масштабных задач, последовательно итерации до полного выполнения всех пунктов спецификации.
📜 Transcript
en · 3 540 слов · 43 сегментов · clean
Показать текст транскрипта
A term that's popping up more and more in the AI space right now is harness engineering. It's the next big thing for this year, just like context engineering was for last year. And it is really important, but just like context engineering, it's starting to turn into this buzzword that people are throwing around without really knowing what it means. And so that begs the question, is this skill or even mindset, like I'll get into in a little bit, worth learning or adopting? And the answer is yes. And so I want to get into that. with you today helping you understand what harness engineering is there are a couple of layers to it that are really worth knowing and so i'm going to break this down nice and simple for you in less than 15 minutes and of course like usual i've got some examples and demos to really make it concrete all right so let's get right into it harness engineering is all about building the wrapper around the model so any agent is the combination of the underlying large language model like gpt or clod and then the wrapper around it that gives it the context and defines your processes. And so I'm mostly going to be focusing on AI coding assistance for this video, but really this idea of harness engineering can be extrapolated out to any agent that you build for anything. And there are really two parts of harness engineering. We have within an individual AI coding assistant session, and then we have really the real evolution here that I'm more focused on right now. This is combining multiple coding agent sessions in a larger workflow to handle a larger task. And so we'll get there, but I want to start more foundational here. And one really important thing to understand is that this first idea of harness engineering within a single session, a lot of the ideas here are very similar to context engineering. This is a direct evolution of context engineering. How do we give the right ecosystem of context to our coding agent? But there are some important differences here that I want to key in on. But first of all, I think this diagram explains it really well. We start with the underlying large language model. This is the reasoning for our agent. And then the first wrapper around it is not something you build yourself. It's actually the tool that you use, the coding agent that you choose. And so Cloud Code, Codex, Pi, you name the millions of coding agents out there, all of them are actually harnesses that a company has engineered around their model. And so this might not feel like harness engineering because you're not defining anything, but you're picking the harness when you choose the tool. Some people think Cloud Code is the best harness for coding. Some people think Codex is. There's a lot of debate right now. But what's even more important than the coding agent you pick is the AI layer. This is the ultimate wrapper. around any coding agent session and this is what you get to actually build. And so when we think about what goes into the AI layer, it's really defining all of our context and processes for our coding agents. So our global rules, our skills and MCP servers, all the capabilities we give, code-based searching like LSP or knowledge graphs, our hooks and our sub-agents. Really like these six components that are pretty much built into every single AI coding assistant now make up your AI layer. So no matter how you want to inject your process or your rules, you're going to do it through one of these six things. So there are a couple of articles I really want to lean into here to help you understand harness engineering. I'll link to them in the description. This first one has an analogy that I want to zoom in on here. I love this. So on the left-hand side, we have a representation of what the model can do by itself, like Claude or GPT. And spoiler, it's not that much. We take for granted all of the capabilities that AI coding assistants like Cloud Code and Codex give to the model out of the box. And LLM by itself, it doesn't have any way to access a file system or get or run any commands. That's everything that comes with that first harness layer built into the tools that we download and use out of the box. And so that's what these top bridges represent here. It's all of the capabilities that these tools give to the model to make it so it can really act as an AI coding assistant, right? It's the capabilities plus the system prompt built into these tools. And then as we get to the lower bridges here, this is where we start to get into that higher level AI layer where we get to define things like the MCP servers we use, the skills that we build or incorporate, rules, things like that. Even going down to RALF loops, like we'll talk about towards the end of this video when we get into stringing multiple coding agent sessions together, the ultimate kind of harness engineering. So stay tuned for that. But the point here is that each of these bridges are tools that allow the large language model to function, and act as an AI coding assistant. All right, cool. So with that definition, I now want to address the elephant in the room. The question you might be asking yourself is, Cole, isn't a lot of this here just context engineering? Like, I thought we were covering this in 2025. And the answer is actually yes to an extent. And that's why I think that harness engineering is becoming such a buzzword right now. Most people don't really understand how this is truly an evolution from context engineering. That's what I want to argue with you. right now. And so there are two important distinctions. So first of all, Most of the harness around the model, like this article outlines, it is just context engineering. Your context injection, your actions through tools and MCPs, persistence, observability. The one thing that really is different is control. Like Ralph Loops, orchestrating different coding agent sessions and sub-agents, I think that is a true evolution from sub-agents. And so we'll talk about that next. But the other really important distinction that this article outlines... is the skill issue reframe. So I alluded to at the start of the video the fact that harness engineering is not just a skill, it's also sort of a mindset, right? A reframe. So the author here says, there's a pattern I watch engineers fall into. The agent does something dumb, the engineer blames the model, and the blame gets filed under wait for the next version. As in, you know, Claude screws up here, well, he better wait for Opus 5, or GPT messes up, let's wait for GPT 6. And, you know, personally, I see this all the time as well. I'm also tempted to think this myself, and you probably are as well. But the harness engineering mindset rejects that default. And by the way, I call this system evolution. It's very in line with something that I've been focusing on a lot recently. So here's what he says. The failure is usually legible. The agent didn't know about a convention, so you add it to agents.md. Or the agent ran a destructive command, so you add a hook that blocks it. Basically, the idea here is every mistake becomes a rule, or the way I like to put it is every mistake becomes an opportunity to improve your harness, improving the security through your hooks, your processes through updating your skills, anything in your harness so that the next coding agent session, that issue you encountered, is less likely to come up. And that is super powerful. That system evolution means that you are taking ownership and improving the performance of your coding agent over time with the AI layer that you control over the coding assistant that you chose. And so really, harness engineering is all about claiming that agency, taking ownership of your system so that when something goes wrong, you're not just blaming your AI coding assistant and feeling helpless. Because things will come up. Just like working with human developers, there are going to be issues. But we need to make sure that we have a way to learn from that and not just be at the mercies of the next session, not encountering that problem again. We want to be the human steering. system, feeding forward. So the initial generation, we have our principles and other kinds of contexts we feed in, and then we have our sensors for feedback, our hooks, our review agents, the skills that we give it for that self-correction, evolving our AI layer over time. The sponsor of today's video is Google Cloud, specifically their new agent CLI. And I'm excited for this because nowadays it's optimal to build your agents with other agents, right? Using your AI coding assistants like Cloud Code and Codex. Now, the easy part is getting the idea for an agent, but actually building it out and deploying it to production, that is a different beast. But Google has made this so incredibly easy now with their agent CLI. It's a collection of skills that I can bring into my code. coding agent that give it full, clear instructions on how to build agents with the Google Agent SDK and even deploy them to production and monitor them. And so right here in my cloud code, for example, I can say use the agent CLI to build a research agent that searches the web. Obviously a simple example, but it's going to use the instructions to really help you build any agent that you want. Then with the help of the skills, your coding agent will create all of the code. It's one-shotted a lot of different tests that I've given it here. And then we also have our local development environment. We can spin up the agent here so that we can test everything locally with a full chat application before we deploy our agent. And then when you're ready to take your agent to production, it is a single command to deploy your Google agent SDK agent to the Google Cloud. Super easy. And your agent gets its own identity in the cloud. You have the playground here to test it live and you have trace... full observability. So everything you need for a production deployment, but it's not extremely difficult to get all this set up like it used to be. And the best part is the Asian CLI is free and open source. You can take these skills, bring it into any coding agent and see how easy it is right now to build any AI agent. I'll have a link in the description. I'd highly recommend checking it out. So I also have this companion repo for our video, giving you a super concrete idea of what an AI layer can look like. And this is a really good representation, everything here of the AI layer that I'll bring into and evolve in each of my code bases. I want to cover this quick before we get into the last evolution of harness engineering, the really powerful stuff, building workflows where we are bringing together and orchestrating many coding agent sessions. And I have examples for that, like with the Ralph loop in this repo as well. And so I did promise that this video is going to be shorter, so I'm not going to dive extremely deep into each one of the components of the AI layer, but I do have a video that I'll link to right here, where I cover in more detail the rules and skills and LSP and hooks, each one of the components. I want to stay just really high level, give you some golden nuggets here, and then you can, of course, just give this repo to your coding agent and have it help you implement things and understand everything here. So really, the foundation of your AI layer is all of the rules, the constraints and conventions. that you want your coding agent to follow, your patterns. And so that's your global rules and any other kinds of on-demand context that you have as markdown, confluence documents, that kind of thing. And then your skills, these are the workflows that you have for your coding agent. Like here's how you want it to plan, implement, and validate. And I have a separate skill for each because what I really want to do, and I highly, highly recommend this, is you want to do your planning, implementation, and validation all in separate coding agent sessions to keep each one of them token efficient and focused. And so each one of these skills is going to output some kind of artifacts. So you can use it as a handoff to the next session. And so this is kind of getting into stringing coding agent sessions together, but we're still talking about doing this manually, right? Like you'll run the plan and you'll create the plan with the coding agent with one skill, and then you'll take this markdown and then you'll give it to the next coding agent session for the implement skill, right? You go through that systematically yourself. And so we'll talk about in a little bit how we can really bring all that together. And then as far as hooks go, these are honestly pretty underused. I love using hooks for a few different things. First of all, for security. So a pre-tool use hook. Basically, this is a piece of code that's going to trigger before the coding agent executes any tool call, like writing out to a file, running any kind of command. And so this is where we can build in security things, like not reading .env files, because we really don't want that in the LLM's context, or removing directories in a very destructive way, for example. I also like... having some kind of stop validation hook. So when the coding agent says it's done with the implementation, I want to deterministically run a set of tests. Like are the unit tests, the linting, the type checking, is all that really passing? Because if it's not, I want to force the coding agent to iterate on it until it is. And that's what this hook does. And then last but not least, just running a quick lint after every single file edit is really good just to keep your code base nice and clean, which also helps make your coding agents more reliable in the future. So there you go, just a couple of golden nuggets for the AI layer that we have here. And I even have instructions that read me for just running a super basic PIVLU. This is the foundational approach to agentic engineering. So you plan with the plan skill, just sending in the feature that you want. You iterate on the plan, you produce that markdown document that you then hand off to the implement skill in a separate coding agent session. And you also have your validation strategy for the agent to check its own work in the markdown as well. So feel free to try that out for yourself. But now finally... I wanna get to the peak evolution of harness engineering. So I'm gonna jump back to the diagram here. Let's talk about orchestrating coding agent sessions. This is when we can really scale the amount of work that we get done with coding agents. So the main idea here is you don't just want to take a massive task or PRD and hand it to a single coding agent session. It's not going to be token efficient and the underlying large language model in the harness is going to be completely overwhelmed. It does not matter how good the AI layer is that you created here. With things like your skills and rules, if you send much into the llm at once it is going to fall flat on its face and so what we're doing here orchestrating many coding agent sessions together is we're giving each coding agent a very focused task and so these can be sub-agents as well but like you'll see in the ralph loop it's actual coding agent sessions that are handing things off to each other so we can explore the implementation that comes in from a user requirement, have one agent that writes the plan, send that plan artifact into an implementation, and then for example, this is just an example harness, but we could have many code review agents running in parallel. This one focuses on security, this one correctness of the implementation, and this one making sure it's as simple as it can be. And then if everything passes, you create the pull request, otherwise you would iterate and keep working on the implementation. And you can do all this manually, like I was talking about earlier. We can go to Claude Code once and create the plan, and then open up another Claude Code and do the implementation. But the real power with Harness Engineering here is we can automate all of this. We can create a system that automatically hooks together all these coding agent sessions with the handoff. documents and creating the pull request and everything like that. And that's what the Ralph loop does. So let's go back to our example repo here. I want to show you what this actually looks like. So the Ralph loop is just one example of an agent harness, but Jeffrey Huntley, the creator of Ralph, he really is one of the pioneers here. This is one of the first examples showing in a very basic sense how we can automate stringing together many instances of cloud code, codex. I mean, you could do this with any coding assistant because basically all it is, I'm not going to get too tech. here, but I just want to show you a little bit, is we have a simple script. This can be a Python script, it can be a bash script. I'm not going to go into the code here, but essentially you give it a larger scope of work, like a massive PRD, and it's going to be responsible for splitting that up into individual tasks and then running coding agent sessions to handle them one at a time until everything is done. And so you give it a prompt. This is the user input here. These are the different items that I want to build in the spec. And then it's going to produce a plan here. This is the fixed plan. So this is what it's going to do in iteration one of the loop, iteration two, iteration three. It's going to keep working, kind of like build up a log as the different cloud code sessions here are running. And then when it decides it's done, it's going to produce some kind of indicator of that. Like here I'm using a done.text. And so this is where it decides like, all right, all eight spec items from the original prompt. are done. We are satisfied, we can now exit the loop. And that's the only condition. The only way that we can exit the main while loop here is if we have this done.txt and the coding agent is confident that the implementation is done and all the validation is there. And so just trying to show you a RELF loop to give you one example of a harness. But you can see the idea here of we are using many coding agent sessions to keep each one very, very focused. But also we're automating it so we don't have to babysit our coding agent as we're... handling these longer tasks. And this really is the future of agentic engineering, building these harnesses to handle larger scopes of work as the models and the underlying tools are getting more powerful. Like this is the way. And so lean into this here. I mean, there's so many resources out there for harness engineering. And then there's Archon, my open source harness builder, free to use. This is the easiest way to get started with agentic engineering, building your own harnesses like the Ralph Loot, but more custom to you, your exact process. and software development lifecycle. So I'd highly recommend checking this out. Otherwise, I hope this video was helpful for you in general, just understanding what is harness engineering. What's the fluff? What's really worth knowing? If you found this useful in any way, I would really appreciate a like and a subscribe. And with that, I will see you in the next video.
⚙️ Pipeline jobs
| Stage | Status | Att. | Updated | Error |
|---|---|---|---|---|
| download | done | 1/3 | 2026-07-20 11:46:25 | |
| transcribe | done | 1/3 | 2026-07-20 11:46:39 | |
| summarize | done | 1/3 | 2026-07-20 11:47:04 | |
| embed | done | 1/3 | 2026-07-20 11:47:06 |
📄 Описание YouTube
Показать
Everyone says "harness engineering" is the most important skill in AI coding for 2026, but almost nobody can tell you what it actually is. A coding agent is the model plus the harness around it, and the harness is the part that's more important! One teardown of Claude Code found roughly 98% of it is the harness, not the model. It has two levels. The first is the AI layer around a single session: your rules, skills, hooks, subagents, etc. And here it's all about using every LLM mistake as an opportunity to evolve your AI layer (your harness). The second, where the real leverage is, is orchestrating multiple coding agent sessions into one workflow, each with a focused job. This makes it possible to reliably take on larger tasks and is how you really scale coding agents. ~~~~~~~~~~~~~~~~~~~~~~~~~~ - Check out the new Google Cloud Agents CLI! I've been using their new Agents CLI to scaffold and deploy production agents straight from my terminal with the model-agnostic ADK, no cloud console required: https://fandf.co/3PsygXP ~~~~~~~~~~~~~~~~~~~~~~~~~~ - The Dynamous Agentic Coding Course is now FULLY released - learn how to build reliable and repeatable systems for AI coding: https://dynamous.ai/agentic-coding-course - Companion repo (the harness, built): https://github.com/coleam00/harness-engineering-demo - Harness Engineering article 1: https://addyosmani.com/blog/agent-harness-engineering/ - Harness Engineering article 2: https://martinfowler.com/articles/harness-engineering.html ~~~~~~~~~~~~~~~~~~~~~~~~~~ 0:00 What is Harness Engineering? 0:47 Defining the Agent Wrapper 1:55 The Three Layers of AI 4:39 Harness vs. Context Engineering 5:46 The Harness Engineering Mindset 7:45 Sponsor: Google Cloud Agents CLI 9:25 Components of the AI Layer 12:52 Orchestrating Coding Agent Sessions 14:34 Automating with the Ralph Loop 16:34 Final Thoughts ~~~~~~~~~~~~~~~~~~~~~~~~~~ Join me as I push the limits of what is possible with AI. I'll be uploading videos weekly - at least every Wednesday at 7:00 PM CDT!