The more I work on my agent harness, the more I am coming to the realisation that agent harnesses are mini operating systems.
What starts off as a simple loop of tool calls soon grows into an application that needs to wrap core OS concepts. Why, the simple act of displaying the chat transcript can itself teach you so much. As the chat transcript grows we understand that we can’t keep re-rendering it each time. Perhaps we need some sort of ‘windowing’ where the working memory only knows about the displayed text. When the number of concurrent chats you need to update grows, you get the same problem. How do we narrow the working memory for UI updates to visible text only? How do we deal with things coming into and going out of visibility in terms of their ‘caches’. When exactly do we repaint?
As you build your own harness to suit yourself, you will find yourself building features that take in more and more of your workflow. At some point you might want to build a kanban board inside your agent. Congratulations, you have now unlocked the ‘component architecture’ skill tree. At some time you might want the agent to receive events from outside and have subscribers react to those events. Congratulations, you have unlocked ‘distributed systems’ skill tree. With compos, I wanted on place to manage everything. Whether it be code, recruiting or AI playtime, I want it all inside one consistent interface, with predictable context feeding to the agents. Building out a whole UI layer in the browser with all state projected from the backend has been such a learning experience for me.
An agent harness humbles you in its infinite capacity for complexity. A simple feature that we take for granted – steering messages, for example – will take you multiple iterations before you get it right. Oh, we should be able to cancel waiting messages, and so on. And the more you focus on performance, the more of the skill tree unlocks. If there’s one rule you want to up your programming game it is this - write fast software!
All software is becoming harnesses. A UI and an agent that deeply understands that UI. You should totally build your own harness and blog about it. May a million harnesses bloom.