Build a Multi-Agent Orchestrator in Python

DEV Community

Build a multi-agent orchestrator from scratch in Python before reaching for LangGraph

Most teams do not have a LangGraph problem. They have a design problem that they first notice while they are inside LangGraph. The demo looks fine: a researcher hands work to a writer, the writer hands work to a reviewer, and a report appears. Then a tool times out, a human rejects an approval two days later, or a specialist’s scratchpad leaks into the next agent, and suddenly you are debugging control flow through someone else’s abstractions. CrewAI’s role-and-task model fails in a different way — it is a very fast afternoon, and a very slow incident. I am not arguing that you should avoid those frameworks. I use them, and I teach them. I am arguing that you should build a small orchestrator yourself first, so that LangGraph or CrewAI becomes an adapter around a design you already understand. What you are actually building A multi-agent system is not a group chat with job titles. It is a piece of software that has to do at least five things well: Run a perceive-reason-act loop against a model, without hiding the messages and tool calls. Call tools that have types, permissions, and idempotency keys. Dispatch specialists with isolated context, then merge only the results you trust. Pause for a human, resume after a crash, and refuse a replay that would double-submit. Leave a trace you can audit, and a way to shut the whole thing down. If your framework does those things, good. If you cannot point to the module that does each one, you do not own the system yet. Start with one agent and a typed tool The smallest useful core is boring, which is the point. A Message type and an Agent that takes messages and returns messages. An LLMClient you can swap. For tests, use a scripted or echo backend. For a real model, point the same client at a local OpenAI-compatible server (Ollama is enough). Tools as ordinary functions with a schema, not as free-form prompt text. The model may request create_ticket(title=..., project=...). Your runtime is what actually calls it. Two properties are worth enforcing on day one. Types. If the argument is not the shape you declared, the tool does not run. A surprising amount of “the agent went crazy” is a string where an identifier should have been. Idempotency. Every tool that has a side effect takes a key. A retry with the same key returns the original result. Without that, every eval, every crash recovery, and every “please try again” is a potential duplicate email, ticket, or payment. ReAct and Reflection are then just loops over this core: reason, maybe call a tool, observe, repeat; or produce an answer, critique it, revise. You do not need a framework to learn those patterns. You need a test that the loop terminates and that a tool error becomes an observation rather than an uncaught exception. Isolation is the reason to have more than one agent A second agent is worth it when you need isolation, not when you need more personality. Give the specialist a brief, not a transcript: the goal, the constraints, the tools it may use, and the artefacts it may read. It should not see the coordinator’s failed tool calls, and it should not see another client’s memory. When it finishes, it returns a structured result — findings, citations, open questions — which the coordinator merges on purpose. This is also how retries become possible. If the researcher failed halfway through, you resume that node from a checkpoint and you do not force the writer to re-read a pile of obsolete errors. Teams and swarms are variations on the same idea. A team has named roles and a merge policy. A swarm needs a tighter decision rule, because “let everyone talk” is how context and cost explode. In all cases the coordinator owns the state. Peer-to-peer chatter is optional and usually overrated. Put the control plane outside the model A prompt that says “ask before you refund” is not a control. The tool gateway should refuse a risky call unless it carries a valid approval, and that approval should be bound to a hash of the tool nam...

Powered by Readybot.io