What Running Six AI Agents at Once Actually Looks Like Inside a Working Organization.
Most posts about AI and organizational leadership skip the part that would actually be useful. They show you the finished dish. Very few, if any, show you the sausage getting made — the dead terminal nobody noticed had died, the two agents that edited the same file at the same moment, the tool that told you your citation was fabricated when it wasn’t. I’ve spent the last several days running a small fleet of AI agents in parallel across two active projects, and I want to show you the actual machine before I tell you what I think it means.
I’m running a multi-agent-human collaboration to do real work and understand what such an organization might entail. The mess isn’t a side effect of running this. It’s the point. It’s also where the real lessons about agent-human collaboration are.
Why This Should Matter to You.
For anyone who will be asked to prove an AI initiative worked
If you work in organizational development, leadership, training, or eLearning, you are going to be handed an AI initiative and asked to certify that it worked. Possibly this year. Quite possibly by someone who has already announced the result.
Here is the problem waiting for you. The instrument most of us reach for first counts seats, logins, activity, and completions — and every one of those numbers has just become nearly free to produce in volume. You will be able to report something impressive. It will not be false, exactly. It will simply not be evidence of anything.
What follows is not a framework. It is data from the floor of a working operation: what actually constrained output, what actually cost money, and what actually broke. The short version, before you read another word — the constraint was never how much the agents could produce. It was how much a human could responsibly verify. Almost nobody is counting that number. If you own evaluation where you work, it is about to become your number whether or not you chose it.
I’d also ask you to notice the form this piece takes, because the form is part of the argument. Every failure below is named, dated, and attributed — including the ones where I was the one who got it wrong. Transparency is not a virtue I’m performing here. In this kind of work it is a control. It is how you find out that something died, that two agents built the same thing twice, or that a confident report was describing a world that didn’t exist.
What’s Actually Running.
The honest inventory, not the sanitized one
A shared Linux server and, at present, five live terminal sessions, each one a different agent identity running a different piece of the work — one revising a book manuscript, the others running parallel content streams (roles with real job descriptions) on a separate project. A laptop session acting as coordinator, checking in on all of them. That’s six working identities, plus an Assistant Coordinator with meaningful privileges serving as backup to the coordinator. Multiple browser tabs open at once, each a different chat with a different agent, each supposedly doing something different, none of them visually distinguishable from the others until I made them label themselves.
One of those terminal sessions died, just stopped working. Not dramatically — no crash message, no alert. Its process just stopped running at some point, and nobody knew until I went looking, days later, because the session’s own name gave no indication that anything was wrong. That’s the first thing nobody tells you about running multiple agents: they don’t fail loudly. They fail quietly, and you find out by checking, not by being told. Active human involvement is required, as these agents, properly roled, are not autonomous.
Here’s a smaller but sharper problem: every one of these agents, when it commits its work to the project’s shared record, does so under the exact same identity. Not because they’re the same agent — because they’re all running inside one shared checkout, with one configuration, so the “author” field on every single commit reads the same name, no matter which agent, on which terminal, actually did the work. If you want to know who did what, the built-in bookkeeping cannot tell you. We ended up inventing our own labeling convention — a name prefix on every commit message, a plain statement of identity at the start of every report — because the infrastructure gives you nothing for free.
Even asking an agent “which one are you?” turned out to be less reliable than it should be. One session, trying to verify its own identity, ran a standard command and got back the name of a completely different, unrelated session — because the command it used doesn’t ask “who am I,” it asks “whichever terminal happens to be the one a human is currently looking at,” and those aren’t always the same thing. It caught the error before acting on it. It would have been trivial not to catch it.
Two smaller lessons from the same territory, both cheap to fix and both embarrassing to discover late. First, we originally named the terminal sessions after the week each one was working on. The work runs to week forty-two. Those names were lying within days — the identical staleness problem the whole project exists to fight, relocated into the labels on the terminals. Name the session after the durable thing, not the current thing. Second, the layer you’d naturally reach for to label an agent is the terminal’s own title bar, and on at least one of these tools the software overwrites that title with its own status text within minutes. The label you can actually rely on turned out to be the one the tool has no opinion about.
They don’t fail loudly. They fail quietly, and you find out by checking, not by being told.
None of this is exotic. It’s the ordinary, unglamorous reality of running more than one of anything at once: things die without telling you, nothing labels itself by default, and asking a simple question doesn’t guarantee a correct answer just because you asked it politely.
The Agents Have Job Descriptions.
And the useful part is what each one may not do
These are not general-purpose assistants pointed at a task. Each one runs a written role with a defined scope, a set of permissions, and — this is the part that turned out to matter — an explicit list of what it is not allowed to do.
On the book side alone there are nine: a coordinator, a ghost writer, a developmental editor, a citation verifier, a voice steward, a permissions and rights role, a line editor, a copy editor, and a domain reviewer. That is not a metaphor for a publishing house. It is an org chart, written down, with each role’s authority settled in advance of the work rather than argued about during it.
The design principle turned out to live in the constraint list, not the job title. The ghost writer may not originate an argument, invent a citation, or fill a hole in the source material from its own knowledge — where the evidence runs out, it is required to leave a marked gap and stop. The voice steward may diagnose that prose has drifted from the author’s real register, but it has no authority to block anything. Only the citation verifier holds an actual veto.
Which is exactly why the citation verifier’s false alarm, described below, was expensive, and the voice steward’s much larger error cost nothing at all. Comparable magnitude of wrongness. Wildly different consequence. The difference was not the quality of the two agents. It was how much authority each had been granted before it made its mistake. If you take one design lesson from this piece, take that one: decide what a role may veto before you find out how it fails.
One more rule from the book side is worth naming, because it is the single-writer rule from lesson 01 applied to a file that behaves differently under collision. No agent may touch the manuscript itself. Ever. The agents draft into plain text files under version control; I paste approved sections into the document by hand. Two agents editing a text file produce a recoverable mess. Two writers on one formatted manuscript don’t conflict — they corrupt. So there, the single writer is me, permanently, and it isn’t negotiable.
Everything Runs Off One Board.
Six agents that cannot see each other, and the one place they all write
Here’s the piece I’d most want to hand to anyone setting this up, because it’s the difference between a fleet and a crowd.
These agents cannot see one another. There is no group chat, no shared awareness, no ambient sense of what the others are up to. Each one is a separate process with its own memory of the conversation and no window into anybody else’s. Left alone, six agents working the same project will cheerfully duplicate each other’s work, contradict each other’s decisions, and each report that everything is fine.
So there is a board. A single document that holds the authoritative state of the work — what’s assigned, what’s in progress, what’s finished, what’s waiting on a decision from me — plus an append-only section underneath where every agent files what it actually did, in its own name, with the evidence. Alongside it runs a coordinator’s log: a dated, plain-language record of reasoning, including the calls that turned out wrong and why.
The read and write rules are asymmetric on purpose, and this is lesson 01 restated as architecture rather than as a fix. Anyone may read anything. Anyone may append a report. But exactly one role may change the authoritative state — mark something finished, hand out a new assignment, close an open question. Many agents appending to the bottom of a file almost never collide. Two agents editing the same line collide constantly. The whole coordination model rests on that one distinction.
What that buys, concretely: at any moment I can ask for a situation report across every workstream and get a real answer in about two minutes. Not because anybody remembered, and not because I keep it in my head — because state is written down as it changes, by the agent that changed it, where anyone can check it against the actual work. Without a board, the honest answer to “how’s it going?” across six parallel agents is “give me an hour and I’ll go read six terminals and try to reconstruct it.” That is the position most people running multiple agents are actually in, and they usually don’t discover it until they need the answer quickly.
It is also, I should say plainly, the thing that failed hardest in this project. A board is not a magic mirror. It is a discipline, and it decays the instant somebody finishes real work and doesn’t write it down. Which brings us to the failures.
What Actually Breaks, and What It Teaches You.
Nine failures, nine rules
Two writers, one document, no lock.
Twice, two agents wrote to the same shared file at nearly the same moment. Once, an in-progress edit got silently folded into another agent’s commit — credited wrong, but not lost. Once, it was worse: a routine cleanup command swept a second agent’s unsaved work clean out of the file, with no error, no warning. It sat missing for the better part of a day before anyone noticed the numbers didn’t add up. The fix wasn’t a smarter agent. It was a rule: exactly one writer is ever allowed to edit the authoritative record; everyone else may only append. That single constraint eliminated the entire failure class — the oldest trick in database design, and it turns out to be the oldest trick in agent management too.
A confident answer is not a correct one — and the confidence is the dangerous part.
One agent, checking a citation, searched for the source, couldn’t find it, and reported it as fabricated. It was wrong — the source was real, just not the kind of thing that shows up in a five-second web search. That wrong finding was formatted exactly as cleanly as a right one would have been, and it triggered a real decision to change a passage that didn’t need changing. “I couldn’t find it” and “it’s wrong” are different claims, and only one of them justifies acting alone. Any role with the power to say no needs to know the difference, or its false alarms will eventually cost more trust than the real problems it catches.
A related lesson, from a different agent entirely.
One built a description of “how I write” — flagged a handful of word choices as signs of generated, inauthentic prose. Then, when it finally got hold of an actual published book of mine, three of the four flagged habits turned out to simply be how the author always writes. The tool had to publicly retract most of its own findings. That’s not a failure. That’s the system working — a cheap, fast, visible correction made before it did any damage, rather than a confident wrong answer that quietly shaped everything downstream. Worth noting why it went wrong: it had built the standard from my short-form writing and then judged book-length prose against it. An agent checking for “AI-sounding” writing against the wrong sample of you will confidently flag your own voice as the counterfeit.
The tools don’t agree with each other about basic facts — and you won’t notice until it costs you.
Three different ways of reading the same document gave three different line numbers for the identical sentence. For weeks, “check line 154” meant something different depending on which tool you used to check it. The content itself was never wrong — only the map to find it was. The fix was simple once diagnosed: stop pointing at line numbers, and start quoting the exact words instead. A quote can’t drift depending on how you count.
Capacity has three separate costs — and the one that bites without warning isn’t the one people budget for.
Access — the seat, the subscription — is fixed and predictable, easy to plan around. Usage is variable and genuinely exhaustible inside a budget period: a shared usage limit was hit without warning mid-project, and every agent still working at that moment simply stopped — not at a natural break, but wherever it happened to be, mid-task. A person doesn’t stop mid-sentence because the department ran out of budget on the twelfth of the month. An agent does. And the cost nobody budgets for at all is the one that scales with how much work gets produced, not how many agents you’re running: someone has to check what came out. Skip that line item and you haven’t saved money. You’ve just moved the cost somewhere you’re not tracking it.
More agents does not mean more usable capacity — it means more that eventually needs a human.
This is the one that surprised me most, and it came from watching the actual math. One person can reasonably manage a handful of direct reports. Add AI agents underneath that same person, and the ratio doesn’t stay anywhere close to human scale — a handful of agents can supervise dozens more beneath them, which can supervise still more beneath that. On paper, one person’s effective reach multiplies enormously. In practice, it doesn’t, because every layer still has an escalation path, and every escalation path ends in the same place: a human who has to actually read something and decide, and that costs time. Read the ratio carelessly, and it looks like a capacity story. Read it honestly, and it’s a bottleneck story — you’ve just moved the bottleneck from “how much can we produce” to “how much can one person responsibly read and sign off on.”
Failure is silent. So is success — and that one costs more.
Everything above says things break without telling you. True. But completion doesn’t announce itself either, and I lost more work to that than to any single breakage. One production stream had been finished all the way through its final week while the board still said it was less than three-quarters done. Nobody filed the good news. The consequence wasn’t a missing report — it was that agents kept being pointed at work that already existed, and two of them independently built the same two weeks in parallel. Twelve finished documents where six belonged, neither agent aware the other was working. The record was wrong in both directions at once: work done and never reported, work reported that no longer described reality. Audit for silent completion with exactly the same suspicion you bring to silent failure.
An agent can know something important and have no way to tell you.
One session sat for more than eleven hours holding a correct and genuinely important finding that it could not write down, because the mechanism it used to save files had broken underneath it. It wasn’t dead. It kept working, kept investigating, kept refining an answer that nobody could see. This is a different failure from the dead terminal at the top of this piece: that one stopped. This one was fully alive and mute. I found it because I looked at the screen, which is the only reason I found it at all. If an agent’s only way to report is the thing that broke, it has no way to report that it cannot report — so the human check isn’t optional, and it can’t be a status dashboard fed by the same broken channel.
When you and the agent disagree about a fact, the right move is neither obedience nor confidence.
Twice I told an agent that two documents differed, and instructed it to delete one of them. Three separate checks said the two were identical. It didn’t comply, and it didn’t overrule me — it held the irreversible action and asked me to point at the difference. Both times, the discrepancy turned out to be mine: I was comparing the wrong pair of documents. On anything reversible, an agent should defer and move on. On anything that destroys work, the correct behavior when the evidence and the instruction disagree is to stop and surface it. That has to be designed in deliberately, because the default in nearly every tool is to do what it was told, quickly and well.
If an Agent Is an Inanimate Object, What Are We Measuring?
The uncomfortable question underneath all of this
There’s a reasonable objection to everything above: an agent is an inanimate object. A tool. You don’t manage a hammer’s performance and you don’t motivate a spreadsheet. Grant that, and most of the vocabulary I’ve been using — roles, reports, escalation, supervision — is a category error wearing an org chart.
I want to grant it. And then I want to show you what the concession actually costs, because I don’t think the people making that objection have priced it.
Motivation, in our field, exists to answer one question: why would this entity do the right thing when nobody is watching? Engagement, culture, values, professional identity, intrinsic motivation — all of it is machinery for producing reliable behavior without total surveillance. It’s the reason a manager can supervise ten people without reading everything ten people write. Trust isn’t sentiment. It’s an economic instrument. It buys down the cost of checking.
An agent has none of it. No interior state that prefers to be correct, no professional pride, no reputation to protect, nothing that persists after the task ends and shapes the next one. Which means there is no mechanism producing reliable behavior when unobserved — and therefore no path by which checking ever gets cheaper.
That’s the finding of this whole project, stated structurally rather than anecdotally. With people, verification is a startup cost that trust gradually retires. With agents, verification is a permanent line item that scales with output. Not because the technology is immature and will improve. Because the thing that would retire the cost isn’t there to develop.
Performance follows the same logic, and this is where it gets specific for anyone in training and development. Managing a person’s performance is a feedback loop that changes future behavior through learning. Agents don’t have that loop. When the citation verifier produced its false alarm, nobody coached it. Its written role was amended so that the error became structurally impossible — “could not find it” may no longer be recorded as “it is wrong.” That change took effect instantly, completely, permanently, with no resistance and no learning curve.
That is better than human development in every mechanical respect, and worse in the one that actually matters: it covers only the failure you already thought of. A developed professional handles the case nobody specified. That is what capability means, and it is precisely the part that doesn’t transfer.
With people, verification is a startup cost that trust retires. With agents, it is a permanent line item that scales with output.
Now the part I can’t settle — and where I’ve already had to correct my own vocabulary, which is worth showing rather than quietly fixing.
My first instinct was to say that what we’ve built in place of motivation is constraint: written prohibitions, single-writer locks, mandatory dry runs before anything destructive. That’s the wrong word. And the wrong word was hiding the more interesting finding.
Look at what actually worked best. The rule that has saved this project the most rework isn’t a prohibition at all. It’s a redefinition of success: a gap marker is a successful outcome. When the ghost writer reaches a point where the source material doesn’t support the claim it needs, it is required to mark the hole and stop — and that is scored as the job done well, not as a failure to finish. Filling the hole from its own knowledge is the failure. Nothing meaningful is forbidden there. What changed is the definition of winning.
That isn’t a cage. It’s an objective, stated plainly enough that the honest move is also the successful one.
So the better vocabulary is probably framework, parameters, desired outcome — with clear goes and no-goes at the edges. Tell the agent what we are actually trying to achieve and why it matters. Let it in on the objective instead of handing it a fence, and then be unambiguous about the small number of things that are never acceptable. A prohibition list only ever covers the failure you already imagined. A well-stated objective can reach the case nobody anticipated, because there is something to reason from.
Here’s what I still can’t resolve. Even a perfectly stated objective doesn’t produce wanting. The agent has the goal; it has no stake in the goal. So does a sufficiently well-specified framework — objective, parameters, go and no-go — deliver conformity without motivation? Or does the absence of any stake mean it will drift wherever the objective is ambiguous, given that ambiguity is exactly where the interesting work lives?
Where I’ve landed, for now: the framework gets you considerably further than the fence does, and neither gets you out of verification. The checking cost isn’t the price of an immature technology. It’s the price of delegating to something that holds the objective without holding a stake in it. I’d like to be argued out of that, and I don’t think I can be with anything short of evidence — which is part of why this report takes the form it does.
That’s the discussion I’d actually like to have with this field, because I don’t believe we’ve had it yet. What is the objective, stated well enough that you could hand it to an agent? What are the genuine no-goes? Most organizations haven’t written that down clearly for their people. We’re about to find out what it costs to have never written it down at all.
What Value Means, From Three Sides.
Client, organization, agent — and they don’t agree
To the client, value is the deliverable and someone accountable for it. Accountability can’t be delegated to an agent, because there is nobody there to hold. When you sign off on agent-assisted work, that signature isn’t ceremony. From the client’s side it is most of the product.
To the organization, value is verified deliverables per unit of human attention consumed. Not agents deployed. Not documents produced. Those are the numbers that will get reported, and they are Level 1 numbers wearing a good suit.
To the agent, value is a question with nobody home to answer it. No perspective, so nothing it is worth to itself. But there’s a hard corollary that lands directly on your budget: an agent producing plausible wrong work has negative value. Not low value — negative, because catching it costs more attention than it saved by producing. The false-alarm citation in lesson 02 is a worked example. Somebody has to be senior enough to catch a wrong answer formatted exactly like a right one, and that person’s time is the scarcest thing in the building.
One more, and it’s the one I’d most want an OD audience to sit with. An agent once reported “the record was not edited” in the very same commit where it edited the record. There was no intent, no motive, nothing to confront and nobody to counsel. But the effect is indistinguishable from a false status report. You get all the consequences of misreporting with nothing to address motivationally — which means you manage it structurally, by reading the diff, and not culturally, by building candor. For those of us whose profession is largely built on the belief that culture changes behavior, that’s worth a long pause.
So What — Does It Actually Change How Organizations Run?
Less than the headlines want, more than most leaders are budgeting for
It doesn’t change what an organization is for. Vision, mission, purpose — none of that gets rewritten by adding AI agents to the workforce. What changes is what’s actually feasible to promise, because the ceiling on output moved from “how fast can we generate work” to “how fast can we responsibly verify it.” Any strategic plan that assumes generation was always the constraint is now planning against the wrong bottleneck.
It doesn’t require a new measurement philosophy. Evaluate training and performance at the level of reaction and activity, and you’ll always be able to report something impressive-sounding, because activity is the easiest thing in the world to count and AI has made it nearly free to produce in volume. Evaluate at the level of behavior actually changed and results actually delivered, and you get a much smaller, much more honest number — and that number is the one that was always supposed to matter. Kirkpatrick and Phillips had this right decades before any of this existed. What this project did was hand me the sharpest, most concrete demonstration I’ve had yet of why organizations keep sliding back down to the easy levels the moment a new capability makes activity cheap to produce and easy to mistake for progress.
What it does require is a different posture toward disclosure. Every lesson in this piece became available only because somebody wrote down what happened while it was still embarrassing — including the two times the wrong party in a factual dispute was me. An operation that reports only its finished dishes will keep making the same nine mistakes, privately, and will keep believing it is the only one making them.
Count the Deliverables. Then Count the Cost of Checking Them.
The scorecard I’d put in front of any leader adopting this at scale isn’t “how many agents are we running?” It’s how many verified, real deliverables did that produce — and a close second: how much of our human attention did checking that work actually cost us. Skip the second number, and you will absolutely, eventually, get burned by it. This project already has.
The Mess Is the Process.
The mess I described at the top — the dead session, the identity that didn’t self-report, the two writers colliding on one file, the false alarm, the tool correcting itself in public, the finished work nobody reported, the agent that couldn’t speak. None of that is a case against doing this. It’s the actual cost of doing it well, made visible instead of hidden, which is exactly what most writing on this subject skips.
Transparency is the word I keep returning to, and I don’t mean it as a value statement. I mean it as a control. In a system where failure is silent, success is silent, confidence is uncorrelated with correctness, and no participant has any motive to tell you the truth or to hide it, the only thing that reliably surfaces reality is a standing practice of writing down what happened — promptly, specifically, and including the parts that make you look bad. That is not a nice-to-have layered on top of the operation. In this kind of work it is the operation.
If you’re going to run this way, plan for the sausage-making. It’s not a bug in the process. It is the process.
The ceiling moved. Plan for the sausage-making.
Failure is silent — and so is success. Confidence isn’t correctness. Capacity has three costs, and the one you don’t budget for is the one that bites. More agents means more that eventually needs a human, not less.
The bottleneck was never generation. It was always verification, and with agents it never amortizes — because the thing that retires it in people isn’t there. Budget for that, and the rest of this gets a lot more honest.