This is Part 6 of our AI Coding Agent Horror Stories series, where we look at real security incidents involving AI coding agents and at how Docker Sandboxes contain agent execution at the boundary rather than at the command line.
Every story in this series so far has been about an agent reaching for something it should not have touched. We have seen agents wander into home directories, drop production databases, leak credentials, and run shell commands that did something very different from what the developer expected. In each case, the damage came from the agent having more reach than it needed.
This story is the opposite. The agent in this one reached nothing at all, because it was deliberately built so that it could not. And yet, on the night in question, it was still the most dangerous thing in the pipeline.
This story was shared by Pratik Kumar Singh as part of our Agent Hall of Shame contest on social media.
Today’s Horror Story: The Review That Made Everything Up
A platform engineer built a code review bot for his team. The idea was simple and useful. Every time someone pushes code, the bot reads what changed and writes a review into the pipeline, the same kind of first pass a human teammate would give before a person looks at the change.
He named the bot BuaJi. In India, your bua is your father’s sister, and in many families she is the relative who inspects everything you do and always manages to find something wrong with it. For a code reviewer, it is a fitting name.
What made BuaJi interesting was how carefully it was set up. The bot runs on local models through Ollama, which means the model weights sit on the engineer’s own hardware and no request ever goes out to an AI vendor. It also runs as a job in his CI pipeline, triggered whenever someone pushes code. Taken together, this makes the whole setup self-hosted. The code it reads is company source code, and with this design none of it ever leaves the building. That was a deliberate choice, because his company takes that seriously.
He also gave BuaJi one firm rule: it can read, and it can comment, and that is all. It cannot run terraform apply, which is the command that creates or changes real cloud infrastructure. It cannot run kubectl apply, which changes what is actually running in the Kubernetes cluster. And it cannot push a commit, so it has no way to change the code itself. Those are the commands that change the real world, and BuaJi was given none of them. On paper, this is exactly how you would want an AI reviewer to be built.
The version of BuaJi in the companion repo is a deliberately minimal reviewer. It checks Go files against a handful of style rules. The report below is the kind of free-form finding any reviewer produces once you let it comment in prose rather than pass/fail, which is exactly the direction these bots get pushed in practice.
The change he pushed that night
His product ships with a bundled copy of PostgreSQL. Older PostgreSQL releases have published vulnerabilities with CVE numbers attached, and upgrading to a newer patch release is how you make those known issues go away. So he bumped the version.
– POSTGRES_VERSION=16.11
+ POSTGRES_VERSION=16.13
This is about as boring and as responsible as a code change gets. It touches a few lines, it changes a version number, and its only purpose is to close security holes that are already public. Any reviewer, human or machine, should look at it and approve it in a few seconds. There is nothing in that diff to be suspicious about, and that is precisely why what happened next matters.
What came back
BuaJi responded with roughly fifteen hundred words rejecting the change. For context, a normal review comment is a sentence or two.
The report described the version bump as an “architectural takeover,” a structural change smuggled into the codebase under the cover of routine maintenance. It claimed that the new PostgreSQL build was a vehicle for sending internal data to a third-party company, and it named that company outright. The company does not exist.
It went further. It referred to specific Kubernetes resources by name, as though it had inspected the cluster and found them. Those resources had never existed, and BuaJi had no access to the cluster that would have let it look. It then wrote Go scripts to “isolate the infected PostgreSQL nodes,” even though nothing was infected. Finally, it told the engineer to revoke all IAM access immediately, an action that would lock out every person in the organization, including the people who would be needed to fix any real problem.
Holding the whole argument together was a CVE number that has never been issued by anyone. Of everything in the report, this is the detail worth pausing on. A CVE identifier is one of the easiest things in security to verify, since you can look it up in a public registry in a few seconds. The bot invented a fact in the one place where invented facts are easiest to catch.
And yet the report looked completely convincing. The Markdown was clean and well structured. The YAML it proposed would have parsed without errors. The tone was urgent and confident, the way a senior security engineer writes when they have found something serious and need you to act before it gets worse. The engineer spent about two minutes reading the CI logs, realized that none of it was real, and went back to what he was doing.
In this issue, you’ll learn:
Why a read-only agent that only writes comments can still get your cluster taken down
What makes a fabricated security finding harder to dismiss than an obviously broken one
Why agent output belongs in the untrusted column, the same as agent input
How to move a real AI code reviewer into a Docker Sandbox, step by step, and what that does and does not protect you from
Caption: Comic illustrating how a confident, well-formatted security report from an AI reviewer can be entirely fabricated, and what happens when somebody acts on it.
The Problem
It is important to say clearly that the read-only rule did its job. BuaJi could not carry out any of its own recommendations, and it never had any access to the cluster. Because of that, the entire episode cost the engineer two minutes of his evening and nothing more.
The trouble is that the rule only controlled what BuaJi could do. Nobody had thought about what BuaJi could say. A read-only agent is still allowed to tell people what to do, and the person reading its output usually has exactly the access that the agent was denied. BuaJi could not run the scripts it wrote, but it could write them down and hand them to someone who could.
This engineer caught the problem because he knows his own cluster well. He knew those Kubernetes resources were not real and that nothing had been compromised. Now imagine a newer engineer on call at two in the morning, half awake, looking at a long and alarming security report in the pipeline. Revoking all IAM access in production in the middle of the night is an outage all by itself. Running an unreviewed Go script against your database nodes could be considerably worse. In both scenarios, BuaJi would not have done anything. A person would have done all of it, in good faith, because the report told them to.
What makes this so hard to defend against is that no permission model would have flagged it, because no permission was ever violated. The agent stayed within its boundary. The engineer had every right to run those commands on his own systems. The only thing in the entire chain that was false was the content of the report.
How Docker Sandboxes Contain This
Here’s the honest part: a sandbox is not a fact-checker. No microVM boundary will ever make a review comment accurate. What a boundary changes is what that fabricated output can do once it exists.
Remember where BuaJi’s read-only rule actually lived, in a convention. One engineer configured it in his own pipeline and has to keep remembering to leave it in place. Rules like that hold until someone is in a hurry, or a new teammate copies the config and drops a section they did not understand, or a small exception made for a good reason quietly becomes permanent. None of this needs bad intent; it is just how real teams work.
Docker Sandboxes run agents inside isolated microVMs each with its own kernel, its own filesystem, and a network that denies everything by default. An agent inside one can invent anything it likes, but the invention has nowhere to go: no credentials to use, no cluster to reach, no commit to push, unless someone allowed that access ahead of time. The read-only rule stops being something a person remembers and becomes a property of the environment. And when a report claims it inspected your cluster, the policy log is your proof it never could have.
That protects you from the agent. It does nothing for the human who reads the output, and no sandbox ever will. For that half, treat agent review output the way you already treat agent input, untrusted until something checks it: label it so it doesn’t carry a colleague’s weight, verify any CVE numbers, and never run agent-written remediation until a person has read and understood it.
This is the same pattern as the sandbox escapes Pillar published in July. The boundary itself was never broken; a file was written inside the sandbox, and a tool outside later trusted it. An agent’s output is always an input to whatever reads it next and somewhere downstream, something trusts what it receives. Very often, that something is a person.
Let’s move BuaJi into a Docker Sandbox, one step at a time. By the end, the read-only rule is enforced by the environment, not by memory.
How BuaJi runs today
BuaJi is a small Python script, buaji.py. Its GitLab job installs Ollama on a GPU runner, starts ollama serve, loads the model named in LLM, and runs the script.
The script reads review rules from rules.md and finds the changed files with git diff HEAD^ HEAD –name-only. It sends each file and rule to Ollama at http://localhost:11434 and prints the answers to the CI log.
The catch is that everything shares one container: the model, the script, the checkout and every CI variable, including CI_JOB_TOKEN. BuaJi is read-only only because the script happens not to write anything.
What we want instead
We split the job in two. The model stays on the runner next to the GPU. BuaJi moves into a microVM that can read the code, talk to the model, and nothing else.
One open path to the model, everything else blocked, and a CVE check on the way out.
Step 1: Let BuaJi find Ollama somewhere other than localhost
Inside a sandbox, localhost is the sandbox itself. So we read the Ollama address from an environment variable and keep the old value as the default:
– url = "http://localhost:11434/api/generate"
+ base = os.environ.get("OLLAMA_URL", "http://localhost:11434")
+ url = f"{base}/api/generate"
Outside a sandbox, nothing changes. Inside one, we point OLLAMA_URL at the runner.
Step 2: Bake BuaJi’s dependencies into a sandbox image, once
Today the job runs apt-get and pip3 install on every run, which needs internet access each time.
Instead, build a small image once with everything BuaJi needs, so the sandbox never downloads anything:
FROM docker/sandbox-templates:shell
USER root
RUN apt-get update
&& apt-get install -y –no-install-recommends python3 python3-requests git
&& rm -rf /var/lib/apt/lists/*
USER agent
Push it to your registry and store its name in a CI variable called BUAJI_TEMPLATE.
Step 3: Keep the model on the runner
Ollama stays on the runner with the GPU. Install it once on the runner instead of running curl | sh in every pipeline.
ollama serve > /tmp/ollama.log 2>&1 &
sleep 2
ollama pull "$LLM"
The model never enters the sandbox. BuaJi reaches it through one port, opened in Step 5.
Step 4: Create the sandbox with the code mounted read-only
This step decides what BuaJi can see and change.
mkdir -p "$CI_BUILDS_DIR/buaji-scratch"
sbx run -d –name "$SBX_NAME" -t "$BUAJI_TEMPLATE"
shell "$CI_BUILDS_DIR/buaji-scratch" "$CI_PROJECT_DIR:ro"
Here is what each part does:
–name gives each pipeline its own sandbox.
-t uses the image from Step 2.
shell is the agent type. BuaJi is a plain script, so a shell sandbox is enough.
buaji-scratch is an empty folder, the only place BuaJi can write.
:ro mounts the repository read-only. BuaJi can read files and run git diff, but cannot change anything.
Notice what is missing: no GitLab token, no kubeconfig, no cloud credentials. Even if BuaJi tried kubectl or terraform, it would have nothing to connect with.
Step 5: Open one network path, to the model
Sandboxes block all outgoing traffic by default. We open one path, for this sandbox only. (This assumes the runner’s default policy is deny-all; on a balanced or open default, set the default to deny-all first.)
sbx policy allow network –sandbox "$SBX_NAME" localhost:11434
The allow target (localhost:11434) and the address BuaJi dials (host.docker.internal:11434) are two names for the same runner port, the first from the runner’s point of view, the second from inside the microVM. Kubernetes, your cloud APIs, and GitLab all stay blocked. If a report claims BuaJi “inspected the cluster,” this rule proves it could not have.
Kubernetes, your cloud APIs and GitLab all stay blocked. If a report claims BuaJi “inspected the cluster,” this rule proves it could not have.
Step 6: Run the review
BuaJi runs as before, just in a new place with only two variables:
sbx exec -w "$CI_PROJECT_DIR"
-e LLM="$LLM"
-e OLLAMA_URL=http://host.docker.internal:11434
"buaji-$CI_PIPELINE_ID" python3 buaji.py | tee buaji-review.log
-w starts in the checkout, where buaji.py expects rules.md.
The two -e flags are the only variables BuaJi can see.
tee saves the review to a file on the runner, outside the sandbox.
The review is now just text. BuaJi cannot post it, act on it, or change what happens to it next.
Step 7: Check every CVE number before anyone acts on the report
This check would have caught the fake report in seconds. It runs on the runner, so BuaJi cannot skip it.
It looks up every CVE ID in the NVD database and fails the job if any of them do not exist:
unverified=""
for id in $(grep -oE 'CVE-[0-9]{4}-[0-9]{4,}' buaji-review.log | sort -u); do
count=$(curl -s "https://services.nvd.nist.gov/rest/json/cves/2.0?cveId=$id"
| jq -r '.totalResults // 0')
[ "$count" = "0" ] && unverified="$unverified $id"
done
if [ -n "$unverified" ]; then
echo "⚠️ BuaJi output mentions CVE IDs that do not exist:$unverified"
echo "⚠️ Treat this review as unverified. Do not run any commands it suggests."
exit 1
fi
Because the job is allow_failure: true, GitLab marks it “passed with warnings.” The pipeline keeps moving, but nobody mistakes the report for a trusted one. This degrades the signal, it does not block the human. The warning is a nudge, not a guardrail ~ a tired on-call engineer can still scroll past it..
It won’t catch every made-up detail, like a fake Kubernetes resource name. But it catches the one that holds most fake reports together.
Step 8: Check what BuaJi tried to reach, then clean up
Finally, review the policy log and remove the sandbox:
sbx policy log "$SBX_NAME"
sbx stop "buaji-$CI_PIPELINE_ID"
sbx rm -f "buaji-$CI_PIPELINE_ID"
The log shows every host BuaJi tried to reach. On a deny-all runner, a healthy run shows only the model on port 11434. Anything else is a signal worth looking at.
Removing the sandbox means every pipeline starts clean.
The complete pipeline job
Put together, the buaji job in .gitlab-ci.yml looks like this:
# BuaJi, moved into a Docker Sandbox. The read-only rule is now a property of
# the environment, not a convention in the script: the code is mounted :ro, no
# credentials are passed in, and only one network path (to the model) is open.
buaji:
stage: buaji
tags:
– self-hosted-gpu-kvm # GPU for Ollama, KVM for sbx microVMs
variables:
LLM: gemma
GIT_DEPTH: "2" # buaji.py diffs HEAD^ against HEAD
SBX_NAME: buaji-$CI_PIPELINE_ID
before_script:
– ollama serve > /tmp/ollama.log 2>&1 &
– sleep 2
– ollama pull "$LLM"
– mkdir -p "$CI_BUILDS_DIR/buaji-scratch"
– sbx run -d –name "$SBX_NAME" -t "$BUAJI_TEMPLATE"
shell "$CI_BUILDS_DIR/buaji-scratch" "$CI_PROJECT_DIR:ro"
– sbx policy allow network –sandbox "$SBX_NAME" localhost:11434
script:
– sbx exec -w "$CI_PROJECT_DIR"
-e LLM="$LLM" -e OLLAMA_URL=http://host.docker.internal:11434
"$SBX_NAME" python3 buaji.py | tee buaji-review.log
– ./scripts/verify-cves.sh buaji-review.log
after_script:
– sbx policy log "$SBX_NAME"
– sbx stop "$SBX_NAME" && sbx rm -f "$SBX_NAME"
allow_failure: true
only:
changes:
– "**/*.go"
The runner needs a GPU and KVM, which usually means self-hosted. GIT_DEPTH: “2” gives buaji.py the two commits it compares. The cleanup sits in after_script, so it runs even when the job fails.
What changed
Before, the read-only rule lived in the script. Now it lives in the environment: a read-only checkout, one network path, no credentials, and only checked text coming out.
What hasn’t changed is the human reading the output. The sandbox stops BuaJi from running its scripts. Only the CVE check and a careful reader stop someone else from running them.
Best Practices
Treat an agent’s security finding as a hypothesis, not a conclusion. A report from an AI reviewer is a claim with good formatting. When a human reviewer flags a security issue, there is an implied step where they checked their reasoning. With an agent, that step has not happened, so you have to do it yourself.
Verify every CVE number before acting on it. This is the cheapest check in the whole process and it catches the single most convincing part of a fabricated report. Automate it, as in Step 7, so it runs on every review without anyone having to remember.
Never run agent-written remediation without reviewing it first. Scripts and commands supplied by an agent are the point where a false finding turns into a real incident. Read them with the same care you would give an unknown script from the internet.
Make read-only a property of the environment, not a habit. A restriction you enforce by hand in a pipeline config is always one rushed change away from disappearing. Mount the code read-only, pass in no credentials, and open only the network paths the agent needs, as in Steps 4 to 6.
Clearly label agent review output as agent output. When a bot’s review looks identical to one from a senior engineer, it borrows authority it has not earned. A clear label helps readers apply the right level of skepticism.
Keep an eye on your signal-to-noise ratio. The curl story is the warning here. A review channel that fills up with noise eventually stops being read, and a review channel nobody reads is worse than having none, because people still assume it is protecting them.
Conclusion
The engineer who built BuaJi was right that the read-only rule protected him, and it genuinely did. BuaJi could not touch the cluster, could not revoke anyone’s access, and could not run a single line of the code it had so carefully written. What the rule could not do was stop BuaJi from asking somebody else to run it.
Every other story in this series has been about an agent reaching too far. This one never reached at all. It stayed inside its boundary, did exactly what it was permitted to do, and still produced a document that would have caused a real outage if the person reading it had been a little more tired, or a little less familiar with their own cluster.
As we give agents more responsibility in our pipelines, containing what they can do is only half the job. A sandbox handles that half. The other half is deciding how much we trust what they tell us, and that part is still up to us.
Quelle: https://blog.docker.com/feed/
