Codebot reviews ran on our machines
A colleague's AI code reviewer went from part of Wix to every repo in one morning. A large part of it ran on a cluster my team runs, and his team never ran a node.

Ronen Amiel's team built Codebot, the AI reviewer that flagged about 25K issues Wix developers fixed in their PRs last month (his post). Each review runs an agent on a machine we hand out from a warm pool. That is the half of the story his post doesn't tell.
The morning it went to 100%
One morning in late September Ronen pinged us. They were opening Codebot to all of Wix that day. They were at about 10 reviews a minute. Could we take 25?
The machines were already warm and waiting. A review grabs one, runs, and gives it back. So the question was only whether the pool could hand them out faster than reviews arrived. I checked our numbers and added more warm machines. Sergei Senko is the one rebuilding how that pool hands them out.
Ronen asked "Safe to start?" I said "5 minutes", then "go". By lunch Codebot was on for every repo at Wix.
A third of the memory
The same day I looked at what the reviews actually used. Most used about a third of the memory each machine reserved. Reserved memory is what decides how many machines fit on a server, so we were paying for servers to hold memory nobody touched. I cut the reservation to match, kept the upper limit where it was, and we needed about a third of the servers.
Keys outside the sandbox
Running code nobody reviewed was never new for us. Velo does it for Wix users every day, each run in its own gVisor sandbox. A review agent reads untrusted PRs, so it gets the same treatment.
The new part was keys. An agent that reads a prompt-injected PR can be talked into printing whatever it can see. In July Ofer Belinsky asked to pull the AI key out of the review machines. Now Max Yegorov's proxy adds the key on the way out, outside the sandbox. The agent never holds it, so a prompt-injected review has no key to leak.
It reviewed my PR
The next morning Codebot reviewed one of my own PRs and caught a real bug in it. Running on my machines, reviewing my code, and it was right.
Ronen's team still hasn't had to think about a node.
Thanks to Ronen Amiel, Sergei Senko, Ofer Belinsky and Max Yegorov.



Comments