← All articles

Human in the loop: AI governance starts with people

By Gianmarco Virzì ·

More and more developers find themselves handing the writing of code over to AI agents. In April Google said that 75% of its new code is generated by AI and then reviewed by engineers. And Google apparently isn’t alone, since in a New Relic survey of 200 technology leaders at US companies, two in three say that AI writes or substantially rewrites between 51% and 75% of the code their organization produces each week.

People who started writing code long before this technology arrived are facing more than legitimate questions, such as: “is it really right to hand most of the code over to AI?”, “what would a responsible pipeline for using AI at a company look like?”, “how do I keep control of my code?” and “is this really sustainable?”

AI has become part of the everyday work of people who build software, often before their company decides how to use it. The Stack Overflow Developer Survey 2026 (run from June to August 2026 with about 30,900 developers) shows this with two questions that, read together, say a lot and lead straight to the principle of human in the loop.

What the data says

The first question asked how much developers trust the output of AI tools in their work. 48% answered “I trust it when I can easily verify the output”. Only 7% answered “I trust it for many tasks, including important work decisions”.

The second asked how AI tools are used in their organization. The most common answer was “AI use is mostly optional and left up to individuals”, at almost 30%. “My organization provides approved AI tools and guidance for how to use them” got 24%.

Bar chart with all the answers to two questions of the Stack Overflow Developer Survey 2026: 48% trust AI when they can easily verify the output; in almost 30% of organizations AI use is mostly optional and left up to individuals
All the answers to the two questions of the Stack Overflow Developer Survey 2026.

Our reading

The first figure, for us (counterintuitive as it may seem), is good news. Trust in AI is conditional. Almost half of developers are happy to use it, but they want to be able to check what it produces. And rightly so. A model can be wrong with great confidence, and the role of people in supervising it is essential, for safety and for ethics.

The figure that worries us more is the second one. In almost one organization in three, according to the people who work there, AI use is left up to each individual. There, nobody decides which tools are used, which data goes into them and who checks the result. One developer pastes code into a chat, another uses AI in the IDE, a third from the terminal, and someone doesn’t use it at all. Everyone does as they please, even when it comes to code covered by copyright or by an agreement with a publisher, or to player data (in the case of games).

More than the technology, the crux is policy, meaning who decides and where those rules live. Artificial intelligence is moving fast and has “forcefully” entered everyday life. It’s often discussed only as a matter of legal regulation, but it’s a discussion that concerns companies and the way they work first of all, and perhaps we tend to ignore a problem we are all too involved in.

Human in the loop: AI proposes, a person decides

“Human in the loop” is the principle that in a process using AI there is always a person who looks at the result before it takes effect: they approve it, correct it or discard it. AI speeds up the work, people conduct the orchestra.

The 48% who trust AI when they can verify its output are, in the end, asking for exactly this. Human in the loop turns that check from a personal habit into a rule of the process, a planned step that no longer depends on whether someone feels like checking.

Human in the loop or human on the loop?

The two expressions are often confused, but they describe two different ways of keeping a person in charge. A clear definition can be found in a 2012 Human Rights Watch report, written during the debate on autonomous weapons, which explains three levels of human involvement:

  • human in the loop: the system acts only on a person’s command;
  • human on the loop: the system acts on its own, under the supervision of a person who can override its actions;
  • human out of the loop: the system acts without any human involvement.
Diagram of the three levels: human in the loop, where a person approves between the AI and the action; human on the loop, where a person supervises and can stop the action; human out of the loop, where the person is absent
The three levels of human involvement described by Human Rights Watch in 2012.

In practice, with a person in the loop the check comes before the action; with a person on the loop it comes during or after, and it only works if whoever supervises can really stop the system. The same report warns that, if supervision is too limited, a system with a person “on the loop” is in fact a system without people.

The distinction also has practical consequences. Article 14 of the EU AI Act requires high-risk AI systems to be designed so that people can oversee them effectively. Those who oversee them must stay aware of the tendency to over-rely on the output (the Act calls it “automation bias”), must be able to disregard or reverse that output and must be able to stop the system. The article covers high-risk systems, but for us the principle applies outside that scope too. Automation bias, in fact, is something developers will soon learn to recognize, and becoming aware of it is essential.

There is a risk tied to this bias that is worth knowing. A person in the loop only helps if they really check. If they approve every AI proposal without reading it, they are there, but they aren’t checking. That’s why a good rule goes beyond “someone approves” and says who that is, what they check and in which cases.

It’s the principle we design our tools around. In Gameheim the built-in AI writes drafts, explains and answers, but never submits, deletes or approves anything on its own. In practice:

  • a changelist description written by the AI is a draft, and it’s always a person who submits it;
  • changes to the shared instructions and memory of the team’s assistants, meaning what the AI learns about the project and saves for next time, are proposed by the assistant and approved by a person. AI-oriented tools should make approving instructions and memory a deliberate choice, not something the AI writes on its own and everyone forgets;
  • the explanation of a crash is marked as AI-generated and possibly wrong, so it should be taken as a first reading to be confirmed in the debugger.
Gameheim screenshot of a project’s memory: an entry proposed by an assistant via Claude Code sits under “Proposals to review”, waiting for a person’s approval
A change to the memory proposed by an assistant and waiting for approval, in an example project on Gameheim.

The agents of the local LLM app we’re about to release also ask for confirmation before acting.

How we see it at Virsion

Rules belong where the work happens. A policy written in a document nobody opens again isn’t much use. In Gameheim, using the built-in AI on project content and connecting external assistants such as Claude or ChatGPT are off in all the predefined roles, until someone decides to grant them. The only AI feature that starts on its own, the explanation of new crashes, can be switched off for each project. A connected assistant only sees the projects chosen by the person who connects it and can only do what that person can do. Changes made through an assistant show up in the activity log with the app’s name.

Say what leaves and what doesn’t. When Gameheim’s AI explains an Unreal crash it receives the report fields, such as the crash type and the first lines of the callstack, but never the minidump, the game log or the machine ID. And what a team sends to the built-in AI, together with what the AI answers, is never used to train models, ours or anyone else’s.

Be honest about the limits. No tool can stop a person from pasting data into an external chatbot. Gameheim governs the AI that goes through its own tools, not the AI outside it. That’s why a written, shared rule is still needed, and why the approved path has to be the convenient one, because when the right way is also the easiest, fewer people work around it.

An AI policy fits on one page

An AI policy can fit in a few lines, as long as it answers six questions: which tools, who uses them, what data they can see, what the AI can do on its own, who checks the result, what gets recorded.

The six questions of an AI policy: which tools, who uses them, what data they can see, what the AI can do on its own, who checks the result, what gets recorded
The six questions an AI policy has to answer.

For game studios we’ve written a guide with a one-page template to copy and the Gameheim settings for each answer: AI policy template for game studios.

Tools will keep changing and will become ever more capable. If only those who feel like it check, the work to be done isn’t on the tool: it’s on the process around it and on training people.