Perspective

The Biggest Obstacle to Autonomous AI Might Be Our Need for Control

Why trust, not technology, could become the biggest bottleneck in AI transformation.

Originally published on LinkedIn on 6 November 2026.

A person hesitates to release a slack tether attached to a calm, self-running network of connected nodes.

What if we're approaching AI transformation the wrong way?

Imagine an organization where AI agents can design, build, test, deploy, and continuously improve software with minimal human intervention. Now imagine that every important decision still requires a manager's approval. Have we actually transformed anything?

We've spent decades building management structures around human limitations: our capacity to process information, coordinate activities, make decisions, and execute work. But AI operates on a completely different timescale. Activities that previously required days or weeks can sometimes be completed in hours or minutes. And yet, we continue trying to organize and supervise AI as though it were another human team.

I believe this is one of the fundamental mistakes we risk making in AI transformation. The challenge isn't simply making AI more autonomous. It's learning how to build organizations that can trust autonomy.

When humans become the bottleneck

Work items flow quickly through connected stations, then queue at a gate where one person checks each item.

Coming from an Agile and Lean background, I've always been interested in understanding how work flows through an organization. Where are the bottlenecks? What creates delays? What activities genuinely add value?

Now consider an AI-powered software factory, like Black Box App Factory, the one I built for my own projects. An agent analyzes requirements, another develops features, while others run automated tests, simulate different user personas, evaluate quality, deploy changes, and monitor production. Potentially, this entire workflow could operate continuously, with very little human intervention. (Black Box App Factory still keeps a few explicit human validation points, and deciding where those should sit is exactly what this article is about.)

But if every feature requires a human to validate it, we've introduced a fundamental constraint. We've accelerated development without accelerating the rest of the system.

This is a classic Lean problem: optimizing individual activities doesn't necessarily improve the performance of the entire system. When one bottleneck disappears, another often becomes visible. In AI-driven organizations, human decision-making could increasingly become that bottleneck.

So should we remove humans from the process? I think that's the wrong question. The more interesting question is: at which level should humans intervene to create the most value?

From managing execution to designing systems

Traditionally, managers coordinate work, assign responsibilities, monitor progress, and validate decisions. But imagine supervising hundreds of AI agents making thousands of decisions every day. Micromanagement becomes impossible, and even if it were technically possible, it would defeat much of the purpose of automation.

This leads me to a different vision of management. Instead of managing individual activities, we should focus on designing the system in which those activities happen. We define the objectives, establish the boundaries, select meaningful metrics, identify risks, and design mechanisms to monitor and mitigate them. Then we progressively allow the system to operate autonomously.

In other words, we shift from controlling what happens to designing the conditions under which things happen.

This doesn't mean giving AI unlimited freedom. Quite the opposite. It means investing more effort upfront in understanding processes, defining governance, and building reliable feedback loops.

An autonomous system needs more than the ability to execute. It needs the ability to detect when something goes wrong, respond appropriately, and learn from the outcome. We also need independent verification mechanisms, because having one AI validate another doesn't automatically guarantee correctness. The real challenge is designing this entire ecosystem coherently.

Trust cannot be imposed. It must be built.

A person holds a small system closely, then loosens the hold, then steps back while a larger system runs on its own.

I've been experimenting with autonomous AI workflows in my own projects, and one thing I've learned is that you don't need to start with a fully autonomous system. Actually, I believe you shouldn't.

I started with small, manageable processes. Something I could understand, observe, and control. Then I progressively expanded them, introducing additional capabilities, connecting different activities, adding specialized agents, and gradually reducing the amount of direct supervision required. Over time, individual automations evolved into more complex workflows.

What's interesting is that this progressive approach doesn't only improve technical reliability. It also changes our relationship with the technology. At first, you verify almost everything. Then you realize that certain activities work reliably, so you start checking less frequently and give the system more room to operate. Progressively, you become comfortable delegating activities you previously wanted to supervise.

Trust becomes the result of experience, rather than a prerequisite for adoption.

I think this is one of the most important lessons for organizations undergoing AI transformation. We cannot simply introduce autonomous systems and expect employees or managers to trust them. We need to build that confidence progressively, through observation, measurement, experimentation, and continuous improvement.

It's a familiar approach for anyone working with Agile or Lean methodologies, and I believe it's particularly relevant to AI.

But what happens when AI makes a mistake?

This is where the discussion becomes more complicated. AI can make mistakes, and because it operates at much greater speed and scale, it can potentially amplify the consequences of those mistakes.

Imagine an autonomous development system introducing a security vulnerability, making an incorrect business decision, or consuming an unexpectedly large budget. By the time a human notices, significant damage might already have occurred. Some consequences may even be irreversible.

That's a legitimate concern, but I don't think the answer is necessarily to introduce human validation everywhere. Instead, we should ask how to design systems capable of detecting risks before they become incidents, limiting their potential impact, and correcting problems as quickly as possible.

That might involve independent monitoring agents, automated tests, spending limits, permission boundaries, anomaly detection, rollback mechanisms, and escalation rules. The appropriate safeguards depend on the consequences of failure.

But there's also a cultural dimension. In many organizations, when something goes wrong, the first question is often "Who made the mistake?" I think we should ask something different: "Why did our system allow this mistake to happen, and how can we prevent it from happening again?"

That's a fundamental principle of continuous improvement, and it applies whether the work was performed by a human or an AI agent.

Of course, legal and organizational accountability still matters. An AI agent cannot currently assume responsibility in the same way a person or company can. But being accountable for a system doesn't necessarily mean personally approving every decision it makes. We can remain responsible for objectives, governance, risk management, and outcomes while delegating execution.

That's an important distinction.

The real transformation is organizational, not technological

I suspect many companies will approach AI transformation primarily through productivity. They'll ask how much faster they can develop, how many activities they can automate, and how many working hours they can save. These are legitimate questions, but I believe they miss a much bigger opportunity.

AI isn't simply another tool that makes existing processes faster. It gives us an opportunity to fundamentally rethink how organizations operate.

Why do we have so many approval layers? Why are certain decisions centralized? Why do teams depend on managers to coordinate activities? Which constraints are genuinely necessary, and which ones exist because our organizations were designed around human capabilities?

I'm not suggesting that every hierarchy should disappear or that every decision should be delegated to AI. But I do believe we should challenge the assumption that our existing organizational structures remain appropriate when execution becomes increasingly autonomous.

And perhaps the biggest challenge won't be technological feasibility. It will be our capacity to change the way we think about control, responsibility, and trust.

A different role for humans

A person draws the boundaries and paths of an orderly network, like an architect.

This doesn't mean humans become irrelevant. Quite the opposite. I believe their role could become more focused on purpose, direction, creativity, and the design of systems.

Some people will want to define strategic objectives and let AI handle execution. Others will prefer to remain closely involved in building products, solving problems, or experimenting. And that's fine. There isn't necessarily one ideal level of human involvement.

The objective shouldn't be to remove humans, but to give them the freedom to participate where they genuinely want to contribute, without making their constant intervention a condition for the system to function.

And to achieve that, we need something more valuable than better prompts or more powerful models. We need organizations designed for autonomy.

So where do we start?

I don't believe there's a universal blueprint for an autonomous AI organization, but I strongly believe in a progressive approach. Understand your processes, identify bottlenecks, define objectives and meaningful success criteria, and start building small autonomous workflows. Introduce appropriate safeguards, measure performance, learn from failures, and gradually expand autonomy as confidence and reliability increase.

Not because AI needs the same working cycles as humans, but because organizations need to learn how to operate with a fundamentally different kind of intelligence.

Perhaps the future of management isn't about supervising increasingly powerful AI agents. Perhaps it's about designing systems that no longer require constant supervision.

And that brings me to a question I'd genuinely like to explore:

If an AI system consistently delivered better results than your team, what would it take for you to trust it enough to stop approving its decisions?

How this article was created: the ideas are mine, the words are ChatGPT's.

Jean-Philippe Arné
AI Transformation Lead
I lead AI transformation: challenging existing workflows, then building the automation where it belongs.
← Back to home