If I cannot guarantee outcomes (people usually get demoted or fired when they don’t exactly do what the boss says), then I need something in the loop which I can hold accountable to ensure I get the outcomes I want.
If an agent keeps giving me bad information or code, how do I demote or fire that agent if I am subjected to vendor lock-in? Everyone is using the same models and APIs across the board.
We need people for quality assurance even beyond AI developing cognitive flexibility on par with humans. I can hold a person, or group of people, accountable for their mistakes. I cannot hold AI agents or APIs accountable for shit.
Let’s also not forget that some mistakes have very little fault tolerance, and software we write has never been able meet that demand. See all the recent plane crashes, for example.
If human capabilities are the bottleneck, then those capabilities need to augmented.
A few points I mostly agree with, with one nuance:
Humans are in the loop today because accountability is clear. You can coach, discipline, replace, or escalate a person. You can’t meaningfully “hold an API responsible” in the same way.
But companies don’t always solve reliability with a person reviewing everything. Over time they often shift to process-based controls: stronger testing, monitoring, audits, fallback procedures, and contractual guarantees. That’s how they already manage critical dependencies they also can’t “fire” overnight (cloud services, core software vendors, etc.).
Vendor lock-in is real—but it’s also a choice firms can mitigate. Multi-vendor options, portability clauses, and keeping an exit path in the architecture are basically the equivalent of being able to replace a bad supplier.
High fault-tolerance domains will keep humans involved longer. The likely change is not “no humans,” but fewer humans overseeing more automated work, with people focused on exceptions, risk ownership, and sign-off in the most sensitive areas.
So yes: we need humans where the downside is serious and someone has to own the risk. My claim is just that as reliability and controls improve, organisations will try to shrink the amount of human review, because that review starts to look like the expensive part of the system.
The problem being that AI can scale faster, and compound problems exponentially, than any human can review or oversee things. There need to be better control mechanisms.
