Large parts of human civilization rests on our ability to make something unreliable less unreliable through organisational structure and processes.
At the end of the day, if I am spending X$s for automation, I want to be able to sleep at night knowing my factory will not build a WMD or delete itself.
If its simply a tool that is a multiplier for experts, then do I really need it? How much does it actually make my processes more efficient, faster, or more capable of earning revenue?
There is a LOT that is forgiven when tech is new - but at some point the shiny newness falls off and it is compared to alternatives.
Review and oversight does address reliability directly, and hence why we make use of those in processes to improve the reliability of mechanical processes as well, and why they are core elements of AI harnesses.
> If its simply a tool that is a multiplier for experts, then do I really need it? How much does it actually make my processes more efficient, faster, or more capable of earning revenue?
You can ask the same thing about all the supporting staff around the experts in your team.
> There is a LOT that is forgiven when tech is new - but at some point the shiny newness falls off and it is compared to alternatives.
Only teams without mature processes are not doing that for AI today.
Most of the deployments of AI I work on are the outcome of comparing it to alternatives, and often are part of initiatives to increase reliability of human teams jut as much as increasing raw productivity, because they are often one and the same.
Yes and no. see next point.
> You can ask the same thing about all the supporting staff around the experts in your team.
I have a good idea of the shape of errors for a human based process, costing and the type of QA/QC team that has to be formed for it.
We have decades, if not centuries of experience working with humans, which LLMs are promising to be the equivalents/superiors of.
I think you and me, would both agree with the statement "use the right tool for the job".
However, the current hype cycle has created expectations of reliability from LLMs that drive 'Automated Intelligence' styled workflows.
On the other hand:
> part of initiatives to increase reliability of human teams
is a significantly more defensible uses of LLMs.
For me, most deployments die on the altar of error rates. The only people who are using them to any effect are people who have an answer to "what happens when it blows up" and "what is the cost if something goes wrong".
(there is no singular thread behind my comment. I think we probably have more in agreement than not, and its more a question of finding the precise words to declare the shapes we perceive.)
I moved this up top, because I agree, despite the length of the below:
> However, the current hype cycle has created expectations of reliability from LLMs that drive 'Automated Intelligence' styled workflows.
Because for a lot of things it works. Today. I have a setup doing mostly autonomous software development. I set direction. I don't even write specs. It's not foolproof yet by any means - that is on the edge of what is doable today. Dial it back just a little bit, and I have projects in production that are mostly AI written, that have passed through rigorous reviews from human developers.
The key thing is that you can't "vibecode" that. I'm sure we agree there.
There needs to be a rigorous process behind it, and I think we'll agree on that too.
Those processes are largely the same as the processes required for human developers. Only for human developers we leave a lot of that process "squishy" and under-specified.
We trust our human developers to mostly do the right thing, even though many don't, and to not need written checklists and controls, even though many do.
What is coming out of this is a start of systems that codify processes that are very much feels based with human teams. Partly because we still need to codify them for AI, but also because we can - most people wouldn't want to work in the kind of regimented environment we can enforce on AI.
Sure, there is a lot of hype from people who just want to throw random prompts at an LLM and get finished software out. That is idiocy. Even a super-intelligent future AI can't read minds.
But there are a lot of people building harnesses to wrap these LLMs in process and rigor to squeeze as much reliability as possible from them, and it turns out you can leverage human organisational knowledge to get surprisingly far in that respect.
So many applications of LLMs have even to start with deterministic brain when using a non-deterministic llm and then wonder why it’s not working.
