If we're talking about changing the industry, we should see some clear evidence of that. But despite extensive searching myself and after asking many proponents (feel free to jump in here), I can't find a single open source codebase, actively used in production, and primarily maintained and developed with AI. If this is so foundationally groundbreaking, that should be a clear signal. Personally, I would expect to see an explosion of this even if the hype is taken extremely conservatively. But I can't even track down a few solid examples. So far my searching only reveals one-off pull requests that had to be laboriously massaged into acceptability.
That’s a great point...and completely incompatible with my pitch deck. I’m trying to raise a $2B seed round on vibes, buzzwords, and a slightly fine-tuned GPT-3.5.. You are seriously jeopardizing my path to an absurdly oversized yacht :-))
That's because using AI to write code is a poor application of LLM AIs. LLMs are better suited to summary, advice, and reflection than forced into a Rube Goldberg Machine. Use your favorite LLM as a Socratic advisor, but not as a coder, and certainly not as an unreliable worker.
But for helping me as a partner in neurophilosophy conversations Claude is unrivaled even compared to my neurophilosophy colleagues—speed and the responsivness is impossible to beat. LLMs are at pushing me to think harder. They provides the wall against which to bounce ideas, and those bounces often come from surprising and helpful angles.
Then why is that exact usecase being talked about ad nauseam by many many many "influencers", including "big names" in the industry? Why is that exact usecase then advertised by leading companies in the industry?
Can you give an example of what you mean by this?
This popular repo (35.6k stars) documents the fraction of code written by LLM for each release since about a year ago. The vast majority of releases since version 0.47 (now at 0.85) had the majority of their code written by LLM (average code written by aider per release since then is about 65%.)
So a project that mostly is maintained by people who care about their problem/code (OSS) would be weird to be "primarily maintained by AI" in a group setting in this stage of the game.
There's no question that the predictions around LLMs are shaking up the industry - see mass layoffs and offers for 8 figures to individual contributors. The question is will it materially change things for the better? no idea.
Not only is that a could, I'd argue they already are. The huge new "premier" models are barely any better than the big ticket ones that really kicked the hype into overdrive.
* Using them as a rubber duck that provides suggestions back for IT problems and coding is huge, I will fully cosign that, but it is not even remotely worth what OpenAI is valued at or would need to charge for it to make it profitable, let alone pay off it's catastrophic debt. Meanwhile every other application is a hard meh.
* The AI generated video ads just look like shit and I'm sorry, call me a luddite if you will, but I just think objectively less of companies that leverage AI video/voices/writing in their advertisements. It looks cheap, in the same way dollar store products have generic, crappy packaging, and makes me less willing to open my wallet. That said I won't be shocked at all if that sticks around and bolsters valuations, because tons of companies worldwide have been racing to the bottom for decades now.
* My employer has had a hard NO AI policy for both vetting candidates and communicating with them for our human resources contracting and we've fired one who wouldn't comply. It just doesn't work, we can tell when they're using bots to review resumes because applicants get notably, measurably worse.
LLMs are powerful tools that have a place, but there is no fucking UNIVERSE where they are the next iPhone that silicon valley is utterly desperate for. They just aren't.
The iPhone and subsequent growth of mobile (and the associated growth of social media which is really only possible in is current form with ubiquitous mobile computing) are evidence it did change everything. Society has been reshaped by mobile/iPhone and its consequences.
NFTs were never anything, and there was never an argument they were. The were a financial speculative item, and it was clear all the hype was due to greater fools and FOMO. To equate those two is silly. That's like arguingsome movie blockbuster like Avengers Endgame was going to "change everything" because it was talked about and advertised. It was always just a single piece of entertainment.
Finally for LLMs, a better comparison for them would have been the 80's AI winter. The question should be "why will this time not be like then?" And the answer is simple: If LLMs and generative AI models never improve an ounce - If they never solve another problem, nor get more efficient, nor get cheaper - they will still drastically change society because they are already good enough today. They are doing so now.
Advertising, software engineering, video making. The tech is already for enough that it is changing all of these fields. The only thing happening now is the time it takes for idea diffusion. People learning new things and applying it are the slow part of the loop.
You could have made your argument pre-chatgpt, and possibly could have made that argument in the window of the following year or two, but at this point the tech at the level to change society exists, it just needs time to spread. All it need are two things: tech stays the same, prices roughly stay the same. (No improvements required)
Now there still is a perfectly valid argument to make against the more extreme claims we hear of: all work being replaced..., and that stuff. And I'm as poorly equipped to predict that future as you (or anyone else) so won't weigh in - but that's not the bar for huge societal change.
The tech is already bigger than the iPhone. I think it is equivalent to social media, (mainly because I think most people still really underestimate how enormous the long term impact of social media will be in society: Politics, mental health, extremism, addiction. All things they existed before but now are "frictionless" to obtain. But that's for some other post...).
The question in my mind is will it be as impactful as the internet? But it doesn't have to be. Anything between social media and internet level of impact is society changing. And the tech today is already there, it just needs time to diffuse into society.
You're looking at Facebook after introducing the algorithm for engagement. It doesn't matter that society wasn't different overnight, the groundwork had been laid.
LLMs in their current state have integrated into the workflows for many, many IT roles. They'll never be niche, unless governing bodies come together to kill them.
> I can't find a single open source codebase, actively used in production, and primarily maintained and developed with AI
Straw man argument - this is in no way a metric for validating the power of LLMs as a tool for IT roles. Can you not find open source code bases that leverage LLMS because you haven't looked, or because you can't tell the difference between human and LLM code?
> If this is so foundationally groundbreaking, that should be a clear signal.
As I said, you haven't been paying attention.
Denialism - the practice of denying the existence, truth, or validity of something despite proof or strong evidence that it is real, true, or valid
The money and the burden of proof are on the side of the pushers. If LLM code is as good as you say it is, we won't be able to tell that it's merged. So, you need to show us lots of examples of real world LLM code that we know is generated, a priori, to compare
So far most of us have seen ONE example, and it was that OAuth experiment from Cloudflare. Do you have more examples? Who pays your bills?
1) https://github.com/domino14/Webolith/pull/523/files (Yes, the CSS file sucks. I tried multiple times to add dark mode to this legacy app and I wasn't able to. This works, and is fine, and people are using it, and I'm not going to touch it again for a while)
2) https://github.com/domino14/macondo/pull/399 - A neural net for playing Scrabble. Has not been done before, in at least an open-source way, and this is a full-fledged CNN built using techniques from Alpha Zero, and almost entirely generated by ChatGPT o3. I have no idea how to do it myself. I've gotten the net to win 52.6% of its games against a purely static bot, which is a big edge (trust me) and it will continue to increase as I train it on better data. And that is before I use it as an actual evaluator for a Monte Carlo bot.
I would _never_ have been able to put this together in 1-2 weeks when I am still working during the day. I would have had to take NN classes / read books / try many different network topologies and probably fail and give up. Would have taken months of full-time work.
3) https://github.com/woogles-io/liwords/pull/1498/files - simple, but one of many bug fixes that was diagnosed and fixed largely by an AI model.
That is an exaggeration, it is integrated into some workflows, usually in a provisional manner while the full implications of such integrations are assessed for viability in the mid to long term.
At least in the fields of which i have first hand knowledge.
> Straw man argument - this is in no way a metric for validating the power of LLMs as a tool for IT roles. Can you not find open source code bases that leverage LLMS because you haven't looked, or because you can't tell the difference between human and LLM code?
Straw man rebuttal, presenting an imaginary position in which this statement is doesn't apply doesn't invalidate the statement as a whole.
> As I said, you haven't been paying attention.
Or alternatively you've been paying attention to a selective subset of your specific industry and have made wide extrapolations based on that.
> Denialism - the practice of denying the existence, truth, or validity of something despite proof or strong evidence that it is real, true, or valid
What's the one where you claim strong proof or evidence while only providing anecdotal "trust me bro" ?
Having a niche is different from being niche. I also strongly believe you overstate how integrated they are.
> Straw man argument - this is in no way a metric for validating the power of LLMs as a tool for IT roles. Can you not find open source code bases that leverage LLMS because you haven't looked, or because you can't tell the difference between human and LLM code?
As mentioned, I have looked. I told you what I found when I looked. And I've invited others to look. I also invited you. This is not a straw man argument, it's making a prediction to test a hypothesis and collecting evidence. I know I am not all seeing, which is why I welcome you to direct my eyes. With how strong your claims and convictions are, it should be easy.
Again: You claim that AI is such a productivity boost that it will rock the IT industry to its foundations. We cannot cast our gaze on closed source code, but there are many open source devs who are AI-friendly. If AI truly is a productivity boost, some of them should be maintaining widely-used production code in order to take advantage of that.
If you're too busy to do anything but discuss, I would instead invite you to point out where my reasoning goes so horrendously off track that such examples are apparently so difficult to locate, not just for me, but for others. If one existed, I would additionally expect that it would be held up as an example and become widely known for it with as often as this question gets asked. But the world's full of unexpected complexities, if there's something that's holding AI back from seeing adoption reflected in the way I predict, that's also interesting and worth discussion.
As I stated, you haven't been paying attention.
For instance, one might point out that the tools for really GOOD AI code authoring have only been available for about 6 months so it is unreasonable to expect that a new project built primarily using such tools has already reached the level of maturity to be relied on in production.
I do however have time to put forth my arguments now that I use LLMs to make my job easier - if it weren't for them, I wouldn't be here right now.
