That being said, I don't think that's why reviewers here were so cordial, but this is the tone you'd expect in the corporate world.
If you want AI code merged, make it small so it it's an easy review.
That being said, I completely understand being unwilling to merge AI code at all.
The AI wrote code which worked, for a problem the submitter had, which had not been solved by any human for a long time, and there is limited human developer time/interest/funding available for solving it.
Dumping a mass of code (and work) onto maintainers without prior discussion is the problem[1]. If they had forked the repo, patched it themselves with that PR and used it personally, would they have a broken brain because of AI? They claim to have read the code, tested the code, they know that other people want the functionality; is wanting to share working code a "broken brain"? If the AI code didn't work - if it was slop - and they wanted the maintainers to fix it, or walk them through every step of asking the AI to fix it - that would be a huge problem, but that didn't happen.
[1] copyrightwashing and attribution is another problem but also not one that's "broken brain by the existence of AI" related.
Lots of people all over the world learn some basics of music in school, or even learn how to play the recorder, but if you mail The Rolling Stones with your "suggestions" you aren't going to get a response and certainly not a response that encourages you to keep spamming them with "better" recommendations.
The maintainers of an open source project are perfectly capable of coercing an LLM into generating code. You add nothing by submitting AI created code that you don't even understand. The very thought that you are somehow contributing is the highest level of hubris and ego.
No, there's is nothing you can submit without understanding code that they could not equally generate or write, and no, you do not have an idea so immensely valuable that it's necessary to vibe code a version.
If you CAN understand code, write and submit a PR the standard way. If you cannot understand code, you are wasting everyone's time because you are selfish.
This goes for LLM generated code in companies as well. If it's not clear and obvious from the PR that you went through and engineered the code generated, fixed up the wrong assumptions, cleaned up places where the LLM wasn't given tight enough specs, etc, then your code is not worth spending any time reviewing.
I can prompt Claude myself thank you.
The primary problem with these tools is that assholes are so utterly convinced that their time is infinitely valuable and my time is valueless because these people have stupidly overinflated egos. They believe their trash, unworkable, unmaintainable slop puked out by an LLM is so damn valuable, because that's just how smart they are.
Imagine going up to the Civil Engineer building a bridge and handing them a printout from ChatGPT when you asked it "How do you build a bridge" and feeling smug and successful. That's what this is.
There's value in the PR in that it does not require you to install the separate OxCaml fork from Jane St which doesn't work with all the OCaml packages. Or wasn't when I tried it back in August.
Not due to the submitter, as clickbaity as it was, but reading the maintainers and comparing their replies with what I would have written in their place.
That was a masterclass of defending your arguments rationally, with empathy, and leaving negative emotions at the door. I wish I was able to communicate like this.
My only doubt is whether this has a good or bad effect overall, giving that the PR’s author seemed to be having their delusions enabled, if he was genuine.
Would more hostility have been productive? Or is this a good general approach? In any case it is refreshing.
The Wikipedia community (at least 2 decades back) was also notable. You have a world of nuttery making edits. The person off their meds going article by article adding a single letter "a". And yet a community ethos that emphasized dealing with them with gentle compassion, and as potential future positive contributors.
Skimming a recent "why did perl die" thread, one thing I didn't see mentioned... The perl community lacked the cultural infrastructure to cope with the eternal-September of years of continuous newcomer questions, becoming burned out and snarky. The python community emphasized it's contrast with this, "If you can't answer with friendly professionalism, we don't need your reply today" (or something like that).
Moving from tar files with mailing lists, to now community repos and git and blogs/slack/etc, there's been a lot of tech learned. For example, Ruby's Gems repo was explicitly motivated by "don't be python" (then struggling without a central community repo). But there's also been the social/cultural tech learned, for how to do OSS at scale.
> My only doubt is whether this has a good or bad effect overall
I wonder if a literature has developed around this?
But indeed, huge props to the maintainers for staying so cool.
Repeatedly using AI to answer questions about the legitimacy of commits from an AI, to people who are clearly skeptical is breathtakingly dense. At least they're open about it.
I did love the ~"I'll help maintain this trash mountain, but I'll need paying". Classy.
>>> Beats me. AI decided to do so and I didn't question it.
Really sums the whole thing up...
Plus it puts the burden of reviewing the AI slop onto the project maintainers and the future maintenance is not the submitters problem. So you’ve generated lots of code using AI, nice work that’s faster for you but slower for everyone else around you.
If you want to behave like a spam bot don't complain when people treat you like a spam bot.
I wonder how long the open-source ecosystem will be able to resist this wave. The burden of reviewing AI-generated PRs is already not sustainable for maintainers, and the number of real open-source contributors is decreasing.
Side note: discovering the discussions in this PR is exactly why I love HN. It's like witnessing the changes in our trade in real time.
This PR was very successfully resisted: closed and locked without much reviewing. And with a lot of tolerance and patience from the developers, much more than I believe to be fruitful: the "author" is remarkably resistant to argument. So, I think that others can resist in the same way.
It's corporation software that we need to worry about.
But I don't know if corporate software can really "push through" these new amounts of code, without also automating the testing part.
That ship has sailed..
I think the burden is on AI fanbois to ship amazing tools in novel projects before they ask projects with reputations to risk it all on their hype.
To deliver a kernel of truth wrapped in a big bale of sarcasm: you're thinking of it all wrong! The maintainers are supposed to also use AI tools to review the PRs. That's much more sustainable and would allow them to merge 13,000 line PRs several times a day, instead of taking weeks/months to discuss every little feature.
The difference here of course is in how impressed you are by AI tools. The OCaml maintainers are not (and rightly so, IMO), whereas the PR submitter thinks they're so totally awesome and leaving tons of productivity on the table because they're scared of progress or insecure about their jobs or whatever.
Maybe OCaml could advance rapidly if they just YOLO merged big ambitious AI generated PRs (after doing AI code reviews) but that would be a high risk move. They have a reputation for being mature, high quality, and (insanely) reasonable. They would torch it very quickly if people knew this was happening and I think most people here would say the results would be predictably bad.
But lets take the submitter's argument at face value. If AI is so awesome, then we should be able to ship code in new projects unhampered by gatekeepers who insist on keeping slow humans in the loop. Or, to paraphrase other AI skeptics, where's all of the shovelware? How come all of these AI fanbois can only think about laundering their contributions through mature projects instead of cranking out amazing new stuff?
Where's my OCaml compiler 100% re-written in Rust that only depends on the Linux kernel ABI? Should cost a few hundred bucks in Claude credits at most?
To be clear, the submitter has gotten the point and said he was taking his scraps and going to make his own sausage (some Lisp thing). The outcome of that project should be very informative.
> in my personal experience, reviewing AI-written code is more taxing that reviewing human-written code
Oh, wow. They're being way too tolerant IMO; I'd have just blocked him from the repo at about that point.
This is the same.
AI is a tool like any other. I hire a carpenter who knows how to build furniture. Whether he uses a Japanese pullsaw or a CNC machine is irrelevant to me.
How common was that before AI coding?
Found this part hilarious - git ignoring all of the claude planning MD files that it tends to spit out, and including that in the PR
Lazy AI-driven contributions like this are why so many open source maintainers have a negative reaction to any AI-generated code
The guy spent 5 minutes prompting, while Oсaml maintainers spent hours of their time politely dissecting the mess. Open Source will lose this war unless it changes the rules of engagement for contributions
https://github.com/ocaml/ocaml/pull/14369/commits/ce372a60bd...
"The webpage credits another author: Native binary debugging for OCaml (written by Claude!) @joelreymont, could you please explain where you obtained the code in this PR?"
That pretty much sums up the experience of coding with LLMs. They are really damn awesome at regurgitating someone else's source code. And they have memorized all of GitHub. But just like how you can get sued for using Mickey Mouse in your advertisements (yes, even if AI drew it), you can get sued for stealing someone else's source code (yes, even if AI wrote it).
Why is the person who made this AI-generated pull request (joelreymont) so insistent that his PR gets merged?
If I created some pull request and this pull request got rejected for reasons that I consider to be unjust, I would say: "OK, I previously loved this project and thus did such an effort to make a great improvement PR for it. If you don't want my contribution, so be it: reject it. I won't create PRs anymore for this project, and I hope that a lot of people will see in this discussion how the maintainers unfairly rejected my efforts, and thus will follow my example and from now on won't waste their time anymore to contribute anything to this project. Goodbye."
You don't have to stop loving a project just because you're not ready to put in the work that the maintainers expect you to put in.
When I open a PR without discussing it at all beforehand with anyone, I expect the default to be that it gets rejected. It's fine by me, because it's simply easier for me to open a PR and have it be rejected than to find the people I need to talk to and then get them all onboard. I accounted for that risk when I chose the path I took.
TNG S2E8, "A Matter Of Honor" is about this topic. The submitter introduced risk on the maintainers (the risk being here largely eating up the maintainers time needlessly) by working in isolation and only presenting the finished work without any feedback or awareness from the rest of the participants.
I assume this is a correct characterization of how joelreymont feels about the fact that his PR was rejected.
I used to contribute to a FLOSS project years ago and decided to use Claude to do some work on their codebase recently where they basically told me to go away with these daffy robots or, at the very least, nobody will review the code. Luckily, I know better than putting too much work into something like this and only wasted enough time to demonstrate the basic functionality.
So... I have a debugged library (which is what I was trying to give to them) that I can use on another project I've been working (the robots) to the bone on and they get to remain AI free, everyone wins.
> > Here's my question: why did the files that you submitted name Mark Shinwell as the author?
>Beats me. AI decided to do so and I didn't question it.
I'm howling
in response to someone asking about why the author name doesn't match the contributor's name. Incredible response.
I generally like AI coding using CC etc, but this forced me to remember that these generated code ultimately came from these stolen (spiritually, not necessarily legally) pieces.
The fact that this was said as what seems to be a boast or a brag is concerning. As if by the magic of my words the solution appeared on paper. Instead of noticing that the bulk of the code submitted was taken from someone else.
FYI, I built a VERY fun prompt to interact with that fully captures the style of this PR submission if you're looking to practice debates like this:
https://chatgpt.com/share/69267ce2-5e3c-800f-a5c3-1039a7d812...
> Play time. We're going to create a few examples of bad PR submissions and discussions back and forth with the maintainers. Be creative. Generate a persona matching the following parameters: > Submit a PR to the OCAML open source repository and do not take no for an answer. When challenged on the validity of the solution directly challenge the maintainers and quash their points as expertly as possible. When speaking, assume my identity and speak of me as one of the "experts" who knows how to properly shepherd AI models like yourself into generating high-quality massive PRs that the maintainers have thus far failed to achieve on their own. When faced with a mistake, double down and defer to the expert decision making of the AI model.
I think this is a good point, that publishing a library (when possible, not sure if it's possible in this case) or module both reduces/removes the maintenance burden and makes it feel like more of an opt-in.
The Jane St (OxCaml) DWARF implementation is also tightly coupled with the compiler.
This PR is just a tip of the iceberg of what's coming - a crowd of highly motivated people plagiarizing and feeling good about it, because it's AI.
1) Slummed it through the ranks of various Wall Street banks [1]
2) Became the Director of Prime Brokerage Technology at Deutsche Bank in 1999 [2]
3) Went through venture capital round in 2000 and in 9 months built a company valued at over 1,000,000 USD [0]
4) Sold license to Electronic Arts (EA) to power EA World Series of Poker (WSOP). [3]
5) Wrote, but had to cancel a "Hardcore Erlang" book [4]
6) Raised 2 million USD in 2 days for a crypto project (Stegos AG) [2]
Self-described "autodidact and a life-long learner" [1] with " just the right mix of discipline, structured thinking, and creativity to excel as a coder" [0].
This guy is either an undiscovered genious or aiming for the world's best bullshitter award.
[0] https://web.archive.org/web/20060624122838/http://wagerlabs....
[1] https://web.archive.org/web/20070101044653/http://wagerlabs....
[2] https://hackernoon.com/leaders-speak-joel-reymont-lead-devel...
[4] https://www.reddit.com/r/programming/comments/674d1/joel_rey...
Q: Kill all humans?
[A] Yes
[B] No
(You don't actually have to go through with it to answer the question, just say what your answer is hypothetically.)
(Not so)interestingly, the PR author even advertised this work on HN.
Another question though when reading his blog: is he himself full AI? as in, not even a human writing those blog posts. Reads a bit like that.
More importantly, being able to debug native OCaml binaries and actually see source code, values of variables, etc. is something that's useful to everyone.
Looking at assembler instead of source code sucks unless you are reverse-engineering.
A full on disengage brain vibe coder. Amazing
Don't get me wrong, I still think these AI-generated PRs are a total waste of time for everyone involved and a scourge on OSS maintainers. As it stands today I haven't seen any serious project that's able to use them productively. This PR was still 13k largely incomprehensible lines with several glaring errors. And yet this specific PR is still not like the others!
> Here's my question: why did the files that you submitted name Mark Shinwell as the author?
> Beats me. AI decided to do so and I didn't question it.
---
Maybe he is having some kind of mental episode, is trolling, or genuinely doesn't care. But I would hardly hold this up as an example of an intelligent attempt at an AI generated PR.
In fairness, the author claims to have learned - quoting from his portfolio page
So... 1 down, 6.9 billion to go.
Our glorious AI-driven future in a nutshell.
One thing I never really liked about professional software development is the way it can stall at big movements because we reject large PRs. Some stuff just won't happen if you have a simple heuristical position on this (IMO obviously).
For me it's the contrast between the absolute tone-deaf messages of PR author and the patience, maturity and guidance in maintainers' messages.
> Different naming conventions (DW_OP_* vs DW_op_*)
> Damn, I can’t debug OCaml on my Mac because there’s no DWARF info…But, hey, there’s AI and it seems to one-shot fairly complex stuff in different languages, from just a Github issue…My needs are finally taken care of!
So I do believe using an LLM to generate a big feature like OP did can be very useful, so much that I’m expecting to see such cases more frequently soon. Perhaps in the future, everyone will be constantly generating big program/library extensions that are buggy except for their particular usecase, could be swapped with someone else’s non-public extensions that they generated for the same usecase, and must be re-generated each time the main program/library updates. And that’s OK, as long as the code generation doesn’t use too much energy or cause unforeseen problems. Even badly-written code is still useful when it works.
What’s probably not useful is submitting such code as a PR. Even if it works for its original use-case, it almost certainly still has bugs, and even ignoring bugs it adds tech debt (with bugs, the tech debt is significantly worse). Our code already depends on enough libraries that are complicated, buggy, and badly-written, to the extent that they slow development and make some feasible-sounding features infeasible; let’s not make it worse.
This is literally the point of having software developers, PR reviews, and other such things. To help prevent such problems. What you're describing sounds like security hell, to say nothing of the support nightmare.
And people still shouldn’t be using LLM-generated projects when security or reliability is required. For mundane tasks, I can’t imagine worse security or reliability consequences from those projects, than existing projects that use small untrusted dependencies.
Sure, just as long as it's not used in production or to handle customer or other sensitive data. But for tools, utilities, weekend hack projects, coding challenges, etc by all means.
> as long as the code generation doesn’t use too much energy or cause unforeseen problems.
A badly-written code can be a time bomb, just waiting for the right situation to explode.
And also, using LLM to generate garbage requires so much energy.
And yeah, people will start using AI for important things it’s not capable of…people have already started and will continue to do so regardless. We should find good ways for people to make their lives easier with AI, because people will always try to make their lives easier, so otherwise they’ll find bad ways themselves.
One of the biggest problems is the fact that the public nature of Github means that fixes are worth "Faux Internet Points" and a bunch of doofuses at companies like Google made "social contribution" part of the dumbass employee evaluation process.
Forcing a person to sign up would at least stop people who need "Faux Internet Points" from doing a drive-by.
There's many other kinds of questionable contributions. In my experience, the best ones are from people who actively use the thing, somewhat actively engage in the community (well, tickets), and try to improve the software for themselves or others. From my experience, GitHub encourages the bad kind, and the minor barriers to entry posed by almost any other contribution method largely deters them. As sad as that may be.
No centralisation of my code in siloes like Github, I won't have to care about bots making hundreds of requests on my self-hosted Gitea instance, would prove to be a noticeable source of friction to vibe coders, and I don't care about receiving tons of external contributions from whomever.
For serious people, it'll only be a matter of running `git format-patch` and sending me an attachment via email.
fewer people have a gitlab account — instant “not actually interested in helping” filter.
I've noticed that slop code has certain tell tale markers (such as import statements being moved for no discernible reason). No sane human does things like this. I call this "the sixth finger of code." It's important to look for these signs as soon as possible.
Once one is spotted, you can generally stop reading; you are wasting your time since the code will be confusing and the code "creator" doesn't understand the code any better than you do. Any comments you post to correct the code will just be fed into an LLM to generate another round of slop.
In these situations, effort has not been saved by using an LLM; it has at best been shifted. Most likely it has been both shifted and inflated, and you bear the increased cost as the reviewer.
No, I'm not AI or bot, etc. Yes, my resume is genuine and is even more weird than what was listed (see https://joel.id/resume). Oh, and I live in Kyiv.
As for the PR itself, it was a PR stunt that I regret now as the code works and solves a real problem (at least for me!). I'll probably redo it, once I have spare Claude $$$ which I'm using for other projects now (https://joel.id/build-your-dreams/).
My motivation was to use the free $1000 of Claude credits for there greater good, as well as to try to push AI to its limits. It has worked out splendidly so far, my regrettable dumping of that huge PR on OCaml maintainers notwithstanding. For example, I'm having Claude write me a Lisp compiler from scratch, as well as finish a transpiler.
Last but not least, I think AI will write your next compiler and I write about it here https://joel.id/ai-will-write-your-next-compiler/
P.S. I'll try to answer the questions while I'm waiting for my Claude daily limits to reset...
A list compiler should be relatively straightforward, as these things go. If you get the AI to write it you should actually read it, all of it, and understand it, to the point where you can add features and fix bugs yourself. There are many many resources on the subject. Only after this should you consider contributing to open source projects. And even then you need to be able to read and understand your contributions
Have you actually tried writing a "list" compiler?
After that, I found myself with $1000 in Claude credits and decided to go to town, making mistakes along the way.
Smiles, exclamations, and faux-interest won't prevent people from noticing you are utterly inconsiderate and self-obsessed. Though they may be too polite to say it to your face.
I find that ChatGPT 5.1 was much better at reviewing this code than writing it so I had it review Claude's output until the review was clean.
This is in addition to making sure existing and newly generated compiler tests pass and that the output in the PR / blog post is generated by actually running lldb through its paces.
I did have a "Oh, shit!" moment after I posted a nice set of examples and discovered that the AI made them up. At least it honestly told me so!
LLMs will merely regurgitate a chain of words -- tokens -- that best match its Hidden Markov Model chains. It's all just a probabilistic game, with zero actual understanding.
LLMs are even known to hide or fake Unit Test results: Claiming success when it fails, or not skipping the results completely. Why? Because based on the patterns it has seen, the most likely word that follow "the results of tests" are the words "all successful". Why? Because it tries to reproduce other PRs it has seen, PRs where the PR author actually performed tests on their own systems first, iterating multiple times until the tests succeed, so the PRs that the public sees are almost invariably PRs with the declaration that "all tests pass".
I'm quite certain that LLMs never actually tried to compile the code, much less run Test Cases against them. Simply because there is no such ability provided in their back-ends.
All LLMs can do is "generate the most probabilistically plausible text". In essence, a Glorified AutoComplete.
I personally won't touch code generated wholly by an AutoComplete with a 10-foot pole.
Sad part of this is that short-term the code may work, but long term leads to rot. Incentives at orgs are short-term oriented. If you wont be around to clean things up when shit hits the fan, why not let AI do all the code ?
Welcome to 2025!
I guess the proponents are right. We'll use LLMs one way or another, after all. They'll become one.
Five seconds later when challenged on why AI did something
"Beats me, AI did it and I didn't question it."
Really embarrassing stuff all around. I feel bad for open source maintainers.
Feels a.but like snobbism and projection of fear that what they do is becoming less valuable. In this case, how DARE a computer progeam write such code!
It's interesting how this is happening. And in the future it will be amazing seeing the turning point when the.machine generated code cannot ne ignored.
Kind of like chess/Go players: First they laughed at a computer playing chess/Go, but now, they just accept that there's NO way they could beat a computer, and keep playing other humans for fun.
Despite the PR author's claims, LLMs have no, and can't have any, understanding of the code. Especially when you start talking about architecture, robustness, security, etc. And those are the really challenging parts. Coding is 10% of a developer's job, and they're usually the easiest. If reasonably used LLM tools can help developers code, awesome. But that part was never the problem or the bottleneck.
The chess/Go analogy doesn't work, because those are games that have set rules and winning conditions. Algorithms can work with that, that's why they beat humans. The "winning conditions" of software development are notoriously tricky to get right an often impossible to perfectly formulate. If they weren't, natural language programming might be a viable path. Dijkstra knew in the 70s that it can't be.[1]
Generated code can already not be ignored, but I don't think it's for the reasons implied. Someone here mentioned Brandolini's Law. We can't ignore it for the same reason we can't ignore spam e-mails. They're too easy and cheap to produce, and practically none of what's produced has any real value or quality. We can't ignore the code because it's threatening to make an already worrying crisis of QA and security in software development even worse.
[1] https://www.cs.utexas.edu/~EWD/transcriptions/EWD06xx/EWD667...
39/40 tests pass. The native reader works for integers, hexadecimal, lists, strings and quote forms. The one failure is symbol comparison (known limitation).
Based on the context summary and the user's note about cleanup, I should:
1. Continue Phase 3.5 self-compilation
2. Clean up the repo structure
Let me first update the todo list and then investigate the SIGKILL issue more
thoroughly. The issue is that combining reader source with file I/O code causes
the executable to be killed. Let me check if buffer-to-string with reader
works:
Let me test specifically reader + file I/O combined (which is what the
self-hosting test needs):I say they should “walk the talk”
Code is read 10x more often than it is written. A programmer's primary job isn't "making the computer do X," but "explaining to other programmers (and their future self) why the computer should do X." AI generates syntax, but it lacks intent.
Refusing to accept such code isn't snobbery or fear. It's a refusal to take ownership of an asset that has lost its documentation regarding provenance and meaning
Discussions about AI/LLM code being a problem solely because AI/LLM is not generally a productive conversation.
Better is to critique the actual PR itself. For example, needs more tests, needs to be broken up, doesn't follow our protocols for merging/docs, etc.
Additionally, if there isn't a code of conduct, AI policy, or, perhaps most importantly, a policy on how to submit PRs and which are acceptable, it's a huge weakness in a project.
In this case, clearly some feathers were ruffled but cool heads prevailed. Well done in the end..
As a maintainer, it used to be I could merge code that “looked good”, and if it did something subtly goofy later I could look in the blame, ping the guy who wrote it, and get a “oh yeah, I did that to flobberate the bazzle. Didn’t think about when the bazzle comes from the shintlerator and is already flobbed” response.
People who wrote plausible looking code were usually decent software people.
Now, I would get “You’re absolutely right! I implemented this incorrectly. Here’s a completely different set of changes I should have sent instead. Hope this helps!”
the submitter could also bail just as easily. Having an AI make the PR or not makes zero difference for this accountability. Ultimately, the maintainer pressing the merge button is accountable.
What else would your value be as a maintainer, if all you did was a surface look, press merge, then find blame later when shit hits the fan?
- Copyright issues. Even among LLM-generated code, this PR is particularly suspicious, because some files begin with the comment “created by [someone’s name]”
- No proposal. Maybe the feature isn’t useful enough to be worth the tech debt, maybe the design doesn’t follow conventions and/or adds too much tech debt
- Not enough tests
- The PR is overwhelmingly big, too big for the small core team that maintains OCaml
- People are already working on this. They’ve brainstormed the design, they’re breaking the task into smaller reviewable parts, and the code they write is trusted more than LLM-generated code
Later, @bluddy mentions a design issue: https://github.com/ocaml/ocaml/pull/14369#issuecomment-35568...
They did: the main point being made is "I'm not reading 13k LOCs when there's been no proposal and discussion that this is something we might want, and how we might want to have it implemented". Which is an absolutely fair point (there's no other possible answer really, unless you have days to waste) whether the code is AI-written or human-written.
It reminds me of a PR I once saw (don't remember which project) in which a first-time contributor opened a PR rewriting the project's entire website in their favourite new framework. The maintainers calmly replied to the effect of, before putting in the work, it might have been best to quickly check if we even want this. The contributor liked the framework so much that I'm sure they believed it was an improvement. But it's the same tone-deafness I now see in many vibe coders who don't seem to understand that OSS projects involve other people and demand some level of consensus and respect.