undefined | Better HN

0 pointscrazygringo4mo ago0 comments

> Humans are just better at communicating about their process.

I wish that were true.

In my experience, most of the time they're not doing the things you talk about -- major architectural decisions don't get documented anywhere, commit messages give no "why", and the people who the knowledge got socialized to in unrecorded conversations then left the company.

If anything, LLM's seem to be far more consistent in documenting the rationales for design decisions, leaving clear comments in code and commit messages, etc. if you ask them to.

Unfortunately, humans generally are not better at communicating about their process, in my experience. Most engineers I know enjoy writing code, and hate documenting what they're doing. Git and issue-tracking have helped somewhat, but it's still very often about the "what" and not the "why this way".

0 comments

recitedropper4mo ago

"major architectural decisions don't get documented anywhere" "commit messages give no "why""

This is so far outside of common industry practices that I don't think your sentiment generalizes. Or perhaps your expectation of what should go in a single commit message is different from the rest of us...

LLMs, especially those with reasoning chains, are notoriously bad at explaining their thought process. This isn't vibes, it is empiricism: https://arxiv.org/abs/2305.04388

If you are genuinely working somewhere where the people around you are worse than LLMs at explaining and documenting their thought process, I would looking elsewhere. Can't imagine that is good for one's own development (or sanity).

crazygringoOP4mo ago

I've worked everywhere from small startups to megacorps. The megacorps certainly do better with things like initial design documents that startups often skip entirely, but even then they're often largely out-of-date because nobody updates them. I can guarantee you that I am talking about common industry practices in consumer-facing apps.

I'm not really interested in what some academic paper has to say -- I use LLM's daily and see first-hand the quality of the documentation and explanations they produce.

I don't think there's any question that, as a general rule, LLM's do a much better job documenting what they're doing, and making it easy for people to read their code, with copious comments explaining what the code is doing and why. Engineers, on the other hand, have lots of competing priorities -- even when they want to document more, the thing needs to be shipped yesterday.

recitedropper4mo ago

Alright, I'm glad to hear you've had a successful and rich professional career. We definitely agree that engineers generally fail to document when they have competing priorities, and that LLMs can be of use to help offload some of that work successfully.

Your initial comment made it sound like you were commenting on a genuine apples-for-apples comparisons between humans and LLMs, in a controlled setting. That's the place for empiricism, and I think dismissing studies examining such situations is a mistake.

A good warning flag for why that is a mistake is the recent article that showed engineers estimated LLMs sped them up by like 24%, but when measured they were actually slower by 17%. One should always examine whether or not the specifics of the study really applies to them--there is no "end all be all" in empiricism--but when in doubt the scientific method is our primary tool for determining what is actually going on.

But we can just vibe it lol. Fwiw, the parent comment's claims line up more with my experience than yours. Leave an agent running for "hours" (as specified in the comment) coming up with architectural choices, ask it to document all of it, and then come back and see it is a massive mess. I have yet to have a colleague do that, without reaching out and saying "help I'm out of my depth".

1 more reply

satvikpendem4mo ago

Idealized industry practices that people wish to follow, but when it comes to meeting deadlines, I too have seen people eschew these practices for getting things out the door. It's a human problem, not one specific to any company.

recitedropper4mo ago

Yes I recognize that, for various reasons, people will fail to document even when it is a profesional expectation.

I guess in this case we are comparing an idealized human to an idealized AI, given AI has equally its own failings in non-idealized scenarios (like hallucination).

j / k navigate · click thread line to collapse

0 pointscrazygringo4mo ago0 comments

> Humans are just better at communicating about their process.

I wish that were true.

If anything, LLM's seem to be far more consistent in documenting the rationales for design decisions, leaving clear comments in code and commit messages, etc. if you ask them to.

0 comments

recitedropper4mo ago

"major architectural decisions don't get documented anywhere" "commit messages give no "why""

LLMs, especially those with reasoning chains, are notoriously bad at explaining their thought process. This isn't vibes, it is empiricism: https://arxiv.org/abs/2305.04388

crazygringoOP4mo ago

I'm not really interested in what some academic paper has to say -- I use LLM's daily and see first-hand the quality of the documentation and explanations they produce.

recitedropper4mo ago

1 more reply

satvikpendem4mo ago

recitedropper4mo ago

Yes I recognize that, for various reasons, people will fail to document even when it is a profesional expectation.

I guess in this case we are comparing an idealized human to an idealized AI, given AI has equally its own failings in non-idealized scenarios (like hallucination).

j / k navigate · click thread line to collapse