undefined | Better HN

0 pointstaurath1mo ago0 comments

Just as with an LLM, a detailed style and format guide helps an incredible amount both for the LLMs and juniors. If you have standards and they’re not written down, you either require everyone to go teach them to anyone new, or you don’t have standards.

0 comments

dapangzi1mo ago

> you don’t have standards.

The problem is that LLM mess up things as basic as math and dates, and that's before the context gets too large and it starts making other mistakes.

Edit: Also LLM over mock tests and juniors trust that...

taurathOP1mo ago

Not very often, and most of the time it shouldn't be generating those but rather formatting code to test that. If you accept the non-determinism and use some of the more recent models, you'll find it can do 99% of it very fast, and with some guardrails and testing it can fairly reliably produce working solutions.

dapangzi1mo ago

> Not very often

> testing

This does not match my experience, have been working with LLM since 2023. We presently use the latest models, I assure you. We can definitely afford it.

I am not saying LLM is worthless, but being able to check its outputs is still necessary at this stage, because as you said, it is non-deterministic.

We have had multiple customer impacting events from code juniors committed without understanding it. Please read my top level comment in this post for context.

I genuinely hope you do not encounter issues due to your confidence in LLM, but again, my experience does not match yours.

Edit: Would also add that LLM is not good at determining line numbers in a code file, another flaw that causes a lot of confusion.

2 more replies

giantg21mo ago

I've never had an LLM create a robust, meaningful test file. I end up rewriting at least half of it.

j / k navigate · click thread line to collapse

0 comments

dapangzi1mo ago

> you don’t have standards.

The problem is that LLM mess up things as basic as math and dates, and that's before the context gets too large and it starts making other mistakes.

Edit: Also LLM over mock tests and juniors trust that...

taurathOP1mo ago

dapangzi1mo ago

> Not very often

> testing

This does not match my experience, have been working with LLM since 2023. We presently use the latest models, I assure you. We can definitely afford it.

I am not saying LLM is worthless, but being able to check its outputs is still necessary at this stage, because as you said, it is non-deterministic.

We have had multiple customer impacting events from code juniors committed without understanding it. Please read my top level comment in this post for context.

I genuinely hope you do not encounter issues due to your confidence in LLM, but again, my experience does not match yours.

Edit: Would also add that LLM is not good at determining line numbers in a code file, another flaw that causes a lot of confusion.

2 more replies

giantg21mo ago

I've never had an LLM create a robust, meaningful test file. I end up rewriting at least half of it.

j / k navigate · click thread line to collapse