undefined | Better HN

0 pointsxmprt9mo ago0 comments

> Using apis I am familiar with but don't have memorized

I think you have to be careful here even with a typed language. For example, I generated some Go code recently which execed a shell command and got the output. The generated code used CombinedOutput which is easier to used but doesn't do proper error handling. Everything ran fine until I tested a few error cases and then realized the problem. In other times I asked the agent to write tests cases too and while it scaffolded code to handle error cases, it didn't actually write any tests cases to exercise that - so if you were only doing a cursory review, you would think it was properly tested when in reality it wasn't.

0 comments

tptacek9mo ago

You always have to be careful. But worth calling out that using CombinedOutput() like that is also a common flaw in human code.

dingnuts9mo ago

The difference is that humans learn. I got bit by this behavior of CombinedOutput once ten years ago, and no longer make this mistake.

csallen9mo ago

This applies to AI, too, albeit in different ways:

1. You can iteratively improve the rules and prompts you give to the AI when coding. I do this a lot. My process is constantly improving, and the AI makes fewer mistakes as a result.

2. AI models get smarter. Just in the past few months, the LLMs I use to code are making significantly fewer mistakes than they were.

3 more replies

j / k navigate · click thread line to collapse