undefined | Better HN

0 pointsverdverm1y ago0 comments

Look to a specialized model instead of a general purpose one

0 comments

Any suggestions? Thanks

I have tried Phind and anything beyond mega junior tier questions it suffers as well and gives bad answers.

You have to think of the LLMs as more of a better search engine than something that can actually write code for you. I use phind for writing obscure regexes, or shell syntax, but I always verify the answer. I've been very pleased with the results. I think anyone disappointed with it is setting the bar too high and won't be fully satisfied until LLMs can effectively replace a Sr dev (which, let's be real, is only going to happen once we reach AGI)

unshavedyak1y ago

Yea, I use them daily and that’s my issue as well. You have to learn what to ask or you spend more time debugging their junk than being productive, at least for me. Devv.ai is my recent try, and so far it’s been good but library changes quickly cause it to lose accuracy. It is not able to understand what library version you’re on and what it is referencing, which wastes a lot of time.

I like LLMs for general design work, but I’ve found accuracy to be atrocious in this area.

verdvermOP1y ago

> library changes quickly cause it to lose accuracy

yup, this is why an LLM only solution will not work. You need to provide extra context crafted from the language or library resources (docs, code, help, chat)

This is the same thing humans do. We go to the project resources to help know what code to write

1 more reply

verdvermOP1y ago

It will be a system, not a single model, and will depend on what programming task you want to perform

probably need routers, RAG, and reranking

I think there is a role for LLM + deterministic code gen as well (https://github.com/hofstadter-io/hof/blob/_dev/flow/chat/pro...)

moomoo111y ago

Interesting. I was hoping for something with a UI like chat gpt or phind.

Something that I can just use as easily as copilot. Unfortunately every single one sucks.

Or maybe that's just how programming is - its easy at the surface/ice berg level and below is just massive amounts of complexity. Then again, I'm not doing menial stuff so maybe I'm just expecting too much.

1 more reply

j / k navigate · click thread line to collapse

0 comments

moomoo111y ago

Any suggestions? Thanks

I have tried Phind and anything beyond mega junior tier questions it suffers as well and gives bad answers.

wing-_-nuts1y ago

unshavedyak1y ago

I like LLMs for general design work, but I’ve found accuracy to be atrocious in this area.

verdvermOP1y ago

> library changes quickly cause it to lose accuracy

yup, this is why an LLM only solution will not work. You need to provide extra context crafted from the language or library resources (docs, code, help, chat)

This is the same thing humans do. We go to the project resources to help know what code to write

1 more reply

verdvermOP1y ago

It will be a system, not a single model, and will depend on what programming task you want to perform

probably need routers, RAG, and reranking

I think there is a role for LLM + deterministic code gen as well (https://github.com/hofstadter-io/hof/blob/_dev/flow/chat/pro...)

moomoo111y ago

Interesting. I was hoping for something with a UI like chat gpt or phind.

Something that I can just use as easily as copilot. Unfortunately every single one sucks.

1 more reply

j / k navigate · click thread line to collapse