undefined | Better HN

0 pointsEGreg2y ago0 comments

How do you know the answer is right?

Because it linked you to the source?

Like a vector database would? Google offered to index sites since 1996.

0 comments

We have internal search. Finding things isn't the problem. It's contextualizing massive amounts of text and making it queryable with natural language.

The question I was trying to solve was -- "what is feature XYZ? How does it work in hardware & software? How is it exposed in our ABC software, and where do the hooks exist to interface with XYZ?"

The answers exist across maybe 30 different Confluence pages, plus source code, plus source code documentation, plus some PDFs. If all of that was indexed by an LLM, it would have been trivial to get the answer I spent hours manually assembling.

artfulmink2y ago

The question to which you are replying still stands. How can you guarantee that the responses generated by the LLM are factually accurate? What if it refers to interfaces that don’t exist? One could argue that in your particular use case some inaccuracies can be tolerated, but in many use cases factual inaccuracies cannot be tolerated.

EGregOP2y ago

So back to the question - how do you know the LLM didnt hallucinate an answer?

What do you think “indexed by an LLM” is?

Perhaps Anthropic with its 100K window can actually do it. But most LLM have such a small comtext window that it’s just Pinecone vector database indexing something and stuffing it in the prompt at prompt time. Come on.

j / k navigate · click thread line to collapse