undefined | Better HN

0 pointsbrunoborges2mo ago0 comments

As long as software is properly documented, and documentation is published in LLM-friendly formats, LLMs may be able to answer most of the beyond basic questions even when docs don't explicitly cover a particular scenario.

Take an API for searching products, one for getting product details, and then an API for deleting a product.

The documentation does not need to cover the detailed scenario of "How to delete a product" where the first step is to search, the second step is to get the details (get the ID), and the third step is to delete.

The LLM is capable of answering the question "how to delete the product 'product name'".

To some degree, many of the questions on SO were beyond basic, but still possible for a human to answer if only they read documentation. LLMs just happen to be capable of reading A LOT of documentation a LOT faster, and then coming up with an answer A LOT faster.

0 comments

al_borland2mo ago

If the LLM is also writing the documentation, because the developers surely don’t want to, I’m not sure how well this will work out.

I have some co-workers who have tried to use Copilot for their documentation (because they never write any and I’m constantly asking them questions as a result), and the results were so bad they actually spent the time to write proper documentation. It failed successfully, I suppose.

brunoborgesOP2mo ago

Indeed, how documentation is written is key. But funny enough, I have been a strong advocate that documentation should always be written in Reference Docs style, and optionally with additional Scenario Docs.

The former is to be consumed by engineers (and now LLMs), while the later is to be consumed by humans.

Scenario Docs, or use case docs, are what millions of blog articles were made of in the early days, then we turned to Stack Overflow questions/answers, then companies started writing documentation in this format too. Lots of Quick Starts for X, Y, and Z scenarios using technology K. Some companies gave away completely on writing reference documentation, which would allow engineers to understand the fundamentals of technology K and then be able to apply to X, Y, and Z.

But now with LLMs, we can certainly go back to writing Reference docs only, and let LLMs do the extra work on Scenario based docs. Can they hallucinate still? Sure. But they will likely get most beyond-basic-maybe-not-too-advanced scenarios right in the first shot.

As for using LLMs to write docs: engineers should be reviewing that as much as they should be reviewing the code generated by AI.

mlinhares2mo ago

"In this imaginary world where everything is perfect and made to be consumed by LLMs, LLMs are the best tool for the job".

JohnBooty2mo ago

    world where everything is perfect and made to be consumed by LLMs

I believe the parent poster was clearly and specifically talking about software documentation that was strong and LLM consumption-friendly, not "everything"

HaZeust2mo ago

Yeah, old news? It's how it is today with humans.

You SHOULD be making things in a human/LLM-readable format nowadays anyway if you're in tech, it'll do you well with AIs resorting to citing what you write, and content aggregators - like search engines - giving it more preferential scores.

j / k navigate · click thread line to collapse