Lessons from building a small-scale AI application (opens in new tab)

(thelis.org)

8 pointsrdli1y ago6 comments

6 comments

"There’s a plethora of developer libraries that claim to make AI development faster and easier. These libraries introduce new abstractions designed to improve your productivity. I tried many of them, and I found that the abstractions work well only if I stayed on a narrow path (the quick starts generally worked great!)."

This tracks my experience with developer libraries. Most of them are very brittle abstractions.

jackmpcollins1y ago

I believe we'll see two main types of AI/LLM libraries/frameworks emerge like has happened for ORMs (at least in Python). - The "Sqlalchemy" level: just bridge the gap between code and LLM - The "Django" level: additionally handle how data is indexed for RAG, standard patterns of connecting the pieces, possibly GUI, support for plugins.

It seems langchain and llamaindex are aiming to be the latter, and I'm building https://magentic.dev to be the former. I'd be interested to get your take on whether these abstractions would allow you stray from the narrow path while still being helpful!

rdliOP1y ago

I took a brief look (~5 minutes). My $0.02 is that it's not clear what problem you're trying to solve. I get what some of the features do (e.g., templated prompts) but it would be v helpful to have an example of how you actually use magentic, versus the non-magentic way. It feels like a lot of syntactic sugar, if I'm being honest (not a bad thing, but something you might want to be clear about, if that's the case.)

rdliOP1y ago

(author here) I didn't put this in my post, but one of my favorite moments was when I read some of the LlamaIndex source code which pointed to the GitHub commit where they copied the code verbatim from LangChain. (LangChain is MIT-licensed, so it's OK, but I still thought it was funny!)

BrandiATMuhkuh1y ago

Same here. I feel like they are not used/tested in a real world scenario.

Given the space moves so quickly, the frameworks always lack massively behind what they try to abstract.

efriis1y ago

Really like your writeup! I work on LangChain and would love to set up some time to chat if you're game. My email is in my profile

j / k navigate · click thread line to collapse

6 comments

datadrivenangel1y ago

This tracks my experience with developer libraries. Most of them are very brittle abstractions.

jackmpcollins1y ago

rdliOP1y ago

BrandiATMuhkuh1y ago

Same here. I feel like they are not used/tested in a real world scenario.

Given the space moves so quickly, the frameworks always lack massively behind what they try to abstract.

efriis1y ago

Really like your writeup! I work on LangChain and would love to set up some time to chat if you're game. My email is in my profile

j / k navigate · click thread line to collapse