Show HN: Rag over all physics-related Wikipedia pages (opens in new tab)

(llm-99.vercel.app)

6 pointsspstoyanov2y ago2 comments

A few weeks ago, during the LK-99 hype, I made a demo app to explain superconductors in simple terms using RAG over relevant Wikipedia pages. I thought it would be cool to extend this to all physics-related Wikipedia pages (which turns out to be ~ 14K). After getting the text and splitting the pages, I ended up with around 100K chunks. I created all the embeddings using OpenAI’s embedding API, which cost around $7. I stored all the vectors in Pinecone.

2 comments

ricardoagzz2y ago

Super cool! And Llmflows looks really interesting, how does it differentiate from Langchain?

spstoyanovOP2y ago

Thank you so much!

The main difference between langchain and LLMFlows is in the philosophy - langchain has a "chain for everything" approach where chains come with multiple LLM calls, opinionated internal logic, and built-in default prompts. On the other hand llmflows has a "simple, explicit, transparent" approach where the goal is to allow developers to easily build their own chains (or flows) and have full control and transparency over the apps.

If you are curious check out the intro section of this blog post I made some time ago: https://llmflows.substack.com/p/introduction-to-llmflows

j / k navigate · click thread line to collapse