Ask HN: What stack to use for LLM on my local Mac?

2 pointsdjangovm2y ago3 comments

For learning purposes, I want to solve the following problem:

Take my notes --> Feed it in a model that can run on my mac (32GB M1 Pro, 10 cores) --> ask questions before my next meeting to give me some context based on my notes.

What stack should I use, and how do I start? Please assume zero knowledge on my part besides python, java, and theoretical understanding of vector search, and basic ML.

3 comments

givenkiban12y ago

Use ollama to manage and download models locally.

there's a python ollama module that can allow you to run inference with these models.

that's the first part.

djangovmOP2y ago

Thank you

friendlynokill2y ago

I use lmstudio: https://lmstudio.ai/

j / k navigate · click thread line to collapse