umtksa

I have an old iMac on which I run Syncthing and Node-RED. I use an M2 for my daily tasks. I SSH into the iMac whenever I need to, and if I require a GUI, I use screen sharing.

My iMac is quite old, dating back to 2012. While I can run most of the LLM models on my M2, I'm struggling to run even the smallest LLMs on my iMac. I've attempted Ollama and LLM.cpp without success. So, is there any local LLM suitable for running on older hardware? I enjoy experimenting with LLMs and writing small shell scripts to interact with them. I'm not looking a replacement for GPT-4; I'm looking for something fun to play with.

iMac's hardware:

- 2.7 GHz Quad-Core Intel Core i5

- 16 GB 1600 MHz DDR3

- NVIDIA GeForce GT 640M 512 MB