A small tool I made for local LLMs: LLM-neofetch-plus

1 pointsHFerrahoglu29d ago2 comments

Hey everyone!

I've had this in mind for a while, and I finally did it: it shows system information like regular NeoFetch, but I've added extra features for those using local LLMs (Ollama, llama.cpp, etc.).

For example: -How much VRAM does your GPU have, which model (NVIDIA, AMD, Intel, Apple M series)? -How many billions of parameters can your machine comfortably run (is 70B or 13B more sensible?) -Which GGUF quantization does what (Q4_K_M vs Q8_0 vs)? -Comparison of Ollama / llama.cpp / vLLM / LM Studio -Disk speed test + JSON/Markdown export

Simple installation: pip install llm-neofetch-plus

llm-neofetch -d 3 ← This is the detailed version, showing suggestions etc.

GitHub: https://github.com/HFerrahoglu/llm-neofetch-plus

If anyone tries it, could you tell me if you liked it or not, and what we should change? Thanks!

2 comments

akssassin90729d ago

Running local LLMs on Apple Silicon has gotten surprisingly capable — the M-series chips handle models that used to require expensive GPU setups, so tools like this that actually speak to that hardware are welcome.

The quantization comparison is the feature I'd use most. It's one of those things that sounds simple but in practice nobody wants to dig through benchmarks just to figure out whether Q4 or Q8 is worth the extra memory on their specific machine.

Does it factor in what else is running in the background when estimating how much your machine can handle? That number can shift a lot depending on what else has memory tied up.

HFerrahogluOP29d ago

No. The tool uses total hardware resources, not currently available ones.

It displays current usage (via progress bars and "used" values), but the LLM model recommendations are based on raw hardware totals, not what's actually free after background processes.

This is a reasonable design choice for a "system specs" tool—it's showing what the machine has, not what it has available right now. But you're right that it could misleadingly suggest a 70B model fits on a 24GB GPU when other apps are already using 4GB.

j / k navigate · click thread line to collapse