However, since I already have pi working with llama.cpp server from a docker container, I did a quick experiment to compare three code bases:
https://gist.github.com/ontouchstart/7483c12efa3c3d3a49e38c2...
https://gist.github.com/ontouchstart/217fe2b8103a5c0bfaee1e9...
Very interesting.
Will do it again next week if I can get unsloth studio working.
curl -LsSf https://astral.sh/uv/install.sh | sh
uv venv unsloth_studio --python 3.13
source unsloth_studio/bin/activate
uv pip install unsloth==2026.3.7 --torch-backend=auto
unsloth studio setup
unsloth studio -H 0.0.0.0 -p 8888
https://gist.github.com/ontouchstart/532312fcba59aec3ce7f6aa...
Here is the error message on my machine:
https://gist.github.com/ontouchstart/86ca3cbd8b6b61fa0aeec75...
It seems we might need more instructions on how to set up python (via uv) in vanilla MacOS.
We have much much in the pipeline!!
Is there an alternative, tutorial, or project you'd recommend that would help me do supervised fine tuning (SFT) with the metal stack / macOS?
(base) unsloth git:(main) unsloth studio setup
╔══════════════════════════════════════╗
║ Unsloth Studio Setup Script ║
╚══════════════════════════════════════╝
Node v25.8.1 and npm 11.11.0 already meet requirements. Skipping nvm install.
Node v25.8.1 | npm 11.11.0
npm run build failed (exit code 2):
> unsloth-theme@0.0.0 build
> tsc -b && vite build
src/features/chat/shared-composer.tsx(366,17): error TS6133: 'status' is declared but its value is never read.curl -LsSf https://astral.sh/uv/install.sh | sh
uv venv unsloth_studio --python 3.13
source unsloth_studio/bin/activate
uv pip install unsloth --torch-backend=auto
unsloth studio setup
unsloth studio -H 0.0.0.0 -p 8888
Works fine on MacOS now (chat only).
On Ubuntu 24.04 with two GPU's (3090+3070), it appears that Llama.cpp sometimes uses the CPU and not GPU. This is judging from the tk/s and CPU load for identical models run with US-studio vs. just Llama.cpp (bleeding edge).
This needs to go on homebrew or be a zip file with an app for manual download.
We come from Python land mainly so packaging and distribution is all very new to us - homebrew will definitely be next!
The pyproject.toml can even handles build env for you, so you no longer need a setup.sh that installs 10 tool in specific order with specific flag to produce working environment. A single uv sync, and the job is done.
Plus the result is reproducible, so if this time uv sync work, then it also work next time.
Highly recommend if you are still on pip.
Note: Take a example that I used to install unsloth with rocm setup that based on unreleased git version dependencies and graphic card specific build flag, all of them can be handled with one command 'uv sync'. This will require a big pile of shell script if doing another way. https://github.com/unslothai/unsloth/issues/4280#issuecommen...
uv init
uv add unsloth
uv run main.py % or whatever
Update: It looks like it doesn't work with the current Python version, you might have to downgrade to Python 3.13 (however even then I still get `error: unexpected argument '--torch-backend' found`)
Also, never saw any Unsloth related software in production to this day. Feels strongly like a non-essential tool for hobby LLM wizards.
Is it like, for AI hobbyists? I.e. I have a 4090 at home and want to fine-tune models?
Is it a competitor to LMStudio?
We're complimentary to LM Studio - they have a great tool as well!
Happy to see unsloth making it even easier for people like me to get going with fine tuning. Not that I am unable to I'm just lazy.
Fine tuning with a UI is definitely targeted towards hobbyists. Sadly I'll have to wait for AMD ROCm support.