Skip to content
Better HN
Top
New
Best
Ask
Show
Jobs
Search
⌘K
Give Me FP32 or Give Me Death?
(opens in new tab)
(arxiv.org)
3 points
spindump8930
9mo ago
2 comments
Share
Give Me FP32 or Give Me Death? | Better HN
2 comments
default
newest
oldest
spindump8930
OP
9mo ago
People often don't understand why LLMs can be non deterministic even with deterministic seeding, temperature, sampling. This paper shows how bad it can be with different hardware and gpu hosts.
incomingpain
9mo ago
Q4 is plenty for me, I dont have the budget for FP32 lol.
If money wasnt a thing, id probably not be going above Q8.
j
/
k
navigate · click thread line to collapse