undefined | Better HN

0 pointsoompydoompy742mo ago0 comments

Remaining dependent on proprietary frontier models that you can only access via an API makes no sense whatsoever. My hope is that the future is open weight models running on local hardware.

0 comments

naasking2mo ago

Eventually, yes. ParoQuant is hopefully the future here, 4-bit weights with no real degradation from FP16:

https://github.com/z-lab/paroquant

j / k navigate · click thread line to collapse

0 comments

naasking2mo ago

Eventually, yes. ParoQuant is hopefully the future here, 4-bit weights with no real degradation from FP16:

https://github.com/z-lab/paroquant

j / k navigate · click thread line to collapse