WebGPU in Chrome Canary now supports float16 for faster local LLM inference (opens in new tab)

(twitter.com)

18 pointsNiekvdMaas2y ago5 comments

5 comments

I hope they gate webGPU behind a per domain allow/deny notification like they do with location and camera access.

This whole thing is just begging to be exploited by cryptominers.

It may be more profitable to use the compute for LLMs or other GPU dependent AI than crypto mining. The downside is that there is no way to verify the client isn't returning random or bad data.

lostmsu2y ago

Cryptomining is mostly gone.

NiekvdMaasOP2y ago

To test, run Chrome Canary with:

  --enable-dawn-features=allow_unsafe_apis,disable_robustness

Then, open: https://webllm.mlc.ai/#chat-demo

hardware2win2y ago

>disable_robustness

What a name, especially since it improves perf

j / k navigate · click thread line to collapse