Rust is a lot better than Python to run ML in production (opens in new tab)

(mectors.medium.com)

5 pointsmectors1y ago2 comments

2 comments

Python Docker images with CUDA, Python, Pytorch,... are 5GB to 10GB of third-party code. Here is an example of bringing this down to 1.13GB thanks to WasmEdge, 10MB of LlamaEdge API Server [compatible with ChatGPT] and 1.17GB for TinyLlama.

verdverm1y ago

If you are not actually running ML code, of course you can remove Python and ML libraries. You could get this down to MB by removing WASM and using a Go binary

apples and oranges

j / k navigate · click thread line to collapse

2 comments

mectorsOP1y ago

verdverm1y ago

If you are not actually running ML code, of course you can remove Python and ML libraries. You could get this down to MB by removing WASM and using a Go binary

apples and oranges

j / k navigate · click thread line to collapse