LLM Microserving: a new RISC-style approach to design LLM serving API (opens in new tab)

(blog.mlc.ai)

4 pointsjinhongyii1y ago1 comments

1 comments

Scale LLM serving with programmable cross-engine serving patterns, all in a few lines of Python

j / k navigate · click thread line to collapse

(blog.mlc.ai)

4 pointsjinhongyii1y ago1 comments

Scale LLM serving with programmable cross-engine serving patterns, all in a few lines of Python

j / k navigate · click thread line to collapse