Skip to content
Better HN
Top
New
Best
Ask
Show
Jobs
Search
⌘K
TensorRT-LLM runtime now open-source | Better HN
TensorRT-LLM runtime now open-source
(opens in new tab)
(github.com)
4 points
mmoskal
1y ago
1 comments
Share
1 comments
default
newest
oldest
mmoskal
OP
1y ago
Previously, the "Executor" runtime was shipped as binary blobs. This is the bit that schedules requests and manages KV cache (similar to vLLM or SGLang server).
j
/
k
navigate · click thread line to collapse