No comments yet.
We've been building a multi-model API for LLMs called OpenRouter since we launched the Window AI extension in April this year [0].
The events of the last week make it clear the LLM landscape is unpredictable, so LLM aggregators are picking up interest. Unlike others, we're making a router on top of a public, explorable dataset.
We recently built a way to rank and visualize this data, including the token counts we see going to and from different models, both open-sourced (like Llama, Mistral, finetunes, and variants) and closed-sourced (OpenAI, Anthropic).
The API supports - 50+ different models: [0] - Consolidated payments for all models - OAuth, so users can pay for usage directly - Upstream latency/throughput tracking - Multiple providers per model, for redundancy (downtime happens!) - Prompt compression, so you don't have to worry as much about context length
Docs: [1]
We support 11 model hosts, including our own, based on vLLM, which we've just open-sourced: [2]
Some users have opted-in to sharing their prompts, which will soon allow us to show which models are best for different tasks.
Let us know your feedback, and if you've worked on a similar problem before!
Alex and Louis
[0] https://news.ycombinator.com/item?id=35481760 [1] https://openrouter/models [2] https://openrouter.ai/docs [3] https://github.com/OpenRouterTeam/openrouter-runner
