Velvet proxies OpenAI calls and stores the requests and responses in your PostgreSQL database. That way, you can analyze logs with SQL (instead of a clunky UI). You can also set headers to add caching and metadata (for analysis).
Backstory: We started by building some more general AI data tools (like a text-to-SQL editor). We were frustrated by the lack of basic LLM infrastructure, so ended up pivoting to focus on the tooling we wanted. So many existing apps, like Helicone, were hard to use as power users. We just wanted a database.
Scale: We’ve already warehoused 50m requests for customers, and have optimized the platform for scale and latency. We’ve built the proxy on Cloudflare Workers, and latency is nominal. We’ve built some “yak shaving” features that were really complex such as decomposing OpenAI Batch API requests so you can track each log individually. One of our early customers (https://usefind.ai/) makes millions of OpenAI requests per day, up to 1500 requests per second.
Vision: We’re trying to build development tools that have as little UI as possible, that can be controlled entirely with headers and code. We also want to blend cloud and on-prem for the best of both worlds — allowing for both automatic updates and complete data ownership.
Here are some things you can do with Velvet logs:
- Observe requests, responses, and latency
- Analyze costs by metadata, such as user ID
- Track batch progress and speed
- Evaluate model changes
- Export datasets for fine-tuning of gpt-4o-mini
(this video shows how to do each of those: https://www.youtube.com/watch?v=KaFkRi5ESi8)
--
To see how it works, try chatting with our demo app that you can use without logging in: https://www.usevelvet.com/sandbox
Setting up your own proxy is 2 lines of code and takes ~5 mins.
Try it out and let us know what you think!
