Ask HN: How to manage AI APIs for SaaS application?

As someone rightly pointed out already, issuing these many keys (a few thousands) is not an issue. But, handling abuse, retries, spikes, fallbacks, logging, monitoring, hacking attempts, etc. ... these are the real issues that you need to focus on.

My suggestion: go with openrouter... it offers management keys that you can use to issue keys on the fly for your customers. You can also set quota per key.

With this setup, everything gets offloaded to them. You just pay 5% commission (when you load credits to their platform).

I've used LiteLLM for my 1k-user service https://LibreThinker.com Works okay-ish, but some vendors now default to MFA on API keys (eg OpenAI) which makes things tricky; I decided to simply let those fail for now.

i did this with aws kms. i had a root key that encrypted & decrypted data keys, which were unique for each customer. those data keys then encrypted & decrypted sensitive customer info. so called envelope encryption

BYOK is becoming more common for AI SaaS. Interested to know how people handle rate limiting and key security at scale.

Honestly the API key part ended up being the easy part for me.

The messy stuff was abuse prevention, retries/fallbacks, and sudden cost spikes once real users started using different workflows.