Show HN: I made a Mac app for rate limiting and monitoring LLM requests (opens in new tab)

(github.com)

3 pointswinchester67882mo ago2 comments

https://github.com/quilrai/LLMWatcher/releases/latest/downlo... - Signed and Notarized app

I use coding agents (claude code, codex, cursor etc) a lot and wanted a lightweight app which

1 - have searchable logs of all conversations 2 - rate limit or get notified if using too much context length (-especially useful for reminding myself to use a new session)

3 - block api keys etc from going to LLM providers

4 - get overview of tokens, tools used, and other interesting information

So, I made LLMWatcher

Using base url feature supported by most of the coding agents, LLMWatcher sits in the middle of coding sessions and can do all the above + some extra goodies

Also has a LLM Gateway feature, which can be used to get monitored/ proxied URLs with controls for any LLMs

Would love some feedback on this and to know if anyone finds it useful

Note: Always wanted to make desktop apps, and Tauri made it very easy to do so. Attempted with python first but got dragged into signing and packaging hell

2 comments

kxbnb2mo ago

Nice approach using the base URL feature as the interception point. That's the same pattern we've been using at toran.sh for API observability - swap a URL and you get visibility without SDK changes.

The "searchable logs of all conversations" piece is interesting. We focused more on the real-time view (watching requests as they happen during debugging) rather than historical search. Curious if you find yourself using the live view vs searching through logs more often?

One thing we ran into: when agents chain multiple API calls, having just the raw logs isn't enough - you need to correlate which calls came from which task/session. Did you add any grouping/correlation to LLMWatcher?

winchester6788OP2mo ago

Only reason I use searchable logs is, when I roll back something in claude conversation but later want to refer it.

Apart from that, may be somebody might find it useful for observing their usage patterns etc.

No grouping or correlation right now. Wanted to get some feedback/ opinions first

j / k navigate · click thread line to collapse