MutedEstate45

I’m building an internal tool that uses a local small language model to handle tasks related to testing that we find annoying like manually creating dummy data from schemas or TypeScript definitions, setting up dummy webworkers to proxy server calls for testing, and/or mapping API dependencies for integration tests, etc.

I’m still at the early stages, but I was wondering if anyone else would find this kind of tool useful (either some aspect or all aspects) because I’m considering open sourcing it on Github when I’m done.

Came across MeMemo, which runs Retrieval-Augmented Generation (RAG) directly on the client side using WebGPU. No server calls, full privacy, and faster responses—sounds great, but I’m wondering:

What’s the catch? How do you scale vector search or manage embeddings locally? Can this handle complex use cases, or is it mostly for lightweight tasks? How do we navigate browser limitations, UX challenges, or security concerns in a setup like this?

Curious if anyone here has tried client-side RAG or sees a compelling use case for it. Is this approach worth exploring for privacy-focused apps, or are we not there yet?