The limitations are very practical. High dimensionality vectors have a huge size cost and this limits scalability. This blows up pretty quickly when you index millions of documents. Also query cost explodes quickly to become impractical in terms of resource usage. The Lucene vector field support has some limitations here. Opensearch uses native libraries to work around that. Elasticsearch only supports whatever Lucene supports. For now. I'd say that will probably evolve over time. It's a relatively new feature for both.

Dedicated products have similar limitations of course. But they have the advantage that they can be optimized for that use case and use some native trickery (GPUs, etc.) to mitigate some of the effects. So, there's a bit of a tradeoff there in what you need and can afford. But ultimately, handling millions/billions of huge vectors has a cost.