Dense Vector and Sparse Vector and Fulltext and Tensor Reranker = Best for RAG? (opens in new tab)

Hi, I'm one of the creators of infinity, and the article has mentioned about the sparse vector vs bm25. While the sparse vector performs well under some evaluations, it is obtained by training a model, which means that it can't fully represent all of the user's keywords/tokens, and those that don't appear in the training set, are truncated. So this is a very big impact for many enterprise vertical scenarios. And bm25 doesn't have such a limitation

A noticeble work to demostrate the effectiveness of hybrid search is blended rag by IBM research (https://arxiv.org/abs/2404.07220), which has shown that 3-way hybrid search could achieve STOA over multiple evaluation datasets. And also, we've reproduced the results of blended rag, as shown in this article. Additionally, blended rag + colbert based reranker could have a much better results.

The major challenges are how to implement and manage such many indices within single database. That's why we build this database start from scratch. Infinity is actually a kind of "indexing" database, based on a columnar store. The executor also requires refined design to fuse these hybrid search approaches effectively.

There are some vector databases that already include both dense vector search and sparse vector search, such as qdrant. A hybrid search of these 2 does not solve many problems well, such as exact queries. Moreover, according to our experiments, as seen in the article, the performance of dense vector + sparse vector, improves only a little bit. In addition to these 2 way recall, infinity offers bm25 as well as colbert reranker, which can make the ranking quality of the hybrid search much better.

Because colbert is not an end-to-end solution. As seen for RAGatouille, it has integrated colbertv2 into this repo. However, it's not a database, we implement tensor within infinity, aim to make an end-to-end solution for late interaction based ranking models.

paradedb could also deliver three-way hybrid search through pg_vector, pg_sparse and pg_search. Compared with paradedb, infinity has following advantages:

1. Performance

The performance of pg_vector is far slower than vector search of Infinity due to the vector index design. The performance of pg_sparse is also slower than sparse vector search of infinity. The performance of pg_search is much slower than full text search of infinity. pg_search is based on Tantivy, which is much slower than the inverted index of infinity.

Detailed benchmark could be seen in this article : https://infiniflow.org/blog/fastest-hybrid-search or github repo.

2. Infinity has all the builtin implementation of the above three search approaches. These indices could work smoothly together with the executor of infinity. The users could use any combination of the search approaches, together with the fused ranking algorithms, in a very efficient approach.

3. Infinity has also builtin support for tensor, which makes it possible to deliver an in-database colbert reranker compared with the cross encoder based reranker outside. The colbert reranker could bring much benefits for search qualities.

4. Infinity is much easier to use, it could be deployed as either a standalone server, or as an embedded python library just through pip install.

5. Infinity is designed start from scratch, it does not have the burden of postgresql, and is evolving fast. It will run on cloud in very near future which could save the cost a lot.