Skip to main content
Version: User Guides (Cloud)

Hybrid Search

Hybrid Search refers to a search method that conducts multiple ANN searches simultaneously, reranks multiple sets of results from these ANN searches, and ultimately returns a single set of results. Using Hybrid Search can enhance the search accuracy. Zilliz Cloud supports conducting Hybrid Search on a collection with multiple vector fields.

Hybrid Search is most commonly used in scenarios including sparse-dense vector searches and multimodal searches. This guide will demonstrate how to conduct a Hybrid Search in Zilliz Cloud with a specific example.

Scenarios​

Hybrid Search is suitable for the following two scenarios:

Different types of vectors can represent different information, and using various embedding models can more comprehensively represent different features and aspects of the data. For example, using different embedding models for the same sentence can generate a dense vector to represent the semantic meaning and a sparse vector to represent the word frequency in the sentence.

  • Sparse vectors: Sparse vectors are characterized by their high vector dimensionality and the presence of few non-zero values. This structure makes them particularly well-suited for traditional information retrieval applications. In most cases, the number of dimensions used in sparse vectors correspond to different tokens across one or more languages. Each dimension is assigned a value that indicates the relative importance of that token within the document. This layout proves advantageous for tasks that involve keyword matching.

  • Dense vectors: Dense vectors are embeddings derived from neural networks. When arranged in an ordered array, these vectors capture the semantic essence of the input text. Note that dense vectors are not limited to text processing; they are also extensively used in computer vision to represent the semantics of visual data. These dense vectors, usually generated by text embedding models, are characterized by most or all elements being non-zero. Thus, dense vectors are particularly effective for semantic search applications, as they can return the most similar results based on vector distance even in the absence of exact keyword matches. This capability allows for more nuanced and context-aware search results, often capturing relationships between concepts that might be missed by keyword-based approaches.

For more details, refer to Sparse Vector and Dense Vector.

Multimodal search refers to the similarity search of unstructured data across multiple modalities (such as images, videos, audio, text, etc). For instance, a person can be represented using various modalities of data such as fingerprints, voiceprints, and facial features. Hybrid Search supports multiple searches simultaneously. For example searching a person with both similar fingerprints and voiceprints.

Workflow​

The main workflow for conducting a Hybrid Search is as follows:

  1. Generate dense vectors through embedding models like BERT and Transformers.

  2. Generate sparse vectors through embedding models like BM25, BGE-M3, SPLADE, etc.

  3. Create a collection in Zilliz Cloud and define the collection schema which includes both dense and sparse vector fields.

  4. Insert sparse-dense vectors into the collection just created in the previous step.

  5. Conduct a Hybrid Search: ANN Search on dense vectors will return a set of top-K most similar results, and text match on sparse vectors will also return a set of top-K results.

  6. Normalization: Normalize the scores of the two sets of top-K results, converting the scores to a range between [0,1].

  7. Choose an appropriate reranking strategy to merge and rerank the two sets of top-K results and ultimately return a final set of top-K results.

UZkfwo6uXhexxpb0uc2cpD7qnxL

Examples​

This section will use a specific example to illustrate how to conduct a Hybrid Search on sparse-dense vectors to enhance the accuracy of text searches.

Create a collection with multiple vector fields​

The process of creating a collection includes three parts: defining the collection schema, configuring the index parameters, and creating the collection.

Define schema​

In this example, multiple vector fields need to be defined within the collection schema. Currently, each collection can include up to 4 vector fields by default. But you can also modify the value of proxy.maxVectorFieldNum to include up to 10 vector fields in a collection as needed.

The following example defines a collection schema, where dense and sparse are the two vector fields:

  • id: This field serves as the primary key for storing text IDs. The data type of this field is INT64.

  • text: This field is used for storing textual content. The data type of this field is VARCHAR, with a maximum length of 1000 characters.

  • dense: This field is used to store the dense vectors of the texts. The data type of this field is FLOAT_VECTOR, with a vector dimension of 768.

  • sparse: This field is used to store the sparse vectors of the texts. The data type of this field is SPARSE_FLOAT_VECTOR.

# Create a collection in customized setup mode
from pymilvus import (
    MilvusClient, DataType
)

client = MilvusClient(
    uri="YOUR_CLUSTER_ENDPOINT",
    token="YOUR_CLUSTER_TOKEN"
)

# Create schema
schema = MilvusClient.create_schema(
    auto_id=False,
    enable_dynamic_field=True,
)
# Add fields to schema
schema.add_field(field_name="id", datatype=DataType.INT64, is_primary=True)
schema.add_field(field_name="text", datatype=DataType.VARCHAR, max_length=1000)
schema.add_field(field_name="sparse", datatype=DataType.SPARSE_FLOAT_VECTOR)
schema.add_field(field_name="dense", datatype=DataType.FLOAT_VECTOR, dim=5)

During sparse vector searches, you can simplify the process of generating sparse embedding vectors by leveraging Full-Text Search capabilities. For more details, see Full Text Search.

Create index​

After defining the collection schema, it is necessary to set up the vector indexes and the similarity metrics. In this example, an index of the AUTOINDEX type is created for both the dense vector field dense, and the sparse vector field sparse.

from pymilvus import MilvusClient

# Prepare index parameters
index_params = client.prepare_index_params()

# Add indexes
index_params.add_index(
    field_name="dense",
    index_name="dense_index",
    index_type="AUTOINDEX",
    metric_type="IP"
)

index_params.add_index(
    field_name="sparse",
    index_name="sparse_index",
    index_type="AUTOINDEX",  # Index type for sparse vectors
    metric_type="IP",  # Currently, only IP (Inner Product) is supported for sparse vectors
    params={"drop_ratio_build": 0.2},  # The ratio of small vector values to be dropped during indexing
)

Create collection​

Create a collection named demo with the collection schema and indexes configured in the previous two steps.

from pymilvus import MilvusClient

client.create_collection(
    collection_name="my_collection",
    schema=schema,
    index_params=index_params
)

Insert data​

Insert the sparse-dense vectors into the the collection demo.

from pymilvus import MilvusClient

data=[
    {"id": 0, "text": "Artificial intelligence was founded as an academic discipline in 1956.", "sparse":{9637: 0.30856525997853057, 4399: 0.19771651149001523, ...}, "dense": [0.3580376395471989, -0.6023495712049978, 0.18414012509913835, ...]},
    {"id": 1, "text": "Alan Turing was the first person to conduct substantial research in AI.", "sparse":{6959: 0.31025067641541815, 1729: 0.8265339135915016, ...}, "dense": [0.19886812562848388, 0.06023560599112088, 0.6976963061752597, ...]},
    {"id": 2, "text": "Born in Maida Vale, London, Turing was raised in southern England.", "sparse":{1220: 0.15303302147479103, 7335: 0.9436728846033107, ...}, "dense": [0.43742130801983836, -0.5597502546264526, 0.6457887650909682, ...]}]

res = client.insert(
    collection_name="my_collection",
    data=data
)

Create multiple AnnSearchRequest instances​

Hybrid Search is implemented by creating multiple AnnSearchRequest in the hybrid_search() function, where each AnnSearchRequest represents a basic ANN search request for a specific vector field. Therefore, before conducting a Hybrid Search, it is necessary to create an AnnSearchRequest for each vector field.

By configuring the expr parameter in an AnnSearchRequest, you can set the filtering conditions for your hybrid sesarch. Please refer to Filtered Search and Filtering.

📘Notes

In Hybrid Search, each AnnSearchRequest supports only one query vector.

Suppose the query text "Who started AI research?" has already been converted into sparse and dense vectors. Based on this, two AnnSearchRequest search requests are created for the sparse and dense vector fields respectively, as shown in the following example.

from pymilvus import AnnSearchRequest

query_dense_vector = [0.3580376395471989, -0.6023495712049978, 0.18414012509913835, -0.26286205330961354, 0.9029438446296592]

search_param_1 = {
    "data": [query_dense_vector],
    "anns_field": "dense",
    "param": {
        "metric_type": "IP",
        "params": {"nprobe": 10}
    },
    "limit": 2
}
request_1 = AnnSearchRequest(**search_param_1)

query_sparse_vector = {3573: 0.34701499565746674}, {5263: 0.2639375518635271}
search_param_2 = {
    "data": [query_sparse_vector],
    "anns_field": "sparse",
    "param": {
        "metric_type": "IP",
        "params": {"drop_ratio_build": 0.2}
    },
    "limit": 2
}
request_2 = AnnSearchRequest(**search_param_2)

reqs = [request_1, request_2]

Since the parameter limit is set to 2, each AnnSearchRequest returns 2 search results. In this example, 2 AnnSearchRequest are created, therefore a total of 4 search results will be returned.

Configure a reranking strategy​

To merge and rerank the two sets of ANN search results, it is necessary to select an appropriate reranking strategy. Zilliz Cloud supports two types of reranking strategy: WeightedRanker and RRFRanker. When choosing a reranking strategy, one thing to consider is whether to there is any emphasis for one or more basic ANN search on the vector fields.

  • WeightedRanker: This strategy is recommended if you require the results to emphasize a particular vector field. The WeightedRanker allows you to assign higher weights to certain vector fields, emphasizing them more. For instance, in multimodal searches, textual descriptions of an image might be considered more important than the colors in this image.

  • RRFRanker (Reciprocal Rank Fusion Ranker): This strategy is recommended when there is no specific emphasis. The RRF can effectively balance the importance of each vector field.

For more details about the mechanisms of these two reranking strategies, refer to Reranking.

The following two examples demonstrate how to use the WeightedRanker and RRFRanker reranking strategies:

  1. Example 1: Using WeightedRanker

    When using the WeightedRanker strategy, you need to input weight values into the WeightedRanker function. The number of basic ANN searches in a Hybrid Search corresponds to the number of values that need to be inputted. The input values should be in the range [0,1], with values closer to 1 indicating greater importance.

    from pymilvus import WeightedRanker

    ranker = WeightedRanker(0.8, 0.3)

  2. Example 2: Using RRFRanker

    When using the RRFRanker strategy, you need to input the parameter value k into the RRFRanker. The default value of k is 60. This parameter helps to determine how the ranks are combined from different ANN searches, aiming to balance and blend the importance across all searches.

    from pymilvus import RRFRanker

    ranker = RRFRanker(100)

Before conducting a Hybrid Search, it is necessary to load the collection into memory. If any vector fields in the collection do not have an index or are not loaded, an error will occur when calling the Hybrid Search method.

from pymilvus import MilvusClient

res = client.hybrid_search(
    collection_name="my_collection",
    reqs=reqs,
    ranker=ranker,
    limit=2
)
for hits in res:
    print("TopK results:")
    for hit in hits:
        print(hit)

The following is the output:

["['id: 844, distance: 0.006047376897186041, entity: {}', 'id: 876, distance: 0.006422005593776703, entity: {}']"]

Since limit=2 is specified in the Hybrid Search, Zilliz Cloud will rerank the four search results from step 3 and ultimately return only the top 2 most similar search results.