Skip to main content

Quickstart to Serving Cluster

A serving cluster is a self-contained server that combines both compute and storage for real-time production serving. Once you have cleaned your data through your Extract-Transform-Load (ETL) pipelines, you can import it into a serving cluster to deliver significant performance gains.

Before you start​

Zilliz Cloud provides a Bring-Your-Own-Cloud (BYOC) solution, allowing organizations to host applications and data in their own cloud accounts rather than on Zilliz Cloud's infrastructure. For details about our BYOC solution, read BYOC Overview.

The following figure illustrates the steps to get started with our BYOC solution.

W1G2wotilhctoibaBlocf2bfngf

Before going through this quick start, ensure that:

  • You have registered an account with Zilliz Cloud.

    For instructions, refer to Register with Zilliz Cloud.

  • You have contacted Zilliz Cloud sales and provided your account to us.

    Notes

    Zilliz BYOC is currently available in General Availability. For access and implementation details, please contact Zilliz Cloud sales.

  • You have created a project in the BYOC organization and deployed the data plane infrastructure for the project.

    Zilliz BYOC operates within your Virtual Private Cloud (VPC), requiring you to initiate the deployment of data plane components. You can deploy the data plane in your VPCs hosted on the following cloud providers:

    If your cloud provider is not available above, contact Zilliz Cloud support.

  • You have determined the way to connect to your BYOC cluster. For details, refer to Prepare for Cluster Connection.

The following procedure assumes that you have already created a serving cluster and obtained its endpoint and access credentials.

Step 1: Set up connection​

Once you have obtained the cluster credentials, you can use it to connect to your cluster.

python
from pymilvus import MilvusClient, DataType

SERVING_CLUSTER_ENDPOINT = "https://{cluster-id}.{region}.vectordb.zillizcloud.com:19530"
TOKEN = "YOUR_ZILLIZ_API_KEY"
# A valid token could be either
# - An API key, or
# - Use your Zilliz Cloud API key

# 1. Set up a Milvus client
client = MilvusClient(
    uri=SERVING_CLUSTER_ENDPOINT,
    token=TOKEN
)
rust
use milvus::v2::prelude::*;

const SERVING_CLUSTER_ENDPOINT: &str = "https://{cluster-id}.{region}.vectordb.zillizcloud.com:19530";
const TOKEN: &str = "YOUR_ZILLIZ_API_KEY";

// 1. Set up a Milvus client
let config = ConnectConfig::new().uri(SERVING_CLUSTER_ENDPOINT).token(TOKEN);
let client = ClientV2::new(&config).await?;
c++
#include "milvus/MilvusClientV2.h"
#include <iostream>

const std::string SERVING_CLUSTER_ENDPOINT = "https://{cluster-id}.{region}.vectordb.zillizcloud.com:19530";
const std::string TOKEN = "YOUR_ZILLIZ_API_KEY";

// 1. Set up a Milvus client
auto client = milvus::MilvusClientV2::Create();
auto status = client->Connect(milvus::ConnectParam(SERVING_CLUSTER_ENDPOINT).WithToken(TOKEN));
if (!status.IsOk()) {
    std::cout << status.Message() << std::endl;
}

Step 2: (Optional) Create a database.​

A serving cluster ships with a default database. If you choose that, skip this step. You can also create a database as follows:

python
# connect to the serving cluster
client = MilvusClient(
    # a cluster-specific endpoint
    uri=SERVING_CLUSTER_ENDPOINT,
    token=TOKEN
)

client.create_database(
    db_name="my_database"
)
rust
client
    .create_database(
        CreateDatabaseRequest::builder()
            .database_name("my_database")
            .build()?,
    )
    .await?;
c++
#include <iostream>

auto status = client->CreateDatabase(
    milvus::CreateDatabaseRequest().WithDatabaseName("my_database"));
if (!status.IsOk()) {
    std::cout << status.Message() << std::endl;
}

Step 3: Create a collection.​

Once the database is ready, you can create managed collections in it. Unlike an external collection that maps collection columns to external data files, a managed collection asks you to import data for significant performance gains.

The following example demonstrates how to set up the collection schema and create a collection.

python
from pymilvus import MilvusClient, DataType

schema = MilvusClient.create_schema()

schema.add_field(
    field_name="product_id",
    datatype=DataType.INT64,
    is_primary=True
)

schema.add_field(
    field_name="product_name",
    datatype=DataType.VARCHAR,
    max_length=512
)

schema.add_field(
    field_name="embedding",
    datatype=DataType.FLOAT_VECTOR,
    dim=768
)
rust
let schema = CollectionSchema::new()
    .add_field(
        FieldSchema::new()
            .name("product_id")
            .data_type(DataType::Int64)
            .primary_key(true),
    )
    .add_field(
        FieldSchema::new()
            .name("product_name")
            .data_type(DataType::VarChar)
            .max_length(512),
    )
    .add_field(
        FieldSchema::new()
            .name("embedding")
            .data_type(DataType::FloatVector)
            .dimension(768),
    );
c++
milvus::CollectionSchemaPtr schema = std::make_shared<milvus::CollectionSchema>();
schema->AddField(milvus::FieldSchema("product_id", milvus::DataType::INT64, "product id", true, false));
milvus::FieldSchema name_field("product_name", milvus::DataType::VARCHAR, "product name");
name_field.SetMaxLength(512);
schema->AddField(name_field);
schema->AddField(milvus::FieldSchema("embedding", milvus::DataType::FLOAT_VECTOR, "embedding").WithDimension(768));

Then you can create a collection with the above schema. If you decide to use the default database, you can safely skip the db_name parameter.

python
client.use_database(
    db_name="my_database"
)

# create the collection
client.create_collection(
    collection_name="prod_collection",
    schema=schema
)
rust
client
    .use_database("my_database")
    .await?;

client
    .create_collection(
        CreateCollectionRequest::builder()
            .collection_name("prod_collection")
            .schema(schema)
            .build()?,
    )
    .await?;
c++
#include <iostream>

auto status = client->UseDatabase("my_database");
if (!status.IsOk()) {
    std::cout << status.Message() << std::endl;
}

status = client->CreateCollection(
    milvus::CreateCollectionRequest()
        .WithCollectionName("prod_collection")
        .WithCollectionSchema(schema));
if (!status.IsOk()) {
    std::cout << status.Message() << std::endl;
}

Step 4: Create indexes.​

You need to create indexes for all vector fields and, optionally, for selected scalar fields.

python
index_params = client.prepare_index_params()

# Add indexes
index_params.add_index(
    field_name="embedding",
    index_type="AUTOINDEX",
    metric_type="COSINE"
)

index_params.add_index(
    field_name="product_name", 
    index_type="AUTOINDEX"
)

client.create_index(
    db_name="my_database",
    collection_name="prod_collection",
    index_params=index_params
)
rust
client
    .create_index(
        CreateIndexRequest::builder()
            .collection_name("prod_collection")
            .index_params(vec![
                IndexParam::new()
                    .field_name("embedding")
                    .index_type(IndexType::AutoIndex)
                    .metric_type(MetricType::Cosine),
                IndexParam::new()
                    .field_name("product_name")
                    .index_type(IndexType::AutoIndex),
            ])
            .build()?,
    )
    .await?;
c++
#include <iostream>

milvus::IndexDesc index_embedding("embedding", "embedding", milvus::IndexType::AUTOINDEX, milvus::MetricType::COSINE);
milvus::IndexDesc index_name("product_name", "product_name", milvus::IndexType::AUTOINDEX);

auto status = client->CreateIndex(
    milvus::CreateIndexRequest()
        .WithDatabaseName("my_database")
        .WithCollectionName("prod_collection")
        .WithIndexes({index_embedding, index_name}));
if (!status.IsOk()) {
    std::cout << status.Message() << std::endl;
}

Step 5: Load the collection.​

Once indexes are ready, load the collection into memory.

python
client.load_collection(
    db_name="my_database",
    collection_name="prod_collection"
)
rust
client
    .load_collection(
        LoadCollectionRequest::builder()
            .collection_name("prod_collection")
            .build()?,
    )
    .await?;
c++
#include <iostream>

auto status = client->LoadCollection(
    milvus::LoadCollectionRequest()
        .WithDatabaseName("my_database")
        .WithCollectionName("prod_collection"));
if (!status.IsOk()) {
    std::cout << status.Message() << std::endl;
}

Step 6: Import data.​

Once everything is set up, you can import the processed data. The following example assumes that you have stored the processed data in an external storage bucket.

For the data format in your bucket or storage integrations, refer to Format Options.

python
from pymilvus.bulk_writer import bulk_import

# The path should be relative to the root 
# of a zilliz cloud volume or an external storage
OBJECT_URLS = [[                                                                                                             
    "https://s3.us-west-2.amazonaws.com/your-bucket/path/in/external/storage.json"                                           
]]                                                                                                                           
                                                                                                                               
ACCESS_KEY = "YOUR_STORAGE_ACCESS_KEY"                                                                                       
SECRET_KEY = "YOUR_STORAGE_SECRET_KEY"

res = bulk_import(
    api_key="YOUR_ZILLIZ_API_KEY",
    url="https://api.cloud.zilliz.com",
    cluster_id="inxx-xxxxxxxxxxxxxxxxxxx",
    db_name="my_database",
    collection_name="prod_collection",
    object_urls=OBJECT_URLS,
    access_key=ACCESS_KEY,
    secret_key=SECRET_KEY
)

# job-xxxxxxxxxxxxxxxxxxxxx
rust
use milvus::v2::bulk_import::{BulkImport, BulkImportConfig, BulkImportRequest};

let config = BulkImportConfig::new()
    .url("https://api.cloud.zilliz.com")
    .api_key("YOUR_ZILLIZ_API_KEY");
let import_client = BulkImport::new(&config)?;

let request = BulkImportRequest::builder()
    .database_name("my_database")
    .collection_name("prod_collection")
    .cluster_id("inxx-xxxxxxxxxxxxxxxxxxx")
    .object_url("https://s3.us-west-2.amazonaws.com/your-bucket/path/in/external/storage.json")
    .access_key("YOUR_STORAGE_ACCESS_KEY")
    .secret_key("YOUR_STORAGE_SECRET_KEY")
    .build()?;

let resp = import_client.bulk_import(request).await?;
// job-xxxxxxxxxxxxxxxxxxxxx
c++
#include "milvus/BulkImport.h"

std::string CLOUD_PLATFORM_ENDPOINT = "https://api.cloud.zilliz.com";
std::string API_KEY = "YOUR_ZILLIZ_API_KEY";

nlohmann::json res = milvus::BulkImport::CreateImportJobs(
    CLOUD_PLATFORM_ENDPOINT,        // url
    "prod_collection",              // collection_name
    {"https://s3.us-west-2.amazonaws.com/your-bucket/path/in/external/storage.json"},  // files
    "my_database",                  // db_name
    API_KEY,                        // api_key
    "",                             // partition_name
    {{"clusterId", "inxx-xxxxxxxxxxxxxxxxxxx"},
     {"accessKey", "YOUR_STORAGE_ACCESS_KEY"},
     {"secretKey", "YOUR_STORAGE_SECRET_KEY"}});

// job-xxxxxxxxxxxxxxxxxxxxx

With the returned job ID, you can monitor its progress.

python
import json
from pymilvus.bulk_writer import get_import_progress

# Get bulk-insert job progress
resp = get_import_progress(
    api_key="YOUR_ZILLIZ_API_KEY",
    url="https://api.cloud.zilliz.com",
    cluster_id="inxx-xxxxxxxxxxxxxxxxxxx",
    job_id="job-xxxxxxxxxxxxxxxxxxxxx",
)

print(json.dumps(resp.json(), indent=4))
rust
use milvus::v2::bulk_import::{GetImportProgressRequest};

let request = GetImportProgressRequest::builder()
    .database_name("my_database")
    .cluster_id("inxx-xxxxxxxxxxxxxxxxxxx")
    .job_id("job-xxxxxxxxxxxxxxxxxxxxx")
    .build()?;

let progress = import_client.get_import_progress(request).await?;
c++
std::string CLOUD_PLATFORM_ENDPOINT = "https://api.cloud.zilliz.com";
std::string API_KEY = "YOUR_ZILLIZ_API_KEY";

nlohmann::json progress = milvus::BulkImport::GetImportJobProgress(
    CLOUD_PLATFORM_ENDPOINT,        // url
    "job-xxxxxxxxxxxxxxxxxxxxx",    // job_id
    "my_database",                  // db_name
    API_KEY);                       // api_key

Step 7: Serve your data.​

Once the import completes, you can invite users to consume your data through searches, queries, and hybrid searches.

python
query_vector = [0.3580376395471989, -0.6023495712049978, 0.18414012509913835, -0.26286205330961354, 0.9029438446296592, ...]
res = client.search(
    db_name="my_database",
    collection_name="prod_collection",
    anns_field="embedding",
    data=[query_vector],
    limit=3,
    output_fields=["product_name"]
)
rust
let query_vector = vec![0.3580376395471989f32, -0.6023495712049978, 0.18414012509913835, -0.26286205330961354, 0.9029438446296592 /* ...remaining dims */];

let search = client
    .search(
        SearchRequest::builder()
            .collection_name("prod_collection")
            .vector_field("embedding")
            .vectors(SearchVectors::Float(vec![query_vector]))
            .output_fields(["product_name"])
            .limit(3)
            .build()?,
    )
    .await?;
c++
#include <iostream>

std::vector<std::vector<float>> query_vectors = {
    {0.3580376395471989f, -0.6023495712049978f, 0.18414012509913835f, -0.26286205330961354f, 0.9029438446296592f /* ...remaining dims */}};

milvus::SearchResponse response;
auto status = client->Search(
    milvus::SearchRequest()
        .WithDatabaseName("my_database")
        .WithCollectionName("prod_collection")
        .WithAnnsField("embedding")
        .WithLimit(3)
        .WithFloatVectors(std::move(query_vectors))
        .AddOutputField("product_name"),
    response);
if (!status.IsOk()) {
    std::cout << status.Message() << std::endl;
}
Contact Sales
Ctrl I