Migrate from Qdrant to Zilliz Cloud

Qdrant is a vector database that provides similarity search capabilities. Migrating data from Qdrant to Zilliz Cloud allows users to leverage Zilliz Cloud's advanced search and analytics features while maintaining compatibility with the multi-vector structure supported by Qdrant.

The migration process is structured into these steps:

1. Connect to data source: Enter your Qdrant cluster endpoint and API key to establish a connection.

2. Select source and target:

   • Choose one or more Qdrant collections for migration.

   • Select an existing Zilliz Cloud cluster as the target.

3. Configure schema: Verify that field types are correctly mapped between Qdrant and Zilliz Cloud. For detailed mapping rules, refer to Mapping rules.

Mapping rules

The following table summarizes how field types in Pinecone are mapped to Zilliz Cloud field types, along with details on any customization options.

Qdrant Field Type	Zilliz Cloud Field Type	Description
Primary key	Primary key	The primary key from Qdrant is automatically mapped as the primary key in Zilliz Cloud. When migrating data, you can enable Auto ID. However, if you do so, the original primary key values from your source collection will be discarded.
Dense vector	FLOAT_VECTOR	Dense vector fields are transferred as FLOAT_VECTOR with no modifications required.
Sparse vector	SPARSE_FLOAT_VECTOR	If the sparse vector field in a sample row is non-empty, it is mapped by default; otherwise, it remains unselected in schema mapping.
Payload	Dynamic field	By default, Qdrant's payload is mapped as a dynamic schema in Zilliz Cloud. For more information, refer to Dynamic Field. When migrating data, consider converting dynamic fields into fixed fields when their patterns have stabilized and you want to enforce strict data types and optimized index configurations for these fields.
Integer	INT64	If a payload field is of type Integer and you convert it to a fixed field, it becomes an INT64 type.
Float	DOUBLE	If a payload field is of type Float and you convert it to a fixed field, it becomes a DOUBLE type.
Bool	BOOL	If a payload field is of type Bool and you convert it to a fixed field, it becomes a BOOL type.
Keyword	VARCHAR	If a payload field is of type Keyword and you convert it to a fixed field, it becomes a VARCHAR type. Note: The maximum length for this field is fixed at 65,535 bytes and cannot be modified. The capacity calculation is determined by the actual field length.
Geo	JSON	If a payload field is of type Geo and you convert it to a fixed field, it becomes a JSON type.
Datetime	VARCHAR	If a payload field is of type Datetime and you convert it to a fixed field, it becomes a VARCHAR type. Note: The maximum length for this field is fixed at 65,535 bytes and cannot be modified. The capacity calculation is determined by the actual field length.
UUID	VARCHAR	If a payload field is of type UUID and you convert it to a fixed field, it becomes a VARCHAR type. Note: The maximum length for this field is fixed at 65,535 bytes and cannot be modified. The capacity calculation is determined by the actual field length.
Array<Integer>	ARRAY<INT64>	Supported for adding as new fixed fields (cannot be converted from dynamic).
Array<Float>	ARRAY<DOUBLE>	Supported for adding as new fixed fields (cannot be converted from dynamic).
Array<Bool>	ARRAY<BOOL>	Supported for adding as new fixed fields (cannot be converted from dynamic).
Array<Keyword>	ARRAY<VARCHAR>	Supported for adding as new fixed fields (cannot be converted from dynamic).
Array<Geo>	Not supported	Array-of-Geo is not supported in the current release.
Array<Datetime>	ARRAY<VARCHAR>	Supported for adding as new fixed fields (cannot be converted from dynamic).
Array<UUID>	ARRAY<VARCHAR>	Supported for adding as new fixed fields (cannot be converted from dynamic).

Before you start

• The source Qdrant cluster is accessible from the public internet.

• If you have an allowlist configured in your network environment, ensure that Zilliz Cloud IP addresses are added to it. For more information, refer to Zilliz Cloud IPs.

• You have obtained the cluster endpoint and API key with necessary permissions to access the target Qdrant cluster.

• You have been granted the Organization Owner or Project Admin role on Zilliz Cloud. If you do not have the necessary permissions, contact your Zilliz Cloud administrator.

• Make sure the CU size can of the target cluster can accommodate your source data. To estimate the required CU size, use the calculator.

Migrate from Qdrant to Zilliz Cloud

1. Log in to the Zilliz Cloud console.

2. Go to the target project page and select Migrations > Qdrant.

   

3. In the Connect to Data Source step, enter the Cluster Endpoint and API Key that can be used as credentials to access the target Qdrant cluster. Then, click Next.

   📘Notes

   Database Authentication can guide you on obtaining the required connection information.

   

4. In the Select Source and Target step, configure settings for the source Qdrant cluster and target Zilliz Cloud cluster. Then, click Next.

   

5. In the Configure Schema step, set up field mappings between Zilliz Cloud and Qdrant:

   

   1. Confirm field mappings:

      • Zilliz Cloud automatically detects and displays your Qdrant fields alongside their corresponding target fields. For details on how these fields are mapped, refer to Mapping rules.

      • Verify that each Qdrant field is correctly paired with its corresponding target field. You can rename fields as needed, but note that the data type cannot be changed.

   2. Handle metadata fields:

      • Your Qdrant payload fields appear in the Metadata section and are set as dynamic fields by default.

        📘Notes

        Dynamic fields store metadata in a JSON format, enabling more flexible and evolving data structures. For details, refer to Dynamic Field.

        Fixed fields are explicitly defined in your schema with a predetermined structure. They allow you to enforce specific data types and index configurations.

      • To convert a payload field into a fixed field, select the field and click the Convert to Fixed Field icon. Note that Zilliz Cloud samples only 100 rows to extract fields from payloads. To add more fields, click the Settings icon.

      • For payload fields converted to fixed fields, configure the following attributes:

        • Nullable: Decide whether a field can accept null values. This feature is enabled by default. For details, refer to Nullable & Default.

        • Default Value: Specify a default value for a field. For details, refer to Nullable & Default.

        • Partition Key: Optionally designate an INT64 or VARCHAR field as the partition key. Note: Each collection supports only one partition key, and the selected field cannot be nullable. For details, refer to Use Partition Key.

   3. (Optional) Adjust shards:

      • Click Advanced Settings to configure the number of shards for your target collection.

      • For datasets of around 100 million rows, a single shard is typically sufficient.

      • If your dataset exceeds 1 billion rows, contact us to discuss optimal shard configuration for your use case.

6. Click Migrate.

Monitor the migration process

Once you click Migrate, a migration job will be generated. You can check the migration progress on the Jobs page. When the job status switches from In Progress to Successful, the migration is complete.

Post-migration

After the migration job is completed, note the following:

• Index Creation: The migration process does not automatically create indexes for vector fields when migrating from external data sources. You must manually create the index for each vector field. For details, refer to Index Vector Fields.

• Manual Loading Required: After creating the necessary indexes, manually load the collections to make them available for search and query operations. For details, refer to Load & Release.

📘Notes

Once you have completed indexing and loading, verify that the number of collections and entities in the target cluster matches the data source. If discrepancies are found, delete the collections with missing entities and re-migrate them.

Cancel migration job

If the migration process encounters any issues, you can take the following steps to troubleshoot and resume the migration:

1. On the Jobs page, identify the failed migration job and cancel it.

2. Click View Details in the Actions column to access the error log.