Skip to main content
Version: User Guides (Cloud)

Migrate from Elasticsearch

This topic offers a step-by-step guide on how to migrate data from Elasticsearch to Zilliz Cloud. By doing so, you can unlock the full potential of business data with enhanced scalability, performance, and ease of use.

Before you start​

Make sure the following steps are performed:

Connect to your Elasticsearch cluster​

To interact with your Elasticsearch cluster on Zilliz Cloud, establish a connection using either of the methods based on your deployment:

  • Connect via Cloud ID: applies to an Elasticsearch cluster that runs on Elastic Cloud. If this is the case, specify the cloud ID and API key of the Elasticsearch cluster.

  • Connect via URL: applies to an Elasticsearch cluster that runs on premises. If this is the case, specify the cluster URL and a username and password credential.

Connect to your cluster and Get API key information can guide you on obtaining the required information.

connect_to_es

Transition from Elasticsearch index to Zilliz Cloud collection​

In Zilliz Cloud, collections are analogous to Elasticsearch indexes. To migrate:

  1. Connect to your Elasticsearch cluster.

  2. Select a source index and desired fields from the left frame; they will appear in the Zilliz Cloud collection on the right frame.

migrate_index

New collections are named Collection_n by default, where n is a unique numerical identifier.

During migration, select:

  • One dense_vector field. If no vector field is selected, data migration will fail.

  • Multiple other fields from the source index. Make sure the index fields you select to migrate have not been excluded in Elasticsearch. Otherwise, data migration will fail. For more information, refer to Elasticsearch documentation.

Considerations:

  • Elasticsearch's dense_vector field maps to FloatVector in Zilliz Cloud. Choose either Euclidean (L2) or Inner Product (IP) as the metric type for the FloatVector field.

  • The vector data dimension is retained from the source index. Ensure the dense_vector field dimension is at least 32 for capacity-optimized CU type clusters to prevent migration errors. See Select the Right CU for details.

  • See Field mapping reference for details on field mappings.

In the Primary Key section, select a field as the primary key for the collection. You can select the metadata field _id or any other field from the source Elasticsearch index as the primary key. If you use _id, set its data type to either Int64 or VarChar.

In the Dynamic Schema section, decide if you want to enable dynamic schema for the collection. For more information, see Enable Dynamic Field.

Verify the migration results​

Once the migration status switches from MIGRATING to SUCCESSFUL, the migration is complete.

📘Notes

Data inserted into the Elasticsearch source index during migration may not synchronously migrate.

verify_collection_es

Field mapping reference​

Review the table below to understand how Elasticsearch field types map to Zilliz Cloud field types.

Elasticsearch Field TypeZilliz Cloud Field TypeDescription
dense_vectorFloatVectorVector dimensions remain unchanged. Specify L2 or IP as the metric type.
keywordVarCharSet Max Length (1 to 65,535). Strings exceeding the limit can trigger migration errors.
textVarCharSet Max Length (1 to 65,535). Strings exceeding the limit can trigger migration errors.
longInt64-
integerInt32-
doubleDouble-
floatFloat-
booleanBool-