Migrate from pgvector to Zilliz Cloud

PostgreSQL is a robust, open-source, object-relational database engine renowned for its extensibility, data integrity, and performance. By utilizing the pgvector extension, PostgreSQL gains the capability to store and manage vector data.

If you have PostgreSQL databases with pgvector installed—whether on-premises or cloud-hosted, you can seamlessly migrate them to your Zilliz Cloud cluster. This migration process involves establishing a connection with your existing source database and replicating its data from the source tables to the corresponding target collections on Zilliz Cloud.

Limits

• Currently, you can migrate the following PostgreSQL data types: vector, text/varchar/date/time/json, bigint, integer, smallint, double precision, real, boolean, array. If your table has fields with unsupported data types, you can choose not to migrate those fields or submit a support ticket. For information on how PostgreSQL data types are mapped to Zilliz Cloud, refer to Field mapping reference.

• For each migration task, you can select only one vector field from each source table.

• Each migration task is limited to a single source PostgreSQL database. You may enable multiple migration jobs if you have data in multiple source databases.

Before you start

Make sure the following prerequisites are met:

• The source PostgreSQL database is accessible from the public internet.

• You have been granted the Organization Owner or Project Admin role. If you do not have the necessary permissions, contact your Zilliz Cloud administrator.

Migrate from PostgreSQL to Zilliz Cloud

You can migrate source data to a Zilliz Cloud cluster of any plan tier, provided its CU size can accommodate the source data.

1. Log in to the Zilliz Cloud console.

2. Go to the target project and select Migrations > From pgvector.

3. In the Connect to Data Source step, enter the endpoint of the source PostgreSQL database in the Database Endpoint field, provide the username and password associated with the database, and click Next.

   📘Notes

   For details on connection information, refer to Connecting to the Database.

4. In the Select Source and Target step, configure settings for the source database and target Zilliz Cloud cluster. Then, click Next.

   📘Notes

   Each table you choose to migrate from PostgreSQL must include a vector field.

5. In the Configure Schema step,

   1. Verify the data mapping between your PostgreSQL data and the corresponding Zilliz Cloud data types. Zilliz Cloud has a default mechanism for mapping PostgreSQL data types to its own, but you can review and make necessary adjustments. Currently, you can rename fields, but cannot change the underlying data types.

   2. (Optional) In Advanced Settings, configure Dynamic Field and Partition Key. For more information, refer to Dynamic data fields and Use Partition Key.

   3. (Optional) In General Information, customize the target collection name and description. The collection name must be unique in each cluster. If the name duplicates an existing one, rename the collection.

6. Click Migrate.

Monitor the migration process

Once you click Migrate, a migration job will be generated. You can check the migration progress on the Jobs page. When the job status switches from IN PROGRESS to SUCCESSFUL, the migration is complete.

📘Notes

After migration, verify that the number of collections and entities in the target cluster matches the data source. If discrepancies are found, delete the collections with missing entities and re-migrate them.

Cancel migration job

If the migration process encounters any issues, you can take the following steps to troubleshoot and resume the migration:

1. On the Jobs page, identify the failed migration job and cancel it.

2. Click View Details in the Actions column to access the error log.

Field mapping reference

Review the table below to understand how PostgreSQL field types map to Zilliz Cloud field types.

PostgreSQL Field Type	Zilliz Cloud Field Type	Description
vector	FloatVector	Vector dimensions remain unchanged. Specify L2 or IP as the metric type.
text/varchar/date/time/json	VarChar	Set Max Length (1 to 65,535). Strings exceeding the limit can trigger migration errors.
bigint	Int64	-
integer	Int32	-
smallint	int16	-
double precision	Double	-
real	Float	-
boolean	Bool	-
array	array	-