Storage Options

Before importing data, it's important to understand the supported cloud storage options and their corresponding URL formats. This ensures that your requests can be properly processed without undergoing validation errors.

📘Notes

Zilliz Cloud now allows you to import data from any object storage service to any Zilliz Cloud cluster, regardless of the cloud provider hosting the clusters. For instance, you can import data from an AWS S3 bucket to a Zilliz Cloud cluster deployed on GCP.

You are advised to use a bucket or a blob container from the same provider and in the same region as the target cluster to ensure a low-latency, stable experience.

Amazon Simple Storage Service (S3)

• Object access URIs

  URI Style	URI Format
  AWS Object URL, virtual-hosted–style	File: https://<bucket_name>.s3.<region-code>.amazonaws.com/<object_name> Folder: https://<bucket_name>.s3.<region-code>.amazonaws.com/<folder_name>/
  AWS Object URL, path-style	File: https://s3.&lt;region-code&gt;.amazonaws.com/&lt;bucket_name&gt;/&lt;object_name> Folder: https://s3.&lt;region-code&gt;.amazonaws.com/&lt;bucket_name&gt;/&lt;folder_name&gt;/
  AWS S3 URI	File: s3://<bucket_name>/<object_name> Folder: s3://<bucket_name>/<folder_name>/

  For more details, see Methods for accessing a bucket.

• Required permissions

  • s3:GetObject

  • s3:ListBucket

  • s3:GetBucketLocation

  • kms:Decrypt

    📘Notes

    If your bucket or specific items in the bucket are encrypted with a custom KMS ID, you must provide decryption permissions for that KMS ID along with the authentication credentials.

• Credential acquisition

  Based on your data security requirements, you can use either long-term credentials or session tokens during data import:

  • If you prefer to authenticate with long-term credentials, read Authenticate using long-term credentials for details.

  • If you prefer to authenticate with short-term credentials, refer to this FAQ.

Google Cloud Storage

• Object access URIs

  URI Style	URI Format
  GSC public URL	File: https://storage.cloud.google.com/<bucket_name>/<object_name> Folder: https://storage.cloud.google.com/<bucket_name>/<folder_name>/
  GSC gsutil URI	File: gs://<bucket_name>/<object_name> Folder: gs://<bucket_name>/<folder_name>/

  For more details, see Share the object.

• Required permissions

  • storage.objects.get

  • storage.objects.list

• Credential acquisition

  Based on your data security requirements, you can use either long-term credentials or session tokens during data import:

  • If you prefer to authenticate with long-term credentials, read Manage HMAC keys for service accounts for details.

  • If you prefer to authenticate with short-term credentials, refer to this FAQ.

Azure Blob Storage

• Object access URIs

  URI Style	URI Format
  Azure storage blob URI	File: https://<storage_account>.blob.core.windows.net/<container>/<blob> Folder: https://<storage_account>.blob.core.windows.net/<container>/<folder>/

  For more details, see Resource URI Syntax.

• Credential acquisition

  Based on your data security requirements, you can use either long-term credentials or session tokens during data import:

  • If you prefer to authenticate with long-term credentials, read View account access keys for details

  • If you prefer to authenticate with short-term credentials, refer to this FAQ.