Skip to main content
Version: User Guides (Cloud)

Manage Volumes (SDK)

This page explains how you can use a volume when managing your data on Zilliz Cloud. For details about managing volumes via web console, see Manage Volumes (Console).

📘Note

You can only create volumes on AWS and Google Cloud. If you need to use a volume on Azure, please contact support.

To import, merge, or migrate data from a volume to a cluster, ensure that the volume and the cluster are within the same cloud region.

Create, list, and delete volumes​

You can manage the lifecycle of a volume by creating a volume, listing all available volumes, and deleting a volume that you do not need, according to your service requirements.

Initiate a volume manager​

A volume manager maintains the connection to Zilliz Cloud's volume service. You need to initiate a volume manager before managing volumes.

from pymilvus.bulk_writer.volume_manager import VolumeManager

volume_manager = VolumeManager(
    cloud_endpoint="https://api.cloud.zilliz.com",
    api_key="YOUR_API_KEY"
)

Create a volume​

A volume is specific to a Zilliz Cloud project. When creating a volume, you need to provide the project ID, region ID, and the name of the volume, as follows:

volume_manager.create_volume(
    project_id="proj-xxxxxxxxxxxxxxxxxxxxxxx", 
    region_id="aws-us-west-1", 
    volume_name="my_volume"
)

print(f"\nVolume my_volume created")

# Volume my_volume created

In the command above,

  • regionId: The region of the volume to create must match the cloud provider and region of the target cluster you plan to import or migrate data into.

  • volumeName: The name of the volume to create must be unique across the organization, no longer than 64 characters, start with a letter or underscore, and contain only letters, digits, hyphens, and underscores.

List volumes​

You can check the volumes already created within a specific Zilliz Cloud project as follows:

volume_list = volume_manager.list_volumes(
    project_id="proj-xxxxxxxxxxxxxxxxxxxxxxx",
    current_page=1, 
    page_size=10
)

print(f"\nlistVolumes results: \n", volume_list.json()['data'])

# listVolumes results: 
# 
# {
#     "count": 1,
#     "currentPage": 1,
#     "pageSize": 10,
#     "volumes": [
#         {
#             "volumeName": "my_volume"
#         }        
#     ]
# }

Delete a volume​

You can delete a volume once it is no longer needed. To delete a volume, do as follows:

volume_manager.delete_volume(
    volume_name="my_volume"
)

print(f"\nVolume my_volume deleted")

# Volume my_volume deleted

Upload data into a volume​

Once a volume is ready, upload your data onto the volume.

Initiate a volume file manager​

A volume file manager maintains the connection to a specific volume on Zilliz Cloud's volume service. You need to initiate a volume file manager before uploading files to the volume.

from pymilvus.bulk_writer.volume_file_manager import VolumeFileManager

volume_file_manager = VolumeFileManager(
    cloud_endpoint='https://api.cloud.zilliz.com',
    api_key='YOUR_API_KEY',
    volume_name='my_volume',
)

Upload files​

Once the volume file manager is ready, use it to upload files to the specified volume. The following example uploads the local file at the source file path to the target file path within the volume.

result = volume_file_manager.upload_file_to_volume(
    source_file_path="/path/to/your/local/data/file", 
    target_volume_path="data/"
)

print(f"\nuploadFileToVolume results: {result}")

# uploadFileToVolume results: 
# 
# {
#     "volumeName": "my_volume",
#     "path": "data/"
# }