Deploy BYOC on GCP
This page describes how to manually create a fully managed Bring-Your-Own-Cloud (BYOC) data plane in your Google Cloud Platform (GCP) Virtual Private Cloud (VPC) using the Zilliz Cloud console and custom GCP configurations.
Zilliz BYOC is currently available in General Availability. For access and implementation details, please contact Zilliz Cloud sales.
Prerequisites
-
You must be a BYOC organization owner.
-
You have enabled the required GCP API services.
Procedure
To deploy BYOC on GCP, Zilliz Cloud needs to assume specific roles to access the Cloud Storage bucket and the GKE cluster within a customer-managed VPC on your behalf. Consequently, Zilliz Cloud needs to gather information about your Cloud Storage bucket, GKE cluster, and VPC, along with the roles necessary for accessing these infrastructure resources.
Within your BYOC organization, click the Create Project and Deploy Data Plane button to start the deployment.
Step 1: Create a project
In this step, you need to set the Zilliz BYOC project name, determine the cloud providers and regions, and the initial project size of your deployment.
-
Set Zilliz BYOC Project Name.
-
Select Cloud Provider and Cloud Region.
-
Configure Initial Project Size.
In a BYOC project, the search service, other database components, and core support services use different Google Compute Engine (GCE) instances. You can set instance types for these services and components.
For details, see Initial project sizes.
-
Determine whether to enable GCP Private Service Connect.
This option allows private connectivity to the clusters within the current project. If you enable this option, you must create a Private Service Connect Endpoint for private connectivity.
Step 2: Set up credentials
In Credential Settings, you must set up the storage and several service accounts for storage access, GKE cluster management, and data-plane deployment.
-
In Google Cloud Platform Project ID, enter the ID of your GCP project.
-
In Storage settings, set Bucket Name and Service Account Email obtained from GCP.
Zilliz Cloud will use the specified bucket as the data-plane storage and access it on your behalf using the specified service account.
For details on setting up the bucket and creating the service account, refer to Create Cloud Storage Bucket and Service Account.
-
In GKE Settings, set GKE Cluster Name and Service Account Email for GKE management.
Zilliz Cloud will use the specified service account to deploy a GKE cluster of the specified name on your behalf and deploy the data plane in the GKE cluster.
For details on creating the service account, refer to Create GKE Service Account.
-
In Cross-Account Settings, set Service Account Name for data-plane deployment.
Once your service account is ready, copy the Zilliz BYOC principal provided in the read-only text box below and paste it into your GCP console to grant Zilliz BYOC the necessary permissions to deploy the data plane of the Zilliz Cloud BYOC project.
For details on creating the cross-account service account, refer to Create a Cross-Account Service Account.
-
Click Next to configure network settings.
Step 3: Configure network settings
In Network Settings, create a VPC and several types of resources, such as subnet names and an optional Private Service Connect Endpoint in the VPC.
-
In Network Settings, set the VPC Name, Subnet Names, and the optional Private Service Connect Endpoint.
In the specified VPC, Zilliz Cloud requires
-
A primary subnet with two secondary subnets,
-
A load balancer subnet, and
-
An optional Private Service Connect endpoint.
Note that Private Service Connect Endpoint is available only when you switch on GCP Private Service Connect in General Settings above.
-
-
Click Next to view the summary.
-
In Deployment Summary, review the configuration settings.
-
Click Create if everything is as expected.
Initial project sizes
The data plane of a Zilliz BYOC project comprises three types of components: Search Services, Other Database Components, and Core Support Services, which use different GCE instances.
In the General settings, determine the GCE instance types for the three data plane components mentioned above. Additionally, you need to specify the number of GCE instances for Core Support Services, which determines the maximum number of clusters that can be created within the project.
There are four predefined project size options, and they are described as follows:
Size | Maximum Cluster Quantity | Maximum Number of Entities (Million) | |
|---|---|---|---|
Performance-optimized CU | Capacity-optimized CU | ||
Small | 3 clusters with 8 to 16 CUs | 10 Million - 25 Million | 40 Million - 80 Million |
Medium | 7 clusters with 16 to 64 CUs | 25 Million - 100 Million | 80 Million - 350 Million |
Large | 12 clusters with 64 to 192 CUs | 100 Million - 300 Million | 350 Million - 1 Billion |
X-Large | 17 clusters with 192 to 576 CUs | 300 Million - 900 Million | 1 Billion - 3 Billion |
You can also choose to customize the settings by selecting Custom in Initial Project Size and adjusting the GCE instance types and counts for all data plane components. If your preferred GCE instance types are not listed, please contact Zilliz support.
View deployment details
After you create a project, you can view its status on the project page.
Suspend & Resume
Suspending a project halts the data plane and terminates all GCE instances associated with the GKE cluster supporting the project. This action does not impact the suspended Zilliz Cloud clusters within the project, which can be resumed once the data plane is restored.
You can only suspend a running project if there are no clusters in the project or all clusters have already been suspended.
Once the status tag on a project card reads Suspended, you cannot manipulate clusters in the project. In such a case, you can click Resume to resume the project. Once the status tag turns to Running again, you can continue manipulating clusters in the project.
Procedures
Create Cloud Storage Bucket and Service Account [READ MORE]
This page describes the procedure for creating and configuring the root storage for your Bring-Your-Own-Cloud (BYOC) project with proper permissions.
Create GKE Service Account [READ MORE]
This page describes how to create and configure a service account for Zilliz Cloud to deploy a Google Kubernetes Engine (GKE) cluster for your Zilliz Cloud project.
Create a Cross-Account Service Account [READ MORE]
This page describes how to create and configure a cross-account service account for Zilliz Cloud to bootstrap your project data plane. This service account grants Zilliz Cloud the necessary permissions to manage VPC resources on your behalf.
Configure a Customer-Managed VPC [READ MORE]
The Zilliz Cloud Bring-Your-Own-Cloud (BYOC) solution enables you to set up a project within your own Virtual Private Cloud (VPC). With a Zilliz Cloud project running in a customer-managed VPC, you gain greater control over your network configurations, allowing you to meet specific cloud security and governance standards required by your organization.
Required Permissions [READ MORE]
This page lists the IAM policies required during the deployment of Zilliz BYOC data plane on your VPC network.
Required GCP API Services [READ MORE]
This page lists the Google Cloud Platform (GCP) API services required to create GCP resources using the Zilliz Cloud Terraform Provider and provides several ways to enable them.