Skip to main content
Version: User Guides (Cloud)

Select the Right CU

Selecting the right Compute Unit (CU) is a crucial step when creating a cluster in Zilliz Cloud. A CU is the basic unit of compute resources used for parallel processing of data, and different CU types comprise varying combinations of CPU, memory, and storage.

Understand CU types

Zilliz Cloud offers these CU types: Performance-optimized, Capacity-optimized, and Cost-optimized.

CU TypeLatencyThroughputCapacity
Performance-optimizedLowHighLow
Capacity-optimizedMediumMediumHigh
Cost-optimizedHigh
Low
High

Performance-optimized CU

  • Tailored for scenarios emphasizing low latency and high throughput.

  • Ideal for real-time applications like generative AI, recommendation systems, chatbots, and more.

Capacity-optimized CU

  • Crafted for handling vast datasets, boasting five times the data capacity of its Performance-optimized counterpart, albeit with subdued search performance.

  • Ideal for large-scale unstructured data search, copyright detection, and identity verification.

Cost-optimized CU

  • A budget-friendly option, best for tasks where response time is not paramount.

  • Provides the same data capacity as the Capacity-optimized CU, but expects higher search latencies.

  • Ideal for offline operations such as data labeling and deduplication.

Select an optimal CU type

Factor in data volume, performance expectations, and budgets while selecting the CU type. Your vector data's magnitude, both in terms of vector count and dimensions, plays a pivotal role in determining cluster resource allocation.

Assess capacity

The table below illustrates the load capacity for each CU type, taking into account the vector dimensions and the total vector count.

Vector DimensionsPerformance-optimized (Max. Vectors per CU)Capacity-optimized (Max. Vectors per CU)Cost-optimized (Max. Vectors per CU)
1288 million25 million25 million
2566 million15 million15 million
5123 million7.5 million7.5 million
7682 million
5 million5 million
10241.5 million3.75 million3.75 million
📘Notes

The above metrics are based on tests considering only primary keys and vectors. If your dataset has extra scalar fields (e.g., id, label, keywords), the actual capacity may deviate. It's prudent to conduct personal tests for a precise evaluation.

Evaluate performance

Performance metrics, notably latency and queries per second (QPS), are vital. The Performance-optimized CU distinctly outperforms its counterparts in latency and throughput, particularly for standard top-k values ranging from 10 to 250.

The following table shows the test result of how each CU type performs in terms of QPS.

top_kQPS for Performance-optimized CU (768-dim 1M vectors)
QPS for Capacity-optimized CU (768-dim 5M vectors)QPS for Cost-optimized CU (768-dim 5M vectors)
1052010030
1004408025
2502706020
10001504010

The following table shows the test result of how each CU type performs in terms of latency.

top_k
Latency of Performance-optimized CU (768-dim 1M vectors)
Latency of Capacity-optimized CU (768-dim 5M vectors)
Latency of Cost-optimized CU (768-dim 5M vectors)
10< 10 ms< 50 ms
< 100 ms
100< 10 ms< 50 ms< 100 ms
250< 10 ms< 50 ms< 100 ms
100010 - 20 ms50 - 100 ms100 - 200 ms

Scenario breakdown

Suppose you are building an image recommendation application with a library of 8 million images. Each image in your library is represented by a 768-dimensional embedding vector. Your goal is to swiftly handle a QPS of 1,000 recommendation requests and deliver the top 100 image recommendations in under 30 milliseconds.

To select the right CU for this requirement, follow these steps:

  1. Evaluate Latency: The Performance-optimized CU is the only type that meets the 30-millisecond latency requirement.

  2. Assess Capacity: A single Performance-optimized CU accommodates 2 million 768-dimensional vectors. To store all 8 million vectors, you would need at least 4 CUs.

  3. Check Throughput: With a top-k setting of 100, the Performance-optimized CU can achieve a QPS of 440. To sustain a consistent 1,000 QPS, you would need to triple the number of replicas.

In conclusion, for this scenario, the Performance-optimized CU is your best bet. A configuration of 3 replicas, with each replica consisting of 4 CUs, should serve you perfectly.