In distributed databases, consistency ensures that every node or replica offers the same data view during read or write operations. Zilliz Cloud provides three consistency levels: Strong, Bounded Staleness, and Eventually, with Bounded Staleness set as the default.
Understanding the Balance: The PACELC Theorem
The PACELC theorem postulates that a distributed database must make trade-offs between consistency, availability, and latency. While high consistency guarantees accurate data, it may come at the cost of longer search latency. Conversely, low consistency offers faster search speeds but may compromise on data accuracy. Thus, the appropriate level of consistency depends on the specific use-case scenario.
The strong level is the strictest level ensuring that users always read the most recent data version, and offers the highest accuracy but may lead to increased latency.
This consistency level is best for functional testing and critical applications like online financial systems where data accuracy is paramount.
The bounded staleness level, as its name suggests, permits temporary data inconsistency but typically achieves global consistency after a short period. It offers a balance between latency and accuracy.
Bounded staleness is suitable for systems like video recommendation platforms, where occasional data inconsistencies don't drastically affect performance.
The level of eventually indicates the most relaxed level, allowing data to converge to a consistent state over time without strict read-write order.
It maximizes search performance by sacrificing immediate consistency and therefore is best suitable for the scenarios prioritizing speed over immediate data accuracy, such as displaying product reviews.
Leveraging Guarantee Timestamp for Consistency
To achieve these consistency levels, Zilliz Cloud employs the concept of the Guarantee Timestamp (GuaranteeTs). This timestamp informs query nodes to delay searches or queries until all data up to the GuaranteeTs are available for viewing. Depending on the selected consistency level, the GuaranteeTs value adjusts:
Strong: Matches the latest system timestamp, ensuring all data up to this point is visible before queries.
Bounded Staleness: Slightly predates the latest system timestamp, permitting queries on slightly older data.
Eventually: Sets a minimal value to bypass consistency checks, enabling immediate queries on available data.
To delve deeper into this mechanism, consult the How GuaranteeTs Works section for a detailed breakdown of consistency levels.