Skip to main content
Version: User Guides (Cloud)

Set Collection TTL

Zilliz Cloud can automatically expire entities through a Time-to-Live (TTL) policy. Expired entities stop appearing in query and search results immediately, and are physically removed from storage on the next compaction cycle — typically within 24 hours.

There are two TTL modes:

  • Collection-level TTL — one retention window shared by every entity, set through the collection.ttl.seconds property.

  • Entity-level TTL — each entity carries its own absolute expiration time in a dedicated TIMESTAMPTZ field, marked as the TTL field through the ttl_field property.

📘Notes

This feature applies only to managed collections.

Limits​

  • The two TTL modes are mutually exclusive. A collection cannot have both collection.ttl.seconds and ttl_field set at the same time. To switch, see Migrate between the two modes.

  • Collection-level TTL applies one window to the whole collection. If a single row needs a different lifetime, use entity-level TTL.

  • The field for entity-level TTL must be TIMESTAMPTZ. Other types are rejected.

  • One TTL field per collection. The schema may contain multiple TIMESTAMPTZ fields, but only one can be named in ttl_field.

  • Dropping ttl_field does not resurface expired entities. To restore an expired entity, upsert it with a NULL or future expiration timestamp.

Overview​

Expand

When to use TTL​

TTL is the right tool when retention is a policy — you know ahead of time that certain entities should eventually go away, and you want the cluster to enforce it without you writing a cron job.

Typical scenarios:

  • Time-windowed datasets. Keep only the last N days of logs, metrics, events, or short-lived feature caches.

  • Multi-tenant collections. Different tenants have different retention windows in the same collection.

  • Per-record retention policies. Per-document lifetime in IoT pipelines, document stores, or MLOps feature stores.

  • Hot / cold data mix. Short-lived entities coexist with long-term ones in the same collection.

  • Compliance-driven expiration. GDPR-style data minimization where each record carries its own "delete by" date.

  • Business-time expiration. An entity represents a record that is only valid until some absolute moment (a campaign ending, a session expiring).

📘Notes

Expired entities will not appear in any search or query results. However, they may stay in the storage until the subsequent data compaction, which should be carried out within the next 24 hours.

TTL modes​

The two modes answer different retention questions:

  • Collection-level TTL applies a single retention duration to every entity. Each entity expires at insert_ts + ttl_seconds.

  • Entity-level TTL lets every entity store its own absolute expiration time in a TIMESTAMPTZ field. A NULL in that field means the entity never expires.

A collection uses one mode at a time — the two are mutually exclusive. Switching between them is a multi-step operation; see Migrate between the two modes.

Use this table to pick a mode:

If your situation is…

Use

Every entity in the collection should follow the same retention window

Collection-level TTL

Retention is "from the moment of insert, keep N seconds"

Collection-level TTL

Different entities need different lifetimes in the same collection (per-tenant, hot/cold, per-document)

Entity-level TTL

Retention is an absolute wall-clock time (for example, 2027-01-01T00:00:00Z)

Entity-level TTL

Retention is driven by a business timestamp, not the insert timestamp

Entity-level TTL

You want to refresh or extend an entity's lifetime after insert

Entity-level TTL

Some entities should never expire while others should

Entity-level TTL (use NULL for the immortal ones)

Set collection-level TTL​

Use collection-level TTL when every entity in the collection should follow the same retention window.

Enable on a new collection​

Pass collection.ttl.seconds (integer, in seconds) through the properties map at creation time.

from pymilvus import MilvusClient, DataType

client = MilvusClient(uri="YOUR_CLUSTER_ENDPOINT")

schema = client.create_schema(auto_id=False, enable_dynamic_field=False)
schema.add_field("id", DataType.INT64, is_primary=True, auto_id=False)
schema.add_field("vector", DataType.FLOAT_VECTOR, dim=128)

index_params = client.prepare_index_params()
index_params.add_index(
    field_name="vector", index_type="AUTOINDEX", metric_type="COSINE"
)

client.create_collection(
    collection_name="my_collection",
    schema=schema,
    index_params=index_params,
    properties={
        "collection.ttl.seconds": 1209600  # 14 days
    },
)

Enable on an existing collection​

Call alter_collection_properties with collection.ttl.seconds in the properties map to apply TTL to a collection that is already in use.

from pymilvus import MilvusClient, DataType

client = MilvusClient(uri="YOUR_CLUSTER_ENDPOINT")

# Assumes "my_collection" was created earlier without TTL
schema = client.create_schema(auto_id=False, enable_dynamic_field=False)
schema.add_field("id", DataType.INT64, is_primary=True, auto_id=False)
schema.add_field("vector", DataType.FLOAT_VECTOR, dim=128)

index_params = client.prepare_index_params()
index_params.add_index(
    field_name="vector", index_type="AUTOINDEX", metric_type="COSINE"
)

if not client.has_collection("my_collection"):
    client.create_collection(
        collection_name="my_collection",
        schema=schema,
        index_params=index_params,
    )

client.alter_collection_properties(
    collection_name="my_collection",
    properties={"collection.ttl.seconds": 1209600},
)

Drop the TTL setting​

If you decide to keep the data in a collection indefinitely, you can simply drop the TTL setting from that collection.

from pymilvus import MilvusClient

client = MilvusClient(uri="YOUR_CLUSTER_ENDPOINT")

client.drop_collection_properties(
    collection_name="my_collection",
    property_keys=["collection.ttl.seconds"],
)

Set entity-level TTL
Private Preview
​

Entity-level TTL lets each entity carry its own absolute expiration time. The time is stored in a dedicated TIMESTAMPTZ column that you declare in the schema, and you mark that column as the TTL field through the ttl_field collection property.

Enable on a new collection​

Enabling entity-level TTL at creation time takes two additions in the same create_collection call: a TIMESTAMPTZ field in the schema, and the ttl_field property pointing to that field.

from pymilvus import MilvusClient, DataType

client = MilvusClient(uri="YOUR_CLUSTER_ENDPOINT")

schema = client.create_schema(enable_dynamic_field=False)
schema.add_field("id", DataType.INT64, is_primary=True, auto_id=False)
schema.add_field("expire_at", DataType.TIMESTAMPTZ, nullable=True)
schema.add_field("vector", DataType.FLOAT_VECTOR, dim=128)

index_params = client.prepare_index_params()
index_params.add_index(field_name="vector", index_type="AUTOINDEX",
                       metric_type="COSINE")

client.create_collection(
    collection_name="my_collection",
    schema=schema,
    index_params=index_params,
    properties={"ttl_field": "expire_at"},
)

Once the collection exists, insert entities with ISO 8601 timestamp strings.

import random
from pymilvus import MilvusClient

client = MilvusClient(uri="YOUR_CLUSTER_ENDPOINT")

# Assumes "my_collection" was created earlier with \`ttl_field\`: "expire_at"
rows = [
    # Never expires
    {"id": 1, "expire_at": None,
     "vector": [random.random() for _ in range(128)]},
    # Expires at 2026-12-31 UTC midnight
    {"id": 2, "expire_at": "2026-12-31T00:00:00Z",
     "vector": [random.random() for _ in range(128)]},
    # Shanghai local time — normalized to UTC internally
    {"id": 3, "expire_at": "2027-01-01T00:00:00+08:00",
     "vector": [random.random() for _ in range(128)]},
]

client.insert("my_collection", rows)

On every query and vector search, the server auto-injects the TTL filter — you never write one yourself, and expired entities never appear in the results:

from pymilvus import MilvusClient

client = MilvusClient(uri="YOUR_CLUSTER_ENDPOINT")

client.load_collection("my_collection")

# Expired rows are filtered out automatically
results = client.query(
    collection_name="my_collection",
    filter="id >= 0",
    output_fields=["id", "expire_at"],
    limit=10,
)
print(results)

The same auto-filter applies to client.search().

To extend an entity's lifetime before compaction physically removes it, upsert with a later expiration timestamp — or None — to return the entity to the queryable set.

import random
from pymilvus import MilvusClient

client = MilvusClient(uri="YOUR_CLUSTER_ENDPOINT")

client.upsert("my_collection", [
    {"id": 2,
     "vector": [random.random() for _ in range(128)],
     "expire_at": "2028-01-01T00:00:00Z"},
])

Enable on an existing collection​

If the collection already exists and does not have collection.ttl.seconds set, add a TIMESTAMPTZ column with add_collection_field, then mark it as the TTL field with alter_collection_properties. Optionally upsert historical rows to backfill their expiration timestamps — rows you do not backfill keep NULL and never expire.

import random
from pymilvus import MilvusClient, DataType

client = MilvusClient(uri="YOUR_CLUSTER_ENDPOINT")

# Step 1 — add a TIMESTAMPTZ column to the schema
client.add_collection_field(
    collection_name="my_collection",
    field_name="expire_at",
    data_type=DataType.TIMESTAMPTZ,
    nullable=True,
)

# Step 2 — mark the new column as the TTL field
client.alter_collection_properties(
    collection_name="my_collection",
    properties={"ttl_field": "expire_at"},
)

# Step 3 (optional) — backfill expiration timestamps for historical rows
client.upsert("my_collection", [
    {"id": 1,
     "vector": [random.random() for _ in range(128)],
     "expire_at": "2026-12-31T00:00:00Z"},
])

Drop the TTL setting​

Call drop_collection_properties with ttl_field in property_keys to stop per-entity expiration. The TIMESTAMPTZ column itself remains on the schema — you can still query on it as a regular field.

from pymilvus import MilvusClient

client = MilvusClient(uri="YOUR_CLUSTER_ENDPOINT")

client.drop_collection_properties(
    collection_name="my_collection",
    property_keys=["ttl_field"],
)

Dropping ttl_field disables the automatic filter for future queries, but entities that had already expired are not automatically surfaced again. To make a previously-expired entity visible, upsert it with a None or future expiration timestamp — that is the only way to restore access to expired rows within the same load session.

Migrate between the two modes
Private Preview
​

The two TTL modes are mutually exclusive, so switching between them is a multi-step operation.

Switch from collection-level to entity-level TTL​

If your collection was created with collection.ttl.seconds and you want to switch to per-entity expiration, follow these four steps. Skipping Step 1 causes Step 3 to fail with collection TTL is already set, cannot be set ttl field.

import random
from pymilvus import MilvusClient, DataType

client = MilvusClient(uri="YOUR_CLUSTER_ENDPOINT")

# Assumes "my_collection" already exists with \`collection.ttl.seconds\` set.
# Step 1 — disable collection-level TTL (mandatory; the two modes are mutually exclusive)
client.drop_collection_properties(
    collection_name="my_collection",
    property_keys=["collection.ttl.seconds"],
)

# Step 2 — add a TIMESTAMPTZ column to the schema
client.add_collection_field(
    collection_name="my_collection",
    field_name="expire_at",
    data_type=DataType.TIMESTAMPTZ,
    nullable=True,
)

# Step 3 — set the ttl_field property on the column you just added
client.alter_collection_properties(
    collection_name="my_collection",
    properties={"ttl_field": "expire_at"},
)

# Step 4 (optional) — backfill expiration timestamps for historical entities
client.upsert("my_collection", [
    {"id": 1,
     "vector": [random.random() for _ in range(128)],
     "expire_at": "2026-12-31T00:00:00Z"},
])

Historical entities for which you do not backfill expire_at will have NULL in that column, meaning they never expire. Backfill only the rows that should have a finite lifetime.

Switch from entity-level to collection-level TTL​

To move in the other direction, drop ttl_field and set collection.ttl.seconds:

from pymilvus import MilvusClient

client = MilvusClient(uri="YOUR_CLUSTER_ENDPOINT")

# Assumes "my_collection" already exists with \`ttl_field\` set.
client.drop_collection_properties(
    collection_name="my_collection",
    property_keys=["ttl_field"],
)
client.alter_collection_properties(
    collection_name="my_collection",
    properties={"collection.ttl.seconds": 1209600},  # 14 days
)

FAQs​

When does data expire due to TTL settings?​

Currently, the data expires based on the time point at which it was inserted or upserted. Expired data will not be displayed in search results. For details, refer to Examples.

When will the expired data be physically deleted?​

Once the data expires, it will not be included in any search results. However, it will be physically deleted only after the subsequent system compaction, according to your cluster's compaction policies.

If you need to delete the data shortly after it expires, contact us.

When will the CU capacity decrease?​

The CU capacity of a cluster is whichever is higher between memory usage and storage usage. If storage usage applies, you can view the decrease in the CU capacity on the Zilliz Cloud console after the expired data is physically deleted.