Alter Collection Schema

As a collection moves from development to production, its schema often changes. You might add scalar fields such as source_uri or review_status for filtering and application logic, add a new vector field for embeddings generated by your application, add a BM25 Function and its generated sparse vector field for lexical search over existing text, or remove fields and Functions that are no longer used. Alter Collection Schema lets you make supported field and Function changes in place instead of recreating the collection.

📘Notes

• This guide covers schema changes for user-defined fields and for Functions with their generated vector fields in managed collections. For field property changes, such as changing max_length on a VARCHAR field or max_capacity on an ARRAY field, refer to Alter Collection Field. For dynamic field behavior, refer to Dynamic Field and Modify Collection.

• This page describes how to add fields to managed collections. To add a field to an external collection, refer to Alter External Collection Schema.

Limits

Add user-defined fields

• Added user-defined fields must be nullable. Set nullable=True when calling add_collection_field(). For existing entities, the added field is NULL unless you add a scalar field with a default_value.

• Adding user-defined scalar fields is supported in Milvus 2.6.x and later. Adding user-defined vector fields is supported in Milvus 2.6.18 and later.

• Field names must be unique among fields in the collection.

Add a Function and its generated vector field

• Each schema update can add only one Function and one generated vector field.

• The supported Function determines the generated vector field type: BM25 generates a SPARSE_FLOAT_VECTOR field, and MINHASH generates a BINARY_VECTOR field.

• The generated vector field must be a new field. It cannot point to a field that already exists in the collection schema.

• The generated vector field cannot be nullable.

• The input fields used by the Function must already exist in the collection. For this existing-collection workflow, BM25 and MinHash inputs must be VARCHAR. Define a BM25 Function that uses TEXT when you create the collection.

Drop user-defined fields

• You cannot drop the primary key field, partition key field, clustering key field, or the last vector field in a collection.

• You can drop a whole ARRAY<STRUCT> field, but you cannot drop an individual sub-field inside an ARRAY<STRUCT> field.

• You cannot directly drop a field that is used as a Function input field or generated as a Function output field. To remove a Function output field, drop the Function that generates it.

Drop a Function and its generated vector field

• In this schema-change workflow, dropping a Function removes the Function, its generated vector field, and the associated index. Function input fields remain in the collection schema.

• Dropping a Function is rejected if removing its generated vector field would leave the collection without any vector field.

📘Notes

For schema changes outside supported add and drop operations, recreate or migrate the collection.

Add fields and Functions to an existing collection

Choose the workflow based on whether you are adding a user-defined field or a Function that generates a vector field:

• Add user-defined scalar fields when you need new metadata for filtering, query output, or application logic.

• Add user-defined vector fields when your application generates embeddings and writes vector values to Zilliz Cloud.

• Add a Function and its generated vector field when Zilliz Cloud should generate vector values from existing fields, such as BM25 sparse vectors or MinHash signatures from text.

In these cases, the total number of fields cannot exceed the Zilliz Cloud field-count limit. For details, refer to Zilliz Cloud Limits.

Add user-defined scalar fields

Use add_collection_field() to add a user-defined scalar field to an existing collection.

This differs from storing arbitrary keys in the dynamic field: after the schema update is available, the new scalar field becomes a regular part of the collection schema. You can insert or upsert values into it, create indexes on it where supported, use it in queries and search filters, and return it in query or search output.

Because existing entities were inserted before the new field existed, every added user-defined scalar field must be nullable:

• If you add a scalar field with nullable=True and no default_value, existing entities return NULL for the new field.

• If you add a scalar field with nullable=True and default_value, existing entities return the default value instead of NULL.

Scalar filter expressions do not match NULL scalar values. For details, refer to Nullable Fields.

Example: Add a nullable scalar field

The following example adds a nullable source field to an existing collection named product_catalog.

python

from pymilvus import DataType, MilvusClient

client = MilvusClient(uri="YOUR_CLUSTER_ENDPOINT")

client.add_collection_field(
    collection_name="product_catalog",
    field_name="source",
    data_type=DataType.VARCHAR,
    max_length=128,
    nullable=True,
)

After the field is added, entities that already existed in the collection return NULL for source. New entities can set source during insert or upsert.

Example: Add a scalar field with a default value

If existing entities should return a concrete value instead of NULL, specify default_value when adding the field. The following example adds a review_status field and uses "unreviewed" as the default value.

python

from pymilvus import DataType, MilvusClient

client = MilvusClient(uri="YOUR_CLUSTER_ENDPOINT")

client.add_collection_field(
    collection_name="product_catalog",
    field_name="review_status",
    data_type=DataType.VARCHAR,
    max_length=32,
    nullable=True,
    default_value="unreviewed",
)

After the field is added, entities that already existed in the collection return "unreviewed" for review_status. New entities can set a different value or use the default value when no value is provided.

Add StructArray fields

Use add_collection_struct_field() to add a StructArray field that accepts arrays of structs. To add a StructArray field, do as follows:

1. Create a StructSchema that contains the necessary subfields of supported data types. For applicable data types, see Data type support.

2. Reference the StructSchema created above and set the maximum capacity of the field in add_collection_struct_field().

3. Set nullable to True in the request.

Example: Add a nullable StructArray field

python

from pymilvus import DataType, MilvusClient

client = MilvusClient(uri="YOUR_CLUSTER_ENDPOINT")

# Create a struct schema
struct_schema = client.create_struct_field_schema()

# add a scalar field to the struct
struct_schema.add_field("text", DataType.VARCHAR, max_length=65535)
struct_schema.add_field("chapter", DataType.VARCHAR, max_length=512)

# add a vector field to the struct with mmap enabled
struct_schema.add_field("text_vector", DataType.FLOAT_VECTOR, mmap_enabled=True, dim=5)
struct_schema.add_field("chapter_vector", DataType.FLOAT_VECTOR, mmap_enabled=True, dim=5)

client.add_collection_struct_field(
    collection_name="books",
    field_name="chunks",
    struct_schema=struct_schema,
    max_capacity=1024,
    nullable=True
)

After the StructArray field is added, entities that already exist in the collection return null for chunks across all its subfields. When you insert a new entity, ensure that all subfields are either null or have valid values. Inserting an entity with some subfields set to null and others to valid values results in errors.

Add user-defined vector fields

Use add_collection_field() to add a user-defined vector field when your application generates embeddings and writes vector values to Zilliz Cloud.

Every added user-defined vector field must be nullable. Existing entities have NULL for the new vector field until you write vector values through upsert or a backfill workflow. New entities can include the vector field during insert. Vector search skips entities whose vector value is NULL. For details, refer to Nullable Fields.

Example: Add a nullable vector field

The following example adds a nullable dense vector field named embedding_v2 to an existing collection. Set dim to the dimensionality of the embeddings generated by your application.

python

from pymilvus import DataType, MilvusClient

client = MilvusClient(uri="YOUR_CLUSTER_ENDPOINT")

client.add_collection_field(
    collection_name="product_catalog",
    field_name="embedding_v2",
    data_type=DataType.FLOAT_VECTOR,
    dim=768,
    nullable=True,
)

After the field is added, create an index on the new vector field before searching it:

python

index_params = client.prepare_index_params()

index_params.add_index(
    field_name="embedding_v2",
    index_type="AUTOINDEX",
    metric_type="COSINE",
)

client.create_index(
    collection_name="product_catalog",
    index_params=index_params,
)

Existing entities have NULL for embedding_v2 and are skipped when you search on this field. To make existing entities searchable through embedding_v2, write non-NULL vector values through upsert workflows. New entities can include embedding_v2 during insert.

Add a Function and its generated vector field

This Milvus 3.0 schema-change workflow is currently documented for Zilliz Cloud On-Demand Clusters. This page does not establish the first supported Cloud patch or Serving Cluster availability.

Use this workflow to generate a new vector field from data already stored in an existing collection. For example, a BM25 Function reads an existing VARCHAR field and generates a SPARSE_FLOAT_VECTOR field for lexical search, while a MinHash Function generates a BINARY_VECTOR field for near-duplicate detection. This workflow does not add or replace the Function input field.

The operation adds a Function definition, a new vector output field, and a bound index definition:

• A Function definition, such as text_bm25, that reads from existing input fields.

• A new vector output field, such as text_sparse, that stores the Function output, together with an index definition bound to that field.

The supported Function determines the generated vector field type:

Function	Generated vector field type	Typical input field
BM25	SPARSE_FLOAT_VECTOR	A VARCHAR field with analyzer enabled
MINHASH	BINARY_VECTOR	A VARCHAR field

For details about how each Function works, refer to BM25 Function and MinHash Function.

The generated vector field must not already exist in the collection, and it cannot be nullable. The Function input field must already exist. For this existing-collection workflow, use a VARCHAR input. A BM25 Function using a TEXT input must be defined when you create the collection; otherwise, recreate or migrate the collection with the Function in its schema.

Example: Add a BM25 Function and its generated sparse vector field

The following example adds a BM25 Function named text_bm25 and its generated sparse vector field named text_sparse to an existing collection. The collection must already have a VARCHAR field named text with analyzer enabled.

python

from pymilvus import DataType, Function, FunctionType, MilvusClient

client = MilvusClient(uri="YOUR_CLUSTER_ENDPOINT")

sparse_field = client.create_field_schema(
    name="text_sparse",
    data_type=DataType.SPARSE_FLOAT_VECTOR,
    desc="BM25-generated sparse vector field",
)

bm25_function = Function(
    name="text_bm25",
    input_field_names=["text"],
    output_field_names=["text_sparse"],
    function_type=FunctionType.BM25,
)

index_params = client.prepare_index_params()

index_params.add_index(
    field_name="text_sparse",
    index_type="SPARSE_INVERTED_INDEX",
    metric_type="BM25",
    params={
        "inverted_index_algo": "DAAT_MAXSCORE",
        "bm25_k1": 1.2,
        "bm25_b": 0.75,
    },
)

client.add_function_field(
    collection_name="product_catalog",
    field_schema=sparse_field,
    func=bm25_function,
    index_params=index_params,
)

The index_params object must contain exactly one index definition for the new Function output field. The Function, its generated vector field, and the bound index definition are submitted in the same schema change. Do not call create_index() separately after add_function_field().

Conceptually, this operation adds the following Function, generated output field, and bound index definitions:

plaintext

New Function:
  name: "text_bm25"
  type: BM25
  input_field_names: ["text"]
  output_field_names: ["text_sparse"]

New generated output field:
  name: "text_sparse"
  data_type: SPARSE_FLOAT_VECTOR
  nullable: false

Bound index:
  field_name: "text_sparse"
  index_type: SPARSE_INVERTED_INDEX
  metric_type: BM25

After the request succeeds, describe_collection() returns both the new text_bm25 Function and its generated text_sparse vector field in the collection schema. For the complete BM25 search workflow, refer to Full Text Search.

MinHash Functions and their generated binary vector fields support near-duplicate detection. A MinHash Function uses FunctionType.MINHASH and writes to a new BINARY_VECTOR output field. For configuration details, refer to MinHash Function.

Drop fields and Functions from an existing collection

You can remove user-defined fields directly when they are no longer part of your collection model. To remove a Function and its generated vector field, drop the Function; the generated field and its index are removed in the same schema change.

Drop user-defined fields

Use drop_collection_field() to remove a user-defined scalar or vector field that is no longer part of your collection model.

Dropping a field first changes the collection schema and field visibility:

• After drop_collection_field() succeeds, the collection schema is updated: describe_collection() no longer returns the dropped field, and queries or searches can no longer return the field in output_fields or use it in expressions.

• Indexes built on the dropped field are cleaned up as part of the schema update.

Storage cleanup is handled separately from schema cleanup. For details, refer to When is storage space reclaimed after dropping a field?

Example: Drop a user-defined scalar field

The following example assumes that experiment_tag is a user-defined scalar field in product_catalog, and drops it from the collection.

python

from pymilvus import MilvusClient

client = MilvusClient(uri="YOUR_CLUSTER_ENDPOINT")

client.drop_collection_field(
    collection_name="product_catalog",
    field_name="experiment_tag",
)

After dropping a field, you can call describe_collection() to verify that the field is no longer part of the schema.

Example: Drop a StructArray field

The following example assumes that the chunks field is a StructArray field in my_collection, and drops it from the collection.

python

from pymilvus import MilvusClient

client = MilvusClient(uri="YOUR_CLUSTER_ENDPOINT")

client.drop_collection_field(
    collection_name="my_collection",
    field_name="chunks",
)

Example: Drop a user-defined vector field

You can drop a vector field with the same drop_collection_field() method, but the collection must still contain at least one vector field after the drop. This is useful for collections that temporarily carry multiple vector representations and later standardize on one of them.

The following example assumes that image_vector is a user-defined vector field in hybrid_catalog, and that the collection still retains another vector field, such as text_vector.

python

from pymilvus import MilvusClient

client = MilvusClient(uri="YOUR_CLUSTER_ENDPOINT")

client.drop_collection_field(
    collection_name="hybrid_catalog",
    field_name="image_vector",
)

If image_vector is the last vector field in the collection, the drop operation is rejected.

Drop a Function and its generated vector field

Use this operation when you no longer need a Function or its generated vector field, such as a BM25 Function and its generated sparse vector field.

Call drop_function_field() with the Function name. The operation removes the Function, its generated vector field, and the associated index while preserving the Function input fields.

Example: Drop a BM25 Function and its generated sparse vector field

The following example assumes that text_bm25 is a BM25 Function in product_catalog and generates a sparse vector output field named text_sparse.

python

from pymilvus import MilvusClient

client = MilvusClient(uri="YOUR_CLUSTER_ENDPOINT")

client.drop_function_field(
    collection_name="product_catalog",
    function_name="text_bm25",
)

After the operation succeeds, describe_collection() no longer returns the dropped Function or its generated vector field. The Function input fields remain in the schema.

If removing the Function output field would leave the collection without any vector field, the operation is rejected.

FAQ

Which method should I use to add a field or Function?

Use add_collection_field() to add a user-defined scalar field when your application provides scalar values for filtering, query output, or application logic.

Use add_collection_field() to add a user-defined vector field when your application generates embeddings and writes vector values to Zilliz Cloud.

Use add_function_field() when vector values should be generated from existing fields. It adds a Function, its generated vector field, and the bound index definition in the same schema change. This guide shows the BM25 path for lexical search; MinHash Functions generate binary vector fields for near-duplicate detection.

Why must added user-defined fields be nullable?

Existing entities were inserted before the new field existed, so they do not have values for that field. Setting nullable=True lets Zilliz Cloud represent the missing value as NULL until your application writes a value or, for scalar fields, until a default value applies.

This rule applies to user-defined scalar fields and user-defined vector fields added with add_collection_field(). It does not apply to a Function's generated vector field, which cannot be nullable.

What happens to existing entities after I add a user-defined field?

For a user-defined scalar field, existing entities return NULL unless you set a default_value. If you set a default_value, existing entities return that default value.

For a user-defined vector field, existing entities have NULL for the new vector field. Vector search on the added field skips entities whose vector value is NULL. To make existing entities searchable through the new vector field, write non-NULL vector values through upsert or a backfill workflow. New entities can include the new vector field during insert.

Can I add a BM25 Function and its generated sparse vector field to an existing collection?

Yes. If the collection already has a VARCHAR field with analyzer enabled, you can add a BM25 Function and its generated sparse vector field for lexical search. This operation adds the Function, the new SPARSE_FLOAT_VECTOR output field, and the bound index definition in the same schema change. You cannot use an existing TEXT field as the BM25 input in this schema-change workflow. To use TEXT, define the field and BM25 Function when you create the collection; otherwise, recreate or migrate the collection with the Function in its schema.

When calling add_function_field(), provide an index_params object that contains one SPARSE_INVERTED_INDEX index with metric_type="BM25" for the new output field. The index definition is bound to the generated field as part of the same schema change.

How do I drop a Function and its generated vector field?

Call drop_function_field() with the Function name. This operation removes the Function, its generated vector field, and the associated index together while preserving the Function input fields.

Do I need to wait after altering a collection schema?

Usually, no manual wait is required. If your next operation depends on the updated schema, you can call describe_collection() first to confirm the schema that Zilliz Cloud currently returns.

In a distributed deployment, there can be a short propagation window while Zilliz Cloud components refresh collection metadata. If an operation immediately after the schema change fails with a schema-related error, refresh the schema and retry the operation.

When is storage space reclaimed after dropping a field?

Dropping a field removes it from the current schema and normal query/search visibility, but historical data for that field is not physically deleted from object storage immediately.

Storage space can be reclaimed later during compaction. Compaction is a background process that reorganizes existing data files into new, more compact files. After a field is dropped, newly compacted files follow the current schema and omit the dropped field. Zilliz Cloud does not guarantee an immediate or fixed-time storage-space reduction after dropping a field.

What happens if I add a scalar field with the same name as a dynamic field key?

If dynamic field is enabled, you can add a scalar field with the same name as an existing dynamic field key. The new scalar field masks the dynamic field key in normal query output, but the original dynamic data is preserved in $meta.

For example, if existing entities store a dynamic key named source, and you later add a scalar field named source, normal output for source refers to the scalar field. To access the original dynamic value, use the $meta path syntax, such as $meta["source"].