Skip to main content
Version: User Guides (BYOC)

Create S3 Bucket and IAM Role

This page describes the procedure for creating and configuring the root storage for a Bring-Your-Own-Cloud (BYOC) project with proper permissions.

📘Notes

Zilliz BYOC is currently available in General Availability. For access and implementation details, please contact Zilliz Cloud sales.

Best practices for the S3 bucket​

The bucket you specify during the project deployment will be used as the root storage for the clusters created in the project. Before you create your S3 bucket, review the following best practices:

  • The S3 bucket must be in the same AWS region as the project deployment.

  • All clusters in a project share the S3 bucket created during the project deployment. Zilliz Cloud recommends using an S3 bucket dedicated to the project and not sharing it with other services and resources.

Procedure​

You can use the AWS console to create the bucket and role. As an alternative, you can use the Terraform script Zilliz Cloud provides to bootstrap the infrastructure for your Zilliz Cloud project on AWS. For details, refer to Terraform Provider.

Step 1: Create the S3 bucket​

In this step, you will create an S3 bucket on AWS for the BYOC project deployment. If you prefer to use an existing S3 bucket, ensure that the bucket is in the same region as the BYOC project. Once it is created, enter the bucket name in Storage settings on the Zilliz Cloud console.

  1. Log in to your AWS console as a user with administrator privileges and go to the S3 service.

  2. On the General purpose bucket tab, click Create bucket.

  3. In Bucket name, enter the name for the bucket and keep the default values for other settings.

  4. Click Create bucket.

  5. Return to the Zilliz Cloud console and paste the bucket name in Bucket under Storage settings.

Step 2: Create an IAM role to access the S3 bucket​

In this step, you will create an IAM role on AWS for Zilliz Cloud to access the S3 bucket that you created in the previous step on your behalf.

  1. Log into your AWS Console as a user with administrator privileges and go to the IAM dashboard.

  2. Expand your account information, and click the copy button before your AWS Account ID.

  3. Click the Roles tab in the left sidebar, then Create Role.

  4. In Select trusted entity, click the Custom trust policy tile. In Common trust policy, paste the trust JSON from below into the editor in the Custom trust policy section and replace {accountId} with your AWS Account ID.

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Principal": {
                    "Federated": "arn:aws:iam::{accountId}:oidc-provider/eks_oidc_url"
                },
                "Action": "sts:AssumeRoleWithWebIdentity",
                "Condition": {
                    "StringLike": {
                        "eks_oidc_url:sub": [
                            "system:serviceaccount:milvus-*:milvus*",
                            "system:serviceaccount:loki:loki*",
                            "system:serviceaccount:index-pool:milvus*"
                        ],
                        "eks_oidc_url:aud": "sts.amazonaws.com"
                    }
                }
            }
        ]
    }

  5. Click Next and skip adding permissions.

  6. In the Name, review, and create step, name the role, review the trusted entities, and click Create role.

  7. Once the role has been created, click View role in the green bar to go to the role details

  8. Click the copy icon in front of the role's ARN.

  9. Go back to the Zilliz Cloud console, paste the role ARN in IAM Role ARN under Storage settings.

Step 3: Add permissions​

This step is solely on the AWS console. In this step, you will create an inline policy for the role created in Step 2.

  1. Go to the details page of the created role. In the Permissions policies section, click Add permissions, and choose Create inline policy.

  2. On the Specify permissions page, click JSON in the Policy editor section to open the policy editor. Then copy the permissions from below and paste it into the policy editor.

    You need to replace {bucketName} with that of the bucket created in Step 1, copy the modified policy JSON, and paste it into Policy editor on AWS.

    {
        "Version": "2012-10-17",
         "Statement": [
            {
              "Effect": "Allow",
              "Action": [
                "s3:ListBucket"
              ],
              "Resource": "arn:aws:s3:::{bucketName}"
            },
            {
                "Sid": "AllowS3ReadWrite",
                "Effect": "Allow",
                "Action": [
                    "s3:GetObject",
                    "s3:PutObject",
                    "s3:DeleteObject"
                ],
                "Resource": [
                    "arn:aws:s3:::{bucketName}/*"
                ]
            }
        ]
    }

  3. In Review and create, enter a policy name, review the permissions, and click Create policy.