Step-by-Step Guide: Setting Up Cross-Account S3 Bucket Replication in AWS

Introduction

AWS Simple Storage Service (S3) offers robust data storage capabilities, and with cross-account replication, you can ensure data redundancy and disaster recovery by replicating objects from a source S3 bucket in one AWS account to a destination S3 bucket in another AWS account. This step-by-step guide will walk you through the process of setting up cross-account S3 bucket replication, highlighting best practices and considerations. S3 replication is very useful to backup S3 data to another bucket, for disaster recovery purposes.

Prerequisites

Before you start, please make sure you have:

• Source and Destination AWS account with valid IAM permissions to access/update S3 buckets.
• Basic know-how of AWS S3 and IAM services.
• Basic knowledge to use AWS Management Console.
• Ensure that versioning is enabled on both the source bucket and destination bucket.

Approach

1. In source AWS account, sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/ 
2. In the Buckets list, choose the name of the bucket that you want to replicate or, create a new one.
3. Open the source bucket, go to the “Management” section, scroll down to find “Replication rules” section, and then select “Create replication rule”.
4. Provide a unique name for your replication rule to help you identify later.
5. Ensure that “Enabled” option is selected.
6. In the “Source bucket” section, select “Apply to all objects in the bucket” option for the scope of rule.
7. For “Destination” option, select the option “Specify a bucket in another account” to replicate the contents of the source bucket to a cross-account destination bucket in another AWS account, and provide the destination AWS account ID and bucket name . Select the option “Change object ownership to destination bucket owner” to make sure that the replicated objects are owned by the destination bucket account. Please note that the destination account bucket should be created before hand.
8. In the “IAM role” section, for “Choose from existing IAM roles”, select “Create new role” to have Amazon S3 create a new IAM role for you. In this case, the ARN of the IAM replication role that got created automatically by S3 service is “arn:aws:iam::XXXXXXXX:role/service-role/s3crr_role_for_source_bucket” and it should have the following policies added:
9. Trust policy for the IAM role:

{
  "Version": "2012–10–17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "s3.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

10. Permission policy for the IAM role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "s3:ListBucket",
        "s3:GetReplicationConfiguration",
        "s3:GetObjectVersionForReplication",
        "s3:GetObjectVersionAcl",
        "s3:GetObjectVersionTagging",
        "s3:GetObjectRetention",
        "s3:GetObjectLegalHold"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::SOURCE_BUCKET",
        "arn:aws:s3:::SOURCE_BUCKET/*",
        "arn:aws:s3:::DESTINATION_BUCKET",
        "arn:aws:s3:::DESTINATION_BUCKET/*"
      ]
    },
    {
      "Action": [
        "s3:ReplicateObject",
        "s3:ReplicateDelete",
        "s3:ReplicateTags",
        "s3:ObjectOwnerOverrideToBucketOwner"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::SOURCE_BUCKET/*",
        "arn:aws:s3:::DESTINATION_BUCKET/*"
      ]
    }
  ]
}

11. There are some extra properties i.e, “Replicate objects encrypted with AWS KMS”, “Change the storage class for the replicated objects”, “Delete marker replication” etc. which can be selected if required for your use case.

12. To complete the setup, select “Save”.

13. Now, Sign in to your Destination AWS account and go to S3 service.

14. Select the destination bucket and update it’s bucket policy to the following:

{
  "Version": "2012–10–17",
  "Id": "PolicyForDestinationBucket",
  "Statement": [
    {
      "Sid": "Permissions on objects",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::XXXXXXXX:role/service-role/s3crr_role_for_source_bucket"
      },
      "Action": [
        "s3:ReplicateDelete",
        "s3:ReplicateObject",
        "s3:ObjectOwnerOverrideToBucketOwner"
      ],
      "Resource": "arn:aws:s3:::DESTINATION_BUCKET/*"
    },
    {
      "Sid": "Permissions on bucket",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::XXXXXXXX:role/service-role/s3crr_role_for_source_bucket"
      },
      "Action": [
        "s3:List*",
        "s3:GetBucketVersioning",
        "s3:PutBucketVersioning"
      ],
      "Resource": "arn:aws:s3:::DESTINATION_BUCKET"
    }
  ]
}

15. Save the bucket policy.

Testing

To test the replication between source bucket and destination bucket, upload some objects to source bucket and you’ll see that they will be automatically replicate to the destination bucket.

Conclusion

Using the above S3 replication approach, the data keeps on getting replicated to different S3 bucket residing in a different AWS account. This is a very useful solution to keep the data replicated and it can be used for disaster recovery purposes as well.