A VPC endpoint facilitates secure, private connections to AWS services and VPC endpoint services using AWS PrivateLink. Amazon VPC instances can communicate with these services without needing public IP addresses, ensuring that data traffic remains within the Amazon network.

These VPC endpoints are essentially virtual devices that are designed to be horizontally scalable, redundant, and highly reliable components within Amazon VPC. They enable seamless communication between instances within an Amazon VPC and services, all while ensuring there are no availability concerns or bandwidth limitations affecting network traffic.

In this guide, we will talk about connecting to Amazon S3 and Secrets Manager from the backend components of the application i.e. private section of the network. By default, all your network calls (API requests) for S3 and Secret Manager go over the internet link which is not very secure. But VPC helps us make it secure using the concept of VPC Endpoints. A VPC endpoint enables you to connect with particular AWS services outside your VPC network through a private link.

Infrastructure Diagram

Creating a VPC Endpoint for S3 and Secret Manager

• Log in to the AWS management console and go to VPC endpoints.
• Select Create Endpoint, give it a name, and choose AWS Services.
• To create endpoints for S3 and Secrets Manager, look for S3 and Secret in the search bar.

• Configure VPC settings.

• AWS allows the creation of a custom endpoint policy to control which services and components have access to the endpoint. You can choose between Full access and Custom. If you go for custom then either you can start writing your policy in the textbox given below or you can use the policy builder tool.

• Click Create Endpoint. This will create an Interface type endpoint for you.

Create a VPC Endpoint for S3 and Secret Manager using Terraform

You can use Terraform code to create the same endpoint as well:

resource "aws_vpc_endpoint" "s3" {
  vpc_id       = "VPC_ID"
  service_name = "com.amazonaws.<aws_region>.s3"

  tags = {
    tagKey = "tagValue"
  }
}

Consuming Endpoint

Create an SDK client using the following Python code snippet and all your network traffic will go through the Private link instead of S3:

import boto

s3_client = boto3.client(

    service_name='s3',

    endpoint_url='https://<vpc-endpoint>'

)

You can use this with other AWS services such as AWS Lambda Function. Please ensure that it has necessary permissions in its basic execution IAM role.

Conclusion

By using AWS VPC Endpoints, we can securely route traffic through a Private link instead of over the internet. This helps create a secure connection in your infrastructure, providing robustness and reliability.