How to Stop Paying for NAT Gateway Traffic You Do Not Need

NAT gateway charges arrive as one anonymous line, and the usual reaction is to accept them as the cost of having private subnets. Often a large part is traffic to AWS services in the same Region, which never needed to leave the VPC and could have taken a private path carrying no data processing charge.

No figures appear below, because published rates change and a stale number in a guide is worse than none. What you get is the method for working out your own saving from your own volume.

What the meter is actually charging for

Two separate things. You are charged for each hour the NAT gateway exists, and for each gigabyte it processes in either direction. The second grows on its own and is the one this fixes, so the saving scales with your gigabytes rather than with how many gateways you run.

Step one, tell AWS-bound from internet-bound in Cost Explorer

AWS publishes a method for this. It compares NAT processing volume against data transfer volume, because traffic leaving for the internet shows up in both while traffic to AWS services shows up only in the first.

In Cost Explorer, set the date range to the last three months with daily granularity. Filter Usage type to NatGateway-Bytes, group by Linked account, and note which account dominates. Add it as a filter, group by Region, add the top Region as a filter too, and record the total usage in gigabytes. Then clear the usage type filter, select every usage type matching DataTransfer-In-Bytes and DataTransfer-Out-Bytes, and record that total.

If the two numbers track each other, the charge is traffic genuinely going to the internet, and the question becomes why your private subnets pull that much from outside AWS. A container image fetched from a public registry on every crash loop is the classic answer.

If the NAT number is much larger, the charge is AWS-bound traffic, which is the case this guide fixes. AWS names the usual causes as S3 without a gateway endpoint, AWS public endpoints where an interface endpoint exists, and your own applications in another VPC reached over the public network.

Step two, find the exact destination in flow logs

Cost Explorer gives the shape and flow logs give the address. The default format is not enough, because behind a NAT the srcaddr and dstaddr fields show the intermediate hop, not the real endpoints.

aws ec2 create-flow-logs \
  --resource-type NetworkInterface \
  --resource-ids eni-0natgatewayeni \
  --traffic-type ALL \
  --log-destination-type s3 \
  --log-destination arn:aws:s3:::my-flow-logs-bucket/natgw/ \
  --log-format '${version} ${flow-direction} ${srcaddr} ${dstaddr} ${pkt-srcaddr} ${pkt-dstaddr} ${pkt-dst-aws-service} ${traffic-path} ${bytes}'

Three fields do the work. pkt-srcaddr and pkt-dstaddr give the packet-level original addresses, so you see which instance and which remote host a flow really connects. pkt-dst-aws-service names the AWS service the destination belongs to when it belongs to one, with values including S3, DYNAMODB, EC2 and CLOUDFRONT. Sum bytes grouped by that field for a ranked list of what to fix. Afterwards traffic-path confirms the change, since 2 or 7 means the egress went through a gateway VPC endpoint and 8 means an internet gateway.

Step three, pick the right endpoint type

Several kinds of VPC endpoint exist and only two are relevant here.

Gateway endpoints work for Amazon S3 and DynamoDB, and nothing else. They are not built on PrivateLink. You create one, select the route tables for the subnets you want, and AWS adds a route whose destination is an AWS-managed prefix list for the service. A gateway endpoint carries no hourly charge and no data processing charge, which makes this change pure saving rather than a trade.

Interface endpoints work for a long list of AWS services, S3 included, and place an elastic network interface in a subnet you choose, one per Availability Zone. These are billed for the endpoint and for data processed, so the decision is a comparison rather than an obvious win. What they add is reach, since unlike gateway endpoints they work from on premises over Direct Connect or VPN, and from another Region over peering or Transit Gateway.

The remaining types answer different questions, with Gateway Load Balancer endpoints routing traffic through virtual appliances for inspection, and resource and service-network endpoints connecting to shared resources such as a database or a VPC Lattice service network.

To see what your Region actually offers:

aws ec2 describe-vpc-endpoint-services \
  --query 'ServiceDetails[].[ServiceName,ServiceType[0].ServiceType]' \
  --output table

Step four, make the change

The S3 gateway endpoint is almost always first, and it is a single command.

aws ec2 create-vpc-endpoint \
  --vpc-id vpc-0abc123 \
  --vpc-endpoint-type Gateway \
  --service-name com.amazonaws.eu-central-1.s3 \
  --route-table-ids rtb-0private1 rtb-0private2

List every private route table. Instances in subnets whose route table is not associated with the endpoint carry on using the public service endpoint and the NAT, which is the most common reason a change like this appears to do nothing. An interface endpoint takes subnets and a security group instead.

aws ec2 create-vpc-endpoint \
  --vpc-id vpc-0abc123 \
  --vpc-endpoint-type Interface \
  --service-name com.amazonaws.eu-central-1.ecr.dkr \
  --subnet-ids subnet-0aaa111 subnet-0bbb222 \
  --security-group-ids sg-0endpoint \
  --private-dns-enabled

Keep --private-dns-enabled on. It makes an SDK calling the ordinary public endpoint resolve to your endpoint's private addresses with no code change, and it needs DNS hostnames and DNS resolution enabled on the VPC.

The traps that make this not work

A bucket in another Region still goes out through the NAT. Prefix lists are Region-specific, so a gateway endpoint only captures traffic to that service in the current Region. Cross-Region S3 needs an S3 interface endpoint, and a gateway endpoint cannot be used from across a VPC peering connection either.

The security group has to allow it. Instances reaching S3 or DynamoDB through a gateway endpoint still use the service's public IP addresses, so the outbound rule must permit the service's prefix list on 443. Network ACLs cannot reference prefix lists, so those rules need the actual address ranges.

ECR needs three endpoints, not two. A private image pull needs ecr.api and ecr.dkr as interface endpoints plus the S3 gateway endpoint, because the layers come from S3. ECS tasks on EC2 need all three, and so do Fargate tasks on platform version 1.4.0 or later. Miss the S3 one and the pull authenticates and then stalls.

Adding the S3 gateway endpoint can briefly drop open connections. AWS documents that containers with live connections to S3 may see them break as the endpoint appears, so schedule it away from peak.

Cross-zone traffic to the NAT is its own charge. Resources in a different Availability Zone from the gateway pay to cross zones before they reach it. Move them, or run a NAT gateway in each zone that has resources, which AWS recommends for fault tolerance as well.

Working out your own number

Take the monthly gigabytes attributable to pkt-dst-aws-service values of S3 and DYNAMODB and multiply by the current NAT data processing rate from the Amazon VPC pricing page. That is the recurring saving from gateway endpoints, with nothing on the other side of the ledger.

An interface endpoint has two sides. Against the same NAT processing charge on that volume, set the endpoint's hourly charge multiplied by the number of Availability Zones you place it in, plus its own data processing charge. Both rates live on the AWS PrivateLink pricing page, and both are worth reading on the day you do the sum rather than trusting a figure from an article, including this one.

If you would rather have the flow logs read and the endpoints put in for you, that is a normal engagement for our cloud cost optimization service, and it sits inside the ongoing work in AWS cloud management.