Skip to main content

How to Reach a Private RDS Without a Bastion Host

6 min read
Replace my bastion host

We set up Session Manager port forwarding, so the database stays private with no inbound port.

A jump host with a public IP and an open SSH port is the most commonly attacked thing in a lot of AWS accounts, and it exists only so somebody can occasionally run a query. Systems Manager forwards a local port through a managed node to any host that node can reach, so the database stays in its private subnet and nothing accepts inbound connections. This covers the exact command, the agent version and permissions it needs, how it works with no NAT gateway at all, and how to drop the stored database password as well.

The bastion is usually the least defensible thing in the account. It has a public IP, an open SSH port, a set of keys that have been copied around for years, and it exists so that two or three people can occasionally run a query against a database that is otherwise perfectly well hidden.

Systems Manager removes the reason for it. Session Manager can forward a port on your laptop through a managed node to any host that node can reach on the network, and the database is one such host. The node has no inbound rules, no public IP and no SSH keys, because the agent on it dials out rather than listening.

What replaces the jump host

You still need one piece of compute inside the VPC. What changes is everything about it. No inbound security group rule, no key pair, no public address, no user accounts, and access controlled by IAM rather than by who has a copy of a private key.

The remote host does not have to be managed by Systems Manager, which is what makes this work for RDS. AWS states that plainly for the port forwarding document, and the only requirement on the RDS side is ordinary network reachability and name resolution from the node.

If you already run ECS, you may not need a new instance at all. Session Manager can target a task directly when ECS Exec is enabled on it, so an existing task can act as the hop.

The prerequisites, in the order they bite

SSM Agent version 3.1.1374.0 or later on the node. That is the minimum for port forwarding to a remote host, and it is higher than the minimum for a plain session. Amazon Linux and recent Ubuntu images ship the agent, but an old AMI will fail here first.

An instance profile with Session Manager permissions. The AmazonSSMManagedInstanceCore managed policy contains everything required.

The Session Manager plugin on your own machine. The AWS CLI cannot run session commands without it, and the error message when it is missing is not obvious.

A security group path from the node to the database. The database security group has to allow the node's security group on the database port. This is the one firewall rule the whole design still needs.

If there is no NAT gateway, add three endpoints

The agent has to reach the Systems Manager service. Where the node sits in a private subnet with no route to the internet, that means interface VPC endpoints.

for SVC in ssm ssmmessages ec2messages; do
  aws ec2 create-vpc-endpoint \
    --vpc-id vpc-0abc123 \
    --vpc-endpoint-type Interface \
    --service-name com.amazonaws.eu-central-1.$SVC \
    --subnet-ids subnet-0aaa111 subnet-0bbb222 \
    --security-group-ids sg-0endpoint \
    --private-dns-enabled
done

The security group on those endpoints must allow inbound 443 from the private subnet where the node lives. Skip that and the node simply never registers, with no error anywhere obvious.

From SSM Agent version 3.3.40.0 onwards the agent prefers the ssmmessages endpoint over ec2messages wherever it is available, so a modern fleet leans on the first two. Creating all three costs nothing extra in complexity and keeps older nodes working.

The command

aws ssm start-session \
  --target i-1234567890abcdef0 \
  --document-name AWS-StartPortForwardingSessionToRemoteHost \
  --parameters '{"host":["mydb.example.eu-central-1.rds.amazonaws.com"],"portNumber":["5432"],"localPortNumber":["5432"]}'

host is the database endpoint, portNumber is the port on the remote host, and localPortNumber is what opens on your machine. Leave portNumber out and Session Manager defaults to 80, which is almost never what you meant. Windows shells want the parameters in key="value" form rather than JSON, because quoting rules differ.

Leave that running in one terminal and connect in another as if the database were local.

psql -h 127.0.0.1 -p 5432 -U app_user -d appdb

For a task instead of an instance, the target takes a composite form.

aws ssm start-session \
  --target ecs:my-cluster_my-container-id_my-container-runtime-id \
  --document-name AWS-StartPortForwardingSessionToRemoteHost \
  --parameters '{"host":["mydb.example.eu-central-1.rds.amazonaws.com"],"portNumber":["5432"],"localPortNumber":["5432"]}'

That path also needs ssmmessages:CreateControlChannel, ssmmessages:CreateDataChannel, ssmmessages:OpenControlChannel and ssmmessages:OpenDataChannel on the task role.

Permissions for the person, not just the machine

Scope ssm:StartSession to specific instances rather than granting it broadly. AWS's own sample policies use the instance ARN as the resource, and pair it with a rule that lets someone end only their own sessions.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["ssm:StartSession"],
      "Resource": "*",
      "Condition": {
        "StringLike": { "ssm:resourceTag/Role": ["db-access"] }
      }
    },
    {
      "Effect": "Allow",
      "Action": ["ssmmessages:OpenDataChannel"],
      "Resource": ["arn:aws:ssm:*:*:session/${aws:userid}-*"]
    },
    {
      "Effect": "Allow",
      "Action": ["ssm:TerminateSession", "ssm:ResumeSession"],
      "Resource": ["arn:aws:ssm:*:*:session/${aws:userid}-*"]
    }
  ]
}

A tag condition scales better than a list of instance IDs, since tagging one new node grants access without anyone editing a policy. Session documents can also be named as resources in the policy, which is how you allow shell sessions and port forwarding to different groups of people.

Drop the stored database password too

Having removed the SSH key, the password in someone's connection string is the next credential worth deleting. RDS supports IAM database authentication for MariaDB, MySQL and PostgreSQL, where the password is a signed token that expires after fifteen minutes.

aws rds modify-db-instance \
  --db-instance-identifier mydb \
  --enable-iam-database-authentication \
  --apply-immediately

export RDSHOST="mydb.example.eu-central-1.rds.amazonaws.com"
export PGPASSWORD="$(aws rds generate-db-auth-token \
  --hostname $RDSHOST --port 5432 --region eu-central-1 --username jane_doe)"

Two documented restrictions shape how you use it. The token must be generated against the DB instance endpoint itself, because a custom Route 53 record will not work. And IAM database authentication needs between 300 and 1000 MiB of extra memory on the instance, which is worth knowing before you enable it on a burstable class.

There is a trade-off in the psql connection too. sslmode=verify-full checks the endpoint you connected to against the certificate the database presents, so connecting to 127.0.0.1 through the tunnel and demanding verify-full will fail on the hostname. Map the real endpoint name to 127.0.0.1 in your hosts file if you want both the tunnel and full certificate verification.

Two things this does not give you

Session logging does not cover port forwarding or SSH sessions. AWS is explicit that the data inside the tunnel is encrypted end to end between the CLI and the Session Manager endpoints, and Session Manager is only carrying it. You will see that a session started, from CloudTrail, but not what was run inside it. If you need query-level auditing, that has to come from the database.

The other is that this is a human access path, not an application one. An application inside the VPC should reach the database directly, with credentials from a task or pod role.

If a public bastion is currently the only route into your database and you would rather have it removed than documented, that work sits inside our AWS cloud management service, and the access review around it is part of security and compliance.

Frequently asked questions

How do I connect to a private RDS database without a bastion host?

Forward a local port through a Systems Manager managed node inside the VPC. Run aws ssm start-session against the node with --document-name AWS-StartPortForwardingSessionToRemoteHost, setting host to the RDS endpoint, portNumber to the database port and localPortNumber to the port opened locally. Leave that running and connect in another terminal as if the database were local, for example psql -h 127.0.0.1 -p 5432 -U app_user -d appdb. The node needs no inbound rule, public IP or SSH key, since the agent dials out.

What SSM Agent version and setup does Session Manager port forwarding to RDS need?

Port forwarding to a remote host needs SSM Agent version 3.1.1374.0 or later on the node, which is higher than the minimum for a plain session, so an old AMI fails here first. The node also needs an instance profile with the AmazonSSMManagedInstanceCore managed policy. On the local machine, the AWS CLI cannot run session commands without the Session Manager plugin, and the error when the plugin is missing is not obvious.

Which VPC endpoints does Session Manager need in a private subnet with no NAT gateway?

Session Manager needs interface VPC endpoints for ssm, ssmmessages and ec2messages, created with --private-dns-enabled. The security group on those endpoints must allow inbound 443 from the private subnet where the node lives, or the node simply never registers and no obvious error appears. From SSM Agent version 3.3.40.0 the agent prefers ssmmessages over ec2messages where available, and creating all three keeps older nodes working.

Does an RDS instance have to be managed by Systems Manager for port forwarding to reach it?

No. The remote host in an AWS-StartPortForwardingSessionToRemoteHost session does not have to be managed by Systems Manager, which is what makes the approach work for RDS. The database only needs ordinary network reachability and name resolution from the managed node. The one firewall rule the design still depends on is the database security group allowing the node's security group on the database port.

How do I restrict who can open Session Manager port forwarding sessions to a database?

Scope ssm:StartSession to specific instances, for example with a tag condition on ssm:resourceTag/Role set to db-access, so tagging a new node grants access without anyone editing a policy. Allow ssm:TerminateSession, ssm:ResumeSession and ssmmessages:OpenDataChannel only on session ARNs that start with the caller's own ${aws:userid}, so people can act only on their own sessions. Session documents can also be named as resources, which is how shell sessions and port forwarding go to different groups.

How do I use IAM database authentication instead of a password on RDS PostgreSQL?

Turn on IAM database authentication with aws rds modify-db-instance --db-instance-identifier mydb --enable-iam-database-authentication --apply-immediately, then get a password from aws rds generate-db-auth-token, a signed token that expires after fifteen minutes. RDS supports the feature for MariaDB, MySQL and PostgreSQL. The token must be generated against the DB instance endpoint itself, since a custom Route 53 record will not work, and the feature needs between 300 and 1000 MiB of extra memory on the instance.

Why does psql sslmode=verify-full fail through an SSM tunnel to RDS?

With sslmode=verify-full, psql checks the endpoint it connected to against the certificate the database presents, so connecting to 127.0.0.1 through the tunnel fails on the hostname. To keep both the Session Manager tunnel and full certificate verification, map the real RDS endpoint name to 127.0.0.1 in the hosts file and connect to that name.

Does Session Manager log the queries run through a port forwarding session?

No. Session logging does not cover port forwarding or SSH sessions, because the data inside the tunnel is encrypted end to end between the CLI and the Session Manager endpoints, with Session Manager only carrying it. CloudTrail shows that a session started, but not what was run inside it. Query-level auditing has to come from the database itself.

Read how we handle it in AWS Cloud Management.