Welcome to the Dark Side of the Gateway
Deploying a Lambda behind a private API Gateway sounds like a straightforward idea — until you actually try to do it.
On paper, it’s AWS’s answer to “secure serverless.” In practice, it’s like trying to assemble a flat-pack IKEA firewall while blindfolded, using only the cries of other engineers as your guide.
Sure, AWS marketing makes it sound effortless: “Just deploy your Lambda to a VPC, attach a PrivateLink endpoint, and enjoy seamless, secure integration!”
What they really mean is: “Hope you enjoy creating seventeen IAM roles, three VPC endpoints, and a CloudFormation stack so large it develops sentience.”
Let’s dive in, shall we?
The Lambda — Your Innocent, Naïve Function
Our protagonist: the humble AWS Lambda.
Just a few lines of code, ready to serve requests, run cron jobs, or silently consume half your free tier because you forgot to disable test events.
Before it becomes part of the Private API Elite, Lambda must lose its innocence. That means no more public internet access — AWS makes sure of that by yanking out the managed network interface and tossing it into your VPC.
You’ll need to assign it subnets and security groups, because apparently serverless doesn’t mean without servers — it means without obvious servers but with just enough networking to ruin your weekend.
So you deploy it into a private subnet.
And now it can’t reach anything. Not the internet, not the API Gateway, not even itself on a bad day.
Welcome to AWS networking. Bring snacks.
The API Gateway — The Gatekeeper of Despair
Next comes the API Gateway — Private mode. Because what’s better than a public API Gateway? One that refuses to speak to anyone who isn’t invited to the party.
A Private API Gateway is only accessible through an Interface VPC Endpoint (AWS PrivateLink). Which sounds fancy until you realise you’ve just created a very expensive tunnel for traffic that used to be free.
You set up the VPC Endpoint, select “Enable Private DNS” (which AWS doesn’t tell you will break your test clients for several hours), and think, “Ah, secure connectivity achieved!”
Except… your Lambda can’t talk to it. Because your Lambda’s in a VPC that doesn’t know this new endpoint exists. And the endpoint’s security group doesn’t allow the Lambda’s security group.
This is the AWS networking equivalent of two introverts at a party — both want to talk, but neither will start the conversation.
Permissions — Because Nothing Screams Security Like 14 Policy Documents
You now have a Lambda in a private subnet and an API Gateway that’s more closed off than a legacy mainframe team. Time to glue them together.
AWS, in its infinite wisdom, requires resource policies for your API Gateway. Yes, another layer of permissioning — because IAM, resource policies, and execution roles apparently didn’t cover all possible failure modes.
Here’s what that beautiful JSON snowflake looks like:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "arn:aws:execute-api:eu-west-1:123456789012:abcd1234/*/*/*",
"Condition": {
"StringEquals": {
"aws:SourceVpce": "vpce-0abc12345def6789f"
}
}
}
]
}
Lovely, right? A perfect JSON snowflake that says: “Only requests from this specific VPC endpoint may enter.”
You deploy it, test it… and get a 403 Forbidden.
You check CloudWatch logs. Nothing helpful. You check your permissions. All good. You check your sanity. Rapidly degrading.
Then you realise: your Lambda’s execution role doesn’t have execute-api:Invoke permission. Because of course it doesn’t.
Add another policy. Redeploy. Success! For about three glorious seconds until you realise your API still isn’t reachable from outside your VPC.
Which was the point — but now you actually need to reach it from your developer laptop.
Time to fire up a bastion host, SSH tunnel, and perhaps a small prayer.
The Testing Phase — Where Hope Goes to Die
You’ve got everything deployed. The Lambda is behind a private API Gateway. Traffic routes through the VPC endpoint. Security policies are locked down tighter than AWS pricing transparency.
You’re ready to test.
Except Postman can’t reach it.
Your laptop can’t reach it.
Even your other AWS services can’t reach it unless they live in the same VPC.
You try the inevitable:
curl https://abcd1234.execute-api.eu-west-1.amazonaws.com/dev/hello
Nothing. Timeout.
You triple-check DNS. Then realise that “Enable Private DNS” wasn’t enough — your Route 53 Resolver needs an inbound endpoint. Because of course it does.
After three hours and one internal meltdown, you finally hit the Lambda successfully from an EC2 instance. You weep softly.
Then you open CloudWatch Logs to confirm the Lambda worked… only to discover your logging permissions are wrong, and no logs were actually written.
AWS: where success is measured in progressively smaller victories.
The “Simplified” Architecture Diagram
[Developer Laptop]
|
[VPN / Bastion Host]
|
[VPC Endpoint (PrivateLink)] <---> [API Gateway (Private)]
|
[Lambda in VPC]
|
[S3, DynamoDB, or Your Tears]
If it feels like overkill for a simple function that returns “Hello, World,” congratulations — you understand AWS architecture.
Every service needs a supporting cast of at least five others.
It’s like a Shakespeare play, but with more billing metrics and fewer coherent plotlines.
CloudFormation: The True Final Boss
Feeling brave? Try automating this circus in CloudFormation.
You’ll write roughly 500 lines of YAML to define a “simple private API.”
It will look like poetry written by a machine that once loved but has since forgotten joy.
Your Resources section will include:
- A Lambda Function
- A VPC with subnets
- A Security Group
- A VPC Endpoint
- An API Gateway
- A Deployment
- A Stage
- A Resource Policy
- Several Roles
- And, naturally, an Output section that doesn’t quite output what you need.
You’ll deploy it once, fail on circular dependency.
Deploy again, fail on missing permissions.
Deploy again, succeed, but the endpoint still won’t resolve.
At this point, you’re no longer building infrastructure.
You’re participating in a performance art piece about futility.
AWS’s Official Line
“We’re excited to bring customers even more secure, flexible, and configurable options for connecting Lambda functions through private APIs. This solution empowers developers to fine-tune their architectures for maximum control.”
Translation: We gave you 14 more knobs to twist. Have fun debugging!
The Bright Side (Yes, There Is One)
Sarcasm aside, once you’ve survived the networking maze, this setup does have its virtues.
A private API Gateway gives you:
- Strong isolation from the public internet
- Predictable security posture for internal microservices
- Easier compliance with corporate auditors who panic at the word public
- And bragging rights on LinkedIn for “hardening your serverless perimeter”
It’s genuinely useful for internal APIs, hybrid architectures, or anything sensitive that shouldn’t be floating around in the digital wild.
Just don’t expect to explain it easily to anyone outside the cloud team. Or your future self, six months from now, when you have to modify it.
The Punchline
Deploying a Lambda behind a private API Gateway is one of those AWS rites of passage — like understanding IAM conditions or figuring out why your S3 bucket is public again.
It’s powerful. It’s secure. It’s… a lot.
The real miracle isn’t that AWS can do this.
It’s that any of us still remember how we did it when it breaks at 2 a.m.
So here’s to you, brave cloud engineer: guardian of the Private API, wielder of VPC endpoints, master of JSON policy indentation.
