IAM Authentication Flow in Detail

Scope: VPC Lattice service/resource APIs and AWS Gateway API Controller; verify the selected release and installed CRDs. Last Updated: September 13, 2026

What This Document Covers

• The four stages a request passes through under Lattice IAM Auth — credential acquisition, request signing, Lattice verification, policy evaluation
• Distinguish credential, signing, policy and network failures using evidence; no failure-frequency ranking is claimed
• What it means to move from connection-scoped mutual authentication to request-scoped signature verification

Why Request Signing?

App Mesh's mTLS checks each side's certificate once, when the connection is established, and trusts that connection from then on. Identity is bound to the connection.

Lattice made a different choice: sign every request and verify every request. Understanding why makes the later constraints follow naturally.

IAM request signing is one documented identity mechanism shared by AWS compute clients. Lambda can also use certificates with appropriate secret/rotation handling, so do not claim that client certificates there are impossible or use that as an explanation of undocumented AWS design decisions.

So Lattice chose to reuse "the identity system every AWS compute platform already has," and that system operates per API request, not per connection. Everything else in this document follows from that choice.

The Four-Stage Sequence

Mapping where a 403 can originate at each stage:

Stage 1 — Credential Acquisition

SigV4 signing needs an access key, a secret key, and a session token. A Pod obtains these in one of two ways.

EKS Pod Identity vs IRSA

Item	EKS Pod Identity (recommended)	IRSA
Trust relationship setup	Mediated by the EKS Auth API. Role trust policy references the pods.eks.amazonaws.com service principal	Register a per-cluster OIDC provider in IAM and write OIDC conditions into the Role trust policy
Work per additional cluster	Roles can be reused	Register an OIDC provider and amend trust policies for every cluster
Credential delivery path	Pod Identity Agent (a node DaemonSet) serves them on a link-local address	Projected service account token → SDK calls AssumeRoleWithWebIdentity
Binding mechanism	ServiceAccount ↔ Role association managed via the EKS API	ServiceAccount annotation eks.amazonaws.com/role-arn
Session tags	Pod/cluster context can be passed as session tags → usable in conditional authorization	Limited
Prerequisites	Pod Identity Agent add-on installed + node Role has AssumeRoleForPodIdentity	OIDC provider association

The practical reason to prefer Pod Identity is multi-cluster. One of the main motivations for adopting Lattice is cross-cluster communication, and IRSA requires registering an OIDC provider per cluster and managing Role trust policies for as many clusters as you have. Pod Identity does not carry that burden.

The STS temporary credential dependency

Both approaches ultimately arrive at temporary credentials issued by STS. This is an important property of the architecture.

• Credentials expire. The SDK caches them and refreshes before expiry, but the refresh path must be alive.
• If the configured credential provider cannot refresh, signing can fail locally; an already-signed expired request can also be rejected. IRSA uses STS, while Pod Identity obtains credentials through the Agent/EKS Auth path.
• In other words, STS becomes a dependency of the East-West data path. This mirrors where SPIRE Server sat in AS-IS, but the owner shifts from the customer to AWS (documents 05 and 06).

The latency implication was covered in document 02 — if refresh blocks the request path, it appears in the p99 tail.

Stage 2 — Request Signing

Canonical request → signing key → Authorization header

SigV4 signing proceeds in three steps.

First, build the canonical request. Canonicalize method, path, query and signed headers. For Lattice, include x-amz-content-sha256: UNSIGNED-PAYLOAD; payload signing is not supported. Use HTTPS to protect the body in transit and add application-level integrity when required.

Second, derive the signing key. Starting from the secret key, apply HMAC-SHA256 four times in sequence: date → region → service name → terminating string. The service name for Lattice is vpc-lattice-svcs.

Because this service name is an input to the signature itself, getting it wrong means the signature will not verify. It is easy to confuse with vpc-lattice (the service name for the Lattice control plane API), but data plane requests must be signed with vpc-lattice-svcs. This is consistent with the service DNS name itself, which takes the form <service>-<id>.<hash>.vpc-lattice-svcs.<region>.on.aws.

Third, attach the headers. The Authorization header carries the algorithm, credential scope, the list of signed headers (SignedHeaders), and the signature value; x-amz-date carries the request time; and when using temporary credentials, x-amz-security-token carries the session token.

Three practical pitfalls

① The Host header is signed — beware with custom domains

In SigV4, the Host header is always included in the signature. Which host the request is addressed to is bound into the signature.

This becomes a problem with custom domains. If you attach a customer domain (api.internal.example.com) to a Lattice service, the client sends requests to that domain and therefore signs with Host: api.internal.example.com. If the value the verifying side expects differs, the signature does not match. Conversely, if you signed with the Lattice-generated domain but the actual request's Host is the custom domain, it also does not match.

The core rule: the Host value used when signing must match the actual request's Host header. When introducing a custom domain, explicitly confirm which value your signing logic uses. This problem surfaces at the moment you attach the custom domain, not at the start of migration, which is why it is easy to miss.

② x-amz-date clock skew

The general AWS SigV4 guide says that requests in most cases must arrive within five minutes of their timestamp. Keep clocks synchronized and inspect the actual expiry/skew error; this is not a separately measured Lattice-specific guarantee.

That makes node clock synchronization a precondition for authentication. On EC2/EKS nodes using the Amazon Time Sync Service this is usually a non-issue, but it becomes a problem when:

• Node or hybrid-host time synchronization is misconfigured
• An application signs with an incorrect timestamp or timezone
• The host resumes with clock drift

This failure is intermittent and node-scoped, making it awkward to diagnose. If "only Pods on one particular node get 403s," check clock synchronization first.

③ Intermediate proxies mutating headers — sign at the last hop

Because the signature is bound to request content, if anything modifies a signed element after signing, verification breaks.

Things that actually cause this:

• Proxies that change signed paths or the actual Host value
• Proxies that add/remove query parameters or alter values/encoding; merely reordering equivalent parameters does not necessarily change the canonical query
• Body mutation is not detected by Lattice SigV4 when using the required UNSIGNED-PAYLOAD; protect it with TLS and application controls

The rule: sign at the last hop before Lattice. No layer that modifies the request may sit between signing and Lattice.

This matters especially when signing via an egress proxy. The aws-samples reference implementation demonstrates the pattern — a sigv4proxy sidecar listening on 8080, with an init container using iptables to redirect only traffic destined for 169.254.171.0/24 (the Lattice range) to local port 8080. The proxy signs and the request goes straight out to Lattice, so nothing sits in between to mutate it. Avoid configurations where a signed request is then handled by another proxy.

Stage 3 — Lattice Verification

On an HTTPS listener, Lattice terminates TLS, parses the headers, recomputes the signature in the Authorization header, and compares.

The single most important constraint of this architecture hides here.

Signature verification requires reading headers, and reading headers requires terminating TLS.

TLS Passthrough does not terminate TLS, so Lattice cannot see the Authorization header and cannot authenticate the caller's SigV4 request signature. This is the constraint discussed in document 06. It does not prohibit every auth policy: TLS listeners support policies limited to anonymous principals, which do not establish authenticated caller identity.

The controller documents policy attachments for Gateway, HTTPRoute and GRPCRoute; check the installed CRD for supported attachment targets. That controller restriction is not a statement that every VPC Lattice TLS auth policy is rejected.

Documented TLS behavior

AWS permits TLS-passthrough auth policies based on anonymous principals, without authenticated SigV4 identity or HTTP header/path inspection. TLS listeners require a custom domain matching plaintext SNI and TCP target groups; ECH/ESNI is not supported. This repository does not recommend wildcard-principal policy examples. See the TLS listener reference.

Authenticated request signing and anonymous network-context authorization are different controls. Design the endpoint authentication and the applicable service-network/service policies explicitly.

One more pitfall: auth policies are only active when authType is AWS_IAM. With NONE, an attached policy is inert. This is the most common cause of "I attached a policy but anyone can still get through."

Stage 4 — Policy Evaluation

For an authenticated principal, evaluate the caller permissions and each applicable Lattice resource policy. Only resources configured with AWS_IAM enforce their auth policy; NONE skips that resource policy. Explicit denies and other IAM controls still apply. Do not turn the three-policy diagram into a universal rule for anonymous or differently configured requests.

Policy	Attached to	Question it answers	Owner	Gateway API resource
identity-based policy	The caller's IAM Role	"Does this Role have permission to perform vpc-lattice-svcs:Invoke?"	Application / platform team	— (directly in IAM)
service network auth policy	Service Network	"Is this principal allowed into this service network?" (coarse-grained)	Network / cloud administrator	IAMAuthPolicy → Gateway
service auth policy	Lattice Service	"Is this principal allowed to call this service?" (fine-grained)	Service-owning team	IAMAuthPolicy → HTTPRoute/GRPCRoute

For service invocation auth policies, the action is vpc-lattice-svcs:Invoke. Resource configurations use a separate access model and do not inherit service-network auth policies.

Available condition keys

These keys can be used as conditions in auth policies. Which keys are present at evaluation time depends on the protocol and on whether the request was SigV4-signed.

Condition key	Filters by
vpc-lattice-svcs:Port	The service port the request was made to
vpc-lattice-svcs:RequestMethod	The request method
vpc-lattice-svcs:RequestPath	The path portion of the request URL
vpc-lattice-svcs:RequestHeader/<header-name>	A header name-value pair in the request
vpc-lattice-svcs:RequestQueryString/<key-name>	A query string key-value pair in the request URL
vpc-lattice-svcs:ServiceArn	The ARN of the target Lattice service
vpc-lattice-svcs:ServiceNetworkArn	The ARN of the service network
vpc-lattice-svcs:SourceVpc	The VPC the request originated from
vpc-lattice-svcs:SourceVpcOwnerAccount	The account owning the source VPC

IAM global condition keys such as aws:PrincipalOrgID and aws:PrincipalTag/<key> can also be used alongside these.

Needs verification

The list above is compiled from the service authorization reference and policy examples in the Gateway API Controller documentation. Lattice gains features over time, so confirm the current list in that reference before finalizing a design.

Having path, method, and header conditions is practically useful — you can enforce a rule like "only these Roles may call POST /refund on the payments service" outside application code. That said, putting path-based authorization into auth policies means API changes trigger policy changes, so decide deliberately which layer expresses authorization.

The dominant 403 failure pattern

A missing vpc-lattice-svcs:Invoke permission is one documented cause of an authenticated request failing. No incident-frequency dataset was supplied, so it is not ranked as the most common cause.

It is common because it is counterintuitive. It is natural to think "the service's auth policy allows this Role, so we're done" — but the calling Role itself also needs Invoke permission. A resource policy alone does not get you through.

Here is the actual error message from the reference implementation:

AccessDeniedException: User: arn:aws:sts::111122223333:assumed-role/eksctl-...-Role1-yz1hNJittmXj/1726632845600682009
is not authorized to perform: vpc-lattice-svcs:Invoke
on resource: arn:aws:vpc-lattice:us-west-2:111122223333:service/svc-0b13d4b53748cbdc7/catalogdetail
because no identity-based policy allows the vpc-lattice-svcs:Invoke action

The last clause — because no identity-based policy allows... — is the key to diagnosis. The message tells you which policy is missing, so read it first when you hit a 403.

403 diagnosis order

Order	What to check	How
1	The last clause of the error message	no identity-based policy → caller Role permissions; otherwise → auth policy
2	Lattice access logs and the returned error	Correlate request ID, caller logs, policy and network evidence
3	Was the request actually signed?	An unsigned request and a failed signature are different problems. Check egress proxy logs
4	Is authType AWS_IAM?	With NONE, policies are inert
5	Node clock	If only one node fails, suspect x-amz-date skew
6	Host header	If you just introduced a custom domain, start here

An easily missed pitfall: calling the k8s Service DNS directly bypasses authorization

The AWS Gateway API Controller documentation states this explicitly:

IAMAuthPolicy can only perform authorization for traffic that travels through Gateways, HTTPRoutes, and GRPCRoutes. The authorization will not take effect if the client sends traffic directly to the k8s service DNS.

Calling http://proddetail.prodcatalog-ns.svc.cluster.local directly bypasses the Lattice data path, so its auth policy does not evaluate that request. This is a design boundary, not a measured ranking of incident causes.

During the migration window, when the AS-IS path (direct in-cluster calls) and the TO-BE path (via Lattice) coexist, there are simultaneously paths where authorization applies and paths where it does not. You need compensating controls such as NetworkPolicy to block direct in-cluster calls, and that belongs in the migration plan.

AS-IS Comparison

Item	AS-IS: App Mesh + SPIRE mTLS	TO-BE: Lattice IAM Auth
Authentication scope	Connection — once at connection setup	Request — every request
Directionality	Bidirectional mutual authentication (both client and server prove identity)	Unidirectional — the client proves itself. The server proves only via its TLS server certificate
Form of identity	SPIFFE ID in an X.509 SVID (a URI)	IAM Role ARN / assumed-role session ARN
Means of proof	Short-lived X.509 certificate (proof of private key possession)	SigV4 signature (proof of secret key possession)
Who verifies	The peer workload's Envoy	Lattice (AWS-managed infrastructure)
Root of trust	A SPIRE Server CA operated by the customer	AWS IAM / STS
Where authorization happens	The receiving Envoy's authorization filter	Lattice's triple policy evaluation
Where TLS terminates	The receiving Pod's Envoy	Lattice (HTTPS listener)
On credential expiry	SVID auto-renewal (SPIRE Agent)	STS credential auto-refresh (SDK)
Observability	Envoy metrics + logs	Lattice access logs (no spans)

The two most important rows

The "Directionality" row is the crux of the review board issue. mTLS had the server prove its identity too. Under Lattice IAM Auth, server-side identity proof is at the level of a TLS server certificate, and there is no step that confirms "is this really the service that team operates" within a workload identity system. Details are in document 05.

The client-to-Lattice HTTPS connection terminates at Lattice. The target protocol is a separate setting: HTTP is plaintext on that segment, and HTTPS adds encryption without Lattice validating the target certificate. Endpoint TLS/mTLS through passthrough changes the trust boundary and cannot expose encrypted HTTP SigV4 identity to Lattice.

Summary

• Lattice chose request signing to reuse the IAM/STS foundation that EKS, ECS, EC2, and Lambda already share. That system operates per request, not per connection.
• The signing service name is vpc-lattice-svcs, and since it is an input to the signature, getting it wrong means verification fails.
• Three practical pitfalls: the Host header is signed (beware with custom domains), x-amz-date 5-minute skew (node clock sync), and sign at the last hop (no mutating proxies in between).
• TLS passthrough cannot authenticate encrypted HTTP SigV4 headers, but AWS supports anonymous network-context auth policies for that path.
• Missing Invoke permission is one possible 403 cause; use the returned reason and correlated logs rather than an unsupported frequency ranking.
• Calling the k8s Service DNS directly bypasses auth policy evaluation. Compensating controls are needed during migration.

Next: Foundations — Link-Local and SNI goes one layer below "why you must terminate TLS to see headers."

References

• Control access to VPC Lattice services using auth policies

• Actions, resources, and condition keys for Amazon VPC Lattice Services

• AWS Gateway API Controller — IAMAuthPolicy API Reference

• aws-samples — Securing the network and implementing AWS IAM authentication

• Implement AWS IAM authentication with Amazon VPC Lattice and Amazon EKS

• EKS Pod Identity / IAM Roles for Service Accounts

• Signing AWS API requests (SigV4)

• Lattice SigV4 authenticated requests — required unsigned payload header and validity window