Load Balancing

Istio provides various load balancing algorithms through Envoy to efficiently distribute traffic.

Table of Contents

1. Why Load Balancing?
2. Load Balancing Overview
3. Load Balancing Algorithms
4. Consistent Hash Details
5. Locality-based Load Balancing
6. Connection Pool Settings
7. Practical Examples
8. Algorithm Selection Guide
9. Best Practices
10. Troubleshooting

Why Load Balancing?

Efficient Resource Utilization

Load balancing distributes traffic across multiple instances to improve overall system throughput and stability.

Without load balancing, all requests hit one service until it is fully loaded while two others sit idle; with a load balancer in front, the same requests are split evenly so every service carries a similar share of the load.Open full screen ↗

Key Benefits

Problem	Without Load Balancing	With Load Balancing
Availability	Single Point of Failure (SPOF)	Automatic failover on failure
Performance	Specific instance overload	Even load distribution
Scalability	Difficult horizontal scaling	Easy scale-out
Response Time	Inconsistent (0-1000ms+)	Consistent response time
Resource Utilization	Inefficient (partial use)	Efficient resource usage

Load Balancing Overview

A client request reaches Istio's load balancing algorithm, which picks one of three pods with different current loads using round robin, least request, or random selection.Open full screen ↗

Load Balancing Algorithms

The released Istio 1.31.0 default is LEAST_REQUEST. Examples are independent DestinationRules, not a combined same-host configuration. Results depend on endpoint health, request cost, connection reuse, and proxy locality; no algorithm guarantees equal CPU load or constant latency. consistentHash is a separate configuration branch, not a simple: CONSISTENT_HASH enum value.

Istio provides the following load balancing algorithms.

Algorithm Comparison

Algorithm	Description	Use Cases	Pros	Cons
ROUND_ROBIN	Sequential distribution (explicit option)	Stateless services	Simple, fair	Possible load imbalance
LEAST_REQUEST	Minimum active requests	High-performance APIs, DB connections	Load equalization	Slight overhead
RANDOM	Random distribution	High traffic volume	Simple, fast	Short-term imbalance possible
PASSTHROUGH	Original destination	TCP proxy, SNI routing	Flexibility	Limited control
CONSISTENT_HASH	Hash-based sticky	Session persistence, cache	Sticky sessions	Possible imbalance

1. ROUND_ROBIN

Distributes requests sequentially to each endpoint.

Four sequential client requests are routed by the load balancer to pod 1, pod 2, pod 3, and then back to pod 1, cycling through the endpoints in order under the ROUND_ROBIN algorithm.Open full screen ↗

Configuration Example:

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: reviews-round-robin
spec:
  host: reviews
  trafficPolicy:
    loadBalancer:
      simple: ROUND_ROBIN

Use Cases:

• Stateless REST APIs
• Pods with identical performance
• When default settings are sufficient

Pros:

• Simple and predictable implementation
• Fair distribution

Cons:

• Does not consider per-pod load differences
• Long requests can cause imbalance

2. LEAST_REQUEST (Default)

For equally weighted endpoints, Envoy normally samples two available hosts and picks the one with fewer active requests. It does not scan every pod or measure its CPU/database query load. Unequal weights use a different weighted algorithm.

Default equal-weight LEAST_REQUEST selects the least busy of two random candidate endpoints.

Configuration Example:

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: api-least-request
spec:
  host: api-service
  trafficPolicy:
    loadBalancer:
      simple: LEAST_REQUEST
      warmup:
        duration: 60s  # 60 second warmup (optional)

Advanced Configuration:

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: api-least-request-advanced
spec:
  host: api-service
  trafficPolicy:
    loadBalancer:
      simple: LEAST_REQUEST
      warmup:
        duration: 120s  # New pod warmup
    connectionPool:
      http:
        http2MaxRequests: 100
        maxRequestsPerConnection: 10

Use Cases:

• APIs with variable response times
• Database connection pools
• Services with heavy processing
• When real-time load balancing is important

Pros:

• Adapts to real-time load
• Improves response time consistency
• Absorbs per-pod performance differences

Cons:

• Slight overhead (tracking active requests)

3. RANDOM

Randomly selects an endpoint.

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: reviews-random
spec:
  host: reviews
  trafficPolicy:
    loadBalancer:
      simple: RANDOM

Use Cases:

• High volume traffic (statistically even)
• When simple and fast selection is needed
• When pod performance is identical

Pros:

• Very fast selection
• Simple implementation
• Statistically even at scale

Cons:

• Short-term imbalance possible
• Unpredictable

4. PASSTHROUGH

Directly connects to the original destination specified by the client.

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: tcp-passthrough
spec:
  host: "*.external-service.com"
  trafficPolicy:
    loadBalancer:
      simple: PASSTHROUGH

Use Cases:

• TCP proxy
• SNI-based routing
• Direct external service connections
• TLS PASSTHROUGH mode

Pros:

• Preserves original destination address
• Flexible routing

Cons:

• Limited load balancing control

DestinationRule PASSTHROUGH uses original-destination load balancing. Gateway tls.mode: PASSTHROUGH controls TLS termination and is a different setting; SNI routing needs a TLS VirtualService route.

5. LEAST_CONN (Deprecated -> LEAST_REQUEST)

Note: LEAST_CONN is deprecated and replaced by LEAST_REQUEST.

Migration:

# Old version (deprecated)
trafficPolicy:
  loadBalancer:
    simple: LEAST_CONN

# New version
trafficPolicy:
  loadBalancer:
    simple: LEAST_REQUEST

Consistent Hash Details

Consistent Hash provides soft affinity for the same key while endpoint views remain stable. Adding/removing endpoints, health changes, or locality differences can remap requests; it is not durable session storage.

Consistent Hash Operation Principle

Each request's key is hashed to a fixed value that always maps to the same pod on the ring, so the same cookie from User A lands on Pod 1 both times while a different cookie from User B lands on Pod 2.Open full screen ↗

1. HTTP Header-based

Calculates hash from a specific HTTP header value.

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: user-hash
spec:
  host: api-service
  trafficPolicy:
    loadBalancer:
      consistentHash:
        httpHeaderName: "x-user-id"

Use Cases:

• Per-user session persistence
• API key-based routing
• Tenant-key affinity; enforce isolation separately

2. HTTP Cookie-based

Calculates hash from cookie value and auto-generates cookie if missing.

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: cookie-hash
spec:
  host: web-service
  trafficPolicy:
    loadBalancer:
      consistentHash:
        httpCookie:
          name: "user-session"
          ttl: 3600s  # 1 hour TTL

Use Cases:

• Web application session persistence
• Shopping cart persistence
• User experience consistency

3. Source IP-based

Calculates hash from client's source IP address.

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: source-ip-hash
spec:
  host: api-service
  trafficPolicy:
    loadBalancer:
      consistentHash:
        useSourceIp: true

Use Cases:

• IP-based session persistence
• Affinity for an external per-IP rate limiter; hashing itself does not enforce a rate limit
• Regional caching

Cautions:

• Clients behind NAT may be routed to the same pod
• When using proxies, verify actual client IP

4. HTTP Query Parameter-based

Calculates hash from query parameter value.

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: query-param-hash
spec:
  host: api-service
  trafficPolicy:
    loadBalancer:
      consistentHash:
        httpQueryParameterName: "user_id"

Use Cases:

• Resource ID-based routing in RESTful APIs
• Cache-friendly routing
• Sharding strategies

5. Minimum Ring Size Setting

Tune the number of virtual nodes on the ring to improve distribution of distinct keys; it does not eliminate remapping or hot keys.

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: hash-with-ring-size
spec:
  host: cache-service
  trafficPolicy:
    loadBalancer:
      consistentHash:
        httpHeaderName: "x-cache-key"
        ringHash:
          minimumRingSize: 1024  # Default: 1024

Explanation:

• Larger ring size provides more even distribution
• Does not guarantee a lower remapping fraction when endpoints change
• Slightly increased memory usage

Illustrative values to benchmark (not universal sizing thresholds):

• Small scale (< 10 pods): 1024 (default)
• Medium scale (10-50 pods): 2048
• Large scale (50+ pods): 4096

Consistent Hash Combination Example

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: advanced-consistent-hash
spec:
  host: api-service
  trafficPolicy:
    loadBalancer:
      consistentHash:
        httpCookie:
          name: "session-id"
          path: "/api"
          ttl: 7200s  # 2 hours
        ringHash:
          minimumRingSize: 2048
    connectionPool:
      http:
        maxRequestsPerConnection: 100
        idleTimeout: 300s

Consistent Hash Cautions

1. Imbalance Risk

When too many users hash to the same range, one pod can end up carrying most of the traffic while the others stay nearly empty — a risk specific to consistent-hash load balancing.Open full screen ↗

Cause: Traffic concentration on specific hash values

Solutions:

• Increase ringHash.minimumRingSize only for uneven distribution of distinct keys; a single hot key still maps to one host
• Use multiple hash key combinations
• Envoy supports bounded-load hashing through hash_balance_factor; Istio DestinationRule does not expose that knob directly

2. Redistribution on Pod Add/Remove

# On pod scale out
# - Before: Pod 1, Pod 2, Pod 3
# - After: Pod 1, Pod 2, Pod 3, Pod 4
# - Result: ~25% of sessions redistributed to different pods

Mitigation Strategies:

• Use graceful shutdown
• Use external session storage (Redis, Memcached)
• Scale gradually

Locality-based Load Balancing

Use either locality distribute percentages or explicit failover, not both in the same setting. Failover needs health detection and reachable healthy endpoints. Region/zone labels describe topology, not measured distance, and DestinationRule does not establish cross-region networking or service discovery. Use actual node topology values on EKS.

Locality-based Load Balancing prioritizes geographically closer endpoints.

Basic Locality Configuration

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: locality-lb
spec:
  host: reviews
  trafficPolicy:
    loadBalancer:
      localityLbSetting:
        enabled: true

Locality Distribution Ratio

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: locality-distribute
spec:
  host: reviews
  trafficPolicy:
    loadBalancer:
      localityLbSetting:
        enabled: true
        distribute:
        - from: us-west/zone-1/*
          to:
            "us-west/zone-1/*": 80  # 80% to same zone
            "us-west/zone-2/*": 20  # 20% to other zone

Locality Failover

Automatically switches to another region when one fails.

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: locality-failover
spec:
  host: api-service
  trafficPolicy:
    loadBalancer:
      localityLbSetting:
        enabled: true
        failover:
        - from: us-west
          to: us-east
    outlierDetection:
      consecutive5xxErrors: 5
      interval: 5s
      baseEjectionTime: 30s

Multi-Region Example

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: global-service-locality
spec:
  host: global-api
  trafficPolicy:
    loadBalancer:
      simple: LEAST_REQUEST
      localityLbSetting:
        enabled: true
        distribute:
        # US West clients
        - from: us-west/*
          to:
            "us-west/*": 90      # 90% local
            "us-east/*": 10      # 10% remote (DR)
        # US East clients
        - from: us-east/*
          to:
            "us-east/*": 90
            "us-west/*": 10
    outlierDetection:
      consecutive5xxErrors: 5
      interval: 30s
      baseEjectionTime: 30s

Use Cases:

• Multi-region deployments
• Latency minimization
• Inter-region disaster recovery
• Cost optimization (same AZ communication)

Connection Pool Settings

Configure Connection Pool alongside load balancing to optimize performance.

HTTP/1.1 Connection Pool

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: http1-connection-pool
spec:
  host: api-service
  trafficPolicy:
    loadBalancer:
      simple: LEAST_REQUEST
    connectionPool:
      tcp:
        maxConnections: 100        # Maximum connections
        connectTimeout: 3s         # Connection timeout
      http:
        http1MaxPendingRequests: 50  # Pending request count
        maxRequestsPerConnection: 100 # Max requests per connection
        idleTimeout: 300s             # Idle connection timeout

HTTP/2 Connection Pool

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: http2-connection-pool
spec:
  host: grpc-service
  trafficPolicy:
    loadBalancer:
      simple: LEAST_REQUEST
    connectionPool:
      tcp:
        maxConnections: 50
      http:
        http2MaxRequests: 1000     # HTTP/2 concurrent requests
        maxRequestsPerConnection: 0 # Unlimited (HTTP/2 multiplexing)
        h2UpgradePolicy: UPGRADE    # Allow HTTP/2 upgrade

Practical Examples

Example 1: High-Performance API Service

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: api-high-performance
  namespace: production
spec:
  host: api-service
  trafficPolicy:
    loadBalancer:
      simple: LEAST_REQUEST
      warmup:
        duration: 60s  # New pod warmup
    connectionPool:
      tcp:
        maxConnections: 200
        connectTimeout: 5s
      http:
        http2MaxRequests: 500
        maxRequestsPerConnection: 100
        idleTimeout: 300s
    outlierDetection:
      consecutive5xxErrors: 5
      interval: 30s
      baseEjectionTime: 30s
      maxEjectionPercent: 50

Use Scenarios:

• High-performance REST APIs
• Requests with variable response times
• Environments with per-pod performance differences

Example 2: User Session-based Routing

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: web-session-affinity
spec:
  host: web-frontend
  trafficPolicy:
    loadBalancer:
      consistentHash:
        httpCookie:
          name: "session-id"
          ttl: 7200s  # 2 hours
        ringHash:
          minimumRingSize: 2048
    connectionPool:
      tcp:
        maxConnections: 500
      http:
        http1MaxPendingRequests: 100
        maxRequestsPerConnection: 50

Use Scenarios:

• Web application session persistence
• Shopping cart consistency
• Per-user cache utilization

Example 3: Multi-Region Global Service

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: global-api-multi-region
spec:
  host: global-api
  trafficPolicy:
    loadBalancer:
      simple: LEAST_REQUEST
      localityLbSetting:
        enabled: true
        distribute:
        # US West
        - from: us-west-1/*
          to:
            "us-west-1/*": 80
            "us-west-2/*": 15
            "us-east-1/*": 5
        # US East
        - from: us-east-1/*
          to:
            "us-east-1/*": 80
            "us-east-2/*": 15
            "us-west-1/*": 5
        # EU
        - from: eu-central-1/*
          to:
            "eu-central-1/*": 90
            "eu-west-1/*": 10
    connectionPool:
      tcp:
        maxConnections: 1000
      http:
        http2MaxRequests: 2000
    outlierDetection:
      consecutive5xxErrors: 5
      interval: 30s
      baseEjectionTime: 60s

Use Scenarios:

• Global SaaS services
• Latency minimization
• Per-region disaster recovery
• Cross-AZ traffic cost reduction

Example 4: Cache Service Optimization

This assumes an HTTP cache service accepting x-cache-key. Native Redis traffic has no HTTP header; use a Redis-aware client/cluster sharding mechanism for Redis keys.

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: cache-service-optimized
spec:
  host: http-cache
  trafficPolicy:
    loadBalancer:
      consistentHash:
        httpHeaderName: "x-cache-key"
        ringHash:
          minimumRingSize: 4096  # Large ring size to minimize redistribution
    connectionPool:
      tcp:
        maxConnections: 100
        connectTimeout: 1s
      http:
        http1MaxPendingRequests: 20
        maxRequestsPerConnection: 1000
        idleTimeout: 600s
    outlierDetection:
      consecutive5xxErrors: 3
      interval: 10s
      baseEjectionTime: 30s

Use Scenarios:

• Maximize cache hit rate
• Consistent cache key routing
• Sharding strategies

Example 5: Database Connection Pool

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: database-connection-pool
spec:
  host: postgres-primary
  trafficPolicy:
    loadBalancer:
      simple: ROUND_ROBIN
    connectionPool:
      tcp:
        maxConnections: 50  # DB connection limit
        connectTimeout: 5s
    outlierDetection:
      consecutive5xxErrors: 3
      interval: 60s
      baseEjectionTime: 120s

Use Scenarios:

• Database connection pool management
• Connection selection only; existing SQL queries are not rebalanced
• Connection limit enforcement

Example 6: High Traffic Processing

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: high-traffic-service
spec:
  host: analytics-ingestion
  trafficPolicy:
    loadBalancer:
      simple: RANDOM  # Fast selection to minimize overhead
    connectionPool:
      tcp:
        maxConnections: 5000
        connectTimeout: 1s
      http:
        http2MaxRequests: 10000
        maxRequestsPerConnection: 1000
        idleTimeout: 60s
    outlierDetection:
      consecutive5xxErrors: 10
      interval: 30s
      baseEjectionTime: 30s
      maxEjectionPercent: 20  # Limited ejection at scale

Use Scenarios:

• Event collection (Analytics)
• Log collection
• Large-scale data processing

Algorithm Selection Guide

Decision Tree

Start with LEAST_REQUEST for HTTP traffic; choose consistent hashing only when soft affinity is needed, and measure other algorithms under representative load.

Recommended Algorithms by Service Type

Service Type	Recommended Algorithm	Reason
REST API	LEAST_REQUEST	Response time consistency
GraphQL API	LEAST_REQUEST	Complex query distribution
gRPC	LEAST_REQUEST	Streaming load balance
Web Frontend	CONSISTENT_HASH (cookie)	Session persistence
WebSocket	Optional reconnect affinity	An established connection stays on its upstream
Cache Service	CONSISTENT_HASH (header)	Cache hit rate
Analytics/Log Collection	RANDOM	Large-scale processing
Database	Database-aware client/pool; TCP policy as needed	Preserve primary/replica semantics
Static Content	ROUND_ROBIN	Simple and sufficient
Message Queue	Broker/client consumer assignment	HTTP active requests do not measure queue load
Batch Processing	LEAST_REQUEST	Job distribution

Selection by Traffic Pattern

# 1. Uniform small requests (< 10ms)
trafficPolicy:
  loadBalancer:
    simple: ROUND_ROBIN  # Simple and efficient

# 2. Variable requests (10ms ~ 1s+)
trafficPolicy:
  loadBalancer:
    simple: LEAST_REQUEST  # Load adaptive

# 3. Very high traffic (10,000+ RPS)
trafficPolicy:
  loadBalancer:
    simple: RANDOM  # Minimize overhead

# 4. Session-based (user state)
trafficPolicy:
  loadBalancer:
    consistentHash:
      httpCookie:
        name: "session-id"
        ttl: 3600s

# 5. Multi-region
trafficPolicy:
  loadBalancer:
    simple: LEAST_REQUEST
    localityLbSetting:
      enabled: true

Best Practices

1. Algorithm Selection Principles

Good Example:

# API with variable response times
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: api-service-best-practice
spec:
  host: api-service
  trafficPolicy:
    loadBalancer:
      simple: LEAST_REQUEST  # Load adaptive
      warmup:
        duration: 60s  # New pod warmup

Bad Example:

# Using ROUND_ROBIN with variable response times
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: api-service-bad-practice
spec:
  host: api-service
  trafficPolicy:
    loadBalancer:
      simple: ROUND_ROBIN  # Causes load imbalance

2. Tune Connection Pools

Tune optional connection-pool limits from measured demand; these are per-proxy limits, not a global service connection budget:

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: complete-lb-config
spec:
  host: api-service
  trafficPolicy:
    loadBalancer:
      simple: LEAST_REQUEST
    connectionPool:  # Optional tuning
      tcp:
        maxConnections: 100
      http:
        http1MaxPendingRequests: 50
        maxRequestsPerConnection: 100

3. Outlier Detection Combination

Use with Circuit Breaker to remove failing pods:

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: lb-with-outlier
spec:
  host: api-service
  trafficPolicy:
    loadBalancer:
      simple: LEAST_REQUEST
    outlierDetection:  # Health-based ejection
      consecutive5xxErrors: 5
      interval: 30s
      baseEjectionTime: 30s

4. Cautions When Using Consistent Hash

# Good example: Using session storage
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: web-with-redis-session
  annotations:
    description: "Uses Redis for session storage"
spec:
  host: web-frontend
  trafficPolicy:
    loadBalancer:
      consistentHash:
        httpCookie:
          name: "session-id"
          ttl: 3600s
    # Using Redis session storage maintains
    # sessions even on pod restart

# Caution: Local session only
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: web-local-session-only
  annotations:
    warning: "No external session storage - sessions lost on pod restart"
spec:
  host: web-frontend
  trafficPolicy:
    loadBalancer:
      consistentHash:
        httpCookie:
          name: "session-id"
          ttl: 3600s
    # Warning: Sessions lost on pod restart

5. Multi-Region Deployment

# Good example: Locality + Failover
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: multi-region-best-practice
spec:
  host: global-service
  trafficPolicy:
    loadBalancer:
      simple: LEAST_REQUEST
      localityLbSetting:
        enabled: true
        failover:  # Required
        - from: us-west
          to: us-east
    outlierDetection:
      consecutive5xxErrors: 5
      interval: 5s
      baseEjectionTime: 30s

6. Monitoring and Metrics

Monitor load balancing effectiveness:

# Per-pod inbound request distribution (assumes scrape labels retain namespace/pod)
sum by (namespace, pod) (rate(istio_requests_total{reporter="destination"}[5m]))

# Per-pod inbound P95 latency
histogram_quantile(0.95,
  sum by (namespace, pod, le) (
    rate(istio_request_duration_milliseconds_bucket{reporter="destination"}[5m])
  )
)

# Raw Envoy connection statistics do not have Istio destination_workload labels
sum by (namespace, pod) (envoy_cluster_upstream_cx_active)

7. Gradual Application

Start with the current LEAST_REQUEST default. Measure representative traffic, then tune the selected algorithm, connection pool, warmup and outlier detection independently. The values in these examples are starting points, not guaranteed performance thresholds.

8. Documentation

The annotation values below are illustrative. Replace the sample latency/load figures with your measurements; they are not benchmark results from this audit. Redis session storage also requires application integration, not a DestinationRule annotation.

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: api-service-lb
  annotations:
    # Configuration rationale
    purpose: "Distribute load based on active requests"

    # Algorithm selection basis
    algorithm-rationale: |
      - LEAST_REQUEST: Response times vary 10ms-500ms
      - warmup.duration: New pods need 60s to warm up cache

    # Test results
    test-results: |
      - Load test: 1000 RPS evenly distributed
      - P95 latency: 150ms (improved from 300ms with ROUND_ROBIN)
      - No pod overload observed

    # Monitoring
    monitoring: |
      - Dashboard: grafana.example.com/d/istio-workload
      - Alert: High P95 latency > 500ms
spec:
  host: api-service

Troubleshooting

Unbalanced Load Distribution

Symptoms:

# Check CPU usage per pod
kubectl top pods -n production

# Output:
# NAME                CPU    MEMORY
# api-pod-1           800m   2048Mi
# api-pod-2           200m   1024Mi
# api-pod-3           150m   1024Mi

Causes and Solutions:

# 1. Change ROUND_ROBIN -> LEAST_REQUEST
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: api-service-fix
spec:
  host: api-service
  trafficPolicy:
    loadBalancer:
      simple: LEAST_REQUEST  # Changed

# 2. Add Warmup
      warmup:
        duration: 60s

# 3. Add Outlier Detection
    outlierDetection:
      consecutive5xxErrors: 5
      interval: 30s
      baseEjectionTime: 30s

Consistent Hash Imbalance

Symptoms:

# Check Envoy metrics
kubectl exec -it pod-name -c istio-proxy -- \
  curl localhost:15000/stats/prometheus | grep upstream_rq_total

# Requests concentrated on specific pod

Solution:

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: hash-fix
spec:
  host: api-service
  trafficPolicy:
    loadBalancer:
      consistentHash:
        httpHeaderName: "x-user-id"
        ringHash:
          minimumRingSize: 4096  # Increase 2048 -> 4096

Locality-based Routing Not Working

Check the node topology that hosts the workload and the locality in Envoy’s endpoint configuration. Kubernetes/EKS normally supplies region/zone labels on Nodes, not application Pods. Correct the actual node/provisioner configuration; do not invent cloud region/zone labels to force routing.

kubectl get pods -o wide
kubectl get nodes -L topology.kubernetes.io/region,topology.kubernetes.io/zone
istioctl proxy-config endpoints <pod-name> -o json
istioctl proxy-config clusters <pod-name> --fqdn api-service.default.svc.cluster.local -o json

References

• Istio Load Balancing

• Envoy Load Balancing

• Consistent Hashing

• Locality Load Balancing

• Primary reference 1

• Primary reference 2

• Primary reference 3

• Primary reference 4

• Primary reference 5

• Primary reference 6