Cloud Native Operations

English

한국어

English

한국어

Appearance

Traffic Management ​

Istio's traffic management capabilities allow fine-grained control over traffic flow within the service mesh.

Table of Contents ​

  1. Gateway and VirtualService
  2. Routing
  3. DestinationRule ⭐ Essential Concept
  4. Traffic Splitting
  5. Retry and Timeout
  6. Load Balancing
  7. Circuit Breaker
  8. Fault Injection
  9. Traffic Mirroring
  10. Session Affinity
  11. Egress Control
  12. ServiceEntry (External Service Management)
  13. WorkloadEntry (VM Registration)

Overview ​

These are sidecar-mode configuration examples for a supported Istio release. Gateway, VirtualService, and DestinationRule are API objects consumed by proxies, not separate network hops. A mirror is an additional copy of selected requests, not the remainder of a 90/10 split; mirrored responses are discarded. The diagrams show logical configuration relationships.

The snippets are alternative examples, not one manifest to apply together. Create the referenced Services and DestinationRule subsets first. A VirtualService with no gateways list applies to the mesh; ingress routes require an explicit Gateway binding and matching hostname. For ambient, use supported Gateway API/waypoint routing.

Traffic management is one of Istio's core features, enabling the following operations without code changes:

Key Features ​

A client request enters through the Gateway, is routed by the VirtualService, has traffic policy applied by the DestinationRule, and is then split across three service versions: the main version, a canary, and a mirrored copy for shadow testing.

A client request enters through the Gateway, is routed by the VirtualService, has traffic policy applied by the DestinationRule, and is then split across three service versions: the main version, a canary, and a mirrored copy for shadow testing.Open full screen ↗

1. Intelligent Routing ​

  • Path-based: /api/v1 → Service A, /api/v2 → Service B
  • Header-based: User-Agent: Mobile → Mobile Version
  • Cookie-based: Route specific users to specific versions
  • Weight-based: Distribute traffic by ratio

2. Deployment Strategies ​

Canary Deployment:

yaml
# Only 10% to new version
route:
- destination:
    host: reviews
    subset: v1
  weight: 90
- destination:
    host: reviews
    subset: v2
  weight: 10

Blue/Green Deployment:

yaml
# Instant switch
route:
- destination:
    host: reviews
    subset: v2  # Switch to Green
  weight: 100

3. Resilience Patterns ​

  • Circuit Breaker: Isolate failing services
  • Retry: Automatic retries
  • Timeout: Response time limits
  • Rate Limiting: Request rate control

4. Testing and Debugging ​

  • Traffic Mirroring: Replicate production traffic for testing
  • Fault Injection: Intentional fault injection
  • A/B Testing: Provide different versions to different user groups

Core Resources ​

Gateway ​

Defines the entry point for external traffic into the mesh.

yaml
apiVersion: networking.istio.io/v1
kind: Gateway
metadata:
  name: my-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "myapp.example.com"

VirtualService ​

Defines how to route requests.

yaml
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
  - reviews
  http:
  - match:
    - uri:
        prefix: "/v2"
    route:
    - destination:
        host: reviews
        subset: v2
  - route:
    - destination:
        host: reviews
        subset: v1

DestinationRule ​

Defines policies for the destination service.

yaml
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: reviews
spec:
  host: reviews
  trafficPolicy:
    loadBalancer:
      simple: LEAST_REQUEST
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2

Practical Examples ​

Safe Canary Deployment ​

yaml
# Step 1: Start with 5% traffic
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: reviews-canary
spec:
  hosts:
  - reviews
  http:
  - route:
    - destination:
        host: reviews
        subset: v1
      weight: 95
    - destination:
        host: reviews
        subset: v2
      weight: 5

After monitoring, gradually increase if no issues:

  • 5% → 10% → 25% → 50% → 100%

Header-based Routing (Developer Testing) ​

yaml
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: reviews-dev
spec:
  hosts:
  - reviews
  http:
  # Developers use new version
  - match:
    - headers:
        x-dev-user:
          exact: "true"
    route:
    - destination:
        host: reviews
        subset: v2
  # Regular users use stable version
  - route:
    - destination:
        host: reviews
        subset: v1

The developer header is a routing hint supplied by the client, not authentication or an authorization boundary.

Circuit Breaker + Retry ​

yaml
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: reviews-resilient
spec:
  host: reviews
  trafficPolicy:
    # Connection Pool settings
    connectionPool:
      tcp:
        maxConnections: 100
      http:
        http1MaxPendingRequests: 50
        maxRequestsPerConnection: 2
    # Circuit Breaker
    outlierDetection:
      consecutive5xxErrors: 5
      interval: 30s
      baseEjectionTime: 30s
---
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: reviews-retry
spec:
  hosts:
  - reviews
  http:
  - route:
    - destination:
        host: reviews
    retries:
      attempts: 3
      perTryTimeout: 2s
      retryOn: 5xx,reset,connect-failure
    timeout: 10s

Traffic Flow ​

A user request enters the ingress Gateway, passes through VirtualService routing (path, header, then weight matching), is shaped by DestinationRule policies (load balancing, circuit breaking, connection pooling), and is finally delivered to one of three backing pods.

A user request enters the ingress Gateway, passes through VirtualService routing (path, header, then weight matching), is shaped by DestinationRule policies (load balancing, circuit breaking, connection pooling), and is finally delivered to one of three backing pods.Open full screen ↗

Learning Path ​

For effective traffic management learning, the following order is recommended:

  1. Gateway and VirtualService ⭐ Starting Point

    • Understanding basic concepts
    • External traffic handling

  2. Routing

    • Advanced routing patterns
    • Conditional routing

  3. DestinationRule ⭐ Essential Concept

    • Understanding Subset concept
    • Traffic Policy basics
    • Integration with VirtualService

  4. Traffic Splitting

    • Canary deployment
    • A/B testing

  5. Retry and Timeout

    • Failure recovery
    • Response time control

  6. Load Balancing

    • Various algorithms
    • Performance optimization

  7. Circuit Breaker

    • Failure isolation
    • Cascading Failure prevention

  8. Fault Injection

    • Failure testing
    • Chaos Engineering

  9. Traffic Mirroring

    • Production testing
    • New version validation

  10. Session Affinity

    • Sticky Session
    • State preservation

  11. Egress Control

    • External service access
    • Security hardening

  12. ServiceEntry

    • External service registration
    • Egress Gateway integration

Best Practices ​

1. Gradual Rollout ​

yaml
# ❌ Bad example: 100% at once
weight: 100

# ✅ Good example: Gradual increase
# 5% → Monitor → 10% → Monitor → ...

2. Set Workload-Appropriate Timeouts ​

yaml
# ✅ Always set timeout
http:
- route:
  - destination:
      host: reviews
  timeout: 10s

Streaming requests may need different timeout settings. A short HTTP deadline is not suitable for every gRPC stream or long-lived response.

3. Use Retry Carefully ​

yaml
# ✅ Only when idempotency is guaranteed
retries:
  attempts: 3
  perTryTimeout: 2s
  retryOn: 5xx,reset,connect-failure

4. Adjust Circuit Breaker Thresholds ​

yaml
# ✅ Adjust according to service characteristics
outlierDetection:
  consecutive5xxErrors: 5      # Adjust per service
  interval: 30s
  baseEjectionTime: 30s
  maxEjectionPercent: 50    # Maximum 50% ejection

5. Monitor Metrics ​

Always monitor when changing traffic management:

  • Request Rate: Change in request count
  • Error Rate: Error ratio
  • Latency: P50, P95, P99 latency
  • Success Rate: Success ratio

Troubleshooting ​

Traffic Not Being Routed ​

bash
# 1. Check VirtualService
kubectl get virtualservice -n <namespace>
kubectl describe virtualservice <name> -n <namespace>

# 2. Check DestinationRule
kubectl get destinationrule -n <namespace>

# 3. Check pod labels
kubectl get pods --show-labels -n <namespace>

# 4. Analyze Istio configuration
istioctl analyze -n <namespace>

Weight Not Being Applied ​

bash
# Check Envoy configuration
istioctl proxy-config routes <pod-name> -n <namespace>

# Check cluster information
istioctl proxy-config clusters <pod-name> -n <namespace>

Next Steps ​

  1. Security: mTLS and authentication/authorization
  2. Observability: Metrics, logs, traces
  3. Resilience: Rate Limiting, Zone Aware Routing

References ​

Quiz ​

To test what you learned in this chapter, try the Istio Traffic Management Quiz.