API Gateway (Istio Service Mesh)

The Earna AI platform uses Istio Service Mesh instead of Kong Gateway for API management, traffic routing, and service communication. Istio provides comprehensive traffic management, security, and observability features.

Architecture Overview

Istio service mesh provides:

• Traffic Management: Request routing, load balancing, circuit breaking
• Security: mTLS, authorization policies, authentication
• Observability: Distributed tracing, metrics collection, logging
• Resilience: Retry policies, timeout management, fault injection

Traffic Management

Virtual Services

Default retry policy for all services:

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: default-retry-policy
  namespace: istio-system
spec:
  hosts:
  - "*"
  http:
  - retries:
      attempts: 3
      perTryTimeout: 5s
      retryOn: 5xx,reset,connect-failure,refused-stream
    timeout: 30s

Circuit Breaker Configuration

Outlier detection and connection pooling:

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: circuit-breaker-default
spec:
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 100
      http:
        http1MaxPendingRequests: 100
        maxRequestsPerConnection: 2
    outlierDetection:
      consecutiveErrors: 5
      interval: 30s
      baseEjectionTime: 30s
      maxEjectionPercent: 50
    loadBalancer:
      simple: LEAST_REQUEST

Rate Limiting

Local Rate Limiting

Envoy-based rate limiting for API protection:

# Rate limiting configuration
token_bucket:
  max_tokens: 1000
  tokens_per_fill: 100
  fill_interval: 1s

Limits Applied:

• 1000 requests per second burst capacity
• 100 requests per second sustained rate
• Applied to all inbound traffic

Security Policies

Authorization Framework

Default deny-all policy with explicit allows:

# Default deny all traffic
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: default-deny-all
  namespace: istio-system
spec:
  {}  # Empty spec = deny all

Allowed Communications

• Ingress Gateway: Traffic from istio-ingressgateway
• Health Checks: /health, /healthz, /ready endpoints
• Service-to-Service: Explicit service account authentication

Service Discovery

External Service Access

Google APIs and external services:

apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
  name: external-googleapis
spec:
  hosts:
  - "*.googleapis.com"
  ports:
  - number: 443
    protocol: HTTPS

Egress Control

Sidecar configuration restricts egress traffic:

• Local namespace services
• istio-system services
• observability namespace
• Explicitly allowed external services only

Fault Injection

Testing Resilience

Built-in fault injection for testing:

# Chaos engineering support
fault:
  delay:
    percentage: 10.0
    fixedDelay: 5s
  abort:
    percentage: 5.0
    httpStatus: 503

Activation: Add header x-test-fault: true

Load Balancing

Algorithms Available

• LEAST_REQUEST (Default): Routes to least loaded backend
• ROUND_ROBIN: Simple round-robin distribution
• RANDOM: Random backend selection
• PASSTHROUGH: Direct connection without load balancing

Connection Pooling

• Max connections per destination: 100
• Max pending HTTP/1.1 requests: 100
• Max HTTP/2 requests: 100
• Max requests per connection: 2

Observability Integration

Metrics Collection

Istio automatically provides:

• Request duration histograms
• Request rate counters
• Error rate tracking
• Circuit breaker status

Distributed Tracing

• Automatic trace header propagation
• Jaeger/Zipkin integration
• End-to-end request tracking

Integration with Platform Services

TigerBeetle Financial Ledger

• High-performance connection pooling
• Circuit breaker protection for financial operations
• mTLS encryption for sensitive transactions

Temporal Workflows

• Retry policies for workflow communication
• Load balancing across temporal workers
• Circuit breaker for workflow engine protection

Plaid Integration

• Rate limiting for API calls
• Timeout protection for external service calls
• Fault injection for testing webhook resilience

Best Practices

Service Configuration

1. Always define DestinationRule for connection pooling
2. Set appropriate timeouts for external service calls
3. Enable circuit breakers for critical service dependencies
4. Use least-request load balancing for better distribution

Security Implementation

1. Start with default deny-all authorization
2. Explicitly allow required communications only
3. Use service accounts for authentication
4. Enable mTLS for all service communication

Monitoring and Alerting

1. Monitor circuit breaker status in Grafana dashboards
2. Set up alerts for high error rates and latency
3. Use distributed tracing for debugging service issues
4. Track request patterns for capacity planning

Comparison with Kong Gateway

Why Istio was chosen:

• Better integration with Kubernetes
• Comprehensive security with mTLS
• Built-in observability and metrics
• More granular traffic policies
• Better suited for microservices architecture