SMS Gateway System High-Level Design

This document presents a scalable SMS gateway architecture designed to handle high-volume message delivery across multiple global carriers while maintaining low latency and high availability for enterprise customers.

1. Executive Summary

Clarifying questions

1. What is the expected read/write ratio for SMS messages?

   • Is this primarily a send-only system or does it handle incoming replies and delivery receipts?

2. What are the consistency requirements for message status tracking?

   • Can we tolerate eventual consistency for delivery confirmations, or must they be strongly consistent?

3. Is this a global system with multi-region support needed?

   • Or is it single-region focused for initial launch?

4. How sensitive is the system to latency?

   • Is sub-100ms response time critical for user experience?

5. What is the expected growth in volume and users?

   • Are we targeting millions of messages per day or more?

6. Are there any external integrations with telecom providers or carriers?

   • If so, what protocols do they support (e.g., SMPP, HTTP APIs)?

7. What is the data ownership model?

   • Should each customer’s data be isolated, or can we use shared infrastructure with multi-tenancy?

Problem being solved

Design a highly available, scalable SMS gateway that enables third-party applications to send and receive SMS messages via a RESTful API. The system must support high throughput, low latency, reliable delivery tracking, and multi-tenant security while integrating with telecom providers.

Back-of-envelope calculations

Assuming 10 million DAU (Daily Active Users), with an average of 5 messages per user per day:

• Peak QPS:

  • Avg. QPS = (10M * 5) / 86400 ≈ 580 QPS
  • Peak QPS = 580 * 10 = 5,800 QPS

• Storage per user/event:

  • Avg. message size = 160 bytes (standard SMS)
  • Metadata overhead per message = ~50 bytes
  • Total storage per message = ~210 bytes
  • Daily messages = 50M
  • Annual storage = 50M * 210 bytes * 365 ≈ 3.7 GB/year

• Bandwidth:

  • Read + Write throughput = 50M * 210 bytes ≈ 10 MB/s (read/write combined)
  • Peak bandwidth = ~20 MB/s

• Cache hit rate:

  • Assume 80% cache hit rate
  • DB QPS = 5,800 * (1 - 0.8) = 1,160 QPS

• Server count at peak:

  • Assuming 5,000 QPS per server
  • Servers needed = 1,160 / 5,000 ≈ 3 servers (rounded up)

SLA targets

• Availability: 99.9% (3.6 minutes downtime/month)
• Latency:
  • p50: <100ms
  • p95: <250ms
  • p99: <500ms
• Throughput: 5,800 QPS average, 10,000 QPS peak
• Durability: At least 99.9% message persistence (RPO < 1 hour)
• Consistency: Eventual consistency for delivery receipts, strong consistency for message send status

Key architectural decisions summary

• Multi-region deployment with active-active failover (for global availability)
• Microservices architecture with domain-driven decomposition (send, receive, status tracking)
• Use of Redis for caching and Kafka for async message processing
• Event-driven architecture with CQRS and eventual consistency for status tracking
• Stateless services to support horizontal scaling
• Support for multi-tenancy with secure isolation and rate limiting per tenant
• Integration with telecom providers via HTTP/SMS API or SMPP protocol
• Use of observability stack (Prometheus, Jaeger, Loki) for monitoring and tracing
• Infrastructure-as-code with Terraform and Kubernetes for deployment automation

2. System Goals & Non-Functional Requirements

Latency Targets

• p50: 200ms for SMS delivery requests
• p95: 500ms for SMS delivery requests
• p99: 1000ms for SMS delivery requests
• End-to-end delivery confirmation: p99 < 5 seconds

Throughput Requirements

• Average QPS: 10,000 SMS deliveries/sec
• Peak QPS: 100,000 SMS deliveries/sec (10x average)
• Daily volume: 8.6 billion SMS messages (assuming 100K peak QPS × 86,400 seconds)
• Concurrent connections: 50,000 simultaneous SMS send operations

Availability Target

• 99.9% (3 9s) availability for SMS delivery service
• Justification: Critical business function with SLA commitments to enterprise customers; acceptable 4.3-minute monthly downtime for maintenance windows and planned outages
• 99.99% (4 9s) availability for enterprise tier customers

Durability and Consistency Requirements

• CAP Trade-off Decision: Prioritize Consistency + Availability (CP) over Partition Tolerance for core delivery operations
• Consistency Model: Strong consistency for delivery confirmation and message status tracking
• Durability: All SMS messages must be persisted to database before acknowledgment sent to client
• Idempotency: Delivery requests must be idempotent to handle retries without duplicate messages

Data Retention and Compliance Requirements

• Message retention: 90 days for delivery confirmation logs, 30 days for raw message content
• Compliance: GDPR, HIPAA (if healthcare use case), SOC2 Type II compliance
• Audit logging: Full audit trail of all SMS send operations with timestamps and user identifiers
• Data deletion: Support for customer data deletion requests within 30 days

Security and Regulatory Requirements

• Authentication: OAuth2/OIDC with JWT tokens for service-to-service communication
• Authorization: RBAC model with roles (admin, developer, operator) for SMS API access
• Encryption: TLS 1.3+ for all network communications; at-rest encryption (AES-256) for message logs
• Compliance:
  • GDPR: Data subject rights, consent management, data portability
  • HIPAA: Protected health information handling, audit logging, breach notification
  • SOC2: Security, availability, processing integrity, confidentiality, privacy principles
• Rate limiting: Per-user and per-application rate limits to prevent abuse and ensure fair usage

3. High-Level Architecture Diagram

C4Context
    title SMS Gateway System - Context Diagram

    Person(user, "End User")
    Person(admin, "System Admin")
    
    System_Boundary(b1, "SMS Gateway System") {
        System(sms_service, "SMS Service", "Core service for sending/receiving SMS")
        System(mobile_gateway, "Mobile Gateway", "Handles carrier-specific protocols")
        System(message_broker, "Message Broker", "Kafka for async messaging")
        System(auth_service, "Auth Service", "Authentication and authorization")
        System(cache_layer, "Cache Layer", "Redis for caching")
        System(db_layer, "Database Layer", "PostgreSQL + Redis")
        System(notify_service, "Notification Service", "Push/email notifications")
    }
    
    System_Ext(carrier_api, "Carrier APIs", "Twilio, AWS SNS, etc.")
    System_Ext(smsc, "SMSC", "Short Message Service Center")
    
    Rel(user, sms_service, "Sends SMS via API")
    Rel(sms_service, mobile_gateway, "Routes to carrier")
    Rel(mobile_gateway, carrier_api, "Uses carrier API")
    Rel(sms_service, message_broker, "Publishes events")
    Rel(message_broker, sms_service, "Consumes events")
    Rel(sms_service, auth_service, "Authenticates requests")
    Rel(sms_service, cache_layer, "Caches user data")
    Rel(sms_service, db_layer, "Stores messages and metadata")
    Rel(sms_service, notify_service, "Sends notifications")
    Rel(admin, sms_service, "Manages system via admin console")

C4Container
    title SMS Gateway System - Container Diagram

    Container_Boundary(c1, "SMS Gateway Service") {
        Container(sms_core, "SMS Core Service", "Spring Boot", "Handles SMS sending/receiving logic")
        Container(sms_processor, "SMS Processor", "Go microservice", "Processes SMS workflows")
        Container(sms_scheduler, "SMS Scheduler", "Node.js", "Handles scheduled messages")
        Container(auth_service, "Auth Service", "Spring Boot", "JWT-based auth")
    }
    
    Container_Boundary(c2, "Infrastructure Layer") {
        Container(message_broker, "Kafka", "Event streaming", "Async message queue")
        Container(cache_layer, "Redis Cluster", "In-memory store", "Caching layer")
        Container(db_master, "PostgreSQL Master", "Relational DB", "Primary data store")
        Container(db_replica, "PostgreSQL Replica", "Relational DB", "Read replicas")
        Container(backup_service, "Backup Service", "Custom", "Database backups")
    }
    
    Container_Boundary(c3, "External Integrations") {
        Container(twilio_api, "Twilio API", "REST API", "Carrier integration")
        Container(aws_sns, "AWS SNS", "REST API", "Carrier integration")
        Container(logging_service, "Logging Service", "ELK Stack", "Centralized logging")
        Container(tracing_service, "Tracing Service", "Jaeger", "Distributed tracing")
    }
    
    Rel(sms_core, sms_processor, "Async processing")
    Rel(sms_processor, sms_scheduler, "Schedule messages")
    Rel(sms_core, auth_service, "Auth check")
    Rel(sms_core, message_broker, "Publish events")
    Rel(message_broker, sms_processor, "Consume events")
    Rel(sms_core, cache_layer, "Cache user data")
    Rel(sms_core, db_master, "Write to DB")
    Rel(sms_core, db_replica, "Read from DB")
    Rel(sms_core, twilio_api, "Send via Twilio")
    Rel(sms_core, aws_sns, "Send via AWS SNS")
    Rel(sms_core, logging_service, "Log events")
    Rel(sms_core, tracing_service, "Trace requests")

Major Components and Services

Core SMS Service: Main service handling SMS sending/receiving, routing to appropriate carriers, and message lifecycle management.

Mobile Gateway: Protocol adapter that translates between standard SMS format and carrier-specific APIs (e.g., SMPP, HTTP).

Message Broker: Kafka-based event streaming for async processing of SMS events, delivery confirmations, and system notifications.

Authentication Service: JWT-based authentication with role-based access control for both end-users and admin interfaces.

Caching Layer: Redis cluster for caching frequently accessed data like user profiles, carrier configurations, and rate limits.

Database Layer: PostgreSQL master for primary writes with read replicas for high-availability reads, plus backup service.

External Integrations: Twilio and AWS SNS for carrier connectivity, ELK stack for logging, Jaeger for tracing.

Traffic Flow and Data Flow

1. User Request Flow: End user sends SMS via API → Auth service validates → SMS Core processes → Mobile Gateway routes to carrier
2. Event Flow: SMS events published to Kafka → SMS Processor consumes → Updates DB and sends notifications
3. Monitoring Flow: All requests logged to ELK → Tracing data sent to Jaeger → Metrics collected for dashboard
4. Carrier Integration: SMS Core communicates with Twilio/AWS SNS via REST APIs for message delivery

Service Communication Patterns

• Synchronous: Auth service integration, database reads/writes
• Asynchronous: Event-driven processing through Kafka, caching updates
• External: REST API calls to Twilio/AWS SNS for carrier communication

Data Flow

1. User sends SMS → Authenticated request → Processed by SMS Core
2. SMS routed to Mobile Gateway → Carrier API integration
3. Delivery confirmation sent back → Processed by SMS Processor
4. Events published to Kafka → Triggers downstream processing
5. Data stored in PostgreSQL with Redis caching for performance

4. Service Architecture

Microservices vs. Monolith vs. Modular Monolith Decision

For an SMS Gateway system, we'll adopt a modular monolith approach with clear service boundaries that can evolve into independent microservices. This decision balances:

• Development velocity - rapid iteration and feature development
• Operational simplicity - single deployment unit with shared infrastructure
• Scalability - ability to extract services later as traffic patterns emerge
• Complexity control - manageable coupling during initial phases

Service Decomposition Strategy

Based on domain-driven design principles, we'll decompose the system into:

1. Message Service - Core SMS processing, routing, and delivery
2. Provider Service - Integration with SMS gateways (Twilio, AWS SNS, etc.)
3. User Service - Account management, authentication, and permissions
4. Billing Service - Usage tracking, billing, and rate limiting
5. Analytics Service - Message analytics, delivery metrics, and reporting

Service Boundaries and Contracts

Each service will maintain clear contracts:

• Message Service:

  • REST API for sending/receiving SMS
  • Event-driven processing for delivery status updates
  • Internal queue for message retry logic

• Provider Service:

  • Adapter pattern for multiple SMS providers
  • Provider-specific configuration and credentials management
  • Rate limiting and throttling per provider

• User Service:

  • JWT-based authentication for API access
  • Account and subscription management
  • Permission model for multi-tenant access

Inter-Service Communication Pattern

Synchronous (REST/gRPC):

• User → Message: Authentication and authorization
• Message → Provider: SMS submission to gateways

Asynchronous (Events):

• Message → Analytics: Delivery status updates
• Provider → Message: Delivery receipts and failures
• Message → Billing: Usage tracking and metering

Service Mesh Consideration

We'll avoid service mesh in Phase 1 to reduce operational complexity. Service mesh (Istio/Linkerd) will be evaluated in Phase 3 for advanced traffic management and security policies.

Service Registry and Discovery Strategy

• Service Registration: Built-in service discovery using Consul or Eureka
• Health Checks: Liveness and readiness probes on all services
• Load Balancing: Client-side load balancing with circuit breaker patterns
• Versioning: API versioning through path prefixes (v1, v2) and content negotiation

Cross-Service Data Ownership

Following the "no shared databases" principle:

• Each service owns its database and manages its own data lifecycle
• Message Service maintains message state and delivery history
• User Service handles authentication tokens and account metadata
• Provider Service stores provider-specific configurations and credentials

CQRS Pattern Application

Message Service will implement CQRS:

• Write Side: Handles message creation and routing
• Read Side: Optimized views for delivery status, analytics, and search

Analytics Service will use CQRS to provide real-time dashboards and historical reporting without impacting core message processing.

Event Sourcing Consideration

Event sourcing will be implemented for:

• Message lifecycle tracking: Complete audit trail of message state changes
• Provider integration logging: Full history of provider interactions and failures

Trade-offs considered:

• Increased complexity in event storage and replay
• Better auditability and debugging capabilities
• Higher storage requirements for event store

Scalability Implications

Each service will be designed to:

• Scale horizontally based on message volume
• Handle burst traffic with proper queueing and rate limiting
• Maintain stateless design where possible for better scaling characteristics

Future Migration Path

The modular monolith design provides a clear migration path:

• Phase 1: All services in single deployment unit
• Phase 2: Extract User and Billing services as independent microservices
• Phase 3: Decompose Message and Provider services for maximum scalability

This approach allows us to start with manageable complexity while maintaining flexibility for future architectural evolution.

5. Data Architecture

Database Selection per Service

1. SMS Gateway Service

• Primary DB: PostgreSQL (v14+) - Relational database for storing SMS metadata, delivery status, and message history
• Justification: ACID compliance required for transactional SMS delivery tracking, strong consistency for message status updates, and complex querying capabilities for analytics and reporting

2. User/Subscriber Service

• Primary DB: MongoDB (v6+) - Document database for storing user profiles, preferences, and subscription information
• Justification: Flexible schema to accommodate varying user profile structures, high write throughput for user updates, and geospatial querying capabilities for location-based routing

3. SMS Template Service

• Primary DB: DynamoDB - Key-value store for template management and caching
• Justification: High read/write throughput, automatic scaling, low latency for template retrieval, and built-in caching capabilities

4. Message Queue Service

• Primary DB: Redis (v7+) - In-memory key-value store for message queuing and caching
• Justification: High-performance in-memory operations, pub/sub capabilities for event-driven architecture, and support for various data structures needed for queue management

5. Analytics Service

• Primary DB: TimescaleDB (v2+) - Time-series database for SMS analytics and reporting
• Justification: Optimized for time-series data with built-in compression, efficient aggregation queries, and horizontal scaling capabilities

Data Partitioning Strategy

Horizontal Sharding:

• Partition Key: Country Code + Mobile Prefix (e.g., US_123)
• Sharding Method: Hash-based sharding for even distribution across shards
• Justification: Ensures balanced load distribution, predictable performance, and supports global scale while maintaining data locality for regional compliance requirements

Range Partitioning:

• Use Case: SMS delivery status tracking by date ranges
• Partition Key: Message creation timestamp (monthly/yearly buckets)
• Justification: Optimizes analytics queries and enables efficient data lifecycle management

Data Replication Strategy

Primary-Replica Model:

• Master-Slave Architecture: PostgreSQL primary with read replicas for analytics and reporting
• Cross-Region Replication: Multi-region read replicas for disaster recovery and low-latency access
• Justification: Provides high availability, reduces read load on primary, and supports global distribution

Multi-Primary (Active-Active):

• Use Case: User service in multi-region deployment
• Replication Method: Synchronous replication with conflict resolution
• Justification: Enables global user access with consistent data across regions

Cross-Service Data Ownership

No Shared Databases Rule:

• Enforcement Strategy:
  • Database-per-service isolation
  • Service-to-service communication via APIs/events only
  • Database access controlled through service accounts with minimal privileges

Data Ownership Model:

• SMS Gateway Service: Owns message metadata, delivery status, and routing information
• User Service: Owns user profiles, preferences, and subscription data
• Template Service: Owns template definitions and content

CQRS Pattern Application

SMS Gateway Service:

• Command Side: Handles message creation, routing, and delivery initiation
• Query Side: Separate read models for analytics dashboards and status reports
• Justification: Separates concerns, enables independent scaling of read/write operations, and supports complex query patterns

Template Service:

• Command Side: Template creation and versioning
• Query Side: Fast template retrieval for message composition
• Justification: Optimizes template access patterns and supports high-concurrency read operations

Event Sourcing Consideration

Use Case: Message delivery tracking and audit trail

• Event Store: PostgreSQL with JSONB columns for event storage
• Justification: Provides immutable audit trail, supports complex event replay, and enables debugging of delivery issues

Trade-offs:

• Pros: Complete auditability, supports complex analytics, enables system recovery
• Cons: Increased storage requirements, complexity in event processing, potential performance overhead

Object Storage Design

Media/Attachment Handling:

• Storage: AWS S3 with lifecycle policies
• Access Patterns:
  • High-frequency access for template images (cache in Redis)
  • Periodic access for analytics reports (archive to Glacier)
• Metadata Separation:
  • Metadata stored in MongoDB
  • Content stored in S3 with metadata tags for indexing

Storage Tiering Strategy

Hot Tier:

• Storage: SSD-backed PostgreSQL and Redis
• Access Frequency: 95% of queries within 10ms
• Justification: Critical for real-time SMS delivery and low-latency operations

Warm Tier:

• Storage: HDD-backed MongoDB and TimescaleDB
• Access Frequency: 5% of queries with acceptable latency (100ms-1s)
• Justification: Analytics and historical data access with cost optimization

Cold Tier:

• Storage: AWS Glacier or S3 Standard-IA
• Access Frequency: <1% of queries, infrequent access (daily/monthly)
• Justification: Long-term archival and compliance data retention

Backup and Restore Strategy

Backup Schedule:

• Full Backups: Daily at 2 AM (UTC)
• Incremental Backups: Hourly, retained for 7 days
• Point-in-Time Recovery: Enabled with 15-minute granularity

RTO/RPO Targets:

• RTO: < 5 minutes for critical services (SMS delivery)
• RPO: < 1 minute for transactional data
• RPO: < 1 hour for analytics and reporting data

Backup Storage:

• Primary Location: AWS S3 (us-east-1)
• Secondary Location: Cross-region replication to eu-west-1
• Encryption: AES-256 encryption at rest and in transit

Restore Process:

• Automated Recovery: 1-click restore for critical services
• Manual Verification: Full data integrity checks before production restore
• Rollback Strategy: Automated rollback to last known good state with 30-minute window

6. Caching Architecture

Caching Strategy Overview

The SMS Gateway system requires a multi-layer caching strategy to handle high QPS, reduce database load, and provide low-latency responses for frequently accessed data patterns. The caching strategy follows a typical multi-tier approach:

Client → CDN → API Gateway → Application Cache → Database

Caching Layers

1. CDN Layer (Edge Caching)

• Use case: Static configuration data, rate limit quotas, and frequently accessed SMS templates
• Storage: CDN edge nodes (e.g., Cloudflare, AWS CloudFront)
• TTL: 1 hour for configuration data, 5 minutes for rate limits
• Benefit: Reduces latency for global users and reduces origin load

2. API Gateway Cache

• Use case: Authentication tokens, client configuration, and rate limit state
• Storage: In-memory cache in API Gateway (e.g., Kong, AWS API Gateway)
• TTL: 15 minutes for client configurations, 1 minute for tokens
• Benefit: Reduces authentication overhead and improves response times

3. Application-Level Cache (L2)

• Use case: User session state, SMS delivery status, and client-specific configurations
• Storage: Redis cluster (multi-AZ for HA)
• TTL: 5 minutes for session state, 1 hour for delivery status
• Benefit: Reduces database queries and provides fast access to frequently accessed data

4. In-Memory Cache (L1)

• Use case: High-frequency lookups like phone number validation, carrier information, and recent SMS IDs
• Storage: Local in-memory cache (e.g., Caffeine, Guava)
• TTL: 30 seconds for phone validation, 1 minute for carrier info
• Benefit: Fastest access for hot data patterns

Cache Sizing Strategy

Based on back-of-envelope calculations:

• Peak QPS: 100,000 (assuming 10x average)
• Average SMS payload size: 150 bytes
• Cache hit rate target: 85% for database queries

Memory requirements per cache tier:

• L1 cache (per instance): 50MB (for phone validation + carrier data)
• L2 cache (Redis cluster): 2GB (session state + delivery status)
• API Gateway cache: 100MB (tokens + configs)
• CDN cache: 500MB (static content)

Cache Eviction Policy

1. LRU (Least Recently Used) for most cache entries
2. TTL-based expiration for time-sensitive data (rate limits, tokens)
3. Size-based eviction with LRU policy for Redis cluster
4. Cache warming: Preload popular phone numbers and carrier data during system startup

Cache Consistency Strategy

1. Write-through caching for critical data (rate limits, session state)
2. Cache invalidation via pub/sub pattern when data changes
3. Event-driven invalidation: When a user's rate limit updates, invalidate the cache entry
4. Cache warming: Load frequently accessed data during system startup and warm-up periods

Cache Warming Strategy

1. Startup warming: Load phone number validation data and carrier information during service startup
2. Periodic warming: Daily warm-up of most frequently accessed phone numbers and carrier data
3. Event-driven warming: Warm cache when new carriers are added or rate limit configurations change

Cache Hit Rate Targets

• Database queries: 85% cache hit rate (reducing DB QPS from 100k to ~15k)
• Rate limit checks: 95% cache hit rate (reducing auth service QPS)
• Phone validation: 90% cache hit rate (reducing external carrier lookups)

Cache Performance Metrics

1. Hit/Miss ratio per cache layer
2. Latency distribution for cache hits vs misses
3. Memory utilization and eviction rates
4. Cache warming success rate

Cache Implementation Considerations

1. Cache stampede prevention: Use cache entry locking or staggered warming
2. Cache partitioning: Separate cache by user segments (high-volume vs low-volume)
3. Cache monitoring: Real-time dashboard for cache health and performance
4. Cache failure handling: Graceful degradation when cache is unavailable

Cache Data Models

1. Rate Limit Cache:

   • Key: rate_limit:{client_id}:{time_window}
   • Value: {remaining_requests}:{reset_time}

2. Phone Validation Cache:

   • Key: phone_validation:{country_code}:{phone_number}
   • Value: {carrier_id}:{valid_flag}:{last_updated}

3. Session State Cache:

   • Key: session:{user_id}:{session_id}
   • Value: {status}:{last_activity}:{rate_limit_data}

This caching architecture is designed to handle peak QPS efficiently while maintaining data consistency and providing predictable performance characteristics for the SMS Gateway system.

7. Messaging & Event Streaming Architecture

Message Broker Selection

For an SMS Gateway system, Apache Kafka is the optimal choice for the message broker due to:

• High throughput: Required for handling millions of SMS messages per day with low latency
• Durability guarantees: Messages are persisted to disk and can be replayed for fault tolerance
• Horizontal scalability: Can scale to handle peak loads with multiple brokers and partitions
• Stream processing capabilities: Enables real-time analytics and alerting on SMS delivery metrics
• Producer/consumer model: Natural fit for decoupling SMS senders from delivery processors

Event Schema Registry and Versioning

• Schema Registry: Confluent Schema Registry for maintaining versioned event schemas
• Versioning strategy: Semantic versioning (v1.0, v1.1) with backward compatibility
• Schema evolution: Allow forward and backward compatibility for schema changes
• Validation: Schema validation at producer side to ensure data integrity

Topic Partitioning and Consumer Group Design

• Partition count: 128 partitions per topic to support high throughput and parallel processing
• Partition key strategy: SMS ID or mobile number hash for consistent partitioning
• Consumer groups:
  • delivery-processing-group: Handles delivery status updates and retries
  • billing-group: Processes billing events from successful deliveries
  • analytics-group: Collects metrics for real-time analytics

Event Ordering Guarantees

• Per-partition ordering: Messages within a partition maintain strict order
• Global ordering: Not required for SMS delivery, but can be achieved via single partition if needed
• Use case: Delivery status events for same SMS ID are processed in order to prevent race conditions

Exactly-once vs. At-least-once Processing Trade-offs

Decision: At-least-once processing with idempotent consumers for SMS delivery events

Justification:

• SMS delivery reliability: Critical to ensure messages are delivered at least once
• Idempotency: SMS delivery systems can be designed to handle duplicate processing gracefully
• Simplified architecture: Avoids complexity of maintaining state for exactly-once semantics
• Cost consideration: Lower operational overhead compared to exactly-once systems

Event Replay and Reprocessing Strategy

• Replay capability: Kafka's retention policy (7 days) enables event replay for debugging
• Manual reprocessing: Admin UI to trigger reprocessing of specific SMS IDs or time ranges
• Replay monitoring: Track replayed events and their processing status
• Failure recovery: Automatic reprocessing of failed delivery events from dead letter queues

Dead Letter Queue and Poison Pill Handling

• DLQ configuration:
  • Primary DLQ topic: sms-delivery-failures
  • Secondary DLQ topic: sms-billing-failures
• Poison pill detection:
  • Retry count threshold: 5 attempts before moving to DLQ
  • Timeout handling: Messages that exceed processing time limits are moved to DLQ
• Manual intervention:
  • Alerting on DLQ queue size
  • Admin dashboard to inspect and reprocess failed messages
• Retry logic:
  • Exponential backoff for DLQ retries (1s, 5s, 30s, 5m, 1h)
  • Maximum retry limit: 10 attempts before permanent failure

Event Flow Architecture

graph TD
    A[Mobile App/Client] -->|SMS Request| B(API Gateway)
    B --> C[SMS Service]
    C --> D[Event Producer]
    D --> E[Kafka Topics]
    E --> F[Delivery Processor]
    E --> G[Billing Processor]
    E --> H[Analytics Processor]
    F --> I[Carrier Gateway]
    I --> J[Delivery Status]
    J --> K[Event Consumer]
    K --> L[Status Update DB]
    G --> M[Billing Service]
    H --> N[Analytics DB]
    K --> O[DLQ Manager]

Message Format and Serialization

• Serialization format: JSON with schema validation
• Message structure:

  {
    "messageId": "uuid",
    "source": "mobile-app",
    "destination": "+1234567890",
    "content": "Hello World",
    "timestamp": "2023-01-01T12:00:00Z",
    "priority": "high",
    "retryCount": 0
  }

Cross-Service Event Integration

• Delivery status events: Trigger billing and analytics services
• Billing events: Integrate with payment processing systems
• Analytics events: Feed real-time dashboard and alerting systems
• Error propagation: Failed delivery events trigger alerting and manual intervention workflows

Retry and Backoff Strategy

• Immediate retry: 1s delay for transient failures
• Exponential backoff: 5s, 30s, 5m, 1h intervals
• Max retries: 10 attempts before moving to DLQ
• Dead letter queue: Separate topic for failed messages requiring manual intervention

Monitoring and Alerting

• Kafka metrics: Consumer lag, producer throughput, partition availability
• Event processing rates: Track successful vs failed message processing rates
• DLQ monitoring: Alert on DLQ queue size exceeding threshold (500 messages)
• Latency monitoring: End-to-end SMS delivery time tracking

Data Retention and Archiving

• Event retention: 7 days for operational purposes
• Archive strategy: Historical delivery events can be archived to object storage (S3/GCS) for compliance
• Compliance: Maintain audit trail of all SMS delivery events for regulatory requirements

Cross-Region Event Handling

• Multi-region support: Kafka clusters in multiple regions for high availability
• Cross-region replication: Event replication between regions for disaster recovery
• Region-aware routing: Route events to nearest Kafka cluster for optimal performance

Performance Optimization

• Batch processing: Process multiple SMS events in batches to reduce overhead
• Async processing: Non-blocking event handling to maintain high throughput
• Partitioning strategy: Consistent hashing for optimal distribution of SMS events across partitions

8. Reliability Engineering

Failure Mode Analysis

Single Points of Failure:

• SMS Gateway API endpoint (if not properly load balanced)
• Primary SMS provider integration (single vendor dependency)
• Centralized message queue (Kafka/SQS) failure
• Shared database cluster for all services
• Global rate limiting service (single point for all traffic)
• Authentication service (if not properly fault-tolerant)

Service-Level Failures:

• Message delivery service can fail due to provider API issues
• Retry logic can cause cascading failures if not bounded
• Cache layer can become a bottleneck during high traffic
• Database connection pool exhaustion under load

Fault Tolerance Patterns

Circuit Breaker Pattern:

• Implemented at SMS provider integration layer (e.g., Twilio, Nexmo)
• Trip after 5 consecutive failures within 10 seconds
• Half-open state to test provider health every 30 seconds
• Prevents cascading failures when upstream providers are down

Bulkhead Pattern:

• Isolated thread pools per SMS provider
• Maximum 100 concurrent requests to each provider
• Prevents one failing provider from consuming all system resources

Timeout & Retry:

• 5-second timeout for SMS provider API calls
• Exponential backoff retry (1s, 2s, 4s, 8s) up to 3 attempts
• Retry only on transient failures (network timeouts, 5xx errors)
• Immediate retry for client-side errors (4xx)

Multi-Region / Multi-AZ Strategy

Active-Active Design:

• Deploy in 3 regions (us-east, eu-west, ap-southeast)
• Each region has independent SMS gateways and provider integrations
• Regional load balancers route traffic to nearest healthy gateway
• Cross-region replication for critical metrics and logs

Failover Mechanism:

• Health checks every 30 seconds to all providers
• Automatic failover to backup region within 2 minutes
• Graceful degradation (send to fallback provider) when primary fails

Chaos Engineering Strategy

GameDays:

• Weekly 2-hour sessions simulating provider outages
• Monthly full-scale chaos experiments (complete region failure)
• Quarterly cross-region disaster recovery drills

Steady-State Hypotheses:

• System maintains 99.9% availability during provider outages
• Message delivery continues with acceptable latency (p99 < 5s)
• No data loss during region failover scenarios

Disaster Recovery Plan

RTO/RPO Targets:

• RTO: 2 minutes for primary region failover
• RPO: <5 minutes for message delivery state
• Recovery time: 15 minutes for complete region restoration

Recovery Procedures:

1. Auto-failover to secondary region within 2 minutes
2. Manual intervention required for cross-region data sync
3. Emergency rate limiting to prevent provider spamming
4. Service degradation mode (accept but don't deliver) for 30 minutes

Runbook Design

Top-5 Failure Scenarios:

Scenario	Impact	Recovery Steps
Provider API Down	50% message delivery delay	Switch to backup provider, trigger circuit breaker
Database Connection Pool Exhaustion	Service unavailability	Scale DB connections, implement connection pooling limits
Cache Cluster Failure	Increased latency	Bypass cache layer, warm cache from DB
Network Partition	Regional service isolation	Failover to alternate region, re-enable cross-region sync
Rate Limiter Exhaustion	Client throttling	Reset rate limits, implement client-side retry logic

Runbook Automation:

• Auto-triggered failover for provider outages
• Alert escalation to on-call engineers (2-minute response)
• Automated health check and recovery scripts for all services
• Pre-defined rollback procedures for each service version

9. Observability Architecture

Logging Stack

• Centralized logging solution: Loki (with Promtail) for log collection and storage, integrated with Grafana for visualization
• Log format: Structured JSON logs with consistent fields (timestamp, service, level, trace_id, span_id, message, error details)
• Log retention: 30 days for operational logs, 90 days for audit logs
• Log aggregation: All services stream logs to Loki via Promtail sidecars in Kubernetes
• Log filtering: Query capabilities by service, level, error code, and time range

Distributed Tracing

• Tracing system: OpenTelemetry (with Jaeger backend) for end-to-end request tracing
• Trace sampling: 1% sample rate for production, 100% for critical paths
• Trace propagation: HTTP headers (traceparent/tracestate) and gRPC metadata for service-to-service calls
• Trace context: Integration with application-level trace IDs for correlation across services
• Span types: Database, HTTP, external service calls, internal processing spans

Metrics Stack

• Metrics collection: Prometheus for time-series metrics with Grafana dashboarding
• Key metrics:
  • SMS send success/failure rates
  • Latency (p50/p95/p99) by provider and region
  • Queue depth and processing rates
  • Error counts by type (rate limiting, provider errors, etc.)
  • Throughput (QPS) and capacity utilization
• Service-level metrics: Per-service error rates, response times, throughput
• Infrastructure metrics: CPU, memory, disk I/O, network usage per pod

Alerting Strategy

• Severity levels:
  • P0 (Critical): SMS delivery failure >5% or provider outage
  • P1 (High): Latency >2s p99, queue depth >10k
  • P2 (Medium): Error rate >1%, cache hit rate <60%
  • P3 (Low): Minor performance degradation or configuration changes
• Routing: Alerts routed to PagerDuty with escalation policies, OpsGenie for secondary notifications
• Silence rules: Alert silencing for scheduled maintenance windows and known issues
• Notification channels: Email, Slack, SMS to on-call engineers

SLI/SLO Definition and Error Budget Tracking

• SLIs:
  • SMS delivery success rate (99.5% SLO)
  • End-to-end latency (p99 < 2s)
  • Provider availability (99.9% SLO)
• SLOs:
  • 99.5% of SMS delivered within 30 seconds
  • 99.9% of requests processed under 2s
• Error budget: 5% for delivery success, 0.1% for latency (allowing 20 minutes of downtime daily)
• Dashboard: Grafana dashboard with real-time error budget burn rate, SLO compliance, and alert history

Capacity Planning and Load Forecasting

• Load forecasting: Historical SMS volume trends, seasonal patterns, and growth projections
• Capacity monitoring: Real-time metrics on queue depth, processing rates, and provider throughput limits
• Auto-scaling correlation: Metrics-driven scaling decisions for message processing services
• Forecasting tools: Prometheus + Grafana with forecasting dashboards and capacity alerts
• Resource utilization: CPU, memory, and I/O monitoring to predict scaling needs before bottlenecks occur

10. API Gateway & Edge Layer

API Gateway Selection

For an SMS Gateway system, we'll select AWS API Gateway as the primary API gateway with custom edge services for advanced routing and protocol handling. This decision balances managed service benefits with the flexibility needed for SMS-specific requirements.

Justification:

• Managed service benefits: Reduced operational overhead, automatic scaling, built-in rate limiting, and monitoring
• Integration with AWS ecosystem: Seamless integration with Lambda, SNS, SQS, and other AWS services
• Protocol support: Native support for REST and WebSocket protocols required for SMS delivery
• Security features: Built-in authentication, authorization, and DDoS protection capabilities

Request Routing Strategy

The API gateway will implement a multi-layered routing strategy:

1. Service-based routing: Route requests to appropriate backend services (SMS delivery, status tracking, user management)
2. Protocol-aware routing: Handle both HTTP/REST and WebSocket connections for real-time status updates
3. Geographic routing: Route based on sender/receiver country codes for optimal carrier selection
4. Priority-based routing: Route high-priority messages through dedicated channels

Authentication and Authorization at Gateway Layer

Authentication:

• API Key-based authentication: Each client (SMS provider, partner) will have unique API keys
• OAuth2/OIDC integration: For enterprise partners requiring OAuth2 flows
• mTLS support: For secure communication between trusted partners and the gateway

Authorization:

• Role-based access control (RBAC): Different permissions for different client types (admin, partner, consumer)
• Rate limit enforcement: Per-client and per-service level limits
• IP whitelisting: For enterprise clients requiring strict IP restrictions

Rate Limiting and Throttling Design

Global rate limiting:

• 10,000 requests/second global limit (configurable per client)
• 500 concurrent connections per client

Per-client limits:

• 1,000 messages/minute (configurable)
• 50,000 messages/hour (configurable)
• 100,000 messages/day (configurable)

Throttling strategies:

• Leaky bucket algorithm: Smooth burst traffic while maintaining average rate limits
• Token bucket algorithm: Allow bursts up to configured limits with strict average rates

Request Transformation and Protocol Translation

Protocol translation:

• SMS over HTTP: Translate between HTTP requests and SMS protocol formats (SMPP, HTTP)
• Format normalization: Standardize incoming message formats to internal SMS format
• Encoding conversion: Handle various character encodings (UTF-8, GSM 7-bit, UCS-2)

Request transformation:

• Message content validation: Validate message length, character set, and content restrictions
• Metadata enrichment: Add timestamps, message IDs, and routing information
• Carrier-specific formatting: Format messages according to carrier requirements

BFF (Backend for Frontend) Design

BFF pattern implementation:

• Partner-specific BFFs: Create specialized gateway services for different partner types (enterprise, mobile operators, marketing platforms)
• Message format adaptation: Transform internal SMS format to partner-specific formats (e.g., JSON, XML)
• Aggregation services: Combine multiple SMS operations into single responses for partners

Partner BFFs:

1. Enterprise Partner BFF: Handle complex enterprise integration requirements, including webhooks and detailed status reporting
2. Marketing Partner BFF: Optimize for high-volume transactional SMS with real-time delivery tracking
3. Mobile Operator BFF: Handle carrier-specific protocols and compliance requirements

Edge Layer Considerations

Edge placement strategy:

• Multi-region deployment: Deploy API gateways in multiple AWS regions (us-east-1, eu-west-1, ap-southeast-1)
• Edge locations: Use AWS CloudFront for global content delivery and caching
• Local edge routing: Route requests to nearest available gateway for optimal latency

Performance optimizations:

• Caching at edge: Cache frequently accessed partner configurations and rate limit information
• Pre-warming: Warm up gateway instances during traffic spikes
• Connection pooling: Maintain persistent connections to backend services

Security and Compliance at Edge Layer

Security measures:

• TLS 1.3 termination: All traffic encrypted with modern TLS standards
• Request validation: Input sanitization and content validation at edge
• Threat detection: Integration with AWS WAF for DDoS protection and malicious traffic filtering

Compliance considerations:

• GDPR compliance: Data handling and processing restrictions for EU customers
• Carrier compliance: Ensure all SMS content meets carrier-specific regulations
• Audit logging: Comprehensive logging of all API interactions for compliance reporting

Monitoring and Metrics at Edge Layer

Metrics collection:

• Request/response times: Track latency for different API endpoints
• Rate limit violations: Monitor and alert on rate limit exceeded scenarios
• Authentication failures: Track failed authentication attempts for security monitoring

Alerting strategy:

• Critical alerts: Rate limit exhaustion, authentication failures, service outages
• Warning alerts: Near-threshold rate limits, high latency trends
• Informational alerts: Successful API calls, configuration changes

This edge layer design ensures that the SMS Gateway system can handle high-volume traffic while maintaining security, compliance, and optimal performance across global deployments.

11. Authentication & Authorization Architecture

Identity Provider Integration

For an SMS Gateway system, we'll integrate with AWS Cognito as the identity provider due to its seamless integration with AWS services, built-in support for SMS/MFA capabilities, and compliance with enterprise security standards. Cognito provides robust user pool management and federated identity through SAML/OIDC, making it ideal for handling both internal system users (admins, developers) and external partners who may need programmatic access.

OAuth2 / OIDC Flow Design

sequenceDiagram
    participant Client
    participant AuthN
    participant Cognito
    participant TokenService

    Client->>AuthN: Request access to SMS Gateway API
    AuthN->>Cognito: Redirect to Cognito login page
    Cognito-->>Client: Login form
    Client->>Cognito: Submit credentials
    Cognito->>Cognito: Validate credentials
    Cognito->>AuthN: Return ID token + Access token
    AuthN->>TokenService: Validate token
    TokenService-->>AuthN: Token valid/invalid
    AuthN->>Client: Return API response or 401

The system will implement the Authorization Code flow with PKCE for mobile and web clients, supporting both direct user authentication and service-to-service flows. For partner integrations, we'll support OAuth2 client credentials flow with API key validation.

Service-to-Service Authentication

For internal microservices communication, we'll implement mutual TLS (mTLS) with SPIFFE/SPIRE for identity management. This ensures that only authorized services can communicate with each other, preventing unauthorized access to sensitive SMS routing and delivery logic.

RBAC / ABAC Model Design

We'll implement a hybrid RBAC/ABAC model:

• RBAC for role-based access control (Admin, Developer, Partner)
• ABAC for attribute-based decisions (sender ID restrictions, time-of-day constraints, rate limits)

The system will support granular permissions like:

• Send SMS from specific sender IDs
• Set rate limits per partner or user
• Access to specific regions or carriers

Token Revocation Strategy

We'll implement a short-lived JWT token strategy (15-30 minutes) with refresh token rotation. When tokens are revoked, they'll be added to a short-lived blacklist (Redis-based) with TTL of 1 hour to handle immediate revocation needs while minimizing performance impact.

The system will support:

• Immediate token invalidation via blacklisting
• Refresh token rotation for enhanced security
• Token expiration with automatic refresh capability

This approach balances security requirements with performance, ensuring that compromised tokens are quickly invalidated while maintaining acceptable user experience for legitimate access patterns.

12. Search Architecture

Search Use Cases and Requirements

The SMS Gateway system does not have core search capabilities as its primary function. The system's main purpose is to route, process, and deliver SMS messages between various providers and end users. While there may be some operational dashboards or administrative interfaces that could benefit from search capabilities, these are not core to the system's functionality.

Search Indexing Pipeline Design

Since search is not a core capability, any indexing pipeline would only be relevant for:

• Admin dashboard search (message history, delivery status)
• Analytics dashboards
• Operational monitoring

If implemented, the indexing pipeline would follow a pattern:

Message Events → Transform (enrich metadata) → Index (Elasticsearch/OpenSearch)

Index Schema and Query Patterns

For any search capability that might be implemented:

• Fields: message_id, user_id, timestamp, status, provider, content_preview, recipient
• Query patterns:
  • User-based message history search (by user_id)
  • Status-based filtering (delivered, failed, queued)
  • Time-range searches for analytics
  • Content-based keyword search (limited to content preview)

Real-time vs. Near-real-time Indexing

Given that SMS delivery is time-sensitive and search is not a primary requirement, the system would prioritize:

• Near-real-time indexing (1-5 minute delay) for operational dashboards
• Batch indexing for analytics and reporting use cases

Relevance Tuning Strategy

If search is implemented:

• Simple field matching for user and status queries
• Content relevance based on message content (limited to preview fields)
• No complex ranking algorithms required

Search Scaling Design

Since search is not a core capability:

• Single Elasticsearch/OpenSearch instance for admin dashboards (with potential for sharding as requirements grow)
• Minimal shard configuration (1-2 shards, 1 replica) for operational dashboards
• No complex sharding strategy required

Alternative Approaches Considered

The system architecture decision was to avoid implementing search capabilities as a core feature, instead focusing on:

• Message routing and delivery reliability
• Provider integration and failover mechanisms
• Real-time monitoring and alerting through observability stack
• Operational dashboards using simple database queries with proper indexing

Implementation Status

Search capability is not implemented as part of the core SMS Gateway system design, as it does not align with the primary use case. Any search functionality would be implemented as a separate administrative dashboard layer with minimal integration to the core messaging system.

The decision to exclude search from the core architecture reduces complexity, improves performance for primary use cases, and maintains clear separation of concerns between messaging delivery and operational analytics.

13. Scalability Deep Dive

Bottleneck Identification per Component

API Gateway Layer:

• Peak QPS: 10,000 (assuming 1000 DAU with 10:1 peak-to-average ratio)
• Throughput: ~2GB/s (assuming 200B average message size)
• Bottleneck: Rate limiting, authentication overhead, TLS termination
• Estimated max concurrent connections: 5000

Message Processing Service:

• Database QPS: 10,000 (assuming 50% cache hit rate)
• Peak DB connections: ~200 (with connection pooling)
• Bottleneck: Database write contention, message queue backpressure

External SMS Provider Integration:

• Rate limit per provider: 1000 messages/second (varies by provider)
• Bottleneck: Provider API throttling, network latency
• Estimated 50% of messages may fail and retry

Database Layer:

• Relational DB (PostgreSQL): 1000-2000 QPS peak
• Cache layer (Redis): 50,000 QPS peak
• Object storage (S3): 1000 PUTs/sec per bucket

Horizontal vs. Vertical Scaling Decisions

Stateless Services (API Gateway, Message Processing):

• Horizontal scaling preferred: Stateless services can scale out easily
• Auto-scaling groups with HPA based on QPS and latency metrics
• Target: 10x horizontal scaling capability with minimal configuration changes

Stateful Services (Database, Cache):

• Vertical scaling for databases: PostgreSQL with read replicas and connection pooling
• Horizontal scaling via sharding for high-volume tables (message logs, user preferences)
• Hybrid approach: Vertical for consistency, horizontal for throughput

Database Scaling Path

Phase 1 (0-1M messages/month):

• Single PostgreSQL instance with read replicas (3)
• Redis cache for session and rate limiting

Phase 2 (1M-10M messages/month):

• Horizontal sharding by user ID (hash-based)
• Read replicas increased to 10
• Separate database clusters for different data types (user, message logs, analytics)

Phase 3 (10M+ messages/month):

• Distributed database (CockroachDB or Spanner)
• Multi-region deployment for availability
• Message queue integration for async processing

Stateless vs. Stateful Service Design Implications

Stateless Services:

• No session affinity required
• Easy to scale and deploy
• Can use Kubernetes horizontal pod autoscaler (HPA)
• Load balancing via round-robin or consistent hashing

Stateful Services:

• Requires careful state management (user sessions, rate limits)
• Use Redis for session storage and distributed locks
• Message queues to ensure reliable processing

Session Affinity and Its Implications

Session Affinity Issues:

• Session stickiness creates hotspots in load balancer
• Can cause uneven distribution of load across service instances
• Requires careful session management to avoid data loss

Mitigation Strategy:

• Use Redis for session storage (distributed cache)
• Implement session timeout and cleanup mechanisms
• Use client-side session tokens with short TTLs (15 minutes)
• Design services to be stateless where possible

Performance Bottlenecks and Mitigation

Database Write Bottleneck:

• Solution: Implement batch writes with message queues (Kafka)
• Use database connection pooling to reduce overhead
• Implement read replicas for reporting queries

Network Latency with External Providers:

• Solution: Use asynchronous processing with message queues
• Implement retry logic with exponential backoff
• Monitor provider SLA and implement fallback mechanisms

Cache Invalidation:

• Solution: Implement cache warming strategies
• Use Redis with TTLs to prevent stale data
• Implement cache invalidation events for critical updates

Scaling Metrics and Triggers

Auto-scaling Triggers:

• CPU utilization > 80% for 5 minutes → scale up
• QPS > 10,000 per second → trigger horizontal scaling
• DB connection pool exhaustion → scale database instances
• Message queue backlog > 10,000 messages → trigger processing service scaling

Capacity Planning:

• 5x headroom for peak traffic (10,000 QPS → target 50,000 QPS capacity)
• 3x database read replicas for handling burst traffic
• 10x cache capacity to handle cold starts and warm-up periods

Cross-Service Scaling Considerations

Message Processing Service:

• Scales independently based on QPS
• Requires coordination with database and cache services
• Implements circuit breaker for external provider failures

API Gateway:

• Scales with traffic volume
• Requires rate limiting and throttling to prevent downstream overload
• Implements circuit breaker for backend services

External Provider Integration:

• Scales based on provider capacity limits
• Requires queue-based buffering for burst traffic
• Implements retry and fallback strategies for reliability

Performance Targeting

QPS Scaling Targets:

• 10,000 QPS (peak) → 50,000 QPS (target capacity)
• 100ms p50 latency → 500ms p99 latency
• 99.9% availability target (30s downtime/month max)
• 50ms average response time for critical paths

Resource Utilization Targets:

• CPU utilization < 70% under normal load
• Memory usage < 80% of allocated capacity
• Database connection pool utilization < 60%
• Cache hit rate > 90% for frequently accessed data

Dependency Management

Service Dependencies:

• External SMS providers (Twilio, AWS SNS) - 1000 messages/sec limit
• Internal message queue (Kafka) - 50,000 messages/sec capacity
• Database cluster - 2000 QPS read/write capacity
• Cache layer (Redis) - 50,000 QPS capacity

Scaling Dependencies:

• External provider limits drive queue-based buffering
• Database scaling affects message processing throughput
• Cache capacity affects API gateway performance
• Message queue capacity affects overall system throughput

Infrastructure Scaling Patterns

Horizontal Scaling:

• Kubernetes pods with HPA based on QPS and latency
• Database read replicas for handling increased load
• Message queue partitions to handle burst traffic

Vertical Scaling:

• Database instances with increased CPU/memory
• Cache cluster with additional nodes for capacity
• API gateway with increased connection handling capability

Cross-Service Scaling:

• Message processing service scales based on queue depth
• Database scaling based on concurrent connection usage
• Cache layer scales based on hit rate and memory utilization

14. Infrastructure & Deployment Architecture

Cloud Provider and Region Selection

Primary Cloud: AWS (with multi-region support) - Justification: AWS provides mature SMS gateway integrations through SNS, extensive global infrastructure, and robust ecosystem for messaging services. Multi-region support ensures high availability and low latency.

Region Strategy:

• Primary: us-east-1 (lowest latency for US market)
• Secondary: eu-west-1 (for EU compliance and redundancy)
• Tertiary: ap-southeast-1 (for APAC expansion)

Kubernetes Cluster Design

Cluster Architecture:

• Multi-Tenant Clusters: Separate clusters per environment (dev/staging/prod) with namespace isolation
• Resource Quota Model: Per-namespace CPU/Memory limits with HPA and VPA enabled
• Cluster Autoscaling: Auto-scaling groups for node pools with spot instances for cost optimization

Namespace Strategy:

• prod-apps - Production services
• staging-apps - Staging services
• dev-apps - Development services
• infrastructure - Monitoring and infrastructure services

Container Image Strategy

Base Images:

• Alpine Linux for minimal size and security
• Official language-specific base images (node:alpine, python:3.11-alpine)

Multi-stage Builds:

• Build stage with dev dependencies
• Runtime stage with minimal production dependencies only
• Image signing and verification using Cosign

Image Registry:

• ECR (Elastic Container Registry) with lifecycle policies
• Image scanning for vulnerabilities using Trivy integration

Deployment Strategy

Primary Strategy: Blue-Green deployment with traffic shifting via Route53 for zero-downtime updates.

Traffic Management:

• 50/50 split during canary rollouts
• Auto-rollback on error rate > 2%
• Health checks with 30s timeout and 5s interval

Deployment Triggers:

• GitHub Actions CI/CD pipeline
• Git tags for versioned releases
• Manual approval gates for prod deployments

Auto-scaling Design

Horizontal Pod Autoscaler (HPA):

• Target: 70% CPU utilization
• Scale out: 5 pods minimum, 100 pods maximum per service
• Scale in: 2 pods minimum

Vertical Pod Autoscaler (VPA):

• Memory and CPU recommendations based on historical usage
• Mode: "Auto" for optimal resource allocation

KEDA Integration:

• Event-driven scaling for message queues (SQS, Kafka)
• Target: 100 messages per queue as scaling trigger
• Minimum scale: 2 pods, maximum: 50 pods

Infrastructure as Code Approach

IaC Tools:

• Terraform for infrastructure provisioning and management
• Terragrunt for module organization and environment-specific configuration
• GitOps with ArgoCD for declarative deployment management

Module Organization:

• vpc - VPC and subnet configuration
• kubernetes - Cluster and namespace setup
• services - Service-specific configurations
• monitoring - Alerting and logging setup

State Management:

• S3 backend with DynamoDB locking for Terraform state
• Version control of infrastructure code in Git

Security Integration:

• IAM roles and policies for service accounts
• Security group rules with least privilege principle
• CloudTrail logging for all infrastructure changes

Environment Separation:

• Separate Terraform workspaces for dev/staging/prod
• Parameterized variables with environment-specific overrides
• Secret management through AWS Secrets Manager integration

15. Security Architecture

VPC Design and Network Segmentation

The system operates within a multi-AZ VPC architecture with public and private subnets. Public subnets host API gateways, load balancers, and edge services that require external connectivity. Private subnets contain application services, databases, and internal microservices that communicate exclusively through private network paths. Network segmentation is enforced using security groups at the instance level and NACLs at subnet level to restrict traffic flows based on service requirements.

Zero-Trust Network Model

All internal communications are secured using mTLS between services, with certificate management handled through a centralized PKI infrastructure. Services authenticate and authorize each other using service-to-service tokens rather than relying on IP-based trust models. All external API calls are authenticated via OAuth2/OIDC flows with JWT validation and token refresh mechanisms.

TLS Termination Strategy

TLS termination occurs at the API gateway layer for external traffic, with strict cipher suites enforced (TLS 1.3+). Internal service-to-service communication uses mTLS with mutual certificate validation. Application services do not terminate TLS directly but rely on the service mesh for secure inter-service communication.

DDoS Protection Strategy

DDoS protection is implemented through AWS Shield Advanced or equivalent cloud provider DDoS mitigation services. Rate limiting and request filtering are applied at the API gateway level to detect and block malicious traffic patterns. Traffic scrubbing is configured for high-volume attack scenarios, with automatic scaling of WAF rules and IP address blacklisting capabilities.

Secrets Management Architecture

Secrets are managed through AWS Secrets Manager or equivalent cloud-native secrets management service. Configuration values, API keys, and database credentials are stored securely with automatic rotation policies. Secrets are injected into services at runtime via sidecar containers or service mesh proxies, avoiding hardcoding in application code or configuration files.

Vulnerability Scanning in CI/CD

Container images undergo automated vulnerability scanning during build process using tools like Trivy or Snyk. Security scans are integrated into the CI pipeline to block deployment of images with critical vulnerabilities. Regular security audits and penetration testing are conducted on production services to identify potential attack vectors.

WAF Rules for Critical Endpoints

Web Application Firewall (WAF) is deployed to protect critical endpoints from common web exploits such as SQL injection, XSS attacks, and DDoS patterns. Rules are customized for the SMS gateway system to block suspicious API access patterns, enforce rate limits on sensitive endpoints, and filter out malformed requests. WAF policies are updated regularly based on threat intelligence feeds.

Data Protection and Encryption

All data at rest is encrypted using AES-256 encryption keys managed through AWS KMS or equivalent key management service. Data in transit is protected using TLS 1.3+ for all external communications and mTLS for internal service-to-service communication. Sensitive data fields (e.g., phone numbers, message content) are encrypted at rest using customer-managed keys with audit logging.

Access Control and Authentication

Authentication is handled through centralized identity providers (e.g., AWS Cognito, Auth0) that support OAuth2/OIDC flows. Service-to-service authentication uses mTLS with SPIFFE/SPIRE for secure identity management. RBAC is implemented across services to enforce least-privilege access control, with granular permissions for different user roles (admin, operator, read-only).

Compliance and Audit

The system maintains detailed audit logs for all SMS delivery operations, including message metadata, recipient information, and delivery status. These logs are stored in secure, immutable storage with retention policies aligned to regulatory requirements (e.g., GDPR, HIPAA). Regular compliance audits are performed to ensure adherence to data protection standards and industry best practices.

16. Deployment & Release Strategy

Deployment Strategy and Traffic Management

The SMS Gateway system will utilize a blue-green deployment strategy for production releases to ensure zero-downtime deployments and quick rollback capabilities. This approach involves running two identical production environments where only one is active at any given time. When deploying a new version, the system will:

• Deploy the new version to the inactive environment
• Run comprehensive health checks and smoke tests
• Perform a controlled traffic cutover to the new version
• Monitor for any anomalies before fully committing to the new version

For traffic management, we'll implement Gradual Traffic Splitting using service mesh capabilities or API gateway routing rules to shift traffic from the old version to the new version in 10% increments over 30 minutes, allowing for quick rollback if issues arise.

Artifact Promotion Model

The system follows a Trunk-Based Development approach with a multi-stage artifact promotion pipeline:

1. Dev Environment: Continuous integration with automated unit and integration tests
2. Staging Environment: Full end-to-end testing including load testing, security scanning, and compliance checks
3. Production Gate: Manual approval process with automated canary analysis before production deployment

Each release goes through the following gates:

• Code review and static analysis
• Automated test suite (unit, integration, performance)
• Security vulnerability scanning (Snyk/Trivy)
• Load testing and capacity validation
• Manual production approval with rollback capability

Zero-Downtime Database Migration Strategy

Database migrations will be handled using Blue-Green Database Strategy:

• Maintain two identical database instances (primary and replica)
• Apply schema changes to the replica instance in a controlled manner
• Validate data consistency and query performance on the new schema
• Switch traffic to the updated database instance with minimal downtime (typically < 5 seconds)
• Decommission the old database instance after validation

For data migrations, we'll use online schema change tools like pt-online-schema-change to minimize table locking and maintain availability during the migration process.

Feature Flag Integration and Progressive Rollout

We'll implement a feature flag system using LaunchDarkly or custom-built solution to support:

• Gradual Rollouts: Start with 5% of users and increase in 10% increments
• Risk-Based Rollouts: Enable features for specific user segments or regions first
• A/B Testing: Compare different implementations or configurations
• Emergency Cutoff: Ability to immediately disable problematic features

Feature flags will be managed through a centralized configuration service with:

• Real-time flag updates without service restarts
• Audit trails and change history
• Rollback capability for flags

Canary Analysis Automation and Auto-Rollback Triggers

Canary analysis will be automated using:

• Metrics-based validation: Monitor p99 latency, error rates, and throughput
• Business logic validation: Validate SMS delivery success rates and content integrity
• Automated rollback thresholds: If error rate exceeds 1% or latency increases by >50%, automatically roll back to previous version

The system will implement auto-rollback triggers:

• Error rate > 1% over 5-minute window
• Latency p99 > 2x baseline for 10 minutes
• Delivery success rate < 95% for 15 minutes

Release Cadence and Branching Strategy

The system follows a Trunk-Based Development with the following release cadence:

• Weekly minor releases: Feature updates and bug fixes
• Monthly major releases: Breaking changes and significant feature additions
• Emergency hotfixes: As needed with immediate deployment to production

Branching strategy:

• main branch as the single source of truth
• Feature branches created for major features (short-lived, merged back to main)
• Release branches created only for patch releases and hotfixes

Release management will be handled through:

• Git tags for version control
• Semantic versioning (MAJOR.MINOR.PATCH)
• Release notes generation and automated documentation updates

Deployment Automation and Infrastructure as Code

All deployments will be managed through Infrastructure as Code (IaC) using Terraform for:

• Kubernetes cluster provisioning and configuration
• Service and namespace creation
• Network policies and security group definitions
• Database instance setup and configuration

Deployment automation includes:

• CI/CD pipeline integration with GitHub Actions or Jenkins
• Automated testing at each stage of the deployment pipeline
• Configuration drift detection and remediation
• Deployment validation and health check integration

Rollback Strategy and Disaster Recovery

In case of deployment failures, the system will support:

1. Immediate Rollback: Automatic rollback to previous stable version within 30 seconds
2. Manual Override: Manual rollback capability through deployment dashboard or CLI
3. Partial Rollback: Ability to rollback specific services while keeping others at new version

Disaster recovery includes:

• Automated backup and restore procedures for all services
• Multi-region deployment strategy with automatic failover capability
• Comprehensive runbook documentation for all failure scenarios

Release Validation and Monitoring

Each release will undergo:

• Pre-deployment validation: Health checks, performance benchmarks, and security scans
• Post-deployment monitoring: Real-time alerting and capacity validation
• Business impact verification: SMS delivery success rate and user experience validation

The system will maintain a release log documenting:

• Release version and timestamp
• Changes included in the release
• Deployment status and rollback actions (if any)
• Performance metrics comparison between versions

17. Cost Architecture

Major Cost Drivers Analysis

Compute Costs (35-40% of total):

• EC2 instances for API services, message processing, and background workers
• Kubernetes cluster nodes (EKS/GKE) with auto-scaling
• Lambda functions for event-driven processing (if using serverless components)

Storage Costs (25-30% of total):

• Database storage (PostgreSQL, Redis, DynamoDB)
• Message queue storage (Kafka/SQS)
• Object storage for media attachments (S3/GCS)
• Backup storage with 30-day retention

Network & Data Transfer Costs (20-25% of total):

• Ingress/egress traffic between services and external providers
• SMS gateway API calls to telecom carriers (per-message costs)
• CDN traffic for content delivery (if applicable)

Database & Messaging Service Costs (15-20% of total):

• Managed database services (RDS, DynamoDB)
• Message broker services (Kafka, SQS/SNS)
• Cache services (ElastiCache, Redis)

Cost Optimization Strategy

Reserved Instances & Spot Pricing:

• Commit to 3-year reserved instances for steady-state workloads (API services, background workers)
• Use spot instances for batch processing and non-critical tasks (message queue consumers, analytics jobs)
• Implement spot instance fallback with graceful degradation for cost-sensitive operations

Storage Tiering Strategy:

• Hot data (active user sessions, recent messages) stored in SSD-backed instances
• Warm data (recent message history, user preferences) stored in cheaper HDD storage
• Cold data (archived messages, long-term analytics) moved to Glacier/Archive storage with 1-5 day retrieval times
• Implement automatic tiering policies based on access frequency

Database & Cache Optimization:

• Use read replicas for high-read workloads (message history, user preferences)
• Implement connection pooling to reduce database overhead
• Use Redis for caching frequently accessed data (user profiles, carrier information)
• Implement database query optimization with proper indexing and materialized views

Reserved vs. Spot Instance Strategy

Reserved Instances (60% of compute):

• 3-year reserved instances for core API services and background workers
• 1-year reserved instances for predictable batch jobs (message analytics, reporting)
• Used for services with consistent CPU/memory requirements and SLA commitments

Spot Instances (30% of compute):

• For non-critical background processing (message analytics, historical data processing)
• For batch jobs that can tolerate interruptions (data exports, system maintenance)
• Spot instance pools with fallback to on-demand instances for critical services

Data Egress Cost Minimization

Internal Traffic Optimization:

• Use private VPC endpoints for inter-service communication to minimize public network costs
• Implement service mesh (Istio) to reduce cross-region traffic and optimize routing
• Use local cache layers to minimize repeated database queries across services

External API Cost Management:

• Implement rate limiting and bulk processing to minimize per-message costs from telecom providers
• Use carrier aggregation strategies to reduce per-message costs (aggregate multiple messages before sending)
• Monitor and optimize SMS content length to reduce carrier charges

FinOps Dashboards and Budget Alerts

Cost Monitoring Dashboard:

• Real-time cost tracking per service, per region, and per resource type
• Monthly cost trend analysis with forecasting capabilities
• Cost allocation tags for team/resource-based cost tracking

Automated Budget Alerts:

• Weekly budget alerts with 80% and 95% thresholds
• Monthly cost overruns alerting to teams with >$10k monthly spend
• Automated budget caps for non-production environments

Cost Optimization Recommendations:

• Weekly auto-scaling recommendations based on historical usage patterns
• Monthly storage optimization suggestions (data archiving, compression)
• Quarterly cost review process with capacity planning alignment

The cost architecture is designed to support a scalable, cost-effective SMS gateway system while maintaining SLA compliance and operational efficiency. The strategy balances cost optimization with reliability, ensuring that cost-saving measures don't compromise system performance or availability.

18. Open Questions, Risks & Trade-offs

Decisions requiring product / engineering alignment

• SMS delivery guarantee levels - What level of delivery assurance is required (best-effort, at-least-once, exactly-once) and how does this impact system complexity and cost?
• International SMS support scope - Should we support all global carriers or focus on specific regions/phone number formats?
• Rate limiting policy enforcement - How strict should rate limits be for individual clients vs. overall system capacity?
• Fallback messaging channels - Should we implement alternative delivery methods (email, push) when SMS fails?

Key architectural risks

Risk	Impact	Likelihood	Mitigation
Carrier-level outages affecting all SMS delivery	High	Medium	Implement multiple carrier partnerships with fallback routing, monitor carrier health continuously
SMS delivery latency exceeding SLA	High	Medium	Implement async processing with retry mechanisms, prioritize critical paths
Message loss during high-volume bursts	High	Medium	Use durable message queues with proper ack/nack handling, implement dead letter queue
Regional compliance violations	High	Low	Implement regional compliance checks, maintain detailed audit logs
Third-party carrier reliability issues	Medium	Medium	Establish SLA thresholds with carriers, maintain backup carrier relationships
Cache invalidation causing delivery failures	Medium	Low	Implement cache warming strategies, use cache tags with TTLs

Trade-offs made with justification

• Consistency vs. Availability - Chose eventual consistency over strong consistency for message status updates to ensure high availability during network partitions, with compensating mechanisms for reconciliation
• Complexity vs. Performance - Prioritized performance for high-throughput SMS delivery over complex multi-step validation workflows, implementing async validation with error reporting
• Single-tenant vs. Multi-tenant - Selected multi-tenant design for cost efficiency and easier scaling, with strict isolation boundaries to prevent cross-client interference
• Real-time vs. Batch processing - Implemented real-time delivery for critical messages with batch processing for non-critical, bulk messaging to optimize resource usage
• Custom vs. Managed services - Used managed messaging services (e.g., AWS SQS) for reliability and operational simplicity over custom implementations, balancing cost vs. control

Alternatives considered and rejected

• Monolithic architecture - Rejected due to scalability limitations and difficulty in maintaining different service SLAs for various SMS use cases
• Full event sourcing - Rejected due to complexity and cost of maintaining full message history for all delivery states, opted for stateless services with audit logging instead
• Custom carrier integrations - Rejected due to high maintenance overhead and complexity of supporting dozens of different carrier APIs vs. using established SMS gateway providers
• In-memory storage for all data - Rejected due to persistence risks and inability to handle system restarts or scaling scenarios
• Synchronous delivery model - Rejected due to poor performance and availability under high load, preferring async processing for better throughput

Technical debt acknowledged

• Carrier-specific logic - Temporary carrier-specific handling logic that will need refactoring once we have a more mature carrier integration framework
• Hardcoded SMS content validation rules - Currently using hardcoded validation logic that will need to be made configurable for future flexibility
• Limited internationalization support - Initial implementation assumes US/Canada phone number formats, requiring future expansion for global support
• Basic rate limiting implementation - Current rate limiting is basic per-client limits that will need more sophisticated traffic shaping and queue management in later phases
• Audit log structure - Initial audit logging is minimal and will need enhanced schema to support complex compliance and forensic analysis requirements

19. Phased Implementation Roadmap

Phase 1: MVP (Minimum Viable Product)

Timeline: 8-12 weeks Focus: Core SMS delivery functionality with basic reliability

Key Deliverables:

• Basic SMS gateway service with REST API for message submission
• Integration with 1-2 SMS providers (Twilio, Nexmo, or local carrier API)
• Core message queue processing (Kafka/SQS) for outbound messages
• Simple database schema for message tracking and delivery status
• Basic authentication (API keys) and rate limiting
• Single-region deployment with basic monitoring (Prometheus + Grafana)
• Basic alerting for delivery failures and system errors
• Core service health checks and liveness probes

Dependencies:

• Cloud infrastructure provisioning (AWS/GCP)
• Basic CI/CD pipeline setup
• Initial security configuration and VPC design

Phase 2: Scale & Reliability (Hardening)

Timeline: 12-16 weeks Focus: Multi-region support, enhanced reliability, and operational maturity

Key Deliverables:

• Multi-region deployment capability (active-active or active-passive)
• Enhanced message delivery reliability with retry logic and dead letter queues
• Service mesh integration (Istio) for cross-service communication
• Advanced caching layer with L1/L2 cache strategy
• Comprehensive logging and distributed tracing (Jaeger/X-Ray)
• Multi-provider redundancy and failover support
• Enhanced monitoring with alerting thresholds and SLO tracking
• Automated chaos engineering experiments (GameDays)
• Backup and restore procedures for message history

Dependencies:

• Multi-region VPC configuration
• Service mesh setup and mTLS implementation
• Advanced observability stack deployment

Phase 3: Excellence & Optimization

Timeline: 16-24 weeks Focus: Production-grade reliability, cost optimization, and operational excellence

Key Deliverables:

• Advanced rate limiting with per-client and global quotas
• Feature flag system for progressive rollouts of new features
• Comprehensive audit logging and compliance tracking
• Optimized storage tiering strategy (hot/warm/cold)
• Cost optimization with reserved instances and spot pricing
• Advanced security measures (DDoS protection, WAF rules)
• Automated capacity planning and load forecasting
• Enhanced search capabilities for message history (Elasticsearch)
• Advanced analytics dashboard for delivery metrics and user behavior
• Zero-downtime deployment strategies with blue-green canary releases

Dependencies:

• Advanced security infrastructure (WAF, DDoS protection)
• Cost optimization tooling and FinOps dashboard
• Advanced analytics platform integration

Critical Path Dependencies:

1. Cloud Infrastructure - VPC, regions, and network configuration must be completed before any service deployment
2. Security Framework - Authentication, authorization, and TLS termination must be established before service integration
3. Database Design - Message schema and indexing strategy must be finalized before message processing services are built
4. Monitoring Stack - Observability tools (logging, tracing, metrics) must be operational before production deployment

Milestones:

• Week 12: MVP deployed to production with basic provider integration
• Week 20: Multi-region support and reliability improvements deployed
• Week 32: Production-grade system with cost optimization and advanced features operational

Risk Mitigation:

• Provider Dependency Risk: Implement provider-agnostic SDKs and maintain fallback mechanisms
• Rate Limiting Risk: Gradual rollout with feature flags to prevent overloading providers
• Compliance Risk: Early integration of audit logging and compliance tracking from MVP phase
• Scalability Risk: Design with horizontal scaling in mind from the beginning, avoiding single points of failure