Timeout Manager System Design

Comprehensive low-level design for a robust timeout management system handling distributed task expiration and cleanup.

PublishedJune 22, 2025
Length10,514 words
Reading time53 min read

Backend LLD

A timeout manager is essential for distributed systems to handle task expiration and resource cleanup. This design addresses concurrent timeout management, persistence concerns, and fault tolerance in high-throughput environments.

1. Problem Statement & Requirements

Clarifying questions

What is the scale of timeout operations expected? Will we be managing millions of concurrent timeouts, or is it more modest in scale (e.g., thousands)?
What consistency guarantees are required? Should timeouts be processed exactly-once, at-least-once, or is eventual consistency acceptable?
What types of timeouts are we managing? Are these application-level timeouts (e.g., job completion, session expiration) or system-level timeouts (e.g., network request timeouts)?
What is the read/write ratio for timeout data? Is it predominantly writes (timeouts being registered) or reads (checking status, querying expiration)?
What latency requirements exist? Are we optimizing for sub-100ms response times or can we tolerate longer delays (e.g., 5–10 seconds)?
What authentication and authorization model is in place? Will all timeout operations be authenticated via JWT, or will some use API keys or service-to-service tokens?
What external systems integrate with the timeout manager? Are there downstream systems that need to be notified of timeouts (e.g., job orchestrators, alerting systems)?

Functional requirements

Register a timeout with a unique identifier, expiration time, and associated metadata (e.g., job ID, callback URL).
Query timeout status by ID.
Cancel a timeout before it expires.
Retrieve all active timeouts for a given context (e.g., user ID, job group).
Notify external systems when a timeout expires.
Support for soft-deleting expired timeouts to allow for audit or recovery.

Non-functional requirements

Latency targets: p50 < 50ms, p95 < 200ms, p99 < 500ms for timeout registration and status queries.
Throughput: Support at least 10,000 timeout registrations/sec under load.
Availability: 99.9% uptime (99.99% for critical timeouts).
Scalability: Horizontally scalable to support multiple timeout manager instances.
Reliability: At-least-once delivery of timeout expiration events to consumers.

Constraints and assumptions

Assumption: The system will be deployed in a multi-region, multi-datacenter environment.
Constraint: Timeout manager must not block the calling service during registration or querying.
Assumption: External systems that receive timeout notifications are assumed to be resilient and idempotent.
Constraint: Timeout manager does not manage timeout persistence itself; it relies on an external database and messaging system.
Assumption: Time-based operations are assumed to be synchronized across all nodes (e.g., via NTP or a time sync service).

Out of scope

Timeout expiration logic implementation (e.g., polling or timer-based expiration in the application).
Timeout manager UI or dashboard.
Timeout history or audit log retention beyond the scope of the core timeout management features.
Integration with specific external systems (e.g., job orchestrators, alerting systems) beyond the generic event publishing.
Timeout migration or data rehydration features.

2. API Design

API Style Selection

REST is selected for this design due to its simplicity, wide adoption, and ease of integration with existing systems. It provides clear semantics for timeout management operations and supports standard HTTP methods for CRUD operations.

Endpoint Definitions

Method	Path	Description	Auth
GET	`/v1/timeouts`	List all timeouts with filtering and pagination	JWT
POST	`/v1/timeouts`	Create a new timeout	JWT
GET	`/v1/timeouts/{id}`	Retrieve a specific timeout by ID	JWT
PUT	`/v1/timeouts/{id}`	Update an existing timeout	JWT
DELETE	`/v1/timeouts/{id}`	Delete a timeout by ID	JWT
POST	`/v1/timeouts/{id}/extend`	Extend timeout duration	JWT
POST	`/v1/timeouts/{id}/cancel`	Cancel an active timeout	JWT
POST	`/v1/timeouts/{id}/trigger`	Manually trigger timeout	JWT

Request and Response Schemas

Create Timeout Request

{
  "resourceId": "string",
  "timeoutDurationMs": "integer",
  "callbackUrl": "string",
  "metadata": {
    "key": "value"
  }
}

Create Timeout Response

{
  "id": "uuid",
  "resourceId": "string",
  "timeoutDurationMs": "integer",
  "callbackUrl": "string",
  "status": "active|expired|cancelled",
  "createdAt": "timestamp",
  "expiresAt": "timestamp",
  "metadata": {
    "key": "value"
  }
}

Extend Timeout Request

{
  "extendDurationMs": "integer"
}

Extend Timeout Response

{
  "id": "uuid",
  "resourceId": "string",
  "timeoutDurationMs": "integer",
  "callbackUrl": "string",
  "status": "active|expired|cancelled",
  "createdAt": "timestamp",
  "expiresAt": "timestamp",
  "metadata": {
    "key": "value"
  }
}

HTTP Status Codes and Error Response Format

Standard HTTP status codes will be used:

200 OK - Successful GET, PUT, DELETE
201 Created - Successful POST
400 Bad Request - Invalid request payload or parameters
401 Unauthorized - Authentication required or invalid token
403 Forbidden - Insufficient permissions
404 Not Found - Resource not found
429 Too Many Requests - Rate limit exceeded
500 Internal Server Error - Server-side error

Standardized error response format:

{
  "error": {
    "code": "string",
    "message": "string",
    "details": "object"
  }
}

API Versioning Strategy

Versioned using path prefix (/v1/) to allow for future evolution without breaking existing clients. Versioning is explicit and follows semantic versioning principles.

Pagination Design

Cursor-based pagination is used for better performance and consistency when dealing with large datasets. It avoids issues with data changes during pagination.

Authentication and Authorization

JWT tokens for authentication
RBAC-based authorization where users must have appropriate permissions to manage timeouts
API keys for service-to-service communication

Rate Limiting

Rate limiting per endpoint:

Create/Update/Delete: 100 requests/minute
List/Get: 500 requests/minute
Extend/Cancel/Trigger: 200 requests/minute

Rate limiting enforced using Redis-based token bucket algorithm.

Idempotency Design

Mutation endpoints support idempotency using:

Idempotency-Key header
Idempotency key stored in Redis with TTL of 1 hour
Idempotent operations return same result for same key

3. Data Model Design

Entity-Relationship Diagram

erDiagram
    TIMEOUT {
        string id PK
        string resourceId
        integer timeoutDurationMs
        string callbackUrl
        string status
        timestamp createdAt
        timestamp expiresAt
        timestamp updatedAt
        json metadata
    }
    
    TIMEOUT_EVENT {
        string id PK
        string timeoutId FK
        string eventType
        timestamp occurredAt
        json payload
    }
    
    TIMEOUT ||--o{ TIMEOUT_EVENT : "has"

Table Schema Definitions

timeout_table

CREATE TABLE timeout (
    id UUID PRIMARY KEY,
    resource_id VARCHAR(255) NOT NULL,
    timeout_duration_ms INTEGER NOT NULL,
    callback_url TEXT NOT NULL,
    status VARCHAR(20) NOT NULL DEFAULT 'active',
    created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
    expires_at TIMESTAMP NOT NULL,
    updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
    metadata JSONB
);

CREATE INDEX idx_timeout_resource_id ON timeout(resource_id);
CREATE INDEX idx_timeout_expires_at ON timeout(expires_at);
CREATE INDEX idx_timeout_status ON timeout(status);
CREATE INDEX idx_timeout_created_at ON timeout(created_at);

timeout_event_table

CREATE TABLE timeout_event (
    id UUID PRIMARY KEY,
    timeout_id UUID NOT NULL REFERENCES timeout(id) ON DELETE CASCADE,
    event_type VARCHAR(50) NOT NULL,
    occurred_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
    payload JSONB
);

CREATE INDEX idx_timeout_event_timeout_id ON timeout_event(timeout_id);
CREATE INDEX idx_timeout_event_occurred_at ON timeout_event(occurred_at);

Primary Key Strategy

UUID is used as primary key to ensure global uniqueness, avoid sequential patterns for security, and support distributed system scenarios where data might be generated across multiple nodes.

Index Design

idx_timeout_resource_id: For fast lookups by resource ID (high read ratio)
idx_timeout_expires_at: For efficient timeout expiration scanning
idx_timeout_status: For filtering active vs. expired timeouts
idx_timeout_created_at: For time-based queries and audit trails

Data Partitioning Strategy

Partition by expires_at date ranges to support efficient querying of timeouts within specific time windows. This allows for optimized scanning and cleanup of expired timeouts.

Soft Delete vs. Hard Delete

Soft delete is implemented using a status field instead of physical deletion to maintain audit trails and allow for recovery if needed.

Audit Trail Design

All state changes are logged in timeout_event table with:

Event type (created, updated, expired, cancelled)
Timestamp of occurrence
Payload containing the change details

Data Archival Strategy

Expired timeouts are archived to a separate table with reduced retention (30 days) and eventually purged after 90 days to maintain system performance.

4. Class Design & Service Layer

UML Class Diagram

classDiagram
    class TimeoutManager {
        +createTimeout(TimeoutRequest) TimeoutResponse
        +extendTimeout(String, ExtendRequest) TimeoutResponse
        +cancelTimeout(String) TimeoutResponse
        +triggerTimeout(String) TimeoutResponse
        +getTimeout(String) TimeoutResponse
        +listTimeouts(TimeoutFilter) List~TimeoutResponse~
    }
    
    class TimeoutService {
        +createTimeout(TimeoutRequest) Timeout
        +extendTimeout(String, ExtendRequest) Timeout
        +cancelTimeout(String) Timeout
        +triggerTimeout(String) Timeout
        +getTimeout(String) Timeout
        +listTimeouts(TimeoutFilter) List~Timeout~
    }
    
    class TimeoutRepository {
        +save(Timeout) Timeout
        +findById(String) Optional~Timeout~
        +findByResourceId(String) List~Timeout~
        +findActiveByExpiresAtBefore(Timestamp) List~Timeout~
        +update(Timeout) Timeout
        +delete(String)
    }
    
    class TimeoutEventPublisher {
        +publish(TimeoutEvent) void
    }
    
    class TimeoutProcessor {
        +processTimeout(String) void
    }
    
    TimeoutManager --> TimeoutService : uses
    TimeoutService --> TimeoutRepository : uses
    TimeoutService --> TimeoutEventPublisher : publishes
    TimeoutProcessor --> TimeoutService : calls
    
    class Timeout {
        +id: String
        +resourceId: String
        +timeoutDurationMs: Integer
        +callbackUrl: String
        +status: TimeoutStatus
        +createdAt: Timestamp
        +expiresAt: Timestamp
        +metadata: Map~String, Object~
    }
    
    class TimeoutEvent {
        +id: String
        +timeoutId: String
        +eventType: String
        +occurredAt: Timestamp
        +payload: Map~String, Object~
    }
    
    enum TimeoutStatus {
        ACTIVE
        EXPIRED
        CANCELLED
    }

State Machine Diagram

stateDiagram-v2
    [*] --> ACTIVE
    ACTIVE --> EXPIRED : timeout occurs
    ACTIVE --> CANCELLED : cancel requested
    ACTIVE --> ACTIVE : extend requested
    EXPIRED --> [*]
    CANCELLED --> [*]

Service Class Structure

public interface TimeoutService {
    Timeout createTimeout(TimeoutRequest request);
    Timeout extendTimeout(String timeoutId, ExtendRequest request);
    Timeout cancelTimeout(String timeoutId);
    Timeout triggerTimeout(String timeoutId);
    Timeout getTimeout(String timeoutId);
    List<Timeout> listTimeouts(TimeoutFilter filter);
}

@Service
public class TimeoutServiceImpl implements TimeoutService {
    private final TimeoutRepository timeoutRepository;
    private final TimeoutEventPublisher eventPublisher;
    private final TimeoutProcessor timeoutProcessor;
    
    public Timeout createTimeout(TimeoutRequest request) {
        Timeout timeout = Timeout.builder()
            .id(UUID.randomUUID().toString())
            .resourceId(request.getResourceId())
            .timeoutDurationMs(request.getTimeoutDurationMs())
            .callbackUrl(request.getCallbackUrl())
            .status(TimeoutStatus.ACTIVE)
            .createdAt(Instant.now())
            .expiresAt(Instant.now().plusMillis(request.getTimeoutDurationMs()))
            .metadata(request.getMetadata())
            .build();
            
        return timeoutRepository.save(timeout);
    }
    
    public Timeout extendTimeout(String timeoutId, ExtendRequest request) {
        Timeout timeout = timeoutRepository.findById(timeoutId)
            .orElseThrow(() -> new TimeoutNotFoundException(timeoutId));
            
        if (timeout.getStatus() != TimeoutStatus.ACTIVE) {
            throw new InvalidTimeoutStateException(timeoutId, timeout.getStatus());
        }
        
        Instant newExpiry = timeout.getExpiresAt().plusMillis(request.getExtendDurationMs());
        timeout.setExpiresAt(newExpiry);
        
        return timeoutRepository.update(timeout);
    }
    
    public Timeout cancelTimeout(String timeoutId) {
        Timeout timeout = timeoutRepository.findById(timeoutId)
            .orElseThrow(() -> new TimeoutNotFoundException(timeoutId));
            
        if (timeout.getStatus() == TimeoutStatus.CANCELLED) {
            return timeout;
        }
        
        timeout.setStatus(TimeoutStatus.CANCELLED);
        return timeoutRepository.update(timeout);
    }
    
    public Timeout triggerTimeout(String timeoutId) {
        Timeout timeout = timeoutRepository.findById(timeoutId)
            .orElseThrow(() -> new TimeoutNotFoundException(timeoutId));
            
        if (timeout.getStatus() == TimeoutStatus.EXPIRED) {
            return timeout;
        }
        
        timeout.setStatus(TimeoutStatus.EXPIRED);
        timeout.setExpiresAt(Instant.now());
        
        return timeoutRepository.update(timeout);
    }
    
    public Timeout getTimeout(String timeoutId) {
        return timeoutRepository.findById(timeoutId)
            .orElseThrow(() -> new TimeoutNotFoundException(timeoutId));
    }
    
    public List<Timeout> listTimeouts(TimeoutFilter filter) {
        return timeoutRepository.findByResourceId(filter.getResourceId());
    }
}

Business Logic Encapsulation Pattern

Strategy pattern is used for timeout processing logic:

public interface TimeoutProcessor {
    void process(String timeoutId);
}

@Component
public class CallbackTimeoutProcessor implements TimeoutProcessor {
    private final RestTemplate restTemplate;
    
    @Override
    public void process(String timeoutId) {
        // Implementation for callback-based timeout processing
    }
}

@Component
public class EventTimeoutProcessor implements TimeoutProcessor {
    private final TimeoutEventPublisher eventPublisher;
    
    @Override
    public void process(String timeoutId) {
        // Implementation for event-based timeout processing
    }
}

Critical-Path Method Implementation

public Timeout createTimeout(TimeoutRequest request) {
    // Transaction boundary - all operations within single DB transaction
    return transactionTemplate.execute(status -> {
        try {
            // 1. Validate request
            validateRequest(request);
            
            // 2. Create timeout object with optimistic locking
            Timeout timeout = Timeout.builder()
                .id(UUID.randomUUID().toString())
                .resourceId(request.getResourceId())
                .timeoutDurationMs(request.getTimeoutDurationMs())
                .callbackUrl(request.getCallbackUrl())
                .status(TimeoutStatus.ACTIVE)
                .createdAt(Instant.now())
                .expiresAt(Instant.now().plusMillis(request.getTimeoutDurationMs()))
                .metadata(request.getMetadata())
                .build();
            
            // 3. Save to database
            Timeout savedTimeout = timeoutRepository.save(timeout);
            
            // 4. Publish event for async processing
            TimeoutEvent event = TimeoutEvent.builder()
                .id(UUID.randomUUID().toString())
                .timeoutId(savedTimeout.getId())
                .eventType("CREATED")
                .occurredAt(Instant.now())
                .payload(Map.of("timeout", savedTimeout))
                .build();
            
            eventPublisher.publish(event);
            
            // 5. Schedule timeout processing (if needed)
            // This could be done via message queue or in-memory scheduler
            
            return savedTimeout;
            
        } catch (Exception e) {
            // Rollback transaction on error
            status.setRollbackOnly();
            throw new TimeoutCreationException("Failed to create timeout", e);
        }
    });
}

Transaction Boundary Design

Single database transaction per operation ensures atomicity. For create operations, all data is persisted within one transaction. For updates, the same principle applies.

Service Interface Contracts

public interface TimeoutService {
    Timeout createTimeout(TimeoutRequest request);
    Timeout extendTimeout(String timeoutId, ExtendRequest request);
    Timeout cancelTimeout(String timeoutId);
    Timeout triggerTimeout(String timeoutId);
    Timeout getTimeout(String timeoutId);
    List<Timeout> listTimeouts(TimeoutFilter filter);
}

Saga / Orchestration Pattern

For distributed timeout processing, a saga pattern is used where each timeout operation can trigger external services through event-driven architecture.

Domain Object Design

@Builder
@Data
@AllArgsConstructor
@NoArgsConstructor
public class Timeout {
    private String id;
    private String resourceId;
    private Integer timeoutDurationMs;
    private String callbackUrl;
    private TimeoutStatus status;
    private Instant createdAt;
    private Instant expiresAt;
    private Instant updatedAt;
    private Map<String, Object> metadata;
    
    public boolean isExpired() {
        return status == TimeoutStatus.EXPIRED || 
               (expiresAt != null && Instant.now().isAfter(expiresAt));
    }
    
    public boolean isActive() {
        return status == TimeoutStatus.ACTIVE;
    }
}

5. Concurrency & Thread Safety

Concurrency Model

Java's thread model with synchronized methods and concurrent collections is used. For high-throughput scenarios, we use thread pools for async processing.

Race Condition Prevention

Race conditions are prevented through:

Database-level optimistic locking using version fields
Atomic operations where possible (e.g., incrementing counters)
Proper transaction isolation levels

Locking Strategy

Optimistic locking is used with version fields in the database to handle concurrent updates. Pessimistic locking is not used due to performance implications.

Deadlock Prevention

Deadlock prevention strategies include:

Consistent ordering of database access (by ID or timestamp)
Timeout configuration on database transactions
Using distributed locks for cross-service operations

Idempotency Implementation

Idempotency is implemented using:

Idempotency keys stored in Redis with TTL
Atomic operations to check and set keys
Return same result for same key within TTL window

6. Sequence Diagrams for Key Flows

Happy Path Sequence Diagram

sequenceDiagram
    participant Client
    participant TimeoutManager
    participant TimeoutService
    participant TimeoutRepository
    participant EventPublisher
    
    Client->>TimeoutManager: POST /v1/timeouts
    TimeoutManager->>TimeoutService: createTimeout()
    TimeoutService->>TimeoutRepository: save(timeout)
    TimeoutRepository-->>TimeoutService: timeout
    TimeoutService->>EventPublisher: publish(event)
    EventPublisher-->>TimeoutService: success
    TimeoutService-->>TimeoutManager: timeoutResponse
    TimeoutManager-->>Client: 201 Created

Error / Edge Case Sequence Diagram

sequenceDiagram
    participant Client
    participant TimeoutManager
    participant TimeoutService
    participant TimeoutRepository
    
    Client->>TimeoutManager: POST /v1/timeouts
    TimeoutManager->>TimeoutService: createTimeout()
    TimeoutService->>TimeoutRepository: save(timeout)
    TimeoutRepository-->>TimeoutService: DatabaseException
    TimeoutService-->>TimeoutManager: TimeoutCreationException
    TimeoutManager-->>Client: 500 Internal Server Error

Async Flow Sequence Diagram

sequenceDiagram
    participant Client
    participant TimeoutManager
    participant TimeoutService
    participant EventPublisher
    participant AsyncProcessor
    
    Client->>TimeoutManager: POST /v1/timeouts
    TimeoutManager->>TimeoutService: createTimeout()
    TimeoutService->>EventPublisher: publish(event)
    EventPublisher->>AsyncProcessor: timeout event
    AsyncProcessor->>ExternalService: callback request
    ExternalService-->>AsyncProcessor: response
    AsyncProcessor->>TimeoutService: update status

7. Caching Design

Cache Strategy

Cache-aside pattern is used for better control over cache lifecycle and consistency.

Cache Key Design

Cache keys follow the pattern: timeout:{id} for direct lookups and timeout:resource:{resourceId} for resource-based queries.

TTL Per Data Type

Data Type	TTL	Reason
Timeout by ID	1 hour	Frequent access, short-lived
Timeout by Resource	30 minutes	Less frequent, but still relevant
Timeout list	5 minutes	For listing operations

Cache Eviction Policy

LRU eviction policy with TTL-based expiration to maintain cache freshness.

Cache Stampede Prevention

Probabilistic early expiry (10% chance of expiring 5 minutes early) and distributed locks to prevent thundering herd.

Distributed Cache Architecture

Redis cluster with replication and automatic failover for high availability.

Cache Invalidation Strategy

Event-driven invalidation where timeout events trigger cache removals to maintain consistency.

8. Repository / Data Access Layer

Repository Pattern Implementation

public interface TimeoutRepository {
    Timeout save(Timeout timeout);
    Optional<Timeout> findById(String id);
    List<Timeout> findByResourceId(String resourceId);
    List<Timeout> findActiveByExpiresAtBefore(Instant threshold);
    Timeout update(Timeout timeout);
    void delete(String id);
}

ORM vs. Raw Query Decision

Raw SQL queries are used for better performance and control over database operations, especially for bulk operations and complex filtering.

Read vs. Write Separation

Reads and writes are separated using read replicas for better performance and scalability.

Connection Pool Configuration

Max pool size: 20
Min idle connections: 5
Connection timeout: 30 seconds
Validation query: SELECT 1

N+1 Query Prevention

Eager loading is used for related entities, and batch operations are implemented to reduce database round trips.

Batch Operation Design

Batch operations for bulk timeout creation and expiration processing using batch inserts and updates.

9. Asynchronous Processing & Messaging

Event-Driven Boundaries

Timeout creation and updates are handled synchronously, while timeout expiration triggers asynchronous callback processing.

Message Schema Design

{
  "version": "1.0",
  "type": "timeout.expired",
  "timestamp": "2023-01-01T00:00:00Z",
  "payload": {
    "timeoutId": "uuid",
    "resourceId": "string",
    "callbackUrl": "string",
    "metadata": {}
  }
}

Dead Letter Queue Strategy

Messages that fail processing after retries are sent to a dead letter queue for manual inspection and reprocessing.

Delivery Guarantee

At-least-once delivery is used to ensure timeout processing occurs, with idempotency handling for duplicate messages.

Retry Policy

Max attempts: 3
Backoff formula: exponential with jitter (1s, 2s, 4s)
Non-retryable errors: validation failures, permanent service errors

Outbox Pattern

Outbox pattern is implemented to ensure reliable event publishing by storing events in the same transaction as database writes.

Async Flow Sequence Diagram

sequenceDiagram
    participant TimeoutManager
    participant TimeoutService
    participant EventPublisher
    participant MessageQueue
    participant AsyncProcessor
    
    TimeoutManager->>TimeoutService: createTimeout()
    TimeoutService->>EventPublisher: publish(event)
    EventPublisher->>MessageQueue: send(timeout.expired)
    MessageQueue-->>AsyncProcessor: timeout.expired event
    AsyncProcessor->>ExternalService: callback request
    ExternalService-->>AsyncProcessor: response
    AsyncProcessor->>TimeoutService: update status

10. Error Handling Design

Error Classification Hierarchy

Domain errors (TimeoutNotFoundException, InvalidTimeoutStateException)
Infrastructure errors (DatabaseException, NetworkException)
Validation errors (InvalidRequestException)

Error Propagation Strategy

Errors are propagated up the call stack to appropriate layers for handling and response generation.

Retry Strategy

Exponential backoff with jitter:

Delay: 1s, 2s, 4s (with ±20% jitter)
Max retries: 3
Retryable errors: Network timeouts, database connection issues

Circuit Breaker Design

Circuit breaker pattern is implemented for external service calls:

Failure threshold: 5 consecutive failures
Timeout: 30 seconds
Half-open state: After timeout, allow one test request

Fallback Behavior

For external service failures, fallback to local processing or return cached results where appropriate.

Error Logging Format

Structured logging with:

trace_id: UUID for distributed tracing
span_id: Current operation identifier
error_code: Standardized error code
user_id: User context if available
entity_id: Timeout identifier

11. Security Design

Input Validation and Sanitization

Input validation using Bean Validation annotations with custom validators for timeout durations and URLs.

SQL Injection Prevention

Prepared statements and parameterized queries are used exclusively to prevent SQL injection attacks.

Authentication Middleware

JWT-based authentication middleware that validates tokens and extracts user information for authorization checks.

Authorization Enforcement

Role-based access control where users must have appropriate permissions to manage timeouts. Permissions are checked at service layer.

Data Encryption

Sensitive data is encrypted at rest using AES-256 encryption and in transit using TLS 1.3.

PII Handling

PII data is masked in logs and audit trails, with only necessary fields retained for compliance.

Secrets Management

AWS Secrets Manager is used to store sensitive configuration values like callback URLs and API keys.

Audit Logging Design

All timeout operations are logged with:

User who performed the action
Timestamp of operation
Operation type (create, update, delete)
Resource identifier and metadata

12. Testing Strategy

Unit Testing Scope

90% code coverage for service layer methods, including edge cases and error conditions.

Integration Testing Strategy

Testcontainers are used to spin up real database instances for integration testing. Embedded Redis is used for caching tests.

Contract Testing

Pact framework is used for contract testing between the timeout manager and external services.

Performance Testing Tool

k6 is used for load testing with key scenarios:

Concurrent timeout creation
Bulk timeout expiration processing
High-frequency timeout extension operations

Test Data Management Strategy

Factory pattern is used to generate test data, with fixtures for common scenarios and seeding scripts for initial test data.

13. Observability Design

Structured Logging Schema

Log entries include:

trace_id: UUID for distributed tracing
span_id: Current operation identifier
user_id: User context if available
entity_id: Timeout identifier
error_code: Standardized error code (if applicable)
PII masking rules applied to sensitive fields

Distributed Tracing Instrumentation

OpenTelemetry is used with custom spans:

Span for timeout creation operation
Span for timeout expiration processing
Span for external callback requests

Metrics Design

RED metrics for key operations:

Rate: Number of timeout operations per second
Errors: Error rates for timeout operations
Duration: Processing time for timeout operations

Health Check Endpoint Design

/health: Basic health check
/ready: Readiness check for database connections
/live: Liveness check for core services

Alerting Thresholds

Warning: 5% error rate on timeout operations
Critical: 10% error rate or >2s average processing time
Escalation path: On-call engineer for critical alerts

14. Open Questions, Risks & Trade-offs

Decisions Requiring Product Alignment

Whether to support timeout cancellation for expired timeouts
Whether to implement timeout pre-expiration notifications
Whether to support timeout groups for batch operations

Known Trade-offs and Justifications

Performance vs. Consistency: Using cache-aside pattern trades some consistency for performance gains
Simplicity vs. Feature Richness: Simple timeout management to start with, extensible for future features
Security vs. Usability: Strong authentication but potentially complex setup for some users

Risks and Mitigation Strategies

Timeout Storm Risk: Implement rate limiting and circuit breakers to prevent cascading failures
Data Loss Risk: Use proper transaction handling and backup strategies for critical timeout data
External Service Failure: Implement circuit breaker patterns and fallback behaviors

Technical Debt Acknowledged

Limited Timeout Types: Only supports callback-based timeouts, not event-driven or webhook-based
Simple Metadata Handling: Basic JSON metadata support without rich querying capabilities
Basic Audit Trail: Simple event logging without full audit trail with detailed change tracking

3. Data Model Design

Entity-Relationship Diagram

erDiagram
    TIMEOUT_MANAGER {
        string id PK
        string name
        string description
        int timeout_duration_ms
        string status
        string owner_id
        timestamp created_at
        timestamp updated_at
        timestamp expires_at
        boolean is_active
    }

    TIMEOUT_EVENT {
        string id PK
        string timeout_id FK
        string event_type
        timestamp occurred_at
        string payload
        boolean is_processed
    }

    TIMEOUT_CONFIG {
        string id PK
        string timeout_manager_id FK
        string config_key
        string config_value
        timestamp created_at
    }

    TIMEOUT_MANAGER ||--o{ TIMEOUT_EVENT : "has"
    TIMEOUT_MANAGER ||--o{ TIMEOUT_CONFIG : "has"

Table Schema Definitions

timeout_managers table

CREATE TABLE timeout_managers (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    name VARCHAR(255) NOT NULL,
    description TEXT,
    timeout_duration_ms INTEGER NOT NULL CHECK (timeout_duration_ms > 0),
    status VARCHAR(50) NOT NULL DEFAULT 'ACTIVE',
    owner_id VARCHAR(255) NOT NULL,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
    expires_at TIMESTAMP WITH TIME ZONE NOT NULL,
    is_active BOOLEAN DEFAULT TRUE,
    
    -- Indexes
    INDEX idx_timeout_managers_owner (owner_id),
    INDEX idx_timeout_managers_status (status),
    INDEX idx_timeout_managers_expires_at (expires_at),
    INDEX idx_timeout_managers_active_expires (is_active, expires_at)
);

timeout_events table

CREATE TABLE timeout_events (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    timeout_id UUID NOT NULL REFERENCES timeout_managers(id) ON DELETE CASCADE,
    event_type VARCHAR(100) NOT NULL,
    occurred_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
    payload TEXT,
    is_processed BOOLEAN DEFAULT FALSE,
    
    -- Indexes
    INDEX idx_timeout_events_timeout_id (timeout_id),
    INDEX idx_timeout_events_occurred_at (occurred_at),
    INDEX idx_timeout_events_processed (is_processed),
    INDEX idx_timeout_events_type_time (event_type, occurred_at)
);

timeout_configs table

CREATE TABLE timeout_configs (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    timeout_manager_id UUID NOT NULL REFERENCES timeout_managers(id) ON DELETE CASCADE,
    config_key VARCHAR(255) NOT NULL,
    config_value TEXT NOT NULL,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
    
    -- Indexes
    INDEX idx_timeout_configs_manager_key (timeout_manager_id, config_key),
    UNIQUE (timeout_manager_id, config_key)
);

Primary Key Strategy

UUID is selected as the primary key for all tables because:

Provides globally unique identifiers without coordination
Enables horizontal sharding based on UUID ranges or hash
Supports distributed system deployment without ID collision concerns
Allows for better audit trail tracking and debugging

Index Design

Index Name	Fields	Type	Purpose
idx_timeout_managers_owner	owner_id	Standard	Fast lookup by owner
idx_timeout_managers_status	status	Standard	Filter active/inactive timeouts
idx_timeout_managers_expires_at	expires_at	Standard	Efficient timeout expiration scanning
idx_timeout_managers_active_expires	is_active, expires_at	Composite	Fast query for active timeouts before expiration
idx_timeout_events_timeout_id	timeout_id	Standard	Fast lookup of events for specific timeout
idx_timeout_events_occurred_at	occurred_at	Standard	Efficient event timeline queries
idx_timeout_events_processed	is_processed	Standard	Filter unprocessed events
idx_timeout_events_type_time	event_type, occurred_at	Composite	Fast query by event type and time
idx_timeout_configs_manager_key	timeout_manager_id, config_key	Composite	Fast config lookup per manager

Data Partitioning / Sharding Strategy

Time-based sharding by expiration date:

Partition by expires_at field using monthly ranges
Shards can be distributed across different database instances or clusters
Allows for efficient cleanup of expired timeouts through partition pruning

Soft Delete vs. Hard Delete Decision

Soft delete is selected because:

Audit trail requirement to track timeout lifecycle
Potential for reactivation of timeouts in future
Historical event data is valuable for analytics and debugging
Compliance requirements may mandate data retention

Audit Trail Design

All changes to timeout_managers are audited with:

created_at and updated_at timestamps
owner_id to track who created the timeout
status field changes can be tracked via event log or trigger-based audit table

Data Archival Strategy and Retention Policy

Archival policy:

Expired timeouts are archived to a separate table or storage system (e.g., S3 with Parquet format)
Archive retention: 90 days for compliance and debugging
Active timeouts are retained indefinitely in the primary database

-- Example archive table structure for expired timeouts
CREATE TABLE timeout_managers_archive (
    id UUID PRIMARY KEY,
    name VARCHAR(255),
    description TEXT,
    timeout_duration_ms INTEGER,
    status VARCHAR(50),
    owner_id VARCHAR(255),
    created_at TIMESTAMP WITH TIME ZONE,
    updated_at TIMESTAMP WITH TIME ZONE,
    expires_at TIMESTAMP WITH TIME ZONE,
    is_active BOOLEAN,
    archived_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
);

4. Class Design & Service Layer

classDiagram
    class TimeoutManagerService {
        +createTimeout(TimeoutRequest) TimeoutResponse
        +extendTimeout(String, Duration) TimeoutResponse
        +cancelTimeout(String) TimeoutResponse
        +getTimeout(String) TimeoutResponse
        +processTimeouts() void
    }
    
    class Timeout {
        -id: String
        -entityId: String
        -timeoutAt: Instant
        -status: TimeoutStatus
        -createdAt: Instant
        -updatedAt: Instant
        +isExpired() boolean
    }
    
    class TimeoutRepository {
        +save(Timeout) void
        +findById(String) Optional~Timeout~
        +findByEntityId(String) List~Timeout~
        +findExpired(Instant) List~Timeout~
        +delete(String) void
    }
    
    class TimeoutEventPublisher {
        +publish(TimeoutEvent) void
    }
    
    class TimeoutProcessor {
        +processExpiredTimeouts() void
    }
    
    class TimeoutStatus {
        PENDING
        ACTIVE
        EXPIRED
        CANCELLED
    }
    
    class TimeoutRequest {
        entityId: String
        duration: Duration
        callbackUrl: String
        metadata: Map~String, String~
    }
    
    class TimeoutResponse {
        timeoutId: String
        entityId: String
        status: TimeoutStatus
        expiresAt: Instant
    }
    
    class TimeoutEvent {
        timeoutId: String
        entityId: String
        eventType: EventType
        timestamp: Instant
    }
    
    class TimeoutScheduler {
        +scheduleTimeout(String, Instant) void
        +cancelScheduledTimeout(String) void
    }
    
    TimeoutManagerService --> Timeout : manages
    TimeoutManagerService --> TimeoutRepository : persists
    TimeoutManagerService --> TimeoutEventPublisher : publishes
    TimeoutManagerService --> TimeoutProcessor : processes
    TimeoutManagerService --> TimeoutScheduler : schedules
    
    Timeout --> TimeoutStatus : has
    TimeoutEventPublisher --> TimeoutEvent : publishes
    
    class TimeoutStrategy {
        +apply(Timeout) void
    }
    
    class DefaultTimeoutStrategy {
        +apply(Timeout) void
    }
    
    TimeoutStrategy <|-- DefaultTimeoutStrategy : implements

stateDiagram-v2
    [*] --> PENDING
    PENDING --> ACTIVE : timer starts
    ACTIVE --> EXPIRED : timeout reached
    ACTIVE --> CANCELLED : cancel requested
    PENDING --> CANCELLED : cancel requested
    EXPIRED --> [*]
    CANCELLED --> [*]

Service Layer Overview

The Timeout Manager service is designed to handle timeout operations for entities with configurable expiration times. The service provides core operations for creating, extending, and canceling timeouts, while maintaining a robust state management system.

Service Class Structure

public interface TimeoutManagerService {
    TimeoutResponse createTimeout(TimeoutRequest request);
    TimeoutResponse extendTimeout(String timeoutId, Duration extension);
    TimeoutResponse cancelTimeout(String timeoutId);
    TimeoutResponse getTimeout(String timeoutId);
    void processTimeouts();
}

public class DefaultTimeoutManagerService implements TimeoutManagerService {
    private final TimeoutRepository timeoutRepository;
    private final TimeoutEventPublisher eventPublisher;
    private final TimeoutProcessor timeoutProcessor;
    private final TimeoutScheduler timeoutScheduler;
    private final TimeoutStrategy timeoutStrategy;
    
    public DefaultTimeoutManagerService(
            TimeoutRepository timeoutRepository,
            TimeoutEventPublisher eventPublisher,
            TimeoutProcessor timeoutProcessor,
            TimeoutScheduler timeoutScheduler,
            TimeoutStrategy timeoutStrategy) {
        this.timeoutRepository = timeoutRepository;
        this.eventPublisher = eventPublisher;
        this.timeoutProcessor = timeoutProcessor;
        this.timeoutScheduler = timeoutScheduler;
        this.timeoutStrategy = timeoutStrategy;
    }
    
    @Override
    @Transactional
    public TimeoutResponse createTimeout(TimeoutRequest request) {
        String timeoutId = UUID.randomUUID().toString();
        Instant now = Instant.now();
        Instant expiresAt = now.plus(request.getDuration());
        
        Timeout timeout = new Timeout()
            .setId(timeoutId)
            .setEntityId(request.getEntityId())
            .setTimeoutAt(expiresAt)
            .setStatus(TimeoutStatus.PENDING)
            .setCreatedAt(now)
            .setUpdatedAt(now);
        
        timeoutRepository.save(timeout);
        eventPublisher.publish(new TimeoutEvent()
            .setEventType(EventType.CREATED)
            .setTimeoutId(timeoutId)
            .setEntityId(request.getEntityId())
            .setTimestamp(now));
        
        timeoutScheduler.scheduleTimeout(timeoutId, expiresAt);
        
        return new TimeoutResponse()
            .setTimeoutId(timeoutId)
            .setEntityId(request.getEntityId())
            .setStatus(TimeoutStatus.PENDING)
            .setExpiresAt(expiresAt);
    }
    
    @Override
    @Transactional
    public TimeoutResponse extendTimeout(String timeoutId, Duration extension) {
        Timeout timeout = timeoutRepository.findById(timeoutId)
            .orElseThrow(() -> new TimeoutNotFoundException(timeoutId));
        
        if (timeout.getStatus() != TimeoutStatus.ACTIVE) {
            throw new InvalidTimeoutStateException(timeoutId, timeout.getStatus());
        }
        
        Instant newExpiry = timeout.getTimeoutAt().plus(extension);
        Instant now = Instant.now();
        
        timeout.setTimeoutAt(newExpiry)
            .setUpdatedAt(now);
            
        timeoutRepository.save(timeout);
        
        eventPublisher.publish(new TimeoutEvent()
            .setEventType(EventType.EXTENDED)
            .setTimeoutId(timeoutId)
            .setEntityId(timeout.getEntityId())
            .setTimestamp(now));
            
        timeoutScheduler.scheduleTimeout(timeoutId, newExpiry);
        
        return new TimeoutResponse()
            .setTimeoutId(timeoutId)
            .setEntityId(timeout.getEntityId())
            .setStatus(TimeoutStatus.ACTIVE)
            .setExpiresAt(newExpiry);
    }
    
    @Override
    @Transactional
    public TimeoutResponse cancelTimeout(String timeoutId) {
        Timeout timeout = timeoutRepository.findById(timeoutId)
            .orElseThrow(() -> new TimeoutNotFoundException(timeoutId));
            
        if (timeout.getStatus() == TimeoutStatus.CANCELLED) {
            return new TimeoutResponse()
                .setTimeoutId(timeoutId)
                .setEntityId(timeout.getEntityId())
                .setStatus(TimeoutStatus.CANCELLED);
        }
        
        Instant now = Instant.now();
        timeout.setStatus(TimeoutStatus.CANCELLED)
            .setUpdatedAt(now);
            
        timeoutRepository.save(timeout);
        
        eventPublisher.publish(new TimeoutEvent()
            .setEventType(EventType.CANCELLED)
            .setTimeoutId(timeoutId)
            .setEntityId(timeout.getEntityId())
            .setTimestamp(now));
            
        timeoutScheduler.cancelScheduledTimeout(timeoutId);
        
        return new TimeoutResponse()
            .setTimeoutId(timeoutId)
            .setEntityId(timeout.getEntityId())
            .setStatus(TimeoutStatus.CANCELLED);
    }
    
    @Override
    public TimeoutResponse getTimeout(String timeoutId) {
        return timeoutRepository.findById(timeoutId)
            .map(this::convertToResponse)
            .orElseThrow(() -> new TimeoutNotFoundException(timeoutId));
    }
    
    @Override
    public void processTimeouts() {
        Instant now = Instant.now();
        List<Timeout> expiredTimeouts = timeoutRepository.findExpired(now);
        
        for (Timeout timeout : expiredTimeouts) {
            if (timeout.getStatus() == TimeoutStatus.ACTIVE) {
                processExpiredTimeout(timeout);
            }
        }
    }
    
    private void processExpiredTimeout(Timeout timeout) {
        Instant now = Instant.now();
        
        timeout.setStatus(TimeoutStatus.EXPIRED)
            .setUpdatedAt(now);
            
        timeoutRepository.save(timeout);
        
        eventPublisher.publish(new TimeoutEvent()
            .setEventType(EventType.EXPIRED)
            .setTimeoutId(timeout.getId())
            .setEntityId(timeout.getEntityId())
            .setTimestamp(now));
            
        timeoutStrategy.apply(timeout);
    }
    
    private TimeoutResponse convertToResponse(Timeout timeout) {
        return new TimeoutResponse()
            .setTimeoutId(timeout.getId())
            .setEntityId(timeout.getEntityId())
            .setStatus(timeout.getStatus())
            .setExpiresAt(timeout.getTimeoutAt());
    }
}

Business Logic Patterns

Strategy Pattern Implementation

public interface TimeoutStrategy {
    void apply(Timeout timeout);
}

@Component
public class DefaultTimeoutStrategy implements TimeoutStrategy {
    private final WebhookService webhookService;
    
    public DefaultTimeoutStrategy(WebhookService webhookService) {
        this.webhookService = webhookService;
    }
    
    @Override
    public void apply(Timeout timeout) {
        // Send webhook notification to callback URL
        webhookService.sendCallback(
            timeout.getEntityId(),
            timeout.getCallbackUrl(),
            "timeout_expired"
        );
    }
}

Critical-Path Method Implementation

@Transactional
public TimeoutResponse createTimeout(TimeoutRequest request) {
    String timeoutId = UUID.randomUUID().toString();
    Instant now = Instant.now();
    Instant expiresAt = now.plus(request.getDuration());
    
    // Validate request
    if (request.getDuration().isNegative() || request.getDuration().isZero()) {
        throw new InvalidTimeoutDurationException("Duration must be positive");
    }
    
    // Check for existing active timeout on same entity
    List<Timeout> existingTimeouts = timeoutRepository.findByEntityId(request.getEntityId());
    Optional<Timeout> activeTimeout = existingTimeouts.stream()
        .filter(t -> t.getStatus() == TimeoutStatus.ACTIVE)
        .findFirst();
    
    if (activeTimeout.isPresent()) {
        throw new DuplicateTimeoutException(
            "Active timeout already exists for entity: " + request.getEntityId());
    }
    
    // Create new timeout
    Timeout timeout = new Timeout()
        .setId(timeoutId)
        .setEntityId(request.getEntityId())
        .setTimeoutAt(expiresAt)
        .setStatus(TimeoutStatus.PENDING)
        .setCreatedAt(now)
        .setUpdatedAt(now)
        .setCallbackUrl(request.getCallbackUrl())
        .setMetadata(request.getMetadata());
    
    // Save to database
    timeoutRepository.save(timeout);
    
    // Publish event for monitoring and audit trail
    eventPublisher.publish(new TimeoutEvent()
        .setEventType(EventType.CREATED)
        .setTimeoutId(timeoutId)
        .setEntityId(request.getEntityId())
        .setTimestamp(now));
    
    // Schedule timeout processing
    timeoutScheduler.scheduleTimeout(timeoutId, expiresAt);
    
    // Return response
    return new TimeoutResponse()
        .setTimeoutId(timeoutId)
        .setEntityId(request.getEntityId())
        .setStatus(TimeoutStatus.PENDING)
        .setExpiresAt(expiresAt);
}

Transaction Boundary Design

The service layer ensures transaction boundaries around all state-changing operations:

Create Timeout: Single transaction for database save and event publishing
Extend Timeout: Transactional update with event logging
Cancel Timeout: Atomic state change and event publishing
Process Timeouts: Batch processing with individual transaction per timeout

Service Interface Contracts

public interface TimeoutRepository {
    void save(Timeout timeout);
    Optional<Timeout> findById(String id);
    List<Timeout> findByEntityId(String entityId);
    List<Timeout> findExpired(Instant now);
    void delete(String id);
}

public interface TimeoutEventPublisher {
    void publish(TimeoutEvent event);
}

public interface TimeoutProcessor {
    void processExpiredTimeouts();
}

public interface TimeoutScheduler {
    void scheduleTimeout(String timeoutId, Instant expiresAt);
    void cancelScheduledTimeout(String timeoutId);
}

Saga / Orchestration Pattern

For distributed operations involving external systems:

public class TimeoutSaga {
    private final TimeoutManagerService timeoutService;
    private final ExternalSystemClient externalClient;
    
    public void handleTimeoutWithExternalSystem(String timeoutId) {
        // Start saga
        Timeout timeout = timeoutService.getTimeout(timeoutId);
        
        try {
            // Process timeout with external system
            externalClient.notifyExternalSystem(
                timeout.getEntityId(), 
                "timeout_expired"
            );
            
            // Update status after successful external operation
            timeoutService.cancelTimeout(timeoutId);
        } catch (Exception e) {
            // Handle failure - potentially retry or escalate
            logger.error("Failed to process timeout with external system", e);
            throw new TimeoutProcessingException(
                "External system failure for timeout: " + timeoutId, e);
        }
    }
}

Domain Object Design

public class Timeout {
    private String id;
    private String entityId;
    private Instant timeoutAt;
    private TimeoutStatus status;
    private String callbackUrl;
    private Map<String, String> metadata;
    private Instant createdAt;
    private Instant updatedAt;
    
    // Getters and setters
    public boolean isExpired() {
        return status == TimeoutStatus.EXPIRED || 
               (timeoutAt != null && Instant.now().isAfter(timeoutAt));
    }
    
    public boolean isActive() {
        return status == TimeoutStatus.ACTIVE;
    }
}

public enum TimeoutStatus {
    PENDING,
    ACTIVE,
    EXPIRED,
    CANCELLED
}

public enum EventType {
    CREATED,
    EXTENDED,
    EXPIRED,
    CANCELLED
}

5. Concurrency & Thread Safety

Concurrency Model

The Timeout Manager will operate in a multi-threaded environment with the following concurrency model:

Thread Pool: Core operations will be handled by a fixed-size thread pool (e.g., 16-32 threads) using java.util.concurrent.ExecutorService for managing background tasks like timeout monitoring and cleanup.
Async/Await Pattern: For I/O-bound operations, we'll leverage CompletableFuture with appropriate ForkJoinPool.commonPool() or custom thread pools to handle asynchronous tasks without blocking the main processing threads.
Lock-Free Data Structures: Where possible, we'll use lock-free concurrent collections like ConcurrentHashMap, CopyOnWriteArrayList, and atomic reference types from java.util.concurrent.atomic to minimize contention.

Race Condition Prevention Strategy

Race conditions in the Timeout Manager can occur during:

Timeout Registration/Update: Multiple threads may attempt to register or update timeouts simultaneously.
Timeout Expiration Processing: Concurrent access to timeout state when expiration occurs.
Cleanup Operations: Simultaneous cleanup of expired timeouts.

Prevention Mechanisms:

Use atomic operations for counters and flags.
Implement read-write locks (ReentrantReadWriteLock) where appropriate to allow concurrent reads while ensuring exclusive access for writes.
Immutable state objects where possible to prevent modification during concurrent operations.

Optimistic vs. Pessimistic Locking Selection

Optimistic Locking is preferred for the Timeout Manager due to:

High Concurrency: The system handles many concurrent timeout registrations and updates.
Low Contention: Most operations are read-heavy or independent, reducing likelihood of conflicts.
Performance: Avoids blocking threads and reduces overhead.

Implementation Details:

Use AtomicLong or AtomicReference for version tracking on timeout entities.
Implement compare-and-swap (CAS) logic in update operations to ensure consistency without blocking.

Deadlock Prevention Strategy

Deadlock prevention is crucial for maintaining system liveness:

Lock Ordering: All threads acquire locks in a consistent order (e.g., by timeout ID or timestamp).
Timeout on Lock Acquisition: Use tryLock() with timeout to prevent indefinite waiting.
Avoid Nested Locks: Minimize nested lock acquisition where possible.
Thread Pool Configuration: Ensure thread pool size is bounded to prevent resource exhaustion.

Idempotency Implementation for Concurrent Duplicate Requests

To handle concurrent duplicate requests, we implement an idempotency key strategy:

public class IdempotencyManager {
    private final Map<String, CompletableFuture<Void>> activeRequests = new ConcurrentHashMap<>();
    
    public CompletableFuture<Void> executeWithIdempotency(String idempotencyKey, Supplier<CompletableFuture<Void>> operation) {
        return activeRequests.computeIfAbsent(idempotencyKey, k -> {
            CompletableFuture<Void> future = operation.get();
            future.whenComplete((result, ex) -> activeRequests.remove(k));
            return future;
        });
    }
}

This ensures that if multiple threads attempt to process the same operation with the same key, only one will execute and others will wait for completion. The activeRequests map acts as a cache of ongoing operations, and the CompletableFuture ensures proper cleanup after completion.

Thread Safety in Core Components

TimeoutManager: Uses ConcurrentHashMap for storing active timeouts and ReentrantReadWriteLock for protecting critical sections during registration and expiration.
TimeoutStore: Thread-safe implementation using CopyOnWriteArrayList for read-heavy operations and atomic updates.
EventPublisher: Uses ExecutorService with bounded thread pool to publish events asynchronously without blocking the main processing flow.

Synchronization Primitives Used

Component	Primitive	Reason
Timeout Map	`ConcurrentHashMap`	High concurrency, low contention
State Updates	`AtomicReference`	Compare-and-swap for versioning
Read-Write Access	`ReentrantReadWriteLock`	Allows concurrent reads, exclusive writes
Async Execution	`CompletableFuture`	Non-blocking async operations
Cleanup Tasks	`ScheduledExecutorService`	Periodic cleanup without blocking main threads

Example: Thread-Safe Timeout Registration

public class TimeoutManager {
    private final Map<String, Timeout> activeTimeouts = new ConcurrentHashMap<>();
    private final ReadWriteLock lock = new ReentrantReadWriteLock();
    
    public void registerTimeout(String id, long durationMs) {
        Timeout timeout = new Timeout(id, System.currentTimeMillis() + durationMs);
        
        // Optimistic update with CAS
        activeTimeouts.put(id, timeout);
        
        // Schedule expiration task
        ScheduledFuture<?> future = scheduler.schedule(
            () -> processTimeout(id), 
            durationMs, TimeUnit.MILLISECONDS);
            
        // Store future for potential cancellation
        timeout.setFuture(future);
    }
    
    private void processTimeout(String id) {
        Timeout timeout = activeTimeouts.get(id);
        if (timeout != null && !timeout.isProcessed()) {
            lock.writeLock().lock();
            try {
                if (activeTimeouts.remove(id) != null) {
                    timeout.markAsProcessed();
                    eventPublisher.publish(new TimeoutExpiredEvent(id));
                }
            } finally {
                lock.writeLock().unlock();
            }
        }
    }
}

6. Sequence Diagrams for Key Flows

sequenceDiagram
    participant Client
    participant TimeoutManager
    participant Storage
    participant Scheduler
    participant EventPublisher

    Client->>TimeoutManager: CreateTimeout(timeoutMs, callbackUrl, payload)
    TimeoutManager->>Storage: Store timeout metadata
    Storage-->>TimeoutManager: Acknowledge storage
    TimeoutManager->>Scheduler: Schedule timeout task
    Scheduler-->>TimeoutManager: Acknowledge scheduling
    TimeoutManager-->>Client: Return timeoutId
    
    Note over Scheduler,Storage: Background task executes after timeout duration
    
    Scheduler->>Storage: Fetch timeout by ID
    Storage-->>Scheduler: Return timeout metadata
    Scheduler->>EventPublisher: Publish timeout event
    EventPublisher-->>Scheduler: Acknowledge event publish
    Scheduler->>Storage: Update timeout status to expired
    Storage-->>Scheduler: Acknowledge update

sequenceDiagram
    participant Client
    participant TimeoutManager
    participant Storage
    participant EventPublisher

    Client->>TimeoutManager: CancelTimeout(timeoutId)
    TimeoutManager->>Storage: Fetch timeout by ID
    Storage-->>TimeoutManager: Return timeout metadata
    TimeoutManager->>Storage: Update status to cancelled
    Storage-->>TimeoutManager: Acknowledge update
    TimeoutManager->>EventPublisher: Publish cancellation event
    EventPublisher-->>TimeoutManager: Acknowledge event publish
    TimeoutManager-->>Client: Return success status

sequenceDiagram
    participant Scheduler
    participant TimeoutManager
    participant Storage
    participant ExternalService
    participant EventPublisher

    Scheduler->>Storage: Fetch expired timeouts
    Storage-->>Scheduler: Return list of expired timeouts
    loop Process each timeout
        Scheduler->>TimeoutManager: HandleTimeout(timeoutId)
        TimeoutManager->>Storage: Fetch timeout details
        Storage-->>TimeoutManager: Return timeout data
        TimeoutManager->>ExternalService: Send callback request
        ExternalService-->>TimeoutManager: Return response
        TimeoutManager->>EventPublisher: Publish timeout completion event
        EventPublisher-->>TimeoutManager: Acknowledge event publish
        TimeoutManager->>Storage: Update timeout status to completed
        Storage-->>TimeoutManager: Acknowledge update
    end

7. Caching Design

Cache Strategy Selection

For timeout management, we'll use cache-aside pattern with write-through behavior for critical timeout state updates. This approach provides:

Consistency guarantee: Cache and database remain synchronized
Performance: Fast reads for timeout status checks
Scalability: Distributed cache can handle high read volumes
Flexibility: Cache can be invalidated or updated in response to timeout events

Cache Key Design

Cache keys will follow a structured namespace pattern:

timeout:{timeoutId}:{type}

Where:

timeoutId is the unique identifier for the timeout
type represents the timeout type (e.g., "payment", "booking", "verification")

TTL Configuration

Data Type	TTL	Reason
Timeout State	30 seconds	Should be short enough to prevent stale state but long enough for typical processing
Timeout Metadata	1 hour	Metadata changes less frequently and can be refreshed periodically
Timeout Events	5 minutes	Event logs need to persist for debugging and audit

Cache Eviction Policy

We'll use LRU (Least Recently Used) eviction policy with a secondary TTL-based cleanup:

// Example Redis configuration for timeout cache
@Cacheable(
    value = "timeout",
    key = "#timeoutId",
    unless = "#result == null",
    sync = true
)
public TimeoutState getTimeoutState(String timeoutId) {
    return timeoutRepository.findById(timeoutId);
}

Cache Stampede Prevention

To prevent cache stampedes:

Probabilistic early expiry: 20% of TTL for cache refresh
Distributed locks: For critical timeout updates

public class CacheStampedePrevention {
    private static final double EARLY_EXPIRY_RATIO = 0.8;
    
    public boolean isCacheStale(String cacheKey, long ttlSeconds) {
        // Check if cache should be refreshed early
        return System.currentTimeMillis() > 
               (getCacheTimestamp(cacheKey) + (long)(ttlSeconds * EARLY_EXPIRY_RATIO * 1000));
    }
    
    public boolean acquireLock(String lockKey, int timeoutSeconds) {
        // Use Redis SET with NX and EX for distributed locking
        String result = redisTemplate.opsForValue()
            .setIfAbsent(lockKey, "locked", Duration.ofSeconds(timeoutSeconds));
        return Boolean.TRUE.equals(result);
    }
}

Distributed Cache Architecture

We'll use Redis Cluster with:

3 master nodes for high availability
3 replica nodes per master for read scaling
Automatic failover capability
Cross-region replication for disaster recovery

Cache Invalidation Strategy

We'll implement event-driven invalidation:

@Component
public class TimeoutCacheInvalidator {
    
    @EventListener
    public void handleTimeoutEvent(TimeoutUpdatedEvent event) {
        String cacheKey = buildCacheKey(event.getTimeoutId(), "state");
        redisTemplate.delete(cacheKey);
        
        // Invalidate related metadata if needed
        String metaKey = buildCacheKey(event.getTimeoutId(), "metadata");
        redisTemplate.delete(metaKey);
    }
    
    private String buildCacheKey(String timeoutId, String type) {
        return String.format("timeout:%s:%s", timeoutId, type);
    }
}

Cache Hit/Miss Metrics

We'll track cache performance with custom metrics:

@MicrometerCounter(name = "timeout.cache.hits", description = "Number of cache hits")
private Counter cacheHits;

@MicrometerCounter(name = "timeout.cache.misses", description = "Number of cache misses")
private Counter cacheMisses;

public TimeoutState getCachedTimeout(String timeoutId) {
    String cacheKey = buildCacheKey(timeoutId, "state");
    
    TimeoutState cached = redisTemplate.opsForValue().get(cacheKey);
    if (cached != null) {
        cacheHits.increment();
        return cached;
    }
    
    cacheMisses.increment();
    return null;
}

This caching strategy provides optimal performance for timeout state lookups while maintaining data consistency and preventing cache stampedes through proper TTL management and distributed locking mechanisms.

8. Repository / Data Access Layer

Repository Pattern Implementation

The repository pattern is implemented to abstract data access logic and provide a clean separation between domain objects and persistence concerns. Each entity will have its own repository interface and implementation.

public interface TimeoutRepository {
    Timeout save(Timeout timeout);
    Optional<Timeout> findById(UUID id);
    List<Timeout> findByStatus(TimeoutStatus status);
    List<Timeout> findByExpiryTimeBefore(Instant cutoffTime);
    void deleteById(UUID id);
    long countByStatus(TimeoutStatus status);
}

@Repository
public class TimeoutRepositoryImpl implements TimeoutRepository {
    private final JdbcTemplate jdbcTemplate;
    private final ObjectMapper objectMapper;

    public TimeoutRepositoryImpl(JdbcTemplate jdbcTemplate, ObjectMapper objectMapper) {
        this.jdbcTemplate = jdbcTemplate;
        this.objectMapper = objectMapper;
    }

    @Override
    public Timeout save(Timeout timeout) {
        String sql = """
            INSERT INTO timeouts (id, status, expiry_time, payload, created_at, updated_at)
            VALUES (?, ?, ?, ?, ?, ?)
            ON CONFLICT (id) DO UPDATE SET
                status = EXCLUDED.status,
                expiry_time = EXCLUDED.expiry_time,
                payload = EXCLUDED.payload,
                updated_at = EXCLUDED.updated_at
            """;
        
        jdbcTemplate.update(sql, 
            timeout.getId(),
            timeout.getStatus().name(),
            timeout.getExpiryTime(),
            objectMapper.writeValueAsString(timeout.getPayload()),
            timeout.getCreatedAt(),
            timeout.getUpdatedAt()
        );
        
        return timeout;
    }

    @Override
    public Optional<Timeout> findById(UUID id) {
        String sql = "SELECT * FROM timeouts WHERE id = ?";
        try {
            return Optional.of(jdbcTemplate.queryForObject(sql, new TimeoutRowMapper(), id));
        } catch (EmptyResultDataAccessException e) {
            return Optional.empty();
        }
    }

    @Override
    public List<Timeout> findByStatus(TimeoutStatus status) {
        String sql = "SELECT * FROM timeouts WHERE status = ? ORDER BY created_at ASC";
        return jdbcTemplate.query(sql, new TimeoutRowMapper(), status.name());
    }

    @Override
    public List<Timeout> findByExpiryTimeBefore(Instant cutoffTime) {
        String sql = "SELECT * FROM timeouts WHERE expiry_time < ? ORDER BY expiry_time ASC";
        return jdbcTemplate.query(sql, new TimeoutRowMapper(), cutoffTime);
    }

    @Override
    public void deleteById(UUID id) {
        String sql = "DELETE FROM timeouts WHERE id = ?";
        jdbcTemplate.update(sql, id);
    }

    @Override
    public long countByStatus(TimeoutStatus status) {
        String sql = "SELECT COUNT(*) FROM timeouts WHERE status = ?";
        return jdbcTemplate.queryForObject(sql, Long.class, status.name());
    }
}

ORM vs. Raw Query Decisions

For this design, raw SQL queries are preferred over ORM for better control over query execution and performance optimization. The decision is based on:

Performance: Raw queries allow precise control over query execution plans and can be optimized for specific use cases
Complexity: The timeout manager has relatively simple CRUD operations with straightforward joins and filtering
Flexibility: Raw queries provide better flexibility for implementing complex business logic like batch operations and atomic updates

Read vs. Write Separation

The current implementation uses a single repository for both read and write operations, but the architecture supports CQRS pattern for future scaling:

// Read-only repository for high-throughput reads
public interface TimeoutReadRepository {
    List<Timeout> findActiveTimeouts();
    Optional<Timeout> getTimeoutById(UUID id);
}

// Write-only repository for transactional writes
public interface TimeoutWriteRepository {
    void createTimeout(Timeout timeout);
    void updateTimeout(Timeout timeout);
    void deleteTimeout(UUID id);
}

Connection Pool Configuration

Connection pool is configured with the following parameters to handle concurrent database operations efficiently:

# application.yml
spring:
  datasource:
    hikari:
      maximum-pool-size: 20
      minimum-idle: 5
      connection-timeout: 30000
      idle-timeout: 600000
      max-lifetime: 1800000
      validation-timeout: 5000
      leak-detection-threshold: 60000

N+1 Query Prevention Strategy

To prevent N+1 query problems, eager loading is used for related entities and batch operations are implemented:

public class BatchTimeoutProcessor {
    private final TimeoutRepository timeoutRepository;
    
    public void processExpiredTimeouts(List<UUID> timeoutIds) {
        // Batch fetch all timeouts in single query
        String sql = "SELECT * FROM timeouts WHERE id IN (" + 
            timeoutIds.stream().map(id -> "?").collect(Collectors.joining(",")) + ")";
        
        // Use batch processing for updates to minimize round trips
        List<Timeout> timeouts = jdbcTemplate.query(sql, new TimeoutRowMapper(), 
            timeoutIds.toArray());
        
        // Process all timeouts in memory to avoid additional queries
        for (Timeout timeout : timeouts) {
            processSingleTimeout(timeout);
        }
    }
}

Batch Operation Design

Batch operations are implemented to handle bulk timeout processing efficiently:

public class BatchTimeoutOperations {
    private final JdbcTemplate jdbcTemplate;
    
    public void batchUpdateStatus(List<UUID> timeoutIds, TimeoutStatus newStatus) {
        String sql = "UPDATE timeouts SET status = ?, updated_at = ? WHERE id IN (" +
            timeoutIds.stream().map(id -> "?").collect(Collectors.joining(",")) + ")";
        
        // Prepare batch update with all parameters
        jdbcTemplate.update(sql, ps -> {
            ps.setString(1, newStatus.name());
            ps.setTimestamp(2, Timestamp.from(Instant.now()));
            
            for (int i = 0; i < timeoutIds.size(); i++) {
                ps.setObject(i + 3, timeoutIds.get(i));
            }
        });
    }
    
    public void batchDeleteExpiredTimeouts(Instant cutoffTime, int batchSize) {
        String sql = "DELETE FROM timeouts WHERE expiry_time < ? LIMIT ?";
        
        int deletedCount;
        do {
            deletedCount = jdbcTemplate.update(sql, cutoffTime, batchSize);
        } while (deletedCount > 0);
    }
}

9. Asynchronous Processing & Messaging

Event-driven boundaries in this feature

The Timeout Manager operates primarily in a synchronous request-response model for immediate timeout registration and cancellation. However, it leverages asynchronous processing for:

Timeout expiration handling - When a timeout reaches its deadline, the system must trigger downstream actions (e.g., notifications, state changes)
Batch cleanup operations - Periodic maintenance of expired timeouts to prevent memory bloat
Metrics aggregation and reporting - Aggregating timeout statistics for observability

The core business logic (registering, cancelling, querying timeouts) remains synchronous to ensure immediate consistency and response. Asynchronous processing is used only for side effects that don't block the main flow.

Message schema design with sample payload

{
  "version": "1.0",
  "event_type": "timeout.expired",
  "timestamp": "2023-12-01T10:30:00Z",
  "timeout_id": "uuid-12345",
  "entity_type": "booking",
  "entity_id": "booking-98765",
  "timeout_duration_ms": 300000,
  "triggered_by": "system",
  "metadata": {
    "source_service": "booking-service",
    "retry_count": 0
  }
}

Versioning strategy uses semantic versioning with backward compatibility guarantees. Each event type has a stable schema that can evolve through minor versions.

Dead letter queue strategy and poison message handling

Poison messages are handled via a dead-letter queue (DLQ) pattern:

Retry mechanism: Messages are retried up to 3 times with exponential backoff (1s, 4s, 9s)
DLQ routing: Messages that fail after max retries are routed to a dedicated DLQ topic
Manual intervention: Failed messages in DLQ are flagged for manual review and reprocessing
Monitoring: Alerting on DLQ queue size and processing rate to detect systemic issues

At-least-once vs. exactly-once delivery

At-least-once delivery is selected for this feature because:

Idempotency in consumers: Timeout expiration handlers are designed to be idempotent
Simplicity: Easier to implement and maintain than exactly-once semantics
Business tolerance: Duplicate timeout events are acceptable and handled gracefully

The system uses message deduplication at the consumer level by tracking processed timeout IDs to prevent duplicate processing.

Retry policy for consumers

Retry configuration:

Max attempts: 3 retries
Backoff strategy: Exponential with jitter (1s, 4s, 9s)
Non-retryable errors:
- Invalid timeout ID (400)
- Service unavailability (503) - retryable
- Authentication/authorization failures (401, 403) - non-retryable

Retry logic uses exponential backoff with jitter to prevent thundering herd effects.

Outbox pattern for reliable event publishing

Outbox pattern is implemented to ensure eventual consistency between timeout state and event publication:

Transaction boundary: When a timeout is created or expired, the operation is wrapped in a database transaction
Event persistence: Events are written to an outbox_events table within the same transaction
Async processing: A separate background worker polls and publishes events to message broker
Idempotency: Events are marked as published to prevent duplicate delivery

This pattern ensures that if the database commit succeeds but message publishing fails, the event will be retried.

Sequence diagram for async flows

sequenceDiagram
    participant TimeoutManager
    participant Database
    participant OutboxService
    participant MessageBroker
    participant Consumer

    TimeoutManager->>Database: Create timeout record
    Database-->>TimeoutManager: Transaction committed
    TimeoutManager->>OutboxService: Insert event to outbox_events table
    OutboxService-->>TimeoutManager: Event persisted
    TimeoutManager->>MessageBroker: Publish timeout.expired event
    MessageBroker-->>Consumer: Deliver event to consumer

    loop Retry logic
        Consumer->>MessageBroker: Acknowledge failed delivery
        MessageBroker->>Consumer: Retry event (1s, 4s, 9s)
    end

    Consumer->>Database: Process timeout expiration
    Database-->>Consumer: Update timeout status

The async flow ensures that timeout expiration events are reliably published to the message broker, and consumers can process these events with appropriate retry logic while maintaining system consistency through the outbox pattern.

10. Error Handling Design

Error Classification Hierarchy

The error handling strategy follows a hierarchical classification system to enable appropriate response handling:

Error
├── DomainError (business logic violations)
│   ├── ValidationError (input validation failures)
│   ├── BusinessLogicError (rule violations, constraints)
│   └── ResourceConflictError (state conflicts, race conditions)
├── InfrastructureError (system-level issues)
│   ├── DatabaseError (connection, query failures)
│   ├── ExternalServiceError (third-party API issues)
│   └── TimeoutError (operation timeouts)
└── SystemError (internal system failures)
    ├── InternalServerError (unexpected runtime errors)
    └── ConfigurationError (misconfiguration issues)

Error Propagation Strategy

Errors follow a layered propagation model:

Validation errors are caught and converted to ValidationError at the API boundary
Business logic errors are handled within service layer and re-thrown as BusinessLogicError
Infrastructure errors bubble up to the service layer for retry logic or fallback handling
System errors are logged and re-thrown as InternalServerError for centralized error handling

Retry Strategy with Exponential Backoff and Jitter

Retry logic is implemented using exponential backoff with jitter to prevent thundering herd:

public class RetryConfig {
    private int maxAttempts = 3;
    private long initialDelayMs = 1000;
    private double backoffMultiplier = 2.0;
    private double jitterFactor = 0.2; // ±20% jitter
    
    public long getDelay(int attempt) {
        double delay = initialDelayMs * Math.pow(backoffMultiplier, attempt - 1);
        double jitter = delay * (Math.random() * 2 * jitterFactor - jitterFactor);
        return Math.round(delay + jitter);
    }
}

Retry configuration:

Max attempts: 3
Initial delay: 1 second
Backoff multiplier: 2x
Jitter: ±20%
Retryable errors: DatabaseError, ExternalServiceError, TimeoutError

Circuit Breaker Design

The circuit breaker pattern prevents cascading failures:

public class CircuitBreaker {
    private final int failureThreshold = 5;
    private final long timeoutMs = 60000; // 1 minute
    private final int successThreshold = 5;
    
    private volatile CircuitState state = CircuitState.CLOSED;
    private int failureCount = 0;
    private long lastFailureTime = 0;
    
    public enum CircuitState {
        CLOSED, OPEN, HALF_OPEN
    }
    
    public boolean canExecute() {
        switch (state) {
            case CLOSED:
                return true;
            case HALF_OPEN:
                return failureCount < successThreshold;
            case OPEN:
                return System.currentTimeMillis() - lastFailureTime > timeoutMs;
        }
        return false;
    }
    
    public void recordSuccess() {
        if (state == CircuitState.HALF_OPEN) {
            failureCount = 0;
            state = CircuitState.CLOSED;
        }
    }
    
    public void recordFailure() {
        lastFailureTime = System.currentTimeMillis();
        failureCount++;
        
        if (state == CircuitState.CLOSED && failureCount >= failureThreshold) {
            state = CircuitState.OPEN;
        } else if (state == CircuitState.OPEN && 
                   System.currentTimeMillis() - lastFailureTime > timeoutMs) {
            state = CircuitState.HALF_OPEN;
        }
    }
}

Fallback Behavior Design

Fallback strategies are implemented per external dependency:

public class TimeoutManagerFallback {
    
    public List<TimeoutRecord> getTimeoutsFallback(String userId) {
        // Fallback to cached data or default values
        return Collections.emptyList();
    }
    
    public boolean cancelTimeoutFallback(String timeoutId) {
        // Log the failure but don't block the operation
        logger.warn("Failed to cancel timeout {}, using fallback", timeoutId);
        return true; // Assume success for graceful degradation
    }
}

Error Logging Format and Alerting Thresholds

Structured logging includes all relevant context:

{
  "timestamp": "2023-12-01T10:30:45.123Z",
  "trace_id": "abc123def456",
  "span_id": "xyz789uvw012",
  "user_id": "user-12345",
  "error_code": "TIMEOUT_DB_CONNECTION_ERROR",
  "error_message": "Failed to connect to database after 3 retries",
  "error_type": "DatabaseError",
  "retry_count": 3,
  "operation": "getTimeouts",
  "service": "timeout-manager",
  "stack_trace": "...",
  "retryable": true
}

Alerting thresholds:

Critical alerts: DatabaseError, TimeoutError (threshold: 5 errors in 5 minutes)
Warning alerts: ExternalServiceError (threshold: 10 errors in 10 minutes)
Info alerts: ValidationError (threshold: 50 errors in 1 hour)

11. Security Design

Input Validation and Sanitization Strategy

All API inputs are validated using a combination of:

Custom validation annotations (e.g., @Min(1), @NotBlank) for primitive types
Regular expression patterns for string format validation (e.g., email, phone number)
Custom validator classes for complex business logic (e.g., valid time ranges, valid status transitions)
OWASP Java HTML Sanitizer for any user-provided content that gets rendered in UIs

SQL Injection Prevention

Prepared statements are used exclusively for all database queries
Parameterized queries with named parameters (e.g., SELECT * FROM timeouts WHERE id = :timeoutId)
No dynamic query building from user inputs
ORM framework (Hibernate) is used to enforce safe query construction

Authentication Middleware Design

JWT-based authentication for API access with 2-hour expiration
OAuth2 integration for third-party service authentication (e.g., for external timeout providers)
Authentication middleware that intercepts all requests and validates the JWT token
Token refresh mechanism with 15-minute sliding window for access tokens

Authorization Enforcement

Role-Based Access Control (RBAC) model:
- ADMIN: Full access to all timeout resources
- USER: Read/write access to their own timeouts only
- SERVICE: Read-only access to timeouts (for system integrations)

Authorization checks performed at service layer before any operation:

public void updateTimeout(String timeoutId, TimeoutUpdateRequest request) {
    Timeout timeout = timeoutRepository.findById(timeoutId);
    if (!hasPermission(currentUser, timeout)) {
        throw new AccessDeniedException("Insufficient permissions");
    }
    // proceed with update
}

Sensitive Data Encryption at Rest and in Transit

At rest: AES-256 encryption using AWS KMS for timeout configuration data (e.g., callback URLs, payload)
In transit: TLS 1.3 with HSTS header enforcement for all API communications
Database encryption using transparent data encryption (TDE) with 256-bit keys

PII Handling and Masking Strategy

PII fields (e.g., user identifiers, callback URLs) are encrypted at rest using AES-256

Logging strips or masks PII fields:

// Log example: "User ID: [MASKED]" instead of actual ID
String maskedUserId = maskUserId(userId);

Secrets Management

AWS Secrets Manager for storing:
- API keys for external timeout providers
- Database credentials (rotated every 90 days)
- JWT signing keys (rotated every 6 months)
Secret rotation policy implemented with AWS Lambda functions that update secrets and notify services

Audit Logging Design

Audit log entries capture:
- User ID (masked)
- Operation type (create, update, delete)
- Timestamp
- IP address (masked)
- Resource ID (timeout ID)
- Change details (before/after)
Log format:

{
  "timestamp": "2023-10-27T14:30:00Z",
  "userId": "[MASKED]",
  "operation": "CREATE_TIMEOUT",
  "resourceId": "timeout-12345",
  "ipAddress": "[MASKED]",
  "details": {
    "timeoutDuration": 300,
    "callbackUrl": "[MASKED]"
  }
}

Retention policy: 90 days for audit logs with daily backups to S3
Alerting on unauthorized access attempts or modification of critical timeout configurations

12. Testing Strategy

Unit Testing Scope and Coverage Targets

Unit tests will cover 85%+ of business logic with 90%+ line coverage for core services. Each service class will have dedicated test suites with mocks for external dependencies. Critical paths like timeout creation, expiration handling, and cleanup operations will target 100% coverage. Test cases will include:

Normal operation scenarios (create, update, delete timeouts)
Boundary condition testing (max timeout duration, zero durations)
Error condition handling (invalid parameters, expired timeouts)
Concurrency edge cases (race conditions during timeout updates)

Integration Testing Strategy

Integration tests will validate end-to-end flows using testcontainers for database and Redis instances. Key integration scenarios include:

Timeout creation and expiration with real database persistence
Cache synchronization between Redis and database
Event publishing to message broker (if applicable)
Cross-service interactions with external systems

Contract Testing

Contract tests will be implemented using Pact for API contract verification between timeout manager and dependent services. Each consumer-provider pair will have a pact file defining expected request/response formats, ensuring compatibility across service boundaries.

Performance Testing Tool

Performance testing will be conducted using k6 with the following key scenarios:

High concurrent timeout creation: 1000 concurrent requests creating timeouts with random durations (1-30 seconds)
Timeout expiration throughput: 500 concurrent timeout expirations with 100ms average latency target
Database query performance: 200 concurrent read operations with <50ms p95 response time
Cache hit ratio validation: 10,000 read operations with >95% cache hit rate

Test Data Management Strategy

Test data will be managed through factory patterns and fixtures:

Factory classes for generating valid timeout entities with configurable parameters
Database fixtures using SQL scripts for consistent baseline data
Test data seeding with deterministic timestamps to enable reproducible test runs
Data cleanup procedures ensuring each test runs in isolation with clean state

// Example factory pattern for timeout entities
public class TimeoutFactory {
    public static Timeout createValidTimeout(String id, long durationMs) {
        return Timeout.builder()
                .id(id)
                .duration(durationMs)
                .createdAt(Instant.now())
                .expiresAt(Instant.now().plusMillis(durationMs))
                .status(TimeoutStatus.ACTIVE)
                .build();
    }
    
    public static Timeout createExpiredTimeout(String id) {
        return Timeout.builder()
                .id(id)
                .duration(1000L)
                .createdAt(Instant.now().minusSeconds(2))
                .expiresAt(Instant.now().minusSeconds(1))
                .status(TimeoutStatus.EXPIRED)
                .build();
    }
}

13. Observability Design

Structured Logging Schema

All logs must follow a consistent schema to enable effective search, filtering, and analysis:

{
  "timestamp": "2023-10-15T14:30:45.123Z",
  "level": "INFO|WARN|ERROR",
  "service": "timeout-manager",
  "trace_id": "12345678-1234-1234-1234-123456789012",
  "span_id": "abcdef12-3456-7890-abcd-ef1234567890",
  "user_id": "user_12345",
  "entity_id": "timeout_98765",
  "operation": "create_timeout",
  "message": "Timeout created successfully",
  "error_code": "TIMEOUT_CREATION_FAILED",
  "error_details": {
    "message": "Invalid timeout duration",
    "stack_trace": "...",
    "cause": "ValidationException"
  },
  "duration_ms": 15,
  "request_id": "req_abc123",
  "metadata": {
    "timeout_duration_seconds": 300,
    "timeout_type": "session"
  }
}

PII masking rules:

User IDs and session IDs are masked in logs (e.g., user_12345 → user_***)
Full email addresses are truncated to domain only (e.g., john.doe@example.com → john.doe@***)
IP addresses are masked to network level (e.g., 192.168.1.100 → 192.168.1.***)

Distributed Tracing Instrumentation

Using OpenTelemetry for distributed tracing:

Custom spans to add:

timeout_manager.create_timeout - Traces the entire timeout creation flow
timeout_manager.check_timeout_status - Monitors timeout status checks
timeout_manager.execute_timeout_action - Records timeout action execution
timeout_manager.cleanup_expired_timeouts - Tracks cleanup operations

Attributes to capture per key operation:

Span currentSpan = Span.current();
currentSpan.setAttribute("timeout_id", timeoutId);
currentSpan.setAttribute("timeout_type", timeoutType);
currentSpan.setAttribute("timeout_duration_seconds", durationSeconds);
currentSpan.setAttribute("user_id", userId);
currentSpan.setAttribute("is_expired", isExpired);

Metrics Design at Service Level

RED Method Metrics:

Metric Name	Type	Labels	Description
`timeout_manager_requests_total`	Counter	`operation`, `status`	Total number of timeout operations
`timeout_manager_errors_total`	Counter	`operation`, `error_type`	Error counts by operation and type
`timeout_manager_duration_seconds`	Histogram	`operation`, `status`	Operation duration in seconds
`timeout_manager_active_timeouts`	Gauge	`timeout_type`	Current active timeout count
`timeout_manager_expired_timeouts_total`	Counter	`timeout_type`	Total expired timeouts
`timeout_manager_processing_queue_size`	Gauge	-	Size of timeout processing queue

Label Design:

operation: create_timeout, check_timeout_status, execute_timeout_action
status: success, error, timeout, retry
error_type: validation_error, database_error, timeout_execution_error

Health Check Endpoint Design

@RestController
public class HealthController {
    
    @GetMapping("/health")
    public ResponseEntity<Map<String, Object>> health() {
        Map<String, Object> response = new HashMap<>();
        response.put("status", "healthy");
        response.put("timestamp", Instant.now());
        response.put("service", "timeout-manager");
        response.put("dependencies", checkDependencies());
        return ResponseEntity.ok(response);
    }
    
    @GetMapping("/ready")
    public ResponseEntity<Map<String, Object>> readiness() {
        Map<String, Object> response = new HashMap<>();
        response.put("status", "ready");
        response.put("timestamp", Instant.now());
        response.put("database_connected", isDatabaseConnected());
        response.put("cache_available", isCacheAvailable());
        response.put("message_queue_ready", isMessageQueueReady());
        return ResponseEntity.ok(response);
    }
    
    @GetMapping("/live")
    public ResponseEntity<Map<String, Object>> liveness() {
        Map<String, Object> response = new HashMap<>();
        response.put("status", "alive");
        response.put("timestamp", Instant.now());
        response.put("uptime_seconds", getUptimeSeconds());
        return ResponseEntity.ok(response);
    }
}

Alerting Thresholds per Key Metric

Metric	Warning Threshold	Critical Threshold	Escalation Path
`timeout_manager_errors_total` (per operation)	> 5 errors/minute	> 20 errors/minute	On-call engineer, SRE team
`timeout_manager_duration_seconds` (p95)	> 500ms	> 2s	On-call engineer, SRE team
`timeout_manager_active_timeouts`	> 100,000 active	> 500,000 active	SRE team
`timeout_manager_processing_queue_size`	> 10,000 items	> 50,000 items	SRE team
`timeout_manager_requests_total` (rate)	> 10,000 req/sec	> 50,000 req/sec	SRE team
`timeout_manager_database_errors_total`	> 1 error/minute	> 5 errors/minute	SRE team
`timeout_manager_cache_errors_total`	> 1 error/minute	> 5 errors/minute	SRE team

Log Rotation and Retention Strategy

Log rotation: 100MB size or 24h rotation
Retention policy: 30 days for INFO level, 7 days for ERROR level
Log aggregation: Centralized via ELK stack or CloudWatch
Log compression: Enabled for long-term storage

Service Monitoring Dashboard

Key dashboard panels:

Request Rate and Latency: Real-time view of timeout operations
Error Rate: Error distribution by operation and type
Active Timeout Count: Trend of active timeouts over time
Queue Size: Processing queue status
Database and Cache Health: Connection pool metrics and latency
Error Rate by User/Timeout Type: Segmented error analysis

Audit Trail Integration

All timeout-related operations are logged to audit trail:

Creation of timeouts
Status changes (expired, cancelled)
Action execution
Manual overrides or admin interventions

Audit logs are stored in a separate audit database with:

Timestamps accurate to microsecond precision
User identity (including service accounts)
Operation type and parameters
Result status and duration

14. Open Questions, Risks & Trade-offs

Decisions requiring product / engineering alignment

Timeout granularity vs. system overhead: Should timeouts be managed at millisecond precision or can we accept coarser granularities (e.g., 100ms buckets) to reduce system overhead? This decision affects both memory usage and the accuracy of timeout delivery.
Global vs. per-service timeout configuration: Should timeout policies be centrally managed or allow service-level overrides? Centralized policies simplify governance but reduce flexibility for services with unique requirements.
Graceful degradation strategy: When system load is high, should we prioritize processing timeouts over new timeout registrations or drop new registrations entirely? This impacts user experience and system reliability.

Known trade-offs and justifications

Trade-off	Option 1	Option 2	Justification
Memory vs. CPU trade-off	Store all timeout entries in-memory with periodic disk sync	Use disk-based storage with memory cache	In-memory provides faster lookups but increases memory pressure. Disk-based approach reduces memory usage but introduces latency on reads. For high-throughput systems, in-memory is preferred with LRU eviction and periodic persistence to disk.
Consistency vs. availability	Enforce strong consistency in timeout state across nodes	Allow eventual consistency with optimistic updates	Strong consistency requires distributed consensus protocols (e.g., Raft), which can impact availability during network partitions. Eventual consistency with optimistic updates allows better availability but introduces potential for temporary inconsistency.
Timeout precision vs. system overhead	Use microsecond-precision timeouts with frequent polling	Use coarse-grained timeouts (e.g., 100ms buckets)	Microsecond precision provides better accuracy but increases CPU usage due to more frequent polling. Coarse-grained timeouts reduce overhead but introduce potential for timeouts to fire slightly earlier or later than intended.
Event-driven vs. polling approach	Poll for expired timeouts periodically	Use event-driven model with timeout notifications	Polling is simpler to implement and debug but introduces latency and CPU overhead. Event-driven systems provide better performance but require more complex infrastructure (e.g., message brokers).

Risks and mitigation strategies

Risk	Impact	Mitigation Strategy
Timeout drift under high load	Timeouts may fire earlier/later than intended, causing service disruption or missed timeouts	Implement clock synchronization with NTP and use monotonic time sources. Add jitter to polling intervals to prevent thundering herd effects.
Memory exhaustion due to timeout accumulation	System may run out of memory if too many timeouts are registered	Implement strict limits on concurrent timeouts per service and enforce TTL-based cleanup. Use memory-efficient data structures (e.g., skip lists or hash maps with LRU eviction).
Race conditions in timeout state updates	Concurrent updates to timeout status can cause inconsistent state	Use atomic operations or locks for critical sections. Implement optimistic locking with version numbers to detect and handle conflicts.
Network partitioning affecting timeout consistency	Distributed timeout systems may lose timeouts during network issues	Implement distributed consensus protocols (e.g., Raft) or use eventual consistency with timeout state reconciliation. Build in retry mechanisms for timeout registration and deletion operations.
Clock skew between nodes	Inconsistent system clocks can cause timeouts to fire at wrong times	Use NTP synchronization across all nodes and implement clock drift detection. Consider using monotonic time sources where possible to avoid wall-clock issues.

Technical debt acknowledged

Hardcoded polling intervals: Current implementation uses hardcoded values for polling intervals and batch sizes. These should be configurable via environment variables or configuration files to allow tuning without code changes.
Limited timeout state persistence: Timeout state is currently only maintained in-memory, with periodic disk sync. This creates a single point of failure and limits the system's ability to recover from restarts without data loss.
No built-in timeout cancellation optimization: The system does not currently optimize for cancellation of timeouts that are about to fire. This could lead to unnecessary processing overhead when timeouts are cancelled shortly before they would have fired.
Lack of timeout dependency tracking: There's no mechanism to track dependencies between timeouts, which could be useful for implementing timeout cascades or hierarchical timeout management.
No metrics aggregation at service level: While individual timeout operations are instrumented, there's no built-in capability to aggregate and report metrics at the service or application level without additional tooling.
Limited support for timeout grouping: The current design does not provide built-in mechanisms to group or batch timeouts for better performance, which could be useful in high-volume scenarios.