Factory Pattern Implementation in Backend Systems

The Factory pattern is a fundamental creational design pattern that provides an interface for creating objects without specifying their exact classes, enabling flexible and maintainable backend system architectures through proper abstraction and decoupling of object creation logic.

1. Problem Statement & Requirements

Clarifying questions

1. What specific domain or system context is the factory pattern being applied to? For example, are we designing a vehicle factory, payment processor factory, or some other domain?
2. What is the expected scale of object creation? Will we be creating thousands of objects per second, or is this more batch-oriented?
3. Are there specific consistency requirements for the factory's output? For example, should all created objects be of a specific type or meet certain criteria?
4. What is the read/write ratio for the factory's operation? Is it primarily used for creation or also for querying/validating?
5. Are there specific performance requirements? For example, should factory creation be sub-10ms for 95th percentile?
6. What authentication and authorization model is expected? Should the factory support role-based access control or API key-based access?
7. What external systems or integrations are involved in the factory process? For example, does it integrate with external suppliers or third-party services?

Functional requirements

• Implement a factory pattern that can dynamically create objects based on input parameters
• Support multiple object types with distinct creation logic for each type
• Allow configuration of factory behavior through external parameters or configuration files
• Provide a unified interface for object creation that abstracts the underlying creation logic

Non-functional requirements

• Latency targets: 50th percentile < 5ms, 95th percentile < 10ms, 99th percentile < 25ms
• Throughput: Support at least 10,000 object creations per second
• Availability: 99.9% uptime with graceful degradation capability
• Scalability: Support horizontal scaling across multiple instances

Constraints and assumptions

• Factory implementation will be part of a larger microservices architecture
• Assumption: Object creation is a frequent operation with predictable patterns
• Assumption: Factory will be used in high-concurrency environments
• Assumption: Configuration can be managed via external configuration service or files

Out of scope

• Authentication and authorization implementation (handled at API layer)
• Database persistence or storage of created objects (handled by downstream services)
• Complex business logic validation beyond basic type checking
• Integration with external systems or third-party APIs (assumed to be handled by other services)
• Real-time monitoring and alerting (handled by observability layer)

2. API Design

API Style Selection

For this system, I'll select REST as the primary API style for the following reasons:

• Familiarity: REST is well-understood by developers and tooling
• Statelessness: Aligns with backend service design principles
• Scalability: REST APIs scale well with proper caching and load balancing
• Interoperability: Works across different client types (web, mobile, third-party)
• Tooling: Rich ecosystem of tools for documentation, testing, and monitoring

API Endpoints

Method	Path	Description	Auth
GET	/api/v1/factories	List all factories	JWT
POST	/api/v1/factories	Create new factory	JWT + Admin Role
GET	/api/v1/factories/{id}	Get factory by ID	JWT
PUT	/api/v1/factories/{id}	Update factory details	JWT + Admin Role
DELETE	/api/v1/factories/{id}	Delete factory	JWT + Admin Role
POST	/api/v1/factories/{id}/products	Create product in factory	JWT + Factory Owner
GET	/api/v1/factories/{id}/products	List products in factory	JWT
POST	/api/v1/factories/{id}/products/{productId}/production	Start production run	JWT + Factory Owner
GET	/api/v1/factories/{id}/production	Get production status	JWT

Request and Response Schemas

Create Factory Request

{
  "name": "Automotive Manufacturing Plant",
  "location": {
    "country": "USA",
    "city": "Detroit"
  },
  "capacity": 1000,
  "type": "automotive"
}

Create Factory Response

{
  "id": "f1234567-89ab-cdef-0123-456789abcdef",
  "name": "Automotive Manufacturing Plant",
  "location": {
    "country": "USA",
    "city": "Detroit"
  },
  "capacity": 1000,
  "type": "automotive",
  "createdAt": "2023-10-15T14:30:00Z",
  "updatedAt": "2023-10-15T14:30:00Z"
}

Create Product Request

{
  "name": "Car Engine",
  "description": "V8 engine for luxury vehicles",
  "specifications": {
    "displacement": "5.0L",
    "power": "400 HP"
  }
}

Create Product Response

{
  "id": "p1234567-89ab-cdef-0123-456789abcdef",
  "name": "Car Engine",
  "description": "V8 engine for luxury vehicles",
  "specifications": {
    "displacement": "5.0L",
:
    "power": "400 HP"
  },
  "createdAt": "2023-10-15T14:30:00Z",
  "updatedAt": "2023-10-15T14:30:00Z"
}

HTTP Status Codes and Error Response Format

Status Code	Usage
200	Successful GET, PUT, PATCH
201	Successful POST (resource created)
204	Successful DELETE
400	Bad request, validation error
401	Authentication required
403	Authorization denied
404	Resource not found
422	Unprocessable entity (validation)
429	Rate limit exceeded
500	Internal server error

Standardized error response format:

{
  "error": {
    "code": "VALIDATION_ERROR",
    "message": "Factory name is required",
    "details": [
      {
        "field": "name",
        "message": "Factory name is required"
      }
    ],
    "timestamp": "2023-10-15T14:30:00Z"
  }
}

API Versioning Strategy

Versioning will be done via URI path prefix:

• /api/v1/factories - Current version
• Future: /api/v2/factories for breaking changes

Pagination Design

Using cursor-based pagination with the following design:

• cursor parameter for continuation
• limit parameter (default 20, max 100)
• direction parameter (forward/backward)

This approach is preferred over offset-based pagination because:

1. More stable with concurrent writes
2. Better performance for large datasets
3. Consistent results even when data changes

Authentication and Authorization

Authentication: JWT tokens with 24-hour expiration Authorization:

• Admin role for factory creation/deletion/update
• Factory owner permissions for product management within their factory
• Read access for all authenticated users

Rate Limiting

Rate limiting per endpoint:

Endpoint	Limit	Period
POST /factories	100/hour	Hour
POST /factories/{id}/products	500/hour	Hour
GET /factories	1000/hour	Hour

Implementation strategy:

• Token bucket algorithm with in-memory or Redis storage
• Per-user and per-API key limits

Idempotency Design

For mutation endpoints, idempotency keys are supported:

• Header: Idempotency-Key: <uuid>
• For duplicate requests, return the same response as first request
• Store key in Redis with TTL of 24 hours

Example Idempotency Flow

POST /api/v1/factories HTTP/1.1
Idempotency-Key: 123e4567-e89b-12d3-a456-426614174000
Content-Type: application/json

{
  "name": "New Factory",
  "location": {
    "country": "USA"
  }
}

If this request is retried with the same key, the system returns the same factory response as the first request.

3. Data Model Design

Entity-Relationship diagram (Mermaid)

erDiagram
    FACTORY {
        string id PK
        string name
        string type
        timestamp created_at
        timestamp updated_at
    }

    PRODUCT {
        string id PK
        string factory_id FK
        string name
        string type
        timestamp created_at
        timestamp updated_at
    }

    PRODUCT_TYPE {
        string id PK
        string name
        string description
        timestamp created_at
        timestamp updated_at
    }

    FACTORY_TO_PRODUCT_TYPE {
        string factory_id FK
        string product_type_id FK
        timestamp created_at
    }

    FACTORY ||--o{ PRODUCT : produces
    FACTORY ||--o{ FACTORY_TO_PRODUCT_TYPE : supports
    PRODUCT_TYPE ||--o{ FACTORY_TO_PRODUCT_TYPE : supported_by

Table / Collection schema definitions with field types, constraints, indices

-- FACTORY table
CREATE TABLE factories (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    name VARCHAR(255) NOT NULL,
    type VARCHAR(100) NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    
    CONSTRAINT chk_factory_type CHECK (type IN ('CAR', 'BICYCLE', 'ELECTRONIC'))
);

-- PRODUCT table
CREATE TABLE products (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    factory_id UUID NOT NULL,
    name VARCHAR(255) NOT NULL,
    type VARCHAR(100) NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    
    CONSTRAINT chk_product_type CHECK (type IN ('SEDAN', 'SUV', 'ROAD_BIKE', 'MOUNTAIN_BIKE', 'SMARTPHONE', 'LAPTOP')),
    FOREIGN KEY (factory_id) REFERENCES factories(id) ON DELETE CASCADE
);

-- PRODUCT_TYPE table
CREATE TABLE product_types (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    name VARCHAR(255) NOT NULL UNIQUE,
    description TEXT,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

-- FACTORY_TO_PRODUCT_TYPE junction table
CREATE TABLE factory_product_types (
    factory_id UUID NOT NULL,
    product_type_id UUID NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    
    PRIMARY KEY (factory_id, product_type_id),
    FOREIGN KEY (factory_id) REFERENCES factories(id) ON DELETE CASCADE,
    FOREIGN KEY (product_type_id) REFERENCES product_types(id) ON DELETE CASCADE
);

Primary key strategy (UUID vs. auto-increment vs. ULID — with justification)

Using UUID as primary keys for all tables to support distributed systems and avoid conflicts in multi-region deployments. UUIDs provide global uniqueness without requiring coordination between systems, which is essential for scalable factory management systems that may span multiple data centers or cloud regions.

Index design (composite, partial, covering indices — with justification per index)

-- Index on factory type for filtering factories by type
CREATE INDEX idx_factories_type ON factories(type);

-- Index on factory_id in products table for efficient joins and filtering
CREATE INDEX idx_products_factory_id ON products(factory_id);

-- Index on product type for efficient querying by product type
CREATE INDEX idx_products_type ON products(type);

-- Composite index on factory_id and type for optimized queries filtering by both
CREATE INDEX idx_products_factory_type ON products(factory_id, type);

-- Index on factory_product_types for fast lookups by factory or product type
CREATE INDEX idx_factory_product_types_factory_id ON factory_product_types(factory_id);
CREATE INDEX idx_factory_product_types_product_type_id ON factory_product_types(product_type_id);

-- Partial index for active products (if we have a status field)
-- CREATE INDEX idx_products_active ON products(id) WHERE status = 'ACTIVE';

Data partitioning / sharding strategy

Partition by factory type for horizontal scaling:

• CAR factories: Partition 1
• BICYCLE factories: Partition 2
• ELECTRONIC factories: Partition 3

This allows for better resource allocation and query optimization based on factory specialization.

Soft delete vs. hard delete decision with justification

Soft delete is implemented using a deleted_at timestamp field in the tables. This approach allows for:

1. Data recovery capability
2. Audit trail preservation
3. Historical analysis of factory and product lifecycle
4. Compliance with data retention policies

Audit trail design (who, what, when)

-- Enhanced factories table with audit fields
CREATE TABLE factories (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    name VARCHAR(255) NOT NULL,
    type VARCHAR(100) NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    created_by VARCHAR(255),
    updated_by VARCHAR(255),
    deleted_at TIMESTAMP,
    
    CONSTRAINT chk_factory_type CHECK (type IN ('CAR', 'BICYCLE', 'ELECTRONIC'))
);

Data archival strategy and retention policy

• Active data (last 30 days): Stored in main tables
• Historical data (30 days to 2 years): Archived to separate storage (e.g., S3 with lifecycle policies)
• Long-term archival (2+ years): Migrated to cold storage or data warehouse
• Retention policy: 5 years for compliance, 2 years for operational purposes

-- Example archive table structure
CREATE TABLE factories_archive (
    id UUID,
    name VARCHAR(255),
    type VARCHAR(100),
    created_at TIMESTAMP,
    updated_at TIMESTAMP,
    deleted_at TIMESTAMP,
    archived_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

4. Class Design & Service Layer

UML Class Diagram

classDiagram
    class VehicleFactory {
        +createVehicle(type: String): Vehicle
    }
    
    class Vehicle {
        <<interface>>
        +start(): void
        +stop(): void
    }
    
    class Car {
        +start(): void
        +stop(): void
    }
    
    class Motorcycle {
        +start(): void
        +stop(): void
    }
    
    class Truck {
        +start(): void
        +stop(): void
    }
    
    VehicleFactory --> Vehicle : creates
    Car --|> Vehicle : implements
    Motorcycle --|> Vehicle : implements
    Truck --|> Vehicle : implements

State Machine Diagram

stateDiagram-v2
    [*] --> Idle
    Idle --> Running : start()
    Running --> Stopped : stop()
    Stopped --> Idle : reset()

Service Class Structure and Responsibilities

The service layer will contain:

• VehicleFactoryService - Implements the factory pattern for vehicle creation
• VehicleService - Contains business logic for vehicle operations
• VehicleRepository - Handles persistence operations

Interface / Abstract Class Design

// Vehicle interface - defines contract for all vehicles
public interface Vehicle {
    void start();
    void stop();
    String getType();
}

// Abstract base class for common vehicle functionality
public abstract class BaseVehicle implements Vehicle {
    protected String id;
    protected String model;
    protected boolean isRunning = false;
    
    public BaseVehicle(String id, String model) {
        this.id = id;
        this.model = model;
    }
    
    @Override
    public String getType() {
        return this.getClass().getSimpleName();
    }
    
    // Common behavior that can be shared across vehicle types
    public void reset() {
        this.isRunning = false;
    }
}

// Factory interface for extensibility
public interface VehicleFactory {
    Vehicle createVehicle(String type);
}

Business Logic Encapsulation Patterns

The Factory pattern is implemented to decouple vehicle creation logic from client code, supporting the Open/Closed Principle.

Critical-Path Method Implementation

public class VehicleFactoryService {
    private final Map<String, Supplier<Vehicle>> vehicleRegistry = new HashMap<>();
    
    public VehicleFactoryService() {
        vehicleRegistry.put("car", Car::new);
        vehicleRegistry.put("motorcycle", Motorcycle::new);
        vehicleRegistry.put("truck", Truck::new);
    }
    
    public Vehicle createVehicle(String type) {
        // Transaction boundary - start of operation
        try {
            // Validate input
            if (type == null || type.trim().isEmpty()) {
                throw new IllegalArgumentException("Vehicle type cannot be null or empty");
            }
            
            // Thread-safe lookup in registry
            Supplier<Vehicle> factory = vehicleRegistry.get(type.toLowerCase());
            if (factory == null) {
                throw new IllegalArgumentException("Unsupported vehicle type: " + type);
            }
            
            // Create and return new vehicle instance
            Vehicle vehicle = factory.get();
            
            // Log creation event (audit trail)
            logVehicleCreation(vehicle);
            
            return vehicle;
        } catch (Exception e) {
            // Transaction boundary - rollback or error handling
            throw new VehicleCreationException("Failed to create vehicle of type: " + type, e);
        }
    }
    
    private void logVehicleCreation(Vehicle vehicle) {
        // In production, this would use structured logging with trace context
        System.out.println("Created vehicle: " + vehicle.getType() + " with ID: " + vehicle.getClass().getSimpleName());
    }
}

Transaction Boundary Design

Transaction boundaries are defined at the service method level for vehicle creation. Each factory operation is atomic - either the vehicle is created successfully or an exception is thrown.

Service Interface Contracts

public interface VehicleService {
    Vehicle createVehicle(String type);
    void startVehicle(String vehicleId);
    void stopVehicle(String vehicleId);
}

Saga / Orchestration Pattern

For complex operations involving multiple systems (e.g., vehicle creation with registration, insurance setup), a saga pattern would be appropriate. However, for simple factory creation, this is not required.

Domain Object / Value Object Design

public class Vehicle {
    private String id;
    private String type;
    private String model;
    private boolean isRunning;
    
    // Constructor, getters, setters
}

public class VehicleType {
    private String name;
    private int maxSpeed;
    
    // Constructor, getters, setters
}

Factory Implementation with Strategy Pattern

public class VehicleFactory {
    private final Map<String, VehicleCreatorStrategy> strategies;
    
    public VehicleFactory() {
        this.strategies = new HashMap<>();
        this.strategies.put("car", new CarCreatorStrategy());
        this.strategies.put("motorcycle", new MotorcycleCreatorStrategy());
        this.strategies.put("truck", new TruckCreatorStrategy());
    }
    
    public Vehicle createVehicle(String type, String model) {
        VehicleCreatorStrategy strategy = strategies.get(type.toLowerCase());
        if (strategy == null) {
            throw new IllegalArgumentException("Unsupported vehicle type: " - type);
        }
        return strategy.create(model);
    }
    
    // Strategy interface
    private interface VehicleCreatorStrategy {
        Vehicle create(String model);
    }
    
    // Concrete strategy implementations
    private static class CarCreatorStrategy implements VehicleCreatorStrategy {
        @Override
        public Vehicle create(String model) {
            return new Car(UUID.randomUUID().toString(), model);
        }
    }
    
    private static class MotorcycleCreatorStrategy implements VehicleCreatorStrategy {
        @Override
        public Vehicle create(String model) {
            return new Motorcycle(UUID.randomUUID().toString(), model);
        }
    }
    
    private static class TruckCreatorStrategy implements VehicleCreatorStrategy {
        @Override
        public Vehicle create(String model) {
            return new Truck(UUID.randomUUID().toString(), model);
        }
    }
}

5. Concurrency & Thread Safety

Concurrency Model

The system uses a thread pool model with Java's ExecutorService for managing concurrent operations. Each operation (e.g., order placement, vehicle entry) runs in a separate thread from the pool. For high-throughput scenarios, we use CompletableFuture with custom ForkJoinPool to handle asynchronous operations efficiently without blocking the main request threads.

Race Condition Prevention Strategy

Race conditions are prevented through:

1. Immutable domain objects - All entities and value objects are immutable to prevent concurrent modification
2. Database-level atomic operations - Using database transactions and row-level locking for critical updates
3. Thread-local state management - Per-request state is isolated using thread-local variables where appropriate
4. Synchronized access to shared resources - Using ReentrantLock for critical sections that require exclusive access

Optimistic vs. Pessimistic Locking Selection

We use optimistic locking for most operations due to:

• High read-to-write ratio in the system (90% reads, 10% writes)
• Low contention scenarios in typical usage patterns
• Better scalability and reduced lock overhead
• Support for conflict detection via version fields

Pessimistic locking is used only for:

• Critical inventory updates where concurrent modifications must be strictly prevented
• Operations that require exclusive database access (e.g., batch processing of orders)

Deadlock Prevention Strategy

Deadlock prevention is implemented through:

1. Consistent lock ordering - All locks are acquired in a predefined order across the system (e.g., by entity ID)
2. Timeout-based lock acquisition - Using tryLock() with 500ms timeout to avoid indefinite blocking
3. Hierarchical lock management - Locks are organized in a hierarchy to prevent circular dependencies
4. Lock monitoring and alerting - Custom metrics track lock wait times and detect potential deadlocks

Idempotency Implementation for Concurrent Duplicate Requests

Idempotency is handled at the service layer using:

1. Idempotency keys - Unique identifiers provided by clients for duplicate request detection
2. In-memory cache with TTL - Stores recent idempotency keys for 5 minutes to handle retries
3. Database-backed storage - For critical operations, idempotency keys are persisted in a dedicated table with TTL
4. Request deduplication - Concurrent requests with same key are queued and only one executes

public class IdempotencyService {
    private final Map<String, CompletableFuture<?>> activeRequests = new ConcurrentHashMap<>();
    private final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
    
    public <T> CompletableFuture<T> executeWithIdempotency(
            String idempotencyKey,
            Supplier<CompletableFuture<T>> operation) {
        
        return activeRequests.computeIfAbsent(idempotencyKey, key -> {
            CompletableFuture<T> future = operation.get();
            // Remove from active requests after completion or timeout
            future.whenComplete((result, error) -> {
                activeRequests.remove(key);
            });
            scheduler.schedule(() -> {
                if (activeRequests.remove(key) != null) {
                    // Log timeout event
                }
            }, 5, TimeUnit.MINUTES);
            
            return future;
        });
    }
}

Thread Safety in Factory Pattern Implementation

The factory pattern implementation ensures thread safety by:

1. Immutable factory configuration - Factory instances are created once and never modified
2. Thread-safe object creation - Using synchronized methods or ReentrantLock for concurrent object creation
3. Caching of created instances - Using thread-safe collections like ConcurrentHashMap for caching
4. Factory initialization synchronization - Ensuring single-threaded initialization of factory components

public class ThreadSafeFactory {
    private final Map<String, Object> cache = new ConcurrentHashMap<>();
    private final ReentrantLock lock = new ReentrantLock();
    
    public <T> T createInstance(String type, Class<T> clazz) {
        return (T) cache.computeIfAbsent(type, key -> {
            lock.lock();
            try {
                return createInstanceInternal(key, clazz);
            } finally {
                lock.unlock();
            }
        });
    }
    
    private <T> T createInstanceInternal(String type, Class<T> clazz) {
        // Implementation logic here
        return null;
    }
}

6. Sequence Diagrams for Key Flows

Happy Path Sequence Diagram

sequenceDiagram
    participant Client
    participant FactoryController
    participant VehicleFactory
    participant Vehicle
    participant EventPublisher
    
    Client->>FactoryController: POST /vehicles
    FactoryController->>VehicleFactory: createVehicle(type)
    VehicleFactory->>VehicleFactory: getInstance(type)
    VehicleFactory-->>FactoryController: return Vehicle
    FactoryController->>EventPublisher: publish vehicle.created event
    EventPublisher-->>FactoryController: event published
    FactoryController-->>Client: 201 Created {id, type, status}

Error / Edge Case Sequence Diagram

sequenceDiagram
    participant Client
    participant FactoryController
    participant VehicleFactory
    participant ErrorLogger
    
    Client->>FactoryController: POST /vehicles?type=invalid_type
    FactoryController->>VehicleFactory: createVehicle(type)
    VehicleFactory->>VehicleFactory: getInstance(type)
    VehicleFactory-->>FactoryController: UnknownVehicleTypeException
    FactoryController->>ErrorLogger: log error
    ErrorLogger-->>FactoryController: error logged
    FactoryController-->>Client: 400 Bad Request {error, message}

Async Flow Sequence Diagram

sequenceDiagram
    participant Client
    participant FactoryController
    participant VehicleFactory
    participant AsyncQueue
    participant VehicleProcessor
    participant EventPublisher
    
    Client->>FactoryController: POST /vehicles
    FactoryController->>VehicleFactory: createVehicle(type)
    VehicleFactory-->>FactoryController: return Vehicle
    FactoryController->>AsyncQueue: queue vehicle.created event
    AsyncQueue-->>VehicleProcessor: consume event
    VehicleProcessor->>EventPublisher: publish vehicle.processed event
    EventPublisher-->>VehicleProcessor: event published
    VehicleProcessor-->>AsyncQueue: event processed

7. Caching Design

Cache Strategy Selection

For this system, we'll use a cache-aside pattern with LRU eviction policy. This approach provides the best balance between simplicity, performance, and cache consistency for our use case where data freshness is important but not mission-critical.

Cache Key Design

Cache keys will follow a namespace-based pattern:

{namespace}:{entity_type}:{entity_id}

Example key structure:

user:profile:12345
order:status:67890

TTL Strategy

Data Type	TTL	Reason
User Profile	5 minutes	Frequent updates, acceptable staleness
Order Status	1 minute	Critical for UI responsiveness
Product Catalog	30 minutes	Relatively static, expensive to rebuild
Session Tokens	1 hour	Based on session timeout policies
API Rate Limits	1 minute	Short-lived, per-user limits

Cache Eviction Policy

We'll implement LRU (Least Recently Used) eviction policy with a secondary TTL-based cleanup. This ensures that:

• Frequently accessed items stay in cache
• Stale data is automatically cleaned up
• Memory pressure is managed proactively

Cache Stampede Prevention

To prevent cache stampedes:

1. Probabilistic Early Expiry: 20% of TTL for cache entries (e.g., 4 minutes instead of 5)
2. Distributed Locking: Using Redis distributed locks for cache warming operations
3. Cache Entry Versioning: Track version numbers to avoid serving stale data during rebuilds

Distributed Cache Architecture

We'll use Redis Cluster with:

• 3 master nodes for data distribution
• 3 replica nodes per master for high availability
• Automatic failover with Redis Sentinel monitoring
• Cross-region replication for disaster recovery

Cache Invalidation Strategy

We'll implement event-driven invalidation:

1. Write Operations: Immediate cache invalidation on data changes
2. Batch Updates: Periodic cleanup of stale entries (every 5 minutes)
3. Explicit Purge: Admin API endpoints to force cache clearing

Cache Implementation Example (Java)

@Component
public class CachingService {
    private final RedisTemplate<String, Object> redisTemplate;
    private final ObjectMapper objectMapper;
    
    public CachingService(RedisTemplate<String, Object> redisTemplate, 
                         ObjectMapper objectMapper) {
        this.redisTemplate = redisTemplate;
        this.objectMapper = objectMapper;
    }
    
    public <T> T getCachedValue(String key, Class<T> type, Supplier<T> loader) {
        String cacheKey = "cache:" + key;
        
        // Try to get from cache
        Object cachedValue = redisTemplate.opsForValue().get(cacheKey);
        if (cachedValue != null) {
            try {
                return objectMapper.convertValue(cachedValue, type);
            } catch (Exception e) {
                // Log and continue with fresh load
                log.warn("Failed to deserialize cached value for key: {}", key);
            }
        }
        
        // Cache miss - load fresh data
        T result = loader.get();
        
        // Store in cache with TTL
        if (result != null) {
            try {
                Object serializedValue = objectMapper.convertValue(result, Object.class);
                redisTemplate.opsForValue().set(cacheKey, serializedValue, 
                    Duration.ofMinutes(5), TimeUnit.MINUTES);
            } catch (Exception e) {
                log.error("Failed to cache value for key: {}", key, e);
            }
        }
        
        return result;
    }
    
    public void invalidateCache(String key) {
        String cacheKey = "cache:" + key;
        redisTemplate.delete(cacheKey);
    }
    
    public void invalidateByPattern(String pattern) {
        Set<String> keys = redisTemplate.keys(pattern);
        if (keys != null && !keys.isEmpty()) {
            redisTemplate.delete(keys);
        }
    }
}

Cache Performance Considerations

1. Read Throughput: Optimized for high read volume with minimal latency
2. Memory Usage: Configured with appropriate memory limits and eviction policies
3. Network Latency: Redis cluster with proper network configuration for low latency
4. Cache Hit Ratio: Target 85%+ hit ratio for optimal performance

Cache Monitoring and Metrics

We'll track:

• Cache hit/miss ratios
• Cache size and memory usage
• Eviction rates
• Cache latency distributions
• Error rates in cache operations

This caching design provides a robust foundation for performance optimization while maintaining data consistency and system reliability. The combination of cache-aside pattern with LRU eviction provides the right balance between simplicity and performance for our system requirements.

8. Repository / Data Access Layer

Repository Pattern Implementation

The repository pattern is implemented to abstract data access logic and provide a clean separation between domain objects and persistence concerns. Each entity will have its own repository interface and implementation.

// Repository interface for Order entity
public interface OrderRepository {
    Order save(Order order);
    Optional<Order> findById(UUID orderId);
    List<Order> findByUserId(UUID userId);
    void delete(UUID orderId);
    Page<Order> findAll(Pageable pageable);
}

// Repository implementation with JPA
@Repository
public class OrderJpaRepository implements OrderRepository {
    private final EntityManager entityManager;
    private final OrderEntityMapper orderEntityMapper;

    public OrderJpaRepository(EntityManager entityManager, OrderEntityMapper orderEntityMapper) {
        this.entityManager = entityManager;
        this.orderEntityMapper = orderEntityMapper;
    }

    @Override
    @Transactional
    public Order save(Order order) {
        OrderEntity entity = orderEntityMapper.toEntity(order);
        entityManager.persist(entity);
        entityManager.flush(); // Ensure immediate persistence
        return orderEntityMapper.toDomain(entity);
    }

    @Override
    public Optional<Order> findById(UUID orderId) {
        OrderEntity entity = entityManager.find(OrderEntity.class, orderId);
        return Optional.ofNullable(entity).map(orderEntityMapper::toDomain);
    }

    @Override
    public List<Order> findByUserId(UUID userId) {
        TypedQuery<OrderEntity> query = entityManager.createQuery(
            "SELECT o FROM OrderEntity o WHERE o.userId = :userId", 
            OrderEntity.class);
        query.setParameter("userId", userId);
        return query.getResultStream()
                   .map(orderEntityMapper::toDomain)
                   .collect(Collectors.toList());
    }

    @Override
    @Transactional
    public void delete(UUID orderId) {
        OrderEntity entity = entityManager.find(OrderEntity.class, orderId);
        if (entity != null) {
            entityManager.remove(entity);
        }
    }

    @Override
    public Page<Order> findAll(Pageable pageable) {
        TypedQuery<OrderEntity> query = entityManager.createQuery(
            "SELECT o FROM OrderEntity o ORDER BY o.createdAt DESC", 
            OrderEntity.class);
        
        query.setFirstResult(pageable.getOffset());
        query.setMaxResults(pageable.getPageSize());
        
        List<OrderEntity> entities = query.getResultList();
        long total = countAll();
        
        return new PageImpl<>(
            entities.stream().map(orderEntityMapper::toDomain).collect(Collectors.toList()),
            pageable,
            total
        );
    }

    private long countAll() {
        TypedQuery<Long> countQuery = entityManager.createQuery(
            "SELECT COUNT(o) FROM OrderEntity o", 
            Long.class);
        return countQuery.getSingleResult();
    }
}

ORM vs Raw Query Decisions

For this implementation, we're using JPA/Hibernate as the ORM layer because:

1. Provides type safety and compile-time validation
2. Handles complex object mapping automatically
3. Supports lazy loading and eager fetching strategies
4. Allows for easy integration with Spring Data JPA repositories

However, for performance-critical queries like analytics or bulk operations, raw SQL will be used with @NamedNativeQuery or direct JDBC.

Read vs Write Separation

We implement CQRS pattern for read and write operations:

// Read-only repository for analytics queries
@Repository
public class OrderAnalyticsRepository {
    private final EntityManager entityManager;

    public OrderAnalyticsRepository(EntityManager entityManager) {
        this.entityManager = entityManager;
    }

    @Query("SELECT new com.example.dto.OrderSummary(o.userId, COUNT(o), SUM(o.totalAmount)) " +
           "FROM OrderEntity o GROUP BY o.userId")
    public List<OrderSummary> getOrdersByUserGrouped() {
        // Implementation for read-heavy analytics
    }
}

// Write repository for business operations
@Repository
public class OrderWriteRepository {
    private final EntityManager entityManager;

    public OrderWriteRepository(EntityManager entityManager) {
        this.entityManager = entityManager;
    }

    @Transactional
    public Order save(Order order) {
        // Write-specific logic with proper transaction handling
        return entityManager.merge(order);
    }
}

Connection Pool Configuration

# application.yml
spring:
  datasource:
    hikari:
      maximum-pool-size: 20
      minimum-idle: 5
      connection-timeout: 30000
      idle-timeout: 600000
      max-lifetime: 1800000
      validation-timeout: 5000
      leak-detection-threshold: 60000

N+1 Query Prevention Strategy

To prevent N+1 query problems, we use eager fetching and batch loading:

// Using JOIN FETCH to prevent N+1 queries
@Query("SELECT o FROM OrderEntity o " +
       "JOIN FETCH o.items i " +
       "JOIN FETCH i.product p " +
       "WHERE o.userId = :userId")
List<OrderEntity> findByUserIdWithItems(@Param("userId") UUID userId);

// Batch loading for collections
@Query("SELECT i FROM OrderItemEntity i " +
       "WHERE i.order.id IN :orderIds")
List<OrderItemEntity> findItemsByOrderIds(@Param("orderIds") List<UUID> orderIds);

Batch Operation Design

For bulk operations, we implement batch processing:

@Repository
public class OrderBatchRepository {
    private final EntityManager entityManager;

    public OrderBatchRepository(EntityManager entityManager) {
        this.entityManager = entityManager;
    }

    @Transactional
    public void batchSave(List<Order> orders) {
        int batchSize = 50;
        for (int i = 0; i < orders.size(); i += batchSize) {
            int end = Math.min(i + batchSize, orders.size());
            List<Order> batch = orders.subList(i, end);
            
            // Convert to entities and persist in batch
            List<OrderEntity> entities = batch.stream()
                .map(orderEntityMapper::toEntity)
                .collect(Collectors.toList());
            
            for (OrderEntity entity : entities) {
                entityManager.persist(entity);
            }
            
            // Flush and clear to prevent memory issues
            entityManager.flush();
            entityManager.clear();
        }
    }
}

9. Asynchronous Processing & Messaging

Event-driven boundaries in this feature

The factory pattern implementation is primarily a synchronous design pattern focused on object creation. However, in a production system, factory implementations may need to integrate with asynchronous workflows for:

1. Event-driven object creation: When a new product type is registered, notify downstream systems (e.g., inventory, pricing, catalog services)
2. Batch processing: When multiple products are created in bulk, process them asynchronously to avoid blocking the main request
3. Audit logging: Track creation events for compliance and monitoring

For this design, we'll focus on the event-driven aspect of factory creation to demonstrate how asynchronous processing can be integrated.

Message schema design

{
  "version": "1.0",
  "event_type": "product.created",
  "timestamp": "2023-10-27T14:30:00Z",
  "source": "factory-service",
  "payload": {
    "product_id": "uuid-12345",
    "product_type": "car",
    "created_by": "user-98765",
    "metadata": {
      "brand": "Toyota",
      "model": "Camry"
    }
  },
  "correlation_id": "trace-abcde"
}

Dead letter queue strategy and poison message handling

For factory creation events, we implement a dead-letter queue (DLQ) strategy:

1. Retry mechanism: Events are retried up to 3 times with exponential backoff (1s, 2s, 4s)
2. DLQ handling: Events that fail after retries are moved to DLQ with metadata:
   • Original event payload
   • Failure reason and stack trace
   • Retry count and timestamps
3. Manual reprocessing: DLQ messages are flagged for manual inspection and potential reprocessing

At-least-once vs. exactly-once delivery

We use at-least-once delivery for factory creation events because:

1. Product creation is idempotent - creating the same product twice doesn't harm system state
2. Event consumers can handle duplicate events gracefully
3. Simpler implementation vs. exactly-once complexity

Retry policy for consumers

public class EventRetryPolicy {
    private static final int MAX_RETRIES = 3;
    private static final long BASE_DELAY_MS = 1000;
    private static final double BACKOFF_MULTIPLIER = 2.0;
    
    public boolean shouldRetry(int retryCount, Exception exception) {
        if (retryCount >= MAX_RETRIES) return false;
        
        // Don't retry on fatal errors
        if (exception instanceof IllegalArgumentException || 
            exception instanceof IllegalStateException) {
            return false;
        }
        
        return true;
    }
    
    public long getDelay(int retryCount) {
        return (long) (BASE_DELAY_MS * Math.pow(BACKOFF_MULTIPLIER, retryCount));
    }
}

Outbox pattern for reliable event publishing

We implement the outbox pattern to ensure event publishing is atomic with database operations:

1. When a product is created via factory, the event is first persisted to an event_outbox table
2. A background worker processes the outbox entries and publishes to message broker
3. Events are marked as published once successfully delivered
4. Failed events are retried via the DLQ strategy

Sequence diagram for async flows

sequenceDiagram
    participant Client
    participant FactoryService
    participant Database
    participant EventOutbox
    participant MessageBroker
    participant ConsumerService
    
    Client->>FactoryService: CreateProductRequest(productType)
    FactoryService->>Database: SaveProduct(product)
    FactoryService->>EventOutbox: InsertEvent(productCreated)
    FactoryService-->>Client: ProductCreatedResponse
    EventOutbox->>MessageBroker: PublishEvent(event)
    MessageBroker->>ConsumerService: DeliverEvent
    ConsumerService->>Database: ProcessEvent(event)

10. Error Handling Design

Error Classification Hierarchy

The error handling design follows a three-tier classification system:

1. Domain Errors - Business logic violations (e.g., insufficient funds, invalid state transitions)
2. Infrastructure Errors - System-level issues (e.g., database connection failures, timeout errors)
3. Validation Errors - Input validation issues (e.g., missing required fields, malformed data)

public abstract class BaseException extends RuntimeException {
    private final String errorCode;
    private final int httpStatus;

    public BaseException(String errorCode, int httpStatus, String message) {
        super(message);
        this.errorCode = errorCode;
        this.httpStatus = httpStatus;
    }

    // Getters
}

public class DomainException extends BaseException {
    public DomainException(String errorCode, String message) {
        super(errorCode, 400, message);
    }
}

public class InfrastructureException extends BaseException {
    public InfrastructureException(String errorCode, String message) {
        super(errorCode, 500, message);
    }
}

public class ValidationException extends BaseException {
    public ValidationException(String errorCode, String message) {
        super(errorCode, 400, message);
    }
}

Error Propagation Strategy

Errors are handled at the appropriate layer:

• Service Layer: Business logic errors are wrapped and re-thrown
• Controller Layer: Infrastructure and validation errors are caught and converted to appropriate HTTP responses
• Global Exception Handler: Unhandled exceptions bubble up to centralized error handling

Retry Strategy with Exponential Backoff and Jitter

For transient infrastructure errors, a retry mechanism is implemented:

public class RetryableOperation {
    private static final int MAX_RETRIES = 3;
    private static final long BASE_DELAY_MS = 1000;
    private static final double JITTER_FACTOR = 0.2;

    public <T> T executeWithRetry(Supplier<T> operation) {
        int attempts = 0;
        while (true) {
            try {
                return operation.get();
            } catch (InfrastructureException e) {
                attempts++;
                if (attempts >= MAX_RETRIES) {
                    throw e;
                }
                
                long delay = (long) (BASE_DELAY_MS * Math.pow(2, attempts - 1));
                delay = (long) (delay * (1 + JITTER_FACTOR * (Math.random() - 0.5)));
                
                try {
                    Thread.sleep(delay);
                } catch (InterruptedException ie) {
                    Thread.currentThread().interrupt();
                    throw new InfrastructureException("Retry interrupted", "Operation interrupted");
                }
            }
        }
    }
}

Circuit Breaker Design

For external service calls, a circuit breaker pattern is implemented:

public class CircuitBreaker {
    private final int failureThreshold;
    private final long timeout;
    private final int successThreshold;
    
    private volatile CircuitState state = CircuitState.CLOSED;
    private volatile int failureCount = 0;
    private volatile long lastFailureTime = 0;

    public enum CircuitState {
        CLOSED, OPEN, HALF_OPEN
    }

    public <T> T execute(Supplier<T> operation) {
        switch (state) {
            case CLOSED:
                return executeClosed(operation);
            case OPEN:
                return executeOpen(operation);
            case HALF_OPEN:
                return executeHalfOpen(operation);
            default:
                throw new IllegalStateException("Unknown circuit state");
        }
    }

    private <T> T executeClosed(Supplier<T> operation) {
        try {
            T result = operation.get();
            failureCount = 0; // Reset on success
            return result;
        } catch (Exception e) {
            failureCount++;
            lastFailureTime = System.currentTimeMillis();
            if (failureCount >= failureThreshold) {
                state = CircuitState.OPEN;
            }
            throw e;
        }
    }

    private <T> T executeOpen(Supplier<T> operation) {
        if (System.currentTimeMillis() - lastFailureTime > timeout) {
            state = CircuitState.HALF_OPEN;
            return executeHalfOpen(operation);
        }
        throw new InfrastructureException("Circuit breaker open", "Service unavailable");
    }

    private <T> T executeHalfOpen(Supplier<T> operation) {
        try {
            T result = operation.get();
            state = CircuitState.CLOSED;
            failureCount = 0;
            return result;
        } catch (Exception e) {
            state = CircuitState.OPEN;
            throw e;
        }
    }
}

Fallback Behavior Design

Fallbacks are implemented for critical external dependencies:

public class ExternalServiceFallback {
    private final CircuitBreaker circuitBreaker;

    public String getExternalData(String id) {
        return circuitBreaker.execute(() -> {
            // Attempt external service call
            return externalService.getData(id);
        });
    }

    public String getExternalDataWithFallback(String id) {
        try {
            return circuitBreaker.execute(() -> externalService.getData(id));
        } catch (InfrastructureException e) {
            // Fallback to default or cached data
            return "default_" + id;
        }
    }
}

Error Logging Format and Alerting Thresholds

Structured logging is implemented with consistent error fields:

public class ErrorLogger {
    private static final Logger logger = LoggerFactory.getLogger(ErrorLogger.class);

    public void logError(String operation, String errorCode, Exception ex, 
                        Map<String, Object> additionalContext) {
        Map<String, Object> logContext = new HashMap<>();
        logContext.put("operation", operation);
        logContext.put("errorCode", errorCode);
        logContext.put("timestamp", Instant.now());
        logContext.put("exceptionType", ex.getClass().getSimpleName());
        logContext.put("errorMessage", ex.getMessage());
        logContext.putAll(additionalContext);

        logger.error("Error occurred in operation: {}", operation, ex);
    }
}

Alerting thresholds:

• Critical: > 100 errors per minute for any error code
• Warning: > 50 errors per minute for any error code
• Info: Any error with errorCode containing "validation" or "domain"

Standardized Error Response Format

public class ErrorResponse {
    private final String timestamp;
    private final String errorCode;
    private final String message;
    private final int httpStatus;
    private final List<String> details;

    public ErrorResponse(String errorCode, String message, int httpStatus) {
        this.timestamp = Instant.now().toString();
        this.errorCode = errorCode;
        this.message = message;
        this.httpStatus = httpStatus;
        this.details = new ArrayList<>();
    }

    // Getters and setters
}

11. Security Design

Input Validation and Sanitization Strategy

All API inputs are validated using a combination of:

• Apache Commons Validator for standard formats (email, URL, phone)
• Custom regex patterns for domain-specific constraints (e.g., product SKU format)
• Length and character set restrictions to prevent injection attacks
• HTML sanitization using OWASP Java HTML Sanitizer for user-generated content

public class InputValidator {
    private static final Pattern EMAIL_PATTERN = 
        Pattern.compile("^[A-Za-z0-9+_.-]+@([A-Za-z0-9.-]+\\.[A-Za-z]{2,})$");
    
    public static boolean isValidEmail(String email) {
        return email != null && EMAIL_PATTERN.matcher(email).matches();
    }
    
    public static String sanitizeHtml(String input) {
        if (input == null) return null;
        return HtmlSanitizer.sanitize(input);
    }
}

SQL Injection Prevention

All database queries use parameterized prepared statements exclusively:

• JPA/Hibernate with named parameters for complex queries
• MyBatis with parameterized SQL mapping
• Custom query builders that enforce parameterization

// Safe usage - parameterized query
@Query("SELECT u FROM User u WHERE u.email = :email AND u.status = :status")
List<User> findByEmailAndStatus(@Param("email") String email, @Param("status") UserStatus status);

// Unsafe - never used
// String query = "SELECT * FROM users WHERE email = '" + email + "'";

Authentication Middleware Design

Authentication is handled through a centralized filter chain:

• JWT-based authentication for API clients and web sessions
• OAuth2 integration with external providers (Google, GitHub)
• API key validation for service-to-service communication
• Session management with secure cookie handling (HttpOnly, SameSite, Secure flags)

@Component
public class JwtAuthenticationFilter extends OncePerRequestFilter {
    @Override
    protected void doFilterInternal(HttpServletRequest request, 
                                   HttpServletResponse response, 
                                   FilterChain filterChain) throws ServletException, IOException {
        String token = extractToken(request);
        if (token != null && jwtService.validateToken(token)) {
            String username = jwtService.getUsernameFromToken(token);
            UsernamePasswordAuthenticationToken auth = 
                new UsernamePasswordAuthenticationToken(username, null, getAuthorities(token));
            SecurityContextHolder.getContext().setAuthentication(auth);
        }
        filterChain.doFilter(request, response);
    }
}

Authorization Enforcement

Role-Based Access Control (RBAC) is implemented at service layer:

• Permission-based checks using Spring Security annotations
• Resource-level authorization with custom @PreAuthorize expressions
• Dynamic role assignment based on user context and business logic

@PreAuthorize("hasRole('ADMIN') or @userPermissionService.canAccessUser(principal, userId)")
public void updateUserProfile(@AuthenticationPrincipal User currentUser, String userId, UserProfile profile) {
    // Implementation
}

Sensitive Data Encryption

• At-rest encryption: AES-256 with AWS KMS for database and file storage
• In-transit encryption: TLS 1.3 with HSTS header enforcement
• Key rotation policy: 90-day automatic rotation with key versioning

@Service
public class DataEncryptionService {
    private final AesGcmEncryptor encryptor;
    
    public String encrypt(String plaintext) {
        return encryptor.encrypt(plaintext, "user-data-key");
    }
    
    public String decrypt(String ciphertext) {
        return encryptor.decrypt(ciphertext, "user-data-key");
    }
}

PII Handling and Masking Strategy

• PII data classification: Email, phone, SSN, address, name
• Masking rules:
  • Email: "john.doe@example.com" → "j***.d***@example.com"
  • Phone: "+1-555-123-4567" → "+1-555-*-**"
  • SSN: "123-45-6789" → "--***"
• Audit logging of PII access with masking in logs

Secrets Management

• AWS Secrets Manager for storing database credentials, API keys, and encryption keys
• Rotation policy: 90-day automatic rotation with versioned secrets
• Access control: IAM policies restricting secret access to specific services only

Audit Logging Design

• Event types: Login, data access, modification, deletion, system events
• Log format:

  {
    "timestamp": "2023-10-15T14:30:00Z",
    "user_id": "user-123",
    "action": "UPDATE_PROFILE",
    "ip_address": "192.168.1.100",
    "user_agent": "Mozilla/5.0...",
    "data_changed": {
      "email": "johndoe@example.com"
    },
    "session_id": "sess-abc123"
  }

• Retention policy: 90 days for audit logs, 1 year for security events
• Secure storage: Logs written to encrypted cloud storage with access logging

12. Testing Strategy

Unit Testing Scope and Coverage Targets

Unit tests will cover 85%+ code coverage for business logic, with specific focus on:

• Service layer methods (90%+ coverage)
• Core domain object operations (95%+ coverage)
• Factory pattern implementation and registration logic (100% coverage)
• Error handling paths within individual methods (100% coverage)

Integration Testing Strategy

Database Integration Tests:

• Use testcontainers for PostgreSQL/MySQL integration testing
• Mock external services (payment gateways, notification systems) using WireMock or Mockito
• Test transaction boundaries and data consistency across multiple operations

External Service Integration Tests:

• Use embedded Redis for cache integration testing
• Test message broker interactions with embedded Kafka/RabbitMQ instances
• Validate event publishing and consumption workflows

Contract Testing

API Contract Testing:

• Implement Pact-based contract testing between frontend and backend services
• Test all API endpoints with defined request/response schemas
• Validate data format compatibility across service boundaries

External Service Contracts:

• Test integration points with third-party APIs (Stripe, Twilio) using contract tests
• Validate message format and payload structure for event-driven systems

Performance Testing Tool

Load and Stress Testing:

• Use k6 for performance testing with key scenarios:
  • Concurrent order placement (500+ concurrent users)
  • High-volume data retrieval operations
  • Factory pattern instantiation under load (10,000+ requests/sec)
• Test response time targets: p50 < 200ms, p95 < 500ms, p99 < 1000ms

Scalability Testing:

• Validate horizontal scaling capabilities with multiple service instances
• Test database connection pool behavior under concurrent load

Test Data Management Strategy

Test Data Generation:

• Use factory pattern for test data generation (similar to production domain objects)
• Implement data builders with default values and override capabilities
• Generate realistic test fixtures for all domain entities

Test Data Seeding:

• Use database migrations for initial test data setup
• Implement test data cleanup strategies to maintain test isolation
• Use in-memory databases for fast test execution and parallel test runs

Test Data Isolation:

• Use separate test databases or schema namespaces
• Implement database transaction rollback for each test case
• Ensure no cross-contamination between parallel test executions

Test Data Validation:

• Validate data integrity constraints in test environments
• Test edge case scenarios with boundary values and invalid inputs
• Implement data consistency checks after each test operation

13. Observability Design

Structured Logging Schema

All logs follow a structured JSON format with consistent fields to enable automated parsing and analysis:

{
  "timestamp": "2023-10-27T14:30:45.123Z",
  "level": "INFO",
  "service": "booking-service",
  "trace_id": "1234567890abcdef1234567890abcdef",
  "span_id": "abcdef1234567890abcdef1234567890",
  "user_id": "user-123",
  "entity_id": "booking-456",
  "operation": "createBooking",
  "request_id": "req-789",
  "message": "Booking created successfully",
  "duration_ms": 150,
  "error_code": null,
  "error_message": null,
  "stack_trace": null
}

PII masking rules:

• User IDs and entity IDs are logged as hashed values when not required for debugging
• Personal data fields (email, phone) are completely redacted in logs

Distributed Tracing Instrumentation

Using OpenTelemetry with custom spans for key operations:

// Example: Tracing a booking creation flow
Span span = tracer.spanBuilder("createBooking")
    .setAttribute("booking.id", bookingId)
    .setAttribute("user.id", userId)
    .startSpan();

try (Scope scope = span.makeCurrent()) {
    // Business logic here
    bookingService.createBooking(bookingRequest);
    
    span.addEvent("booking.created");
} catch (Exception e) {
    span.recordException(e);
    throw e;
} finally {
    span.end();
}

Key operation spans to capture:

• createBooking - includes user, booking ID, duration
• validatePayment - payment method, amount, success/failure
• sendConfirmationEmail - email address, template ID

Metrics Design

RED Metrics (Rate, Errors, Duration) with appropriate labels:

Metric Name	Type	Labels	Description
booking_requests_total	Counter	{status="success", status="error", operation="create"}	Total booking creation requests
booking_creation_duration_seconds	Histogram	{operation="create", status="success"}	Time to create booking
booking_errors_total	Counter	{error_type="validation", error_type="payment", error_type="internal"}	Count of booking creation errors
active_bookings	Gauge	{status="pending", status="confirmed"}	Current active bookings

Health Check Endpoint Design

@RestController
public class HealthController {
    
    @GetMapping("/health")
    public ResponseEntity<Map<String, Object>> health() {
        Map<String, Object> response = new HashMap<>();
        response.put("status", "healthy");
        response.put("timestamp", Instant.now());
        response.put("service", "booking-service");
        
        // Check database connectivity
        boolean dbHealthy = dataSourceHealthIndicator.isHealthy();
        response.put("database", dbHealthy ? "healthy" : "unhealthy");
        
        // Check cache connectivity
        boolean cacheHealthy = redisHealthIndicator.isHealthy();
        response.put("cache", cacheHealthy ? "healthy" : "unhealthy");
        
        return ResponseEntity.ok(response);
    }
}

Health check endpoints:

• /health - Overall service health (database, cache)
• /ready - Service ready to serve requests (dependencies satisfied)
• /live - Process is alive and responding
• /startup - Startup completion status

Alerting Thresholds

Metric	Warning Level	Critical Level	Escalation
booking_errors_total (5m)	> 10 errors/sec	> 50 errors/sec	On-call engineer
booking_creation_duration_seconds (p99)	> 500ms	> 2s	On-call engineer
active_bookings	> 100k	> 200k	SRE team
database_connection_pool	< 10 free connections	< 5 free connections	SRE team

Alerts are configured with:

• 30-second evaluation intervals
• 5-minute alerting window for stability
• Slack notifications with severity levels (info, warning, critical)
• Automatic incident creation in Jira for critical alerts

14. Open Questions, Risks & Trade-offs

Decisions requiring product / engineering alignment

• Factory vs. Service Locator pattern: The choice between a simple factory and a service locator pattern requires alignment on whether we want to centralize object creation logic or provide more flexible runtime resolution of dependencies. The factory pattern is simpler and more testable, while service locator provides runtime flexibility but can obscure dependencies.

• Type registration strategy: Should factory types be registered at startup via configuration files, annotations, or through a centralized registry? This impacts deployment flexibility and runtime performance.

• Inheritance vs. composition in factory implementation: Whether to use inheritance or composition for the factory base class requires engineering consensus on code reuse patterns and extensibility requirements.

Known trade-offs and justifications

• Performance vs. flexibility: The factory pattern adds a layer of indirection that can slightly impact performance, but provides flexibility to switch implementations at runtime or during configuration. For high-throughput systems, this cost may be unacceptable.

• Complexity vs. maintainability: While the factory pattern reduces coupling between client code and concrete implementations, it introduces additional complexity in the form of factory classes and registration logic. For simple use cases with few implementations, this added complexity may not be justified.

• Testability vs. test overhead: Factory patterns make unit testing easier by allowing easy substitution of implementations, but they also require additional test coverage for factory logic itself.

Risks and mitigation strategies

• Factory state management: If the factory maintains internal state (e.g., caches of created instances), it introduces risks of memory leaks or inconsistent state. Mitigation: Use dependency injection frameworks that handle lifecycle management, or implement proper cleanup mechanisms.

• Runtime type resolution failures: If factory registration is incomplete or incorrect, runtime errors may occur when trying to create unknown types. Mitigation: Implement comprehensive validation during application startup, use a fallback strategy with default implementations, and provide clear error messages.

• Thread safety in concurrent environments: Factory methods may not be thread-safe if they maintain shared mutable state. Mitigation: Use synchronized blocks or concurrent collections, implement proper locking strategies, and design factory methods to be stateless where possible.

Technical debt acknowledged

• Hardcoded type mappings: Current implementation uses hardcoded string-to-class mappings which are brittle and require code changes for new types. Future improvement: Use configuration-driven or annotation-based registration.

• Limited extensibility: The current factory design assumes all implementations are known at compile time. For truly dynamic systems, we may need to support plugin architectures or dynamic class loading.

• No built-in lifecycle management: The factory does not handle object lifecycle management (e.g., cleanup, disposal). This may cause resource leaks in long-running applications.

• Error handling granularity: Factory error handling is basic, catching generic exceptions and returning null or default instances. Better approach would be to provide more specific error types with contextual information for debugging.

• No support for parameterized creation: The current factory pattern supports only simple object creation. For complex scenarios requiring constructor parameters, we'd need to extend the factory interface with parameterized creation methods.