Image Processing Pipeline Design and Implementation

This document presents a production-ready low-level design for an image processing pipeline that efficiently handles image uploads, applies transformations, and delivers optimized images at scale.

1. Problem Statement & Requirements

Clarifying questions

1. What scale and throughput requirements are expected?

   • Expected concurrent image uploads per second (e.g., 1000, 10k)?
   • Peak hourly image processing volume?

2. What consistency and latency requirements exist?

   • What is the target p50/p95/p99 latency for image processing (e.g., 200ms, 1s)?
   • Are there strict SLAs for image availability or processing completion?

3. What are the key entities and their relationships?

   • Are images stored with metadata, user ownership, or tags?
   • Do users have roles (admin, regular) that affect processing permissions?

4. What is the read/write ratio and access pattern?

   • Is image retrieval more frequent than uploads?
   • Are there batch processing or bulk operations?

5. How sensitive is the system to latency?

   • Are some image formats or transformations more time-sensitive than others?
   • Is there a need for real-time processing (e.g., live video thumbnails)?

6. What authentication and authorization model is in place?

   • Are users authenticated via OAuth2, JWT tokens, or API keys?
   • Do roles and permissions need to be enforced at the image level?

7. What external systems or integrations are expected?

   • Will the system integrate with cloud storage (S3, GCS)?
   • Are there third-party image processing services or APIs to leverage?

Functional requirements

• Accept image uploads via HTTP POST
• Transform uploaded images (resize, crop, rotate, filter)
• Store processed images in cloud storage or local filesystem
• Support multiple image formats (JPEG, PNG, GIF, WebP)
• Generate thumbnails for uploaded images
• Allow image metadata extraction (EXIF, dimensions, color space)
• Support batch image processing workflows
• Provide image retrieval via public or signed URLs
• Allow users to delete processed images

Non-functional requirements

• Latency targets:

  • p50: <200ms for basic transformations
  • p95: <1s for complex operations (e.g., filters, batch)
  • p99: <5s for large or multi-step operations

• Throughput:

  • Handle up to 10,000 concurrent image uploads per second
  • Support batch processing of up to 100 images in parallel

• Availability:

  • 99.9% uptime for image processing and retrieval services
  • SLA: 99.5% availability for image storage (e.g., S3 integration)

Constraints and assumptions

• Image size limits: Max 50MB per image file
• Supported formats: JPEG, PNG, GIF, WebP (with limitations)
• Assumption: Image processing will be CPU and memory intensive
• Assumption: Images are uploaded via HTTP or through direct file system access
• Assumption: Users have authenticated and authorized access to their images

Out of scope

• Image recognition or AI-based analysis (e.g., object detection)
• Real-time video processing or streaming
• Image compression algorithms beyond standard format support
• Multi-tenant image storage (e.g., per organization or account)
• Image versioning or rollback features
• Image search or indexing capabilities

graph TD
    A[Image Upload] --> B{Validate Format}
    B -->|Valid| C[Extract Metadata]
    C --> D[Process Image]
    D --> E{Transform Type}
    E -->|Resize/Crop| F[Save Resized Image]
    E -->|Filter| G[Apply Filter]
    F --> H[Store in Cloud]
    G --> H
    H --> I[Generate Thumbnail]
    I --> J[Return Public URL]

2. API Design

API Style Selection

gRPC with Protocol Buffers is selected for the image processing pipeline due to:

• Binary serialization provides better performance and smaller payload sizes than JSON
• Strong typing with protobuf definitions enables robust contract validation
• Built-in support for streaming and bidirectional communication for large file transfers
• Native support for versioning through protobuf package namespaces
• Efficient handling of binary image data streams

Endpoint Definitions

Method	Path	Description	Auth	Rate Limit
POST	/v1/images/upload	Upload image file for processing	JWT	100/hour
GET	/v1/images/{id}	Retrieve processed image metadata	JWT	1000/hour
GET	/v1/images/{id}/download	Download processed image file	JWT	500/hour
POST	/v1/images/{id}/process	Trigger manual image processing	Admin JWT	50/hour
DELETE	/v1/images/{id}	Delete image and associated data	JWT	20/hour
GET	/v1/images	List user's processed images	JWT	500/hour
POST	/v1/images/batch/process	Process multiple images in batch	Admin JWT	20/hour

Request and Response Schemas

Upload Image Request

{
  "file": "binary_data",
  "metadata": {
    "name": "sample.jpg",
    "description": "Sample image for processing",
    "tags": ["nature", "landscape"]
  }
}

Image Metadata Response

{
  "id": "uuid-12345",
  "name": "sample.jpg",
  "originalSize": 1024000,
  "processedSize": 512000,
  "format": "JPEG",
  "dimensions": {
    "width": 1920,
    "height": 1080
  },
  "status": "processed",
  "createdAt": "2023-10-01T10:00:00Z",
  "updatedAt": "2023-10-01T10:05:00Z",
  "processedAt": "2023-10-01T10:05:00Z",
  "thumbnails": [
    {
      "size": "small",
      "url": "/images/uuid-12345/thumbnail/small.jpg"
    }
  ]
}

Batch Processing Request

{
  "imageIds": ["uuid-12345", "uuid-67890"],
  "processingOptions": {
    "resize": {
      "width": 800,
      "height": 600
    },
    "format": "WEBP"
  }
}

HTTP Status Codes and Error Format

Standard HTTP Status Codes:

• 200 OK - Successful GET, PUT, PATCH
• 201 Created - Successful POST (resource created)
• 400 Bad Request - Invalid request body or parameters
• 401 Unauthorized - Missing or invalid auth token
• 403 Forbidden - Insufficient permissions
• 404 Not Found - Resource not found
• 429 Too Many Requests - Rate limit exceeded
• 500 Internal Server Error - Unexpected server error

Standardized Error Response Format:

{
  "error": {
    "code": "IMAGE_PROCESSING_ERROR",
    "message": "Failed to process image due to invalid dimensions",
    "details": {
      "field": "width",
      "value": 2000,
      "reason": "Exceeds maximum allowed width of 1920"
    },
    "timestamp": "2023-10-01T10:00:00Z",
    "requestId": "req-abc123"
  }
}

API Versioning Strategy

Versioning follows semantic versioning with path-based versioning:

• /v1/images/upload - Current stable API
• Future: /v2/images/upload for breaking changes

Versioning strategy ensures backward compatibility while allowing for future enhancements.

Pagination Design

Cursor-based pagination is used because:

• More consistent and predictable than offset-based pagination
• Performs better with large datasets
• Handles data changes during pagination gracefully
• Eliminates the need to recompute offsets when items are inserted/deleted

Pagination Response Format:

{
  "data": [/* image metadata array */],
  "pagination": {
    "cursor": "eyJpZCI6IjEyMzQ1In0=",
    "hasNext": true,
    "hasPrev": false
  }
}

Authentication and Authorization

Authentication:

• JWT tokens with HS256 signing algorithm
• Token expiration set to 24 hours
• Refresh token support for long-lived sessions

Authorization:

• JWT claims validation (user_id, roles)
• RBAC model with roles: USER, ADMIN
• Admin endpoints require ADMIN role
• User can only access their own images

Rate Limiting Strategy

Rate Limiting Per Endpoint:

• Upload: 100 requests/hour
• Download: 500 requests/hour
• List Images: 500 requests/hour
• Image Processing: 50 requests/hour (admin only)
• Batch Processing: 20 requests/hour (admin only)

Implementation Strategy:

• Token bucket algorithm for fairness
• Distributed rate limiting using Redis
• Per-user rate limits with sliding window

Idempotency Design

Idempotency Key Strategy:

• X-Idempotency-Key header for all mutating operations
• Key format: {user_id}:{operation_type}:{unique_identifier}
• Cache key TTL of 1 hour
• For duplicate requests, return cached response with same status code

Example Idempotency Key:

X-Idempotency-Key: user_12345:upload:img_abc123

3. Data Model Design

Entity-Relationship Diagram

erDiagram
    IMAGE {
        string id PK
        string original_filename
        string storage_key
        string mime_type
        int width
        int height
        int size_bytes
        string status
        string user_id
        timestamp created_at
        timestamp updated_at
        timestamp deleted_at
    }

    IMAGE_PROCESSING_JOB {
        string id PK
        string image_id FK
        string status
        string processor_type
        string output_format
        int width
        int height
        timestamp created_at
        timestamp updated_at
        timestamp started_at
        timestamp completed_at
        string error_message
    }

    IMAGE_PROCESSING_RESULT {
        string id PK
        string job_id FK
        string image_id FK
        string storage_key
        string mime_type
        int width
        int height
        int size_bytes
        timestamp created_at
    }

    USER {
        string id PK
        string email
        string role
        timestamp created_at
    }

    IMAGE ||--o{ IMAGE_PROCESSING_JOB : has
    IMAGE_PROCESSING_JOB ||--o{ IMAGE_PROCESSING_RESULT : produces
    USER ||--o{ IMAGE : owned_by

Table Schema Definitions

images table

CREATE TABLE images (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    original_filename VARCHAR(255) NOT NULL,
    storage_key VARCHAR(512) NOT NULL UNIQUE,
    mime_type VARCHAR(100) NOT NULL,
    width INTEGER,
    height INTEGER,
    size_bytes INTEGER NOT NULL,
    status VARCHAR(50) NOT NULL DEFAULT 'pending',
    user_id UUID NOT NULL REFERENCES users(id),
    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
    deleted_at TIMESTAMP WITH TIME ZONE
);

CREATE INDEX idx_images_user_id ON images(user_id);
CREATE INDEX idx_images_status ON images(status);
CREATE INDEX idx_images_created_at ON images(created_at);
CREATE INDEX idx_images_storage_key ON images(storage_key);

image_processing_jobs table

CREATE TABLE image_processing_jobs (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    image_id UUID NOT NULL REFERENCES images(id) ON DELETE CASCADE,
    status VARCHAR(50) NOT NULL DEFAULT 'pending',
    processor_type VARCHAR(100) NOT NULL,
    output_format VARCHAR(50) NOT NULL,
    width INTEGER,
    height INTEGER,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
    started_at TIMESTAMP WITH TIME ZONE,
    completed_at TIMESTAMP WITH TIME ZONE,
    error_message TEXT
);

CREATE INDEX idx_image_processing_jobs_image_id ON image_processing_jobs(image_id);
CREATE INDEX idx_image_processing_jobs_status ON image_processing_jobs(status);
CREATE INDEX idx_image_processing_jobs_created_at ON image_processing_jobs(created_at);

image_processing_results table

CREATE TABLE image_processing_results (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    job_id UUID NOT NULL REFERENCES image_processing_jobs(id) ON DELETE CASCADE,
    image_id UUID NOT NULL REFERENCES images(id),
    storage_key VARCHAR(512) NOT NULL,
    mime_type VARCHAR(100) NOT NULL,
    width INTEGER,
    height INTEGER,
    size_bytes INTEGER NOT NULL,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
);

CREATE INDEX idx_image_processing_results_job_id ON image_processing_results(job_id);
CREATE INDEX idx_image_processing_results_image_id ON image_processing_results(image_id);

users table

CREATE TABLE users (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    email VARCHAR(255) NOT NULL UNIQUE,
    role VARCHAR(50) NOT NULL DEFAULT 'user',
    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
);

Primary Key Strategy

UUIDs are used for all primary keys to support:

• Distributed system compatibility
• Avoiding sequential ID predictability (security concern)
• Cross-database replication scenarios

Index Design

1. idx_images_user_id - Composite index on user_id to support user image listing queries
2. idx_images_status - For filtering images by processing status (pending, processed, failed)
3. idx_images_created_at - For time-based queries and pagination
4. idx_images_storage_key - Fast lookup by storage key for content delivery
5. idx_image_processing_jobs_image_id - Fast job lookup by image ID for status checks
6. idx_image_processing_jobs_status - Efficient job queue processing by status
7. idx_image_processing_results_job_id - Fast result lookup by job ID for processing completion
8. idx_image_processing_results_image_id - Fast image result lookup by image ID

Data Partitioning / Sharding Strategy

Partition by user_id hash to distribute load across shards:

• Hash(user_id) % NUMBER_OF_SHARDS
• Each shard contains a subset of users' images and processing jobs
• Sharding key is user_id to maintain data locality for user operations

Soft Delete vs. Hard Delete

Soft delete is used because:

• Audit trail required for compliance and debugging
• Image metadata needs to be preserved for analytics (e.g., processing history)
• Recovery capability needed in case of accidental deletion
• User data retention policies require historical tracking

Audit Trail Design

All mutations are logged in a separate audit table:

CREATE TABLE image_audit_log (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    image_id UUID NOT NULL,
    user_id UUID NOT NULL,
    operation VARCHAR(50) NOT NULL, -- 'CREATE', 'UPDATE', 'DELETE'
    changes JSONB,
    timestamp TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
);

Data Archival Strategy and Retention Policy

1. Active images: Keep in main tables for 30 days
2. Processed results: Keep for 1 year (for analytics and user access)
3. Processing jobs: Archive after 90 days
4. Archival process: Daily batch job that moves old records to archive storage (S3 or similar)
5. Retention policy:
   • Images: 30 days (soft delete) → 1 year (hard delete)
   • Processing jobs: 90 days (soft delete) → 1 year (hard delete)
   • Results: 1 year (soft delete) → 2 years (hard delete)

The system uses a combination of database soft deletes and periodic archival jobs to maintain performance while ensuring compliance with data retention requirements.

4. Class Design & Service Layer

classDiagram
    class ImageProcessorService {
        +processImageAsync(ImageRequest) CompletableFuture~ImageProcessingResult~
        +getImageMetadata(String) ImageMetadata
        +deleteImage(String)
    }
    
    class ImageRequest {
        +String id
        +String sourceUrl
        +List~ImageTransformation~ transformations
        +String callbackUrl
    }
    
    class ImageTransformation {
        +String type
        +Map~String, Object~ params
    }
    
    class ImageMetadata {
        +String id
        +String originalUrl
        +int width
        +int height
        +String format
        +long sizeBytes
        +Instant createdAt
    }
    
    class ImageProcessingResult {
        +String id
        +String processedUrl
        +String status
        +Instant completedAt
        +List~ImageTransformation~ transformations
    }
    
    class TransformationStrategy {
        +apply(ImageData, ImageTransformation) ImageData
    }
    
    class ResizeTransformation {
        -int width
        -int height
        +apply(ImageData, ImageTransformation) ImageData
    }
    
    class WatermarkTransformation {
        -String watermarkText
        +apply(ImageData, ImageTransformation) ImageData
    }
    
    class ImageStorageService {
        +storeImage(ImageData, String) String
        +getImageUrl(String) String
        +deleteImage(String)
    }
    
    class ImageData {
        +byte[] data
        +String format
        +int width
        +int height
    }
    
    class EventPublisher {
        +publishEvent(ImageProcessingEvent)
    }
    
    class ImageProcessingEvent {
        +String imageId
        +String status
        +String callbackUrl
    }
    
    class ImageProcessingSaga {
        +start(ImageRequest) void
        +handleSuccess(String, ImageProcessingResult) void
        +handleFailure(String, String) void
    }
    
    class ImageProcessingRepository {
        +save(ImageProcessingResult) void
        +findById(String) Optional~ImageProcessingResult~
        +updateStatus(String, String) void
    }
    
    ImageProcessorService --> ImageProcessingRepository : persists
    ImageProcessorService --> EventPublisher : publishes events
    ImageProcessorService --> ImageStorageService : stores images
    TransformationStrategy <|-- ResizeTransformation
    TransformationStrategy <|-- WatermarkTransformation
    ImageProcessorService --> TransformationStrategy : strategy pattern
    ImageProcessingSaga --> ImageProcessorService : orchestrates

stateDiagram-v2
    [*] --> Pending
    Pending --> Processing : process()
    Processing --> Success : success()
    Processing --> Failed : failure()
    Success --> [*]
    Failed --> [*]

Service Class Structure and Responsibilities

The core service layer for image processing is organized around several key responsibilities:

1. ImageProcessorService: Main orchestrator that handles image processing workflows, including transformation application and result publication.
2. ImageStorageService: Manages storage and retrieval of processed images in cloud storage (e.g., S3).
3. ImageProcessingSaga: Implements saga pattern for managing distributed image processing workflows with idempotency and error recovery.
4. TransformationStrategy: Strategy pattern implementation for applying various image transformations.

Interface / Abstract Class Design

public interface TransformationStrategy {
    ImageData apply(ImageData imageData, ImageTransformation transformation);
}

public abstract class BaseImageProcessorService {
    protected final ImageProcessingRepository repository;
    protected final ImageStorageService storageService;
    protected final EventPublisher eventPublisher;
    protected final List<TransformationStrategy> strategies;

    public BaseImageProcessorService(ImageProcessingRepository repository,
                                   ImageStorageService storageService,
                                   EventPublisher eventPublisher) {
        this.repository = repository;
        this.storageService = storageService;
        this.eventPublisher = eventPublisher;
        this.strategies = new ArrayList<>();
    }
}

Business Logic Encapsulation Patterns

The system uses the Strategy pattern for image transformations:

@Component
public class ResizeTransformation implements TransformationStrategy {
    private static final Logger logger = LoggerFactory.getLogger(ResizeTransformation.class);

    @Override
    public ImageData apply(ImageData imageData, ImageTransformation transformation) {
        try {
            int width = (Integer) transformation.getParams().get("width");
            int height = (Integer) transformation.getParams().get("height");

            // Use ImageIO or similar library to resize image
            BufferedImage originalImage = ImageIO.read(new ByteArrayInputStream(imageData.getData()));
            BufferedImage resizedImage = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);
            
            Graphics2D g2d = resizedImage.createGraphics();
            g2d.drawImage(originalImage, 0, 0, width, height, null);
            g2d.dispose();

            ByteArrayOutputStream baos = new ByteArrayOutputStream();
            ImageIO.write(resizedImage, imageData.getFormat(), baos);
            return new ImageData(baos.toByteArray(), imageData.getFormat(), width, height);

        } catch (IOException e) {
            throw new ImageProcessingException("Failed to resize image", e);
        }
    }
}

@Component
public class WatermarkTransformation implements TransformationStrategy {
    private static final Logger logger = LoggerFactory.getLogger(WatermarkTransformation.class);

    @Override
    public ImageData apply(ImageData imageData, ImageTransformation transformation) {
        try {
            String watermarkText = (String) transformation.getParams().get("text");
            
            // Apply watermark to image using Graphics2D
            BufferedImage originalImage = ImageIO.read(new ByteArrayInputStream(imageData.getData()));
            BufferedImage watermarkedImage = new BufferedImage(originalImage.getWidth(), originalImage.getHeight(), BufferedImage.TYPE_INT_ARGB);
            
            Graphics2D g2d = watermarkedImage.createGraphics();
            g2d.drawImage(originalImage, 0, 0, null);
            
            Font font = new Font("Arial", Font.BOLD, 24);
            g2d.setFont(font);
            g2d.setColor(Color.WHITE);
            g2d.setComposite(AlphaComposite.getInstance(AlphaComposite.SRC_OVER, 0.5f));
            
            FontMetrics fm = g2d.getFontMetrics();
            int x = (originalImage.getWidth() - fm.stringWidth(watermarkText)) / 2;
            int y = originalImage.getHeight() - fm.getHeight();
            
            g2d.drawString(watermarkText, x, y);
            g2d.dispose();

            ByteArrayOutputStream baos = new ByteArrayOutputStream();
            ImageIO.write(watermarkedImage, imageData.getFormat(), baos);
            return new ImageData(baos.toByteArray(), imageData.getFormat(), originalImage.getWidth(), originalImage.getHeight());

        } catch (IOException e) {
            throw new ImageProcessingException("Failed to apply watermark", e);
        }
    }
}

Critical-Path Method Implementation

public CompletableFuture<ImageProcessingResult> processImageAsync(ImageRequest request) {
    String imageId = UUID.randomUUID().toString();
    
    // Start transaction boundary
    return CompletableFuture.supplyAsync(() -> {
        try {
            // 1. Validate request and create processing record
            ImageProcessingResult result = new ImageProcessingResult();
            result.setId(imageId);
            result.setStatus("PENDING");
            result.setTransformations(request.getTransformations());
            
            repository.save(result);
            
            // 2. Fetch original image (with retry and timeout)
            String originalUrl = fetchOriginalImage(request.getSourceUrl());
            
            // 3. Apply transformations sequentially
            ImageData imageData = new ImageData();
            imageData.setData(fetchImageData(originalUrl));
            imageData.setFormat(detectImageFormat(imageData.getData()));
            
            List<ImageTransformation> transformations = request.getTransformations();
            for (ImageTransformation transformation : transformations) {
                TransformationStrategy strategy = findStrategy(transformation.getType());
                imageData = strategy.apply(imageData, transformation);
            }
            
            // 4. Store processed image
            String processedUrl = storageService.storeImage(imageData, imageId);
            
            // 5. Update result and publish event
            result.setProcessedUrl(processedUrl);
            result.setStatus("SUCCESS");
            result.setCompletedAt(Instant.now());
            
            repository.updateStatus(imageId, "SUCCESS");
            
            // 6. Publish event for async callback or notification
            ImageProcessingEvent event = new ImageProcessingEvent();
            event.setImageId(imageId);
            event.setStatus("SUCCESS");
            event.setCallbackUrl(request.getCallbackUrl());
            
            eventPublisher.publishEvent(event);
            
            return result;
            
        } catch (Exception e) {
            // Handle error and update status
            repository.updateStatus(imageId, "FAILED");
            
            ImageProcessingEvent event = new ImageProcessingEvent();
            event.setImageId(imageId);
            event.setStatus("FAILED");
            event.setCallbackUrl(request.getCallbackUrl());
            event.setError(e.getMessage());
            
            eventPublisher.publishEvent(event);
            
            throw new ImageProcessingException("Image processing failed for ID: " + imageId, e);
        }
    });
}

Transaction Boundary Design

The transaction boundary encompasses:

1. Creating processing record in database
2. Fetching original image from source URL (with retry logic)
3. Applying all transformations sequentially
4. Storing processed image in storage system
5. Updating processing status and publishing events

Service Interface Contracts

public interface ImageProcessingService {
    CompletableFuture<ImageProcessingResult> processImageAsync(ImageRequest request);
    ImageMetadata getImageMetadata(String imageId);
    void deleteImage(String imageId);
}

Saga / Orchestration Pattern

@Component
public class ImageProcessingSaga {
    private static final Logger logger = LoggerFactory.getLogger(ImageProcessingSaga.class);
    
    private final ImageProcessorService imageProcessor;
    private final ImageStorageService storageService;
    private final EventPublisher eventPublisher;

    public void start(ImageRequest request) {
        String imageId = UUID.randomUUID().toString();
        
        // Start saga with initial state
        ImageProcessingEvent event = new ImageProcessingEvent();
        event.setImageId(imageId);
        event.setStatus("STARTED");
        event.setCallbackUrl(request.getCallbackUrl());
        
        // Publish initial event
        eventPublisher.publishEvent(event);
        
        // Process asynchronously with retry logic
        imageProcessor.processImageAsync(request)
            .thenAccept(result -> {
                handleSuccess(imageId, result);
            })
            .exceptionally(throwable -> {
                handleFailure(imageId, throwable.getMessage());
                return null;
            });
    }

    private void handleSuccess(String imageId, ImageProcessingResult result) {
        // Update database and publish success event
        ImageProcessingEvent event = new ImageProcessingEvent();
        event.setImageId(imageId);
        event.setStatus("SUCCESS");
        event.setCallbackUrl(result.getCallbackUrl());
        
        eventPublisher.publishEvent(event);
    }

    private void handleFailure(String imageId, String errorMessage) {
        // Update database and publish failure event
        ImageProcessingEvent event = new ImageProcessingEvent();
        event.setImageId(imageId);
        event.setStatus("FAILED");
        event.setError(errorMessage);
        
        eventPublisher.publishEvent(event);
    }
}

Domain Object / Value Object Design

public class ImageRequest {
    private String id;
    private String sourceUrl;
    private List<ImageTransformation> transformations;
    private String callbackUrl;
    
    // Getters and setters
}

public class ImageTransformation {
    private String type;
    private Map<String, Object> params;
    
    // Getters and setters
}

public class ImageMetadata {
    private String id;
    private String originalUrl;
    private int width;
    private int height;
    private String format;
    private long sizeBytes;
    private Instant createdAt;
    
    // Getters and setters
}

public class ImageProcessingResult {
    private String id;
    private String processedUrl;
    private String status;
    private Instant completedAt;
    private List<ImageTransformation> transformations;
    
    // Getters and setters
}

5. Concurrency & Thread Safety

Concurrency Model

The image processing pipeline will utilize a thread pool model with a fixed-size thread pool for concurrent image processing tasks. Each worker thread will handle one image processing job at a time, ensuring thread safety through proper synchronization mechanisms and immutable data structures where possible.

Race Condition Prevention Strategy

Race conditions will be prevented through:

1. Immutable input data structures to avoid shared mutable state
2. Thread-local storage for intermediate processing artifacts
3. Synchronized access to shared resources like database connections and cache entries
4. Atomic operations for counters and status tracking

Optimistic vs. Pessimistic Locking Selection

Optimistic locking will be used for database operations due to:

• Low contention in image processing workflows
• High throughput requirements favoring minimal blocking
• Image metadata updates are infrequent and can be retried on conflict
• Implementation simplicity over complex locking mechanisms

Deadlock Prevention Strategy

Deadlock prevention will be achieved through:

1. Fixed ordering of resource acquisition - All database operations follow a consistent order (e.g., acquire image lock, then metadata lock)
2. Timeout-based acquisition - Set 5-second timeouts on database locks to prevent indefinite waiting
3. Hierarchical lock management - Locks acquired in predetermined order (image ID → metadata ID)
4. Avoid nested locks - Minimize scope of lock acquisition to single operations

Idempotency Implementation for Concurrent Duplicate Requests

Idempotency will be handled through:

1. Request deduplication - Use request ID or content hash to identify duplicate processing requests
2. Database-level uniqueness constraints - Prevent concurrent processing of same image by enforcing unique constraints on image IDs in processing queue
3. State-based idempotency - If an image is already marked as "processed", subsequent identical requests will return the existing result without reprocessing

// Example implementation of idempotent image processing with concurrent safety
public class ImageProcessingService {
    private final Map<String, CompletableFuture<ImageProcessingResult>> processingMap = new ConcurrentHashMap<>();
    private final Semaphore concurrentProcessingSemaphore = new Semaphore(10); // Max 10 concurrent jobs
    
    public CompletableFuture<ImageProcessingResult> processImageAsync(ImageRequest request) {
        String requestId = generateRequestId(request);
        
        // Check if already processing
        CompletableFuture<ImageProcessingResult> existingFuture = processingMap.get(requestId);
        if (existingFuture != null) {
            return existingFuture;
        }
        
        // Create new processing task
        CompletableFuture<ImageProcessingResult> future = CompletableFuture.supplyAsync(() -> {
            try {
                concurrentProcessingSemaphore.acquire();
                
                // Check if already processed (idempotency)
                ImageMetadata metadata = imageMetadataRepository.findByImageId(request.getImageId());
                if (metadata != null && metadata.getStatus() == ImageStatus.PROCESSED) {
                    return new ImageProcessingResult(metadata);
                }
                
                // Acquire exclusive lock on image
                try (Lock lock = imageLockManager.acquireLock(request.getImageId(), 5, TimeUnit.SECONDS)) {
                    // Double-check after lock acquisition
                    metadata = imageMetadataRepository.findByImageId(request.getImageId());
                    if (metadata != null && metadata.getStatus() == ImageStatus.PROCESSED) {
                        return new ImageProcessingResult(metadata);
                    }
                    
                    // Process image
                    ImageProcessingResult result = performImageProcessing(request);
                    
                    // Update metadata and publish events
                    updateImageMetadataAndPublishEvents(result);
                    
                    return result;
                }
            } catch (Exception e) {
                throw new RuntimeException("Image processing failed", e);
            } finally {
                concurrentProcessingSemaphore.release();
            }
        });
        
        processingMap.put(requestId, future);
        return future;
    }
    
    private String generateRequestId(ImageRequest request) {
        return "img-" + request.getImageId() + "-" + 
               DigestUtils.md5Hex(request.getProcessingOptions().toString());
    }
}

6. Sequence Diagrams for Key Flows

sequenceDiagram
    participant U as User
    participant API as ImageUploadAPI
    participant Queue as MessageQueue
    participant Worker as ImageProcessingWorker
    participant Storage as ObjectStorage
    participant DB as Database

    U->>API: POST /images/upload
    API->>DB: Save image metadata (status: PENDING)
    API->>Queue: Publish to image-processing-queue
    Queue-->>Worker: Consume message
    Worker->>Storage: Download original image
    Worker->>Storage: Upload processed images (thumbnail, web, mobile)
    Worker->>DB: Update image metadata (status: PROCESSED)
    Worker->>Storage: Upload thumbnails
    Worker->>DB: Update image metadata (status: PROCESSED)

sequenceDiagram
    participant U as User
    participant API as ImageUploadAPI
    participant Queue as MessageQueue
    participant Worker as ImageProcessingWorker
    participant Storage as ObjectStorage
    participant DB as Database

    U->>API: POST /images/upload
    API->>DB: Save image metadata (status: PENDING)
    API->>Queue: Publish to image-processing-queue
    Queue-->>Worker: Consume message
    Worker->>Storage: Download original image (failed)
    Worker->>DB: Update image metadata (status: FAILED)
    Worker->>Queue: Publish to dead-letter-queue

sequenceDiagram
    participant U as User
    participant API as ImageUploadAPI
    participant Queue as MessageQueue
    participant Worker as ImageProcessingWorker
    participant Storage as ObjectStorage
    participant DB as Database
    participant EventBus as EventBus

    U->>API: POST /images/upload
    API->>DB: Save image metadata (status: PENDING)
    API->>Queue: Publish to image-processing-queue
    Queue-->>Worker: Consume message
    Worker->>Storage: Download original image
    Worker->>Storage: Upload processed images (thumbnail, web, mobile)
    Worker->>EventBus: Publish ImageProcessedEvent
    EventBus-->>NotificationService: Notify user of completion

7. Caching Design

Cache Strategy Selection

For image processing pipeline, I'll use cache-aside pattern with a distributed Redis cluster as the primary cache layer. This approach provides:

1. Flexibility: Cache can be invalidated or updated explicitly
2. Separation of concerns: Clear distinction between cache and data layers
3. Scalability: Redis cluster supports horizontal scaling
4. Rich data structures: Support for image metadata, thumbnails, and processing status

Cache Key Design

Cache keys will follow a structured namespace pattern:

{namespace}:{entity_type}:{entity_id}:{version}:{suffix}

Example keys:

• image:processed:123456789:v1:thumbnail_200x200
• image:metadata:123456789:v1:original
• image:status:123456789:v1:processing

TTL Strategy

Data Type	TTL	Reason
Processed thumbnails (small)	30 days	Frequently accessed, small size
Processed thumbnails (large)	7 days	Less frequently accessed, larger size
Image metadata	1 year	Stable, immutable once processed
Processing status	2 hours	Short-lived, active jobs
Queue status	1 hour	Job tracking, short-term

Cache Eviction Policy

LRU (Least Recently Used) with TTL-based eviction as primary strategy:

• LRU ensures frequently accessed images stay in cache
• TTL prevents stale data from accumulating
• Redis with maxmemory-policy set to allkeys-lru

Cache Stampede Prevention

To prevent cache stampedes:

1. Probabilistic early expiry: Set TTL to 80% of actual value (e.g., 24 hours instead of 30)
2. Distributed locks: Use Redis SETNX with TTL for cache population
3. Cache warming: Pre-populate popular images during low-traffic periods

Implementation example:

public class CacheService {
    private static final String CACHE_LOCK_KEY = "lock:image:processed:";
    private static final int LOCK_TTL_MS = 5000; // 5 seconds
    
    public String getOrCreateThumbnail(String imageId, int width, int height) {
        String cacheKey = buildCacheKey(imageId, width, height);
        String cachedValue = redisTemplate.opsForValue().get(cacheKey);
        
        if (cachedValue != null) {
            return cachedValue;
        }
        
        // Acquire distributed lock
        String lockKey = CACHE_LOCK_KEY + imageId;
        if (tryAcquireLock(lockKey, LOCK_TTL_MS)) {
            try {
                // Double-check after lock acquisition
                cachedValue = redisTemplate.opsForValue().get(cacheKey);
                if (cachedValue != null) {
                    return cachedValue;
                }
                
                // Perform expensive operation
                String thumbnail = processImageThumbnail(imageId, width, height);
                
                // Store in cache with TTL
                redisTemplate.opsForValue().set(
                    cacheKey, thumbnail, 
                    THUMBNAIL_TTL_SECONDS, TimeUnit.SECONDS
                );
                
                return thumbnail;
            } finally {
                releaseLock(lockKey);
            }
        } else {
            // Wait briefly and retry
            Thread.sleep(100);
            return getOrCreateThumbnail(imageId, width, height);
        }
    }
}

Distributed Cache Architecture

Redis Cluster Configuration:

• 3 master nodes with 3 replicas each (9 total instances)
• Automatic failover with Redis Sentinel
• Sharding based on image ID hash
• Memory optimization with compression for image data

Cache Invalidation Strategy

Event-driven invalidation:

1. When original image is updated → invalidate all processed versions
2. When processing configuration changes → invalidate affected thumbnails
3. Manual invalidation via admin API

Implementation:

@Component
public class ImageCacheInvalidator {
    
    @EventListener
    public void handleImageUpdated(ImageUpdatedEvent event) {
        String imageId = event.getImageId();
        
        // Invalidate all processed versions
        Set<String> keysToInvalidate = findProcessedKeysForImage(imageId);
        redisTemplate.delete(keysToInvalidate);
        
        // Publish cache invalidation event for downstream services
        eventPublisher.publish(new CacheInvalidatedEvent(imageId));
    }
}

Cache Performance Considerations

1. Memory efficiency: Store compressed image data (JPEG/PNG) rather than raw pixels
2. Hot/cold cache separation: Separate frequently accessed thumbnails from less-used variants
3. Cache warming: Preload popular image sizes during off-peak hours
4. Cache hit ratio monitoring: Track and optimize for 85%+ hit rates

Cache Tiering Strategy

Multi-tier caching:

1. L1: In-memory cache (local JVM) for most frequently accessed thumbnails
2. L2: Redis cluster for cross-service sharing and persistence
3. L3: External CDN for public thumbnails (optional)

This approach balances performance, cost, and complexity while supporting the image processing pipeline's high-throughput requirements.

8. Repository / Data Access Layer

Repository Pattern Implementation

The repository pattern is implemented to abstract data access logic and provide a clean separation between domain objects and database operations. Each entity has its own repository interface and implementation.

public interface ImageRepository {
    Optional<Image> findById(UUID id);
    List<Image> findByStatus(ImageStatus status);
    List<Image> findByUserId(UUID userId);
    Image save(Image image);
    void delete(UUID id);
    long countByStatus(ImageStatus status);
}

public class ImageRepositoryImpl implements ImageRepository {
    private final JdbcTemplate jdbcTemplate;
    private final ObjectMapper objectMapper;
    
    public ImageRepositoryImpl(JdbcTemplate jdbcTemplate, ObjectMapper objectMapper) {
        this.jdbcTemplate = jdbcTemplate;
        this.objectMapper = objectMapper;
    }
    
    @Override
    public Optional<Image> findById(UUID id) {
        String sql = "SELECT * FROM images WHERE id = ?";
        try {
            return Optional.of(jdbcTemplate.queryForObject(sql, new ImageRowMapper(), id));
        } catch (EmptyResultDataAccessException e) {
            return Optional.empty();
        }
    }
    
    @Override
    public List<Image> findByStatus(ImageStatus status) {
        String sql = "SELECT * FROM images WHERE status = ? ORDER BY created_at DESC";
        return jdbcTemplate.query(sql, new ImageRowMapper(), status.name());
    }
    
    @Override
    public List<Image> findByUserId(UUID userId) {
        String sql = "SELECT * FROM images WHERE user_id = ? ORDER BY created_at DESC";
        return jdbcTemplate.query(sql, new ImageRowMapper(), userId.toString());
    }
    
    @Override
    public Image save(Image image) {
        if (image.getId() == null) {
            return createImage(image);
        } else {
            return updateImage(image);
        }
    }
    
    private Image createImage(Image image) {
        String sql = """
            INSERT INTO images (id, user_id, original_url, processed_urls, status, metadata, created_at, updated_at)
            VALUES (?, ?, ?, ?, ?, ?, ?, ?)
            """;
        
        UUID id = UUID.randomUUID();
        image.setId(id);
        image.setCreatedAt(Instant.now());
        image.setUpdatedAt(Instant.now());
        
        jdbcTemplate.update(sql, 
            id.toString(),
            image.getUserId().toString(),
            image.getOriginalUrl(),
            objectMapper.writeValueAsString(image.getProcessedUrls()),
            image.getStatus().name(),
            objectMapper.writeValueAsString(image.getMetadata()),
            image.getCreatedAt(),
            image.getUpdatedAt()
        );
        
        return image;
    }
    
    private Image updateImage(Image image) {
        String sql = """
            UPDATE images SET 
                original_url = ?,
                processed_urls = ?,
                status = ?,
                metadata = ?,
                updated_at = ?
            WHERE id = ?
            """;
        
        image.setUpdatedAt(Instant.now());
        
        jdbcTemplate.update(sql,
            image.getOriginalUrl(),
            objectMapper.writeValueAsString(image.getProcessedUrls()),
            image.getStatus().name(),
            objectMapper.writeValueAsString(image.getMetadata()),
            image.getUpdatedAt(),
            image.getId().toString()
        );
        
        return image;
    }
    
    @Override
    public void delete(UUID id) {
        String sql = "DELETE FROM images WHERE id = ?";
        jdbcTemplate.update(sql, id.toString());
    }
    
    @Override
    public long countByStatus(ImageStatus status) {
        String sql = "SELECT COUNT(*) FROM images WHERE status = ?";
        return jdbcTemplate.queryForObject(sql, Long.class, status.name());
    }
}

ORM vs. Raw Query Decisions

For the image processing pipeline, we use a hybrid approach:

• Raw SQL queries for performance-critical operations like batch processing and status counting
• JPA/Hibernate for complex domain object relationships (when we introduce more entities)
• JdbcTemplate for simple CRUD operations with better control over SQL execution

This approach provides:

1. Better performance for high-volume operations (batching, counting)
2. Easier debugging and query optimization
3. Reduced abstraction overhead for simple operations

Read vs. Write Separation (CQRS Pattern)

We implement CQRS for read-heavy operations:

public interface ImageReadRepository {
    List<ImageProjection> findRecentImages(int limit);
    ImageProjection findById(UUID id);
    long countByStatus(ImageStatus status);
}

public class ImageReadRepositoryImpl implements ImageReadRepository {
    private final JdbcTemplate jdbcTemplate;
    
    public ImageReadRepositoryImpl(JdbcTemplate jdbcTemplate) {
        this.jdbcTemplate = jdbcTemplate;
    }
    
    @Override
    public List<ImageProjection> findRecentImages(int limit) {
        String sql = """
            SELECT id, user_id, original_url, status, created_at
            FROM images 
            ORDER BY created_at DESC 
            LIMIT ?
            """;
        return jdbcTemplate.query(sql, new ImageProjectionRowMapper(), limit);
    }
    
    @Override
    public ImageProjection findById(UUID id) {
        String sql = """
            SELECT id, user_id, original_url, status, created_at
            FROM images 
            WHERE id = ?
            """;
        return jdbcTemplate.queryForObject(sql, new ImageProjectionRowMapper(), id.toString());
    }
    
    @Override
    public long countByStatus(ImageStatus status) {
        String sql = "SELECT COUNT(*) FROM images WHERE status = ?";
        return jdbcTemplate.queryForObject(sql, Long.class, status.name());
    }
}

Connection Pool Configuration

@Configuration
public class DatabaseConfig {
    
    @Bean
    public DataSource dataSource() {
        HikariDataSource dataSource = new HikariDataSource();
        dataSource.setJdbcUrl("jdbc:postgresql://localhost:5432/image_processing");
        dataSource.setUsername("user");
        dataSource.setPassword("password");
        dataSource.setMaximumPoolSize(20);
        dataSource.setMinimumIdle(5);
        dataSource.setConnectionTimeout(30000);
        dataSource.setIdleTimeout(600000);
        dataSource.setMaxLifetime(1800000);
        dataSource.setLeakDetectionThreshold(60000);
        return dataSource;
    }
}

N+1 Query Prevention Strategy

We implement eager loading for related entities and use batch operations:

public class ImageBatchProcessor {
    private final JdbcTemplate jdbcTemplate;
    
    public void processImagesInBatches(List<UUID> imageIds, ImageProcessor processor) {
        // Batch query to avoid N+1
        String sql = """
            SELECT i.id, i.user_id, i.original_url, i.status, i.metadata,
                   u.email as user_email
            FROM images i 
            JOIN users u ON i.user_id = u.id
            WHERE i.id IN (%s)
            """;
        
        String placeholders = imageIds.stream()
            .map(id -> "?")
            .collect(Collectors.joining(","));
        
        String finalSql = sql.formatted(placeholders);
        
        // Execute batch query with prepared statement
        List<Image> images = jdbcTemplate.query(finalSql, 
            imageIds.toArray(), 
            new ImageWithUserRowMapper());
        
        // Process all images in batch
        images.forEach(processor::process);
    }
}

Batch Operation Design

public class ImageBatchRepository {
    
    public void updateImageStatuses(List<UUID> imageIds, ImageStatus newStatus) {
        String sql = """
            UPDATE images 
            SET status = ?, updated_at = ?
            WHERE id IN (%s)
            """;
        
        String placeholders = imageIds.stream()
            .map(id -> "?")
            .collect(Collectors.joining(","));
        
        String finalSql = sql.formatted(placeholders);
        
        // Prepare batch update
        jdbcTemplate.update(finalSql, ps -> {
            for (int i = 0; i < imageIds.size(); i++) {
                ps.setString(i + 1, newStatus.name());
                ps.setTimestamp(i + 2, Timestamp.from(Instant.now()));
            }
        });
    }
    
    public void createImagesInBatch(List<Image> images) {
        String sql = """
            INSERT INTO images (id, user_id, original_url, processed_urls, status, metadata, created_at, updated_at)
            VALUES (?, ?, ?, ?, ?, ?, ?, ?)
            """;
        
        jdbcTemplate.batchUpdate(sql, images, (ps, image) -> {
            ps.setString(1, image.getId().toString());
            ps.setString(2, image.getUserId().toString());
            ps.setString(3, image.getOriginalUrl());
            ps.setObject(4, image.getProcessedUrls(), Types.VARCHAR);
            ps.setString(5, image.getStatus().name());
            ps.setObject(6, image.getMetadata(), Types.VARCHAR);
            ps.setTimestamp(7, Timestamp.from(image.getCreatedAt()));
            ps.setTimestamp(8, Timestamp.from(image.getUpdatedAt()));
        });
    }
}

9. Asynchronous Processing & Messaging

Event-driven boundaries in this feature

The image processing pipeline uses a hybrid synchronous/asynchronous approach. The core workflow is:

1. Synchronous: Image upload, initial validation, and metadata extraction happen synchronously to ensure immediate feedback to the client.
2. Asynchronous: Image transformations, thumbnail generation, and storage operations are handled asynchronously to avoid blocking the main request thread.

Key events that trigger async processing:

• ImageUploadedEvent - triggers transformation pipeline
• ImageProcessedEvent - triggers thumbnail generation and storage
• ImageProcessingFailedEvent - triggers retry logic or failure notification

Message schema design with sample payload

{
  "version": "1.0",
  "eventId": "uuid-1234567890",
  "eventType": "ImageProcessedEvent",
  "timestamp": "2023-10-15T14:30:00Z",
  "payload": {
    "imageId": "img-1234567890",
    "originalUrl": "https://storage.example.com/original.jpg",
    "processedUrls": {
      "thumbnail": "https://storage.example.com/thumbnail.jpg",
      "medium": "https://storage.example.com/medium.jpg"
    },
    "metadata": {
      "width": 1920,
      "height": 1080,
      "format": "jpg",
      "sizeBytes": 2048000
    }
  }
}

Dead letter queue strategy and poison message handling

Poison messages are handled through a dead-letter queue (DLQ) system:

1. Retry Limit: Maximum 3 retries with exponential backoff (1s, 4s, 16s)
2. DLQ: Messages that fail after max retries are moved to a DLQ topic
3. Manual Review: Failed messages in DLQ are flagged for manual inspection
4. Monitoring: DLQ queue size and failure rate are monitored with alerts

At-least-once vs. exactly-once delivery

At-least-once delivery is chosen for this system because:

1. Idempotency: All processing operations are idempotent (reprocessing same image produces same result)
2. Simpler implementation: Avoids complexity of maintaining state for exactly-once semantics
3. Performance: Lower overhead compared to distributed transaction coordination

Retry policy for consumers

public class ImageProcessingRetryPolicy {
    private static final int MAX_RETRIES = 3;
    private static final long BASE_DELAY_MS = 1000;
    private static final double BACKOFF_MULTIPLIER = 4.0;
    
    public boolean shouldRetry(ImageProcessingEvent event, int attempt) {
        if (attempt >= MAX_RETRIES) {
            return false;
        }
        
        // Don't retry on validation errors or permanent failures
        if (isNonRetryableError(event)) {
            return false;
        }
        
        return true;
    }
    
    public long getDelay(int attempt) {
        return (long) Math.pow(BACKOFF_MULTIPLIER, attempt) * BASE_DELAY_MS;
    }
    
    private boolean isNonRetryableError(ImageProcessingEvent event) {
        return event.getErrorCode() == ErrorCode.VALIDATION_ERROR ||
               event.getErrorCode() == ErrorCode.UNSUPPORTED_FORMAT;
    }
}

Outbox pattern for reliable event publishing

The outbox pattern is implemented to ensure eventual consistency between database writes and message publishing:

@Component
public class ImageProcessingEventPublisher {
    
    @Autowired
    private EventOutboxRepository eventOutboxRepository;
    
    @Transactional
    public void publishImageProcessedEvent(ImageProcessingResult result) {
        // 1. Store image processing result to database
        imageRepository.save(result.getImage());
        
        // 2. Create outbox event record (in same transaction)
        EventOutbox outboxEvent = new EventOutbox();
        outboxEvent.setEventId(UUID.randomUUID().toString());
        outboxEvent.setEventType("ImageProcessedEvent");
        outboxEvent.setPayload(mapper.writeValueAsString(result));
        outboxEvent.setStatus(EventStatus.PENDING);
        eventOutboxRepository.save(outboxEvent);
        
        // 3. Publish to message queue (handled by background processor)
        // This ensures that if DB commit fails, event is not published
    }
}

Sequence diagram for async flows

sequenceDiagram
    participant Client
    participant APIGateway
    participant ImageService
    participant EventPublisher
    participant MessageQueue
    participant ImageProcessingWorker
    participant StorageService
    
    Client->>APIGateway: Upload image
    APIGateway->>ImageService: ProcessUpload(image)
    ImageService->>ImageService: Validate image
    ImageService->>EventPublisher: Publish ImageUploadedEvent
    EventPublisher->>MessageQueue: Queue event
    MessageQueue-->>ImageProcessingWorker: Consume ImageUploadedEvent
    ImageProcessingWorker->>StorageService: Generate thumbnails
    ImageProcessingWorker->>StorageService: Store processed images
    StorageService-->>ImageProcessingWorker: Return URLs
    ImageProcessingWorker->>EventPublisher: Publish ImageProcessedEvent
    EventPublisher->>MessageQueue: Queue event
    MessageQueue-->>APIGateway: Consume ImageProcessedEvent
    APIGateway->>Client: Return processed image URLs

10. Error Handling Design

Error Classification Hierarchy

The system implements a three-tier error classification hierarchy:

1. Domain Errors - Business logic violations (e.g., invalid image format, unsupported dimensions)
2. Infrastructure Errors - System resource constraints (e.g., storage full, network timeout)
3. Validation Errors - Input parameter issues (e.g., missing fields, malformed URLs)

public abstract class BaseException extends RuntimeException {
    private final String errorCode;
    private final int httpStatus;

    public BaseException(String message, String errorCode, int httpStatus) {
        super(message);
        this.errorCode = errorCode;
        this.httpStatus = httpStatus;
    }

    // Getters
}

public class ImageProcessingException extends BaseException {
    public ImageProcessingException(String message, String errorCode) {
        super(message, errorCode, 500);
    }
}

public class ValidationException extends BaseException {
    public ValidationException(String message, String errorCode) {
        super(message, errorCode, 400);
    }
}

public class ResourceExhaustedException extends BaseException {
    public ResourceExhaustedException(String message, String errorCode) {
        super(message, errorCode, 429);
    }
}

Error Propagation Strategy

Errors are handled at the service boundary and propagated upward with appropriate HTTP status codes. Service layer exceptions are converted to standardized API responses.

Retry Strategy

For transient infrastructure errors, exponential backoff with jitter is applied:

public class RetryableOperation {
    private static final int MAX_RETRIES = 3;
    private static final long INITIAL_DELAY_MS = 1000;
    private static final double JITTER_FACTOR = 0.2;

    public <T> T executeWithRetry(Supplier<T> operation) {
        int attempts = 0;
        while (attempts < MAX_RETRIES) {
            try {
                return operation.get();
            } catch (ResourceExhaustedException e) {
                if (attempts >= MAX_RETRIES - 1) throw e;
                
                long delay = (long) (INITIAL_DELAY_MS * Math.pow(2, attempts) 
                    * (1 + JITTER_FACTOR * (Math.random() - 0.5)));
                Thread.sleep(delay);
                attempts++;
            }
        }
        throw new RuntimeException("Max retries exceeded");
    }
}

Circuit Breaker Design

For external dependencies like image storage or processing services, a circuit breaker pattern is implemented:

public class CircuitBreaker {
    private final int failureThreshold = 5;
    private final long timeout = 60000; // 1 minute
    private int failureCount = 0;
    private long lastFailureTime = 0;
    private CircuitState state = CircuitState.CLOSED;

    public enum CircuitState {
        CLOSED, OPEN, HALF_OPEN
    }

    public <T> T execute(Supplier<T> operation) {
        switch (state) {
            case CLOSED:
                try {
                    T result = operation.get();
                    reset();
                    return result;
                } catch (Exception e) {
                    recordFailure();
                    throw e;
                }
            case OPEN:
                if (System.currentTimeMillis() - lastFailureTime > timeout) {
                    state = CircuitState.HALF_OPEN;
                } else {
                    throw new ServiceUnavailableException("Circuit breaker is open");
                }
            case HALF_OPEN:
                try {
                    T result = operation.get();
                    reset();
                    return result;
                } catch (Exception e) {
                    state = CircuitState.OPEN;
                    throw e;
                }
        }
        throw new IllegalStateException("Unexpected circuit state");
    }

    private void recordFailure() {
        failureCount++;
        lastFailureTime = System.currentTimeMillis();
        if (failureCount >= failureThreshold) {
            state = CircuitState.OPEN;
        }
    }

    private void reset() {
        failureCount = 0;
        state = CircuitState.CLOSED;
    }
}

Fallback Behavior Design

For critical external dependencies, fallbacks are implemented:

public class ImageProcessingService {
    private final StorageService storageService;
    private final ProcessingService processingService;
    private final CircuitBreaker circuitBreaker;

    public ImageProcessingResult processImage(ImageRequest request) {
        try {
            return circuitBreaker.execute(() -> {
                // Primary operation
                return processImageInternal(request);
            });
        } catch (ServiceUnavailableException e) {
            // Fallback to default processing or cached result
            return handleFallback(request);
        }
    }

    private ImageProcessingResult handleFallback(ImageRequest request) {
        // Return default processed image or cached version
        return new ImageProcessingResult(
            "default_processed_image.png",
            ImageProcessingStatus.FALLBACK_USED
        );
    }
}

Error Logging Format

Structured logging includes trace context and error details:

@Component
public class ErrorLogger {
    private static final Logger logger = LoggerFactory.getLogger(ErrorLogger.class);

    public void logError(String operation, Exception error, Map<String, Object> context) {
        Map<String, Object> logContext = new HashMap<>(context);
        logContext.put("error_code", extractErrorCode(error));
        logContext.put("error_type", error.getClass().getSimpleName());
        logContext.put("stack_trace", error.getStackTrace()[0].toString());

        logger.error("Image processing error in operation: {}", operation, logContext);
    }

    private String extractErrorCode(Exception e) {
        if (e instanceof BaseException) {
            return ((BaseException) e).getErrorCode();
        }
        return "UNKNOWN_ERROR";
    }
}

Alerting Thresholds

Critical errors trigger immediate alerts:

Error Type	Alert Threshold	Alert Channel
ImageProcessingException	> 10 occurrences/minute	On-call
ResourceExhaustedException	> 5 occurrences/minute	Slack
ValidationException	> 100 occurrences/hour	Email

Error Response Format

Standardized JSON error response:

{
  "error": {
    "code": "IMAGE_FORMAT_UNSUPPORTED",
    "message": "The provided image format is not supported",
    "httpStatus": 400,
    "timestamp": "2023-10-15T14:30:00Z",
    "requestId": "req-12345"
  }
}

11. Security Design

Input Validation and Sanitization Strategy

All image upload endpoints must validate file type, size, and content to prevent malicious uploads. We'll use Apache Tika for MIME type detection and implement size limits (e.g., 10MB max). Input sanitization will be handled through:

• File extension validation (whitelist: .jpg, .jpeg, .png, .gif, .webp)
• Content-type verification using Tika
• Image dimension validation (max 10k x 10k pixels)
• File header checksum verification to detect format mismatches

SQL Injection Prevention

All database operations use parameterized queries with prepared statements. No dynamic query construction or string concatenation. ORM usage is enforced with strict mapping rules to prevent injection vectors.

Authentication Middleware Design

All image processing endpoints require JWT-based authentication. Middleware will:

• Validate JWT signature using RS256 with public key rotation
• Check token expiration and audience claims
• Verify user permissions for specific image operations
• Implement rate limiting per authenticated user

Authorization Enforcement (RBAC)

Role-based access control will be implemented with the following permissions:

• image:upload - allows image upload to user's namespace
• image:process - allows image transformation operations
• image:delete - allows image deletion from user's namespace
• image:read - allows reading processed images

Authorization checks will occur at service layer boundaries, using a centralized permission checker that validates user roles against resource ownership.

Sensitive Data Encryption at Rest and in Transit

• In transit: All image data will be encrypted using TLS 1.3 (minimum) for all network communications
• At rest: Images will be encrypted using AES-256-GCM with keys managed by AWS KMS or similar HSM-backed service
• Metadata: Sensitive metadata (user IDs, timestamps) will be encrypted using the same key management system

PII Handling and Masking Strategy

• User identifiers in image metadata will be hashed using SHA-256 with salted prefixes
• Image filenames will not contain user identifiers or PII
• All logs and audit trails will mask any PII in filenames, paths, or metadata fields
• Image content itself is not considered PII but will be protected with the same encryption policies

Secrets Management

• API keys, database credentials, and external service tokens will be managed through AWS Secrets Manager (or equivalent)
• Secret rotation policy: 90-day automatic rotation with 7-day grace period
• Secrets will be injected into application containers via environment variables or mounted secret volumes

Audit Logging Design

All security-relevant events will be logged with structured JSON format containing:

{
  "timestamp": "2023-12-01T10:30:45Z",
  "event_type": "image_upload_success",
  "user_id": "hashed_user_id",
  "ip_address": "192.168.1.100",
  "file_size": 1024000,
  "file_type": "image/jpeg",
  "request_id": "req-1234567890",
  "trace_id": "trace-abcdef123456"
}

Log retention policy: 90 days for security events, 1 year for audit trails. All logs will be encrypted at rest and sent to centralized logging system (e.g., ELK stack or AWS CloudWatch) with proper access controls.

Additional Security Controls

• Cross-Site Request Forgery (CSRF) protection: Implemented via CSRF tokens for web-based image upload forms
• Content Security Policy (CSP): Strict CSP headers to prevent XSS attacks on image preview pages
• Rate limiting: Per-user rate limits (e.g., 100 uploads/hour) to prevent abuse and DoS attacks
• File integrity verification: SHA-256 checksums will be stored for all uploaded images to detect tampering
• Vulnerability scanning: Automated scanning of image processing libraries for known vulnerabilities (e.g., ImageMagick, libpng)

12. Testing Strategy

Unit Testing Scope and Coverage Targets (%)

• Service Layer: 85% code coverage with focus on business logic and error paths
• Data Access Layer: 90% coverage including repository method tests with mocked database interactions
• Domain Objects: 100% coverage for value objects and immutable domain entities
• API Layer: 75% coverage with focus on request/response validation and error handling
• Event Handlers: 80% coverage for async processing logic with mocked event sources

Integration Testing Strategy

• Database Integration: Test using embedded H2/PostgreSQL in-memory databases with schema migration scripts to validate repository operations and transaction boundaries
• External Service Integration: Use WireMock or Testcontainers for mocking image processing service dependencies (e.g., S3, Redis, message brokers) to test end-to-end flows
• Cross-component Integration: Validate service-to-service communication through mock HTTP clients and real database connections for data consistency checks
• Caching Integration: Test cache interactions with Redis mock or in-memory cache implementations to ensure proper cache population and eviction behavior

Contract Testing

• API Contracts: Implement Pact-based contract testing between image processing service and consumers (e.g., frontend, other microservices) to ensure API stability
• Event Contracts: Use Avro schema registry with Schema Registry client to validate event payload formats and version compatibility for asynchronous flows
• External Service Contracts: Define and test against contract files for S3 bucket access, image processing service endpoints, and queue system interfaces

Performance Testing Tool

• Load Testing: Use k6 with custom scripts to simulate concurrent image uploads and processing requests, targeting 1000 concurrent users with 50ms p95 response time for image processing operations
• Stress Testing: Implement JMeter test plans to validate system behavior under peak load conditions (5000 concurrent users) and identify bottlenecks in database, cache, and message queue layers
• Throughput Testing: Validate that the system can process 1000 image uploads per minute with 95% success rate and <200ms average processing time

Test Data Management Strategy

• Fixtures: Define JSON fixtures for common image metadata (dimensions, format, color space) and test user profiles to ensure consistent baseline testing conditions
• Factories: Implement Factory pattern for generating realistic image processing workflow data including user, image, and processing job entities with configurable parameters
• Seeding: Use database migration scripts (Flyway) to seed test environments with realistic data sets including various image formats, sizes, and processing status combinations
• Data Isolation: Ensure test data isolation using database transaction rollback or dedicated test schemas to prevent cross-test contamination and maintain test reliability

// Example unit test for image processing service
@Test
public void processImage_WithValidInput_ShouldReturnProcessedImage() {
    // Given
    ImageRequest request = new ImageRequest("image.jpg", "resize", "1024x768");
    ImageMetadata metadata = new ImageMetadata("image.jpg", "JPEG", 1024, 768);
    
    // When
    ProcessedImageResult result = imageProcessingService.processImage(request);
    
    // Then
    assertThat(result.getProcessedUrl()).isNotNull();
    assertThat(result.getOriginalDimensions()).isEqualTo("1024x768");
}

13. Observability Design

Structured Logging Schema

All logs must follow a consistent schema to enable effective querying and analysis:

{
  "timestamp": "2023-10-15T14:30:45.123Z",
  "level": "INFO|WARN|ERROR",
  "service": "image-processing-service",
  "trace_id": "1234567890abcdef1234567890abcdef",
  "span_id": "abcdef1234567890",
  "user_id": "user-12345",
  "entity_id": "image-98765",
  "operation": "processImage",
  "message": "Image processing started",
  "duration_ms": 150,
  "error_code": "IMAGE_TOO_LARGE",
  "error_details": "Image size exceeds 50MB limit",
  "request_id": "req-abc123",
  "source_ip": "192.168.1.100"
}

PII masking rules:

• User IDs and email addresses are masked in logs
• IP addresses are truncated to network level (e.g., 192.168.1.0/24)
• Image content is never logged

Distributed Tracing Instrumentation

Using OpenTelemetry for end-to-end tracing:

Custom spans to add:

1. image.upload.start → image.upload.end
2. image.processing.start → image.processing.end
3. image.storage.start → image.storage.end
4. image.thumbnailing.start → image.thumbnailing.end

Attributes to capture per key operation:

Span currentSpan = Span.current();
currentSpan.setAttribute("image.id", imageId);
currentSpan.setAttribute("image.size.bytes", imageSizeBytes);
currentSpan.setAttribute("image.format", imageFormat);
currentSpan.setAttribute("processing.duration.ms", durationMs);

Metrics Design

RED Method metrics (Rate, Errors, Duration):

Metric Name	Type	Labels	Description
image_processing_requests_total	Counter	status, operation	Total image processing requests
image_processing_errors_total	Counter	error_type, operation	Error counts by type and operation
image_processing_duration_seconds	Histogram	operation, status	Processing time distribution
image_upload_size_bytes	Histogram	format, user_type	Distribution of uploaded image sizes
image_thumbnail_count	Gauge	format	Current thumbnail count in system

Additional service-level metrics:

• active_workers (Gauge): Current number of processing workers
• queue_length (Gauge): Pending image processing jobs
• cache_hit_rate (Gauge): Cache hit ratio for thumbnails

Health Check Endpoint Design

GET /health
GET /ready
GET /live
GET /startup

Response format:

{
  "status": "healthy|unhealthy",
  "timestamp": "2023-10-15T14:30:45.123Z",
  "service": "image-processing-service",
  "checks": {
    "database": {"status": "healthy", "details": "Connected to primary DB"},
    "cache": {"status": "healthy", "details": "Redis cluster healthy"},
    "storage": {"status": "healthy", "details": "S3 connectivity OK"},
    "queue": {"status": "healthy", "details": "Message queue operational"}
  },
  "version": "1.2.3"
}

Alerting Thresholds

Metric	Warning Level	Critical Level	Escalation Path
image_processing_errors_total	> 10 errors/minute	> 50 errors/minute	On-call engineer
image_processing_duration_seconds	p99 > 5s	p99 > 10s	SRE team
queue_length	> 100 pending jobs	> 500 pending jobs	SRE team
active_workers	< 2 workers	< 1 worker	SRE team
cache_hit_rate	< 70%	< 50%	SRE team

Service Monitoring Configuration

Metrics collection frequency:

• Histograms: 10s bucket aggregation
• Gauges: 30s sample rate
• Counters: Real-time updates

Log aggregation:

• Forward logs to ELK stack or CloudWatch
• Set up log retention policies (90 days for INFO, 30 days for DEBUG)
• Enable structured log parsing for alerting conditions

Dashboard requirements:

1. Real-time processing throughput
2. Error rate trends by operation type
3. Queue depth and worker utilization
4. Storage usage and thumbnail generation rates

14. Open Questions, Risks & Trade-offs

Decisions requiring product / engineering alignment

• Image format support matrix: Should we support all major formats (JPEG, PNG, GIF, WebP, AVIF) or start with core formats and expand based on usage? This affects storage costs, processing complexity, and client compatibility.
• Resolution tier strategy: How many output resolutions should we pre-generate? Should we support dynamic resizing or fixed tiers (e.g., 320x240, 640x480, 1024x768)? This impacts storage costs vs. latency.
• Processing pipeline orchestration: Should image processing be handled by a single monolithic service or split into specialized microservices (e.g., upload handler, resize processor, watermarking engine)? This affects deployment complexity vs. scalability.
• Storage architecture: Should we store raw images in object storage (S3, GCS) or use a hybrid approach with local caching? This impacts cost, availability, and performance trade-offs.

Known trade-offs and justifications

Trade-off	Option A	Option B	Justification
Storage vs. Compute	Pre-generate all image variants	Generate on-demand	Pre-generation offers better latency but increases storage costs and requires more complex invalidation logic. On-demand processing provides better cost efficiency but introduces latency for first-time users.
Synchronous vs. Asynchronous Processing	Process all image transformations synchronously	Use async queue for heavy operations	Synchronous processing provides immediate feedback but can block user experience. Async processing enables better scalability and user experience but introduces complexity in tracking state and handling failures.
Caching Strategy	Cache all processed images in memory	Cache only most popular variants	Memory caching provides fastest access but requires more memory and is less effective for diverse image usage patterns. Cache-based on popularity provides better resource utilization but requires more complex cache warming and invalidation logic.
Image Format Support	Support all formats natively	Support only web-friendly formats	Full format support increases processing complexity and resource consumption. Limited support reduces processing overhead but may break compatibility with some client applications.

Risks and mitigation strategies

High Storage Costs: Image processing can lead to exponential storage growth (original + multiple variants).

• Mitigation: Implement intelligent image variant generation based on usage analytics, set strict retention policies for temporary files, and use compression techniques.

Processing Time Variance: Different image formats and sizes can cause unpredictable processing times.

• Mitigation: Implement request prioritization, set timeout limits for image operations, and use rate limiting to prevent system overload.

Cache Invalidation Complexity: When an original image changes, all variants need to be invalidated and regenerated.

• Mitigation: Use content-based versioning for image variants, implement cache warming strategies, and design a robust invalidation pipeline with proper tracking.

Service Availability: Image processing service failure can impact entire application functionality.

• Mitigation: Implement circuit breaker patterns, fallback mechanisms (e.g., serving default image sizes), and graceful degradation strategies.

Security Vulnerabilities: Image processing can be a vector for malicious file uploads or DoS attacks.

• Mitigation: Implement strict input validation, size limits, format checking, and sandboxed processing environments.

Technical debt acknowledged

• Limited Format Extensibility: Current design assumes a fixed set of supported formats, making it difficult to add new formats without code changes.
• Hardcoded Image Processing Parameters: Processing quality, compression levels, and output formats are currently hardcoded, requiring configuration changes for adjustments.
• Inconsistent Error Handling: Different processing steps may have inconsistent error handling and reporting mechanisms, making debugging difficult.
• No Image Metadata Indexing: Raw image metadata (EXIF data) is not indexed or stored, limiting search and filtering capabilities for future features.
• Tight Coupling in Processing Pipeline: Current design has tight coupling between image processing steps, making it harder to swap or extend individual processing components.