Video Processing Workflow Design

This document presents a scalable video processing workflow architecture designed to handle large-scale video ingestion, format conversion, and delivery across multiple platforms while maintaining high availability and performance standards expected in FAANG environments.

1. Problem Statement & Requirements

Clarifying questions

1. What scale and throughput requirements are expected?

   • How many videos per day/month will be processed?
   • What is the expected peak concurrent processing capacity?

2. What consistency and latency requirements exist?

   • Are there strict SLAs for video processing completion times?
   • Is real-time processing required, or can batch processing be acceptable?

3. What are the key entities and their relationships?

   • Are there user accounts, video collections, or playlists?
   • How do videos relate to metadata, thumbnails, or transcoded versions?

4. What is the read/write ratio for video data?

   • Is the system primarily used for uploading and processing, or for viewing content?

5. What are the latency sensitivity and performance expectations?

   • What is the maximum acceptable time for video processing (e.g., 5s, 30s, 5min)?
   • Are there specific time windows for content availability?

6. What authentication and authorization model is expected?

   • Will users authenticate via OAuth2 or JWT?
   • Are there role-based permissions (admin, content creator, viewer)?

7. Are there external integrations or dependencies?

   • Will the system integrate with cloud storage (S3, GCS) or CDN services?
   • Are there third-party transcoding services or APIs involved?

Functional requirements

• Video Upload: Users can upload videos in various formats (MP4, MOV, AVI, etc.)
• Video Processing Pipeline: Automatically transcode uploaded videos into multiple resolutions (1080p, 720p, 480p) and formats (H.264, H.265)
• Thumbnail Generation: Create thumbnails at various intervals (1s, 5s, 10s) from the video
• Metadata Extraction: Extract and store metadata such as duration, resolution, codec info, and frame rate
• Video Delivery: Serve processed videos via CDN or direct download links
• Status Tracking: Provide real-time status updates (pending, processing, completed, failed)
• User Management: Associate videos with users and manage access control

Non-functional requirements

Requirement	Target
P50 Latency	< 2s for status updates
P95 Latency	< 5s for video processing
P99 Latency	< 30s for video processing
Throughput	10,000 videos/day
Availability	99.9% (5 minutes of downtime/month)
Concurrency	Support up to 1000 simultaneous video processes

Constraints and assumptions

• Assumptions:

  • Video files are uploaded to cloud storage (e.g., AWS S3 or GCS)
  • Transcoding is handled by a third-party service (e.g., AWS MediaConvert, FFmpeg)
  • Users can upload videos up to 10GB in size
  • Video processing is done asynchronously with status updates via webhooks or polling

• Constraints:

  • The system must support horizontal scaling to handle peak loads
  • Video metadata and thumbnails should be stored in a database for fast retrieval
  • The system must handle failures gracefully with retries and fallbacks

Out of scope

• Content moderation or filtering
• Video transcoding quality tuning or adaptive bitrate streaming (ABR)
• User interface or mobile app integration
• Real-time video streaming or live broadcast capabilities
• Analytics dashboard or user statistics

2. API Design

API Style Selection

gRPC with Protocol Buffers is selected for the video processing workflow due to:

• Binary serialization provides better performance and smaller payload sizes than JSON
• Strong typing with proto definitions ensures type safety across services
• Built-in support for streaming for real-time video processing feedback
• Efficient handling of large binary data (video files) through streaming RPCs
• First-class support for versioning in proto definitions

Endpoint Definitions

Method	Path	Description	Auth	Rate Limit
CreateVideoJob	/v1/video/process	Submit video for processing	JWT + RBAC	100/hour
GetVideoJobStatus	/v1/video/{jobId}/status	Check processing status	JWT + RBAC	1000/hour
GetProcessedVideo	/v1/video/{jobId}/download	Download processed video	JWT + RBAC	50/hour
CancelVideoJob	/v1/video/{jobId}/cancel	Cancel in-progress processing	JWT + RBAC	50/hour
ListVideoJobs	/v1/video/list	List user's video processing jobs	JWT + RBAC	100/hour

Request and Response Schemas

CreateVideoJobRequest

message CreateVideoJobRequest {
  string video_url = 1;
  string video_file_id = 2;
  repeated ProcessingTask tasks = 3;
  string callback_url = 4;
  string user_id = 5;
}

message ProcessingTask {
  TaskType task_type = 1;
  map<string, string> parameters = 2;
}

enum TaskType {
  TRANSCODE = 0;
  THUMBNAIL_GENERATION = 1;
  METADATA_EXTRACTION = 2;
  DURATION_CALCULATION = 3;
}

CreateVideoJobResponse

message CreateVideoJobResponse {
  string job_id = 1;
  string status = 2; // "PENDING", "PROCESSING", "COMPLETED"
  int64 created_at = 3;
  repeated ProcessingTask tasks = 4;
}

GetVideoJobStatusResponse

message GetVideoJobStatusResponse {
  string job_id = 1;
  string status = 2; // "PENDING", "PROCESSING", "COMPLETED", "FAILED", "CANCELLED"
  int32 progress = 3; // 0-100%
  string error_message = 4;
  repeated TaskStatus task_statuses = 5;
  int64 created_at = 6;
  int64 updated_at = 7;
}

message TaskStatus {
  string task_type = 1;
  string status = 2; // "PENDING", "PROCESSING", "COMPLETED", "FAILED"
  int32 progress = 3;
  string error_message = 4;
}

HTTP Status Codes and Error Response

Code	Description
200	Success
201	Created (job submitted)
400	Bad Request - Invalid input
401	Unauthorized - Missing or invalid auth
403	Forbidden - Insufficient permissions
404	Not Found - Job not found
429	Too Many Requests - Rate limit exceeded
500	Internal Server Error

Standardized Error Response

{
  "error": {
    "code": "INVALID_INPUT",
    "message": "Video URL is required",
    "details": [
      {
        "field": "video_url",
        "message": "URL is required"
      }
    ]
  }
}

API Versioning Strategy

Versioned using path prefix /v1/ with semantic versioning. Proto files include version numbers in package names and comments for clear tracking.

Pagination Design

Cursor-based pagination is used because:

• More efficient than offset-based for large datasets
• Consistent results even when data changes during pagination
• Handles better performance for deep page navigation

Authentication and Authorization

Authentication: JWT tokens with OAuth2 integration for third-party access Authorization:

• RBAC roles: USER, ADMIN, SYSTEM
• Permissions: video:read, video:write, video:manage
• JWT claims validation for user_id, roles, and permissions

Rate Limiting Strategy

Per-user rate limiting using Redis with sliding window algorithm:

• 100 requests/hour for job creation
• 1000 requests/hour for status checks
• 50 requests/hour for downloads
• 50 requests/hour for cancellation

Idempotency Design

Idempotency Key Strategy:

• Clients send X-Idempotency-Key header with each request
• Server stores job creation result for the key
• If same key is used, return stored result instead of reprocessing
• Key format: job-{user_id}-{client_generated_uuid}

// Example implementation in Java
public class IdempotencyService {
    private final RedisTemplate<String, String> redisTemplate;
    
    public CreateVideoJobResponse createWithIdempotency(
            String idempotencyKey, 
            CreateVideoJobRequest request) {
        String cacheKey = "idempotency:" + idempotencyKey;
        String cachedResult = redisTemplate.opsForValue().get(cacheKey);
        
        if (cachedResult != null) {
            return parseCachedResponse(cachedResult);
        }
        
        CreateVideoJobResponse response = createVideoJob(request);
        redisTemplate.opsForValue().set(
            cacheKey, 
            serializeResponse(response), 
            Duration.ofHours(1)
        );
        
        return response;
    }
}

3. Data Model Design

Entity-Relationship Diagram

erDiagram
    VIDEO {
        string id PK
        string title
        string description
        string original_file_path
        string thumbnail_path
        string status
        integer duration_seconds
        string created_by
        timestamp created_at
        timestamp updated_at
        timestamp processed_at
    }

    PROCESSING_JOB {
        string id PK
        string video_id FK
        string status
        string task_type
        string progress_percentage
        string error_message
        timestamp created_at
        timestamp updated_at
        timestamp started_at
        timestamp completed_at
    }

    VIDEO_FORMAT {
        string id PK
        string video_id FK
        string format_type
        string file_path
        integer width
        integer height
        integer bitrate
        timestamp created_at
    }

    USER {
        string id PK
        string email
        string full_name
        timestamp created_at
        timestamp updated_at
    }

    VIDEO_TAG {
        string id PK
        string video_id FK
        string tag_name
        timestamp created_at
    }

    VIDEO {
        string id PK
        string title
        string description
        string original_file_path
        string thumbnail_path
        string status
        integer duration_seconds
        string created_by
        timestamp created_at
        timestamp updated_at
        timestamp processed_at
    }

    PROCESSING_JOB {
        string id PK
        string video_id FK
        string status
        string task_type
        string progress_percentage
        string error_message
        timestamp created_at
        timestamp updated_at
        timestamp started_at
        timestamp completed_at
    }

    VIDEO_FORMAT {
        string id PK
        string video_id FK
  }

Table Schema Definitions

videos table

CREATE TABLE videos (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    title VARCHAR(255) NOT NULL,
    description TEXT,
    original_file_path TEXT NOT NULL,
    thumbnail_path TEXT,
    status VARCHAR(50) NOT NULL DEFAULT 'pending',
    duration_seconds INTEGER,
    created_by VARCHAR(255) NOT NULL,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
    processed_at TIMESTAMP WITH TIME ZONE,
    deleted_at TIMESTAMP WITH TIME ZONE
);

CREATE INDEX idx_videos_created_by ON videos(created_by);
CREATE INDEX idx_videos_status ON videos(status);
CREATE INDEX idx_videos_created_at ON videos(created_at);
CREATE INDEX idx_videos_deleted_at ON videos(deleted_at);

processing_jobs table

CREATE TABLE processing_jobs (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    video_id UUID REFERENCES videos(id) ON DELETE CASCADE,
    status VARCHAR(50) NOT NULL DEFAULT 'pending',
    task_type VARCHAR(100) NOT NULL,
    progress_percentage INTEGER DEFAULT 0,
    error_message TEXT,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
    started_at TIMESTAMP WITH TIME ZONE,
    completed_at TIMESTAMP WITH TIME ZONE
);

CREATE INDEX idx_processing_jobs_video_id ON processing_jobs(video_id);
CREATE INDEX idx_processing_jobs_status ON processing_jobs(status);
CREATE INDEX idx_processing_jobs_created_at ON processing_jobs(created_at);

video_formats table

CREATE TABLE video_formats (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    video_id UUID REFERENCES videos(id) ON DELETE CASCADE,
    format_type VARCHAR(50) NOT NULL,
    file_path TEXT NOT NULL,
    width INTEGER,
    height INTEGER,
    bitrate INTEGER,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
);

CREATE INDEX idx_video_formats_video_id ON video_formats(video_id);
CREATE INDEX idx_video_formats_format_type ON video_formats(format_type);

users table

CREATE TABLE users (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    email VARCHAR(255) UNIQUE NOT NULL,
    full_name VARCHAR(255),
    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
);

video_tags table

CREATE TABLE video_tags (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    video_id UUID REFERENCES videos(id) ON DELETE CASCADE,
    tag_name VARCHAR(100) NOT NULL,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
);

CREATE INDEX idx_video_tags_video_id ON video_tags(video_id);
CREATE INDEX idx_video_tags_tag_name ON video_tags(tag_name);

Primary Key Strategy

UUID (Universally Unique Identifier) is selected as the primary key strategy for all tables due to:

1. Distributed system compatibility - UUIDs can be generated without coordination across services
2. Scalability - No need for centralized ID generation, avoiding bottlenecks
3. Security - UUIDs don't reveal information about creation order or system internals
4. Cross-service integration - Easier to merge data from multiple systems

Index Design

Table	Index Name	Columns	Type	Purpose
videos	idx_videos_created_by	created_by	Standard	User-based filtering
videos	idx_videos_status	status	Standard	Status-based queries
videos	idx_videos_created_at	created_at	Standard	Time-range filtering
videos	idx_videos_deleted_at	deleted_at	Partial	Soft-delete queries
processing_jobs	idx_processing_jobs_video_id	video_id	Standard	Video-specific job lookups
processing_jobs	idx_processing_jobs_status	status	Standard	Job status monitoring
video_formats	idx_video_formats_video_id	video_id	Standard	Format lookups by video
video_tags	idx_video_tags_video_id	video_id	Standard	Tag lookups by video
video_tags	idx_video_tags_tag_name	tag_name	Standard	Tag-based filtering

Data Partitioning / Sharding Strategy

Shard by user ID (created_by) for videos table:

1. Partitioning Key: created_by field
2. Shard Range: 100 shards (0-99)
3. Partitioning Logic: hash(created_by) % 100
4. Rationale:
   • Even distribution of user workloads
   • Supports horizontal scaling across database instances
   • Maintains referential integrity within user scope

Soft Delete vs. Hard Delete

Soft delete is implemented with deleted_at timestamp field:

1. Rationale:

   • Data recovery capability for accidental deletions
   • Audit trail preservation for compliance
   • Prevents cascading data loss in related tables

2. Implementation:

   • All queries filter out soft-deleted records by default
   • Admin-level operations can access soft-deleted data

Audit Trail Design

Audit logging approach:

1. Fields captured:

   • user_id (via auth context)
   • operation_type (CREATE, UPDATE, DELETE)
   • entity_id (video ID)
   • timestamp
   • old_values (for updates)
   • new_values (for updates)

2. Storage:

   • Separate audit_log table or embedded in each entity
   • For this design, we'll use a separate audit_log table

CREATE TABLE audit_logs (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    user_id VARCHAR(255),
    operation_type VARCHAR(20) NOT NULL,
    entity_type VARCHAR(50) NOT NULL,
    entity_id UUID NOT NULL,
    old_values JSONB,
    new_values JSONB,
    timestamp TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
);

Data Archival Strategy and Retention Policy

Archival strategy:

1. Archive criteria: Videos that are older than 2 years and have no active processing jobs
2. Archive location: S3-compatible storage with lifecycle policies
3. Retention policy:
   • Active videos: 1 year (with daily backups)
   • Processed videos: 2 years (with monthly snapshots)
   • Failed videos: 30 days (for debugging purposes)

-- Archive trigger logic example (pseudo-code)
IF video.status = 'processed' 
   AND video.created_at < NOW() - INTERVAL '2 years'
   AND NOT EXISTS (SELECT 1 FROM processing_jobs WHERE video_id = video.id)
THEN
    -- Move to archive storage
    -- Update status to 'archived'
END IF;

4. Class Design & Service Layer

classDiagram
    class VideoProcessorService {
        +processVideo(VideoInput) VideoProcessingResult
        +getVideoStatus(String videoId) VideoStatus
        +cancelProcessing(String videoId) boolean
    }
    
    class Video {
        -String id
        -String originalFilePath
        -VideoStatus status
        -String thumbnailUrl
        -List~VideoFormat~ formats
        -Instant createdAt
        -Instant updatedAt
    }
    
    class VideoFormat {
        -String formatType
        -String encodedFilePath
        -int width
        -int height
        -double duration
        -String bitrate
    }
    
    class VideoProcessingResult {
        -String videoId
        -List~VideoFormat~ formats
        -String thumbnailUrl
    }
    
    class VideoStatus {
        -String value
        -static final VideoStatus PENDING
        -static final VideoStatus PROCESSING
        -static final VideoStatus COMPLETED
        -static final VideoStatus FAILED
    }
    
    class VideoProcessingEvent {
        -String videoId
        -VideoStatus status
        -String errorMessage
    }
    
    class VideoProcessor {
        -String ffmpegBinaryPath
        -int maxConcurrentJobs
        -Map~String, VideoProcessorWorker~ workers
        -ExecutorService executorService
        +processVideo(VideoInput) VideoProcessingResult
    }
    
    class VideoProcessorWorker {
        -String workerId
        -VideoProcessingTask task
        -boolean isRunning
        +executeTask()
        +cancelTask()
    }
    
    class VideoProcessingTask {
        -String videoId
        -VideoInput input
        -VideoStatus status
        -List~VideoFormat~ outputFormats
        -String thumbnailPath
        +execute()
        +updateStatus(VideoStatus)
    }
    
    class VideoStorageService {
        -String storageBasePath
        +uploadFile(InputStream, String) String
        +downloadFile(String) InputStream
        +deleteFile(String) boolean
    }
    
    class VideoFormatProcessor {
        -String ffmpegBinaryPath
        +process(VideoInput, VideoFormatConfig) VideoFormat
    }
    
    class VideoFormatConfig {
        -String formatType
        -int width
        -int height
        -int bitrate
    }
    
    class EventPublisher {
        -EventBus eventBus
        +publish(VideoProcessingEvent)
    }
    
    VideoProcessorService --> Video : manages
    VideoProcessorService --> VideoStorageService : uses
    VideoProcessorService --> VideoProcessor : orchestrates
    VideoProcessor --> VideoProcessingTask : creates
    VideoProcessor --> VideoFormatProcessor : delegates
    VideoProcessor --> EventPublisher : publishes
    VideoProcessingTask --> VideoFormatProcessor : processes formats
    VideoProcessingTask --> VideoStorageService : stores files
    VideoProcessingTask --> EventPublisher : publishes events

stateDiagram-v2
    [*] --> PENDING
    PENDING --> PROCESSING : start processing
    PROCESSING --> COMPLETED : success
    PROCESSING --> FAILED : error
    FAILED --> PENDING : retry
    COMPLETED --> [*]

Service Class Structure and Responsibilities

The core service layer for video processing is organized around several key components:

1. VideoProcessorService - Main orchestrator that handles the end-to-end video processing workflow
2. VideoProcessor - Manages concurrent processing workers and task execution
3. VideoProcessingTask - Individual unit of work for processing a single video
4. VideoFormatProcessor - Handles format-specific encoding operations
5. EventPublisher - Manages event publishing for processing status updates

Interface / Abstract Class Design

public interface VideoProcessor {
    VideoProcessingResult processVideo(VideoInput input) throws VideoProcessingException;
    VideoStatus getVideoStatus(String videoId);
    boolean cancelProcessing(String videoId);
}

public interface VideoStorageService {
    String uploadFile(InputStream fileStream, String fileName) throws StorageException;
    InputStream downloadFile(String filePath) throws StorageException;
    boolean deleteFile(String filePath) throws StorageException;
}

public interface VideoFormatProcessor {
    VideoFormat process(VideoInput input, VideoFormatConfig config) throws VideoProcessingException;
}

Business Logic Encapsulation Patterns

Strategy Pattern Implementation for Format Processing

public enum VideoFormatType {
    MP4_720P,
    MP4_1080P,
    WEBM_720P,
    GIF
}

public interface VideoFormatStrategy {
    VideoFormat process(VideoInput input, VideoFormatConfig config) throws VideoProcessingException;
}

public class Mp4720PStrategy implements VideoFormatStrategy {
    private final String ffmpegBinaryPath;
    
    public Mp4720PStrategy(String ffmpegBinaryPath) {
        this.ffmpegBinaryPath = ffmpegBinaryPath;
    }
    
    @Override
    public VideoFormat process(VideoInput input, VideoFormatConfig config) throws VideoProcessingException {
        try {
            // Execute FFmpeg command for MP4 720p conversion
            String command = String.format("%s -i %s -vf scale=1280:720 -c:v libx264 -preset fast %s",
                ffmpegBinaryPath, input.getFilePath(), config.getOutputPath());
            
            ProcessBuilder processBuilder = new ProcessBuilder(command.split(" "));
            Process process = processBuilder.start();
            
            int exitCode = process.waitFor();
            if (exitCode != 0) {
                throw new VideoProcessingException("Failed to process MP4 720p format");
            }
            
            return new VideoFormat("MP4_720P", config.getOutputPath(), 1280, 720, 
                calculateBitrate(input.getDuration(), 1280, 720));
        } catch (Exception e) {
            throw new VideoProcessingException("Error processing MP4 720p format", e);
        }
    }
    
    private String calculateBitrate(double duration, int width, int height) {
        // Simplified bitrate calculation
        return String.format("%dK", (int)(width * height * duration / 1000));
    }
}

public class VideoFormatProcessorFactory {
    private final String ffmpegBinaryPath;
    
    public VideoFormatProcessorFactory(String ffmpegBinaryPath) {
        this.ffmpegBinaryPath = ffmpegBinaryPath;
    }
    
    public VideoFormatStrategy getStrategy(VideoFormatType type) {
        switch (type) {
            case MP4_720P:
                return new Mp4720PStrategy(ffmpegBinaryPath);
            case MP4_1080P:
                return new Mp41080PStrategy(ffmpegBinaryPath);
            case WEBM_720P:
                return new Webm720PStrategy(ffmpegBinaryPath);
            case GIF:
                return new GifStrategy(ffmpegBinaryPath);
            default:
                throw new IllegalArgumentException("Unsupported video format: " + type);
        }
    }
}

Critical-Path Method Implementation

public VideoProcessingResult processVideo(VideoInput input) throws VideoProcessingException {
    // Transaction boundary - all operations within this method are part of one transaction
    String videoId = UUID.randomUUID().toString();
    
    // 1. Validate input and create video record
    Video video = new Video(videoId, input.getOriginalFilePath(), VideoStatus.PENDING);
    
    try {
        // 2. Store original file in storage
        String storedFilePath = videoStorageService.uploadFile(input.getFileStream(), 
            "videos/" + videoId + "_" + input.getFileName());
        
        // 3. Update video record with storage path
        video.setOriginalFilePath(storedFilePath);
        video.setStatus(VideoStatus.PROCESSING);
        
        // 4. Process formats in parallel
        List<CompletableFuture<VideoFormat>> formatFutures = new ArrayList<>();
        
        for (VideoFormatConfig formatConfig : input.getRequestedFormats()) {
            CompletableFuture<VideoFormat> future = CompletableFuture.supplyAsync(() -> {
                try {
                    VideoFormatStrategy strategy = videoFormatProcessorFactory.getStrategy(
                        VideoFormatType.valueOf(formatConfig.getFormatType()));
                    return strategy.process(input, formatConfig);
                } catch (VideoProcessingException e) {
                    throw new RuntimeException(e);
                }
            });
            
            formatFutures.add(future);
        }
        
        // 5. Wait for all formats to complete
        List<VideoFormat> processedFormats = new ArrayList<>();
        for (CompletableFuture<VideoFormat> future : formatFutures) {
            processedFormats.add(future.get());
        }
        
        // 6. Generate thumbnail
        String thumbnailPath = generateThumbnail(input, videoId);
        
        // 7. Update video record with results
        video.setFormats(processedFormats);
        video.setThumbnailUrl(thumbnailPath);
        video.setStatus(VideoStatus.COMPLETED);
        
        // 8. Publish completion event
        VideoProcessingEvent event = new VideoProcessingEvent(videoId, VideoStatus.COMPLETED);
        eventPublisher.publish(event);
        
        return new VideoProcessingResult(videoId, processedFormats, thumbnailPath);
        
    } catch (Exception e) {
        // Handle failure case
        video.setStatus(VideoStatus.FAILED);
        video.setErrorMessage(e.getMessage());
        
        VideoProcessingEvent event = new VideoProcessingEvent(videoId, VideoStatus.FAILED, e.getMessage());
        eventPublisher.publish(event);
        
        throw new VideoProcessingException("Video processing failed", e);
    }
}

Transaction Boundary Design

The transaction boundary encompasses:

1. File storage operations (upload, cleanup)
2. Database state updates (video record creation, status updates)
3. Event publishing for processing status changes
4. Format encoding operations (FFmpeg executions)

All these operations are treated as a single atomic unit with rollback capabilities where possible.

Service Interface Contracts

public interface VideoProcessingService {
    /**
     * Initiates video processing workflow
     * @param input contains source file and requested output formats
     * @return result containing processed video information
     * @throws VideoProcessingException when processing fails
     */
    VideoProcessingResult processVideo(VideoInput input) throws VideoProcessingException;
    
    /**
     * Retrieves current processing status of a video
     * @param videoId unique identifier for the video
     * @return current status of the video processing
     */
    VideoStatus getVideoStatus(String videoId);
    
    /**
     * Cancels ongoing video processing
     * @param videoId unique identifier for the video
     * @return true if cancellation was successful
     */
    boolean cancelProcessing(String videoId);
}

Saga / Orchestration Pattern

The video processing workflow uses a saga pattern for distributed transactions:

1. Start: Create video record in database
2. Process: Execute format conversions in parallel
3. Complete: Update database with results and publish events
4. Compensate: Handle failures by cleaning up resources

Domain Object / Value Object Design

public class Video {
    private final String id;
    private final String originalFilePath;
    private VideoStatus status;
    private String thumbnailUrl;
    private List<VideoFormat> formats;
    private final Instant createdAt;
    private Instant updatedAt;
    
    public Video(String id, String originalFilePath, VideoStatus status) {
        this.id = id;
        this.originalFilePath = originalFilePath;
        this.status = status;
        this.createdAt = Instant.now();
        this.updatedAt = Instant.now();
    }
    
    // Getters and setters with proper encapsulation
}

public class VideoFormat {
    private final String formatType;
    private final String encodedFilePath;
    private final int width;
    private final int height;
    private final double duration;
    private final String bitrate;
    
    public VideoFormat(String formatType, String encodedFilePath, int width, int height, double duration, String bitrate) {
        this.formatType = formatType;
        this.encodedFilePath = encodedFilePath;
        this.width = width;
        this.height = height;
        this.duration = duration;
        this.bitrate = bitrate;
    }
    
    // Getters only - immutable value object
}

public class VideoInput {
    private final InputStream fileStream;
    private final String fileName;
    private final String originalFilePath;
    private final List<VideoFormatConfig> requestedFormats;
    private final double duration;
    
    // Constructor and getters
}

5. Concurrency & Thread Safety

Concurrency Model

The video processing workflow will leverage a hybrid concurrency model combining:

• Thread pools for CPU-intensive video encoding/decoding operations using java.util.concurrent.ThreadPoolExecutor
• Reactive streams for handling asynchronous event processing with Project Reactor
• Async/Await pattern for I/O-bound operations like file system access and external API calls

Race Condition Prevention Strategy

Race conditions will be mitigated through:

1. Immutable domain objects - all video processing entities are immutable to prevent concurrent modification
2. Database-level atomic operations - using optimistic locking with version fields for state transitions
3. Thread-local storage for temporary processing context to avoid cross-thread contamination
4. Synchronized access to shared resources using java.util.concurrent.locks.ReentrantReadWriteLock

Optimistic vs. Pessimistic Locking

Optimistic locking is selected for the video processing workflow because:

• Video processing jobs are typically long-running (minutes to hours)
• Concurrent access is low to moderate
• The system prioritizes throughput over strict consistency during processing
• Version-based conflict resolution is more efficient than database locks for this use case

Implementation pattern:

// Pessimistic locking would require explicit DB locks, which are expensive for long-running jobs
// Optimistic locking uses version field in database

@Entity
@Table(name = "video_jobs")
public class VideoJob {
    @Id
    private UUID id;
    
    @Version
    private Long version; // Used for optimistic locking
    
    private VideoJobStatus status;
    
    // ... other fields
}

Deadlock Prevention Strategy

Deadlock prevention mechanisms:

1. Fixed ordering of resource acquisition - all database operations follow a consistent order (by job ID, then by entity type)
2. Timeout-based lock acquisition - 5-second timeout for database locks
3. Hierarchical job dependency tracking - prevents circular dependencies between jobs
4. Thread pool size limiting - controlled parallelism to prevent resource exhaustion

Idempotency Implementation for Concurrent Duplicate Requests

Idempotency is implemented using:

1. Idempotency keys in request headers for external API calls
2. Database uniqueness constraints on job identifiers and processing parameters
3. State machine guard conditions that prevent duplicate transitions

@Service
public class VideoProcessingService {
    
    private final Map<String, CompletableFuture<Void>> activeOperations = new ConcurrentHashMap<>();
    
    public CompletableFuture<VideoJob> processVideoAsync(VideoProcessingRequest request) {
        String idempotencyKey = generateIdempotencyKey(request);
        
        return activeOperations.computeIfAbsent(idempotencyKey, key -> {
            // Check if job already exists
            VideoJob existing = videoJobRepository.findByJobId(request.getJobId());
            if (existing != null && existing.getStatus() == VideoJobStatus.COMPLETED) {
                return CompletableFuture.completedFuture(existing);
            }
            
            // Proceed with processing
            return CompletableFuture.supplyAsync(() -> {
                try {
                    return processVideoJob(request);
                } catch (Exception e) {
                    throw new RuntimeException("Failed to process video job", e);
                }
            });
        });
    }
    
    private String generateIdempotencyKey(VideoProcessingRequest request) {
        return "video_process_" + 
               request.getVideoId() + "_" + 
               request.getProcessingProfile().getName();
    }
}

Thread Safety in Critical Components

1. Job state transitions - using java.util.concurrent.atomic.AtomicReference for atomic state updates
2. Progress tracking - thread-safe counters using java.util.concurrent.atomic.LongAdder
3. Shared configuration objects - immutable configuration classes with final fields
4. Event publishing - synchronized event publisher that ensures thread-safe event dispatching

Concurrency Control for Batch Operations

Batch video processing operations will use:

• Fixed thread pool with size configurable based on available CPU cores and I/O capacity
• Work-stealing algorithm for efficient load distribution across processing threads
• Semaphore-based resource limiting to prevent overwhelming system resources during peak loads

@Configuration
public class VideoProcessingConfig {
    
    @Bean("videoProcessingExecutor")
    public ExecutorService videoProcessingExecutor() {
        int corePoolSize = Runtime.getRuntime().availableProcessors();
        int maxPoolSize = corePoolSize * 2;
        
        return new ThreadPoolExecutor(
            corePoolSize,
            maxPoolSize,
            60L, TimeUnit.SECONDS,
            new LinkedBlockingQueue<>(1000),
            ThreadFactoryBuilder
                .newBuilder()
                .setNameFormat("video-processing-%d")
                .build(),
            new ThreadPoolExecutor.CallerRunsPolicy()
        );
    }
}

6. Sequence Diagrams for Key Flows

Happy Path Sequence Diagram

sequenceDiagram
    participant U as User
    participant API as Video API Gateway
    participant SVC as Video Processing Service
    participant DB as Database
    participant MQ as Message Queue
    participant FFmpeg as FFmpeg Processor
    participant Storage as Object Storage

    U->>API: Upload video (multipart/form-data)
    API->>SVC: CreateVideoRequest
    SVC->>DB: Save video metadata (status=PENDING)
    SVC->>MQ: Publish VideoProcessEvent
    MQ->>FFmpeg: Consume VideoProcessEvent
    FFmpeg->>Storage: Store processed video (mp4, webm)
    FFmpeg->>DB: Update video metadata (status=PROCESSED, duration=120)
    SVC->>API: Return video ID and status
    U->>API: Poll for processing completion
    API->>DB: Fetch video status
    API->>U: Return final video metadata

Error / Edge Case Sequence Diagram

sequenceDiagram
    participant U as User
    participant API as Video API Gateway
    participant SVC as Video Processing Service
    participant DB as Database
    participant MQ as Message Queue
    participant FFmpeg as FFmpeg Processor
    participant Storage as Object Storage

    U->>API: Upload video (multipart/form-data)
    API->>SVC: CreateVideoRequest
    SVC->>DB: Save video metadata (status=PENDING)
    SVC->>MQ: Publish VideoProcessEvent
    MQ->>FFmpeg: Consume VideoProcessEvent
    FFmpeg->>Storage: Store processed video (mp4, webm)
    FFmpeg->>DB: Update video metadata (status=FAILED, error_msg="Invalid format")
    SVC->>API: Return error response
    U->>API: Retry upload or cancel
    API->>SVC: CancelVideoRequest
    SVC->>DB: Update status=CANCELLED

Async Flow Sequence Diagram

sequenceDiagram
    participant U as User
    participant API as Video API Gateway
    participant SVC as Video Processing Service
    participant DB as Database
    participant MQ as Message Queue
    participant FFmpeg as FFmpeg Processor
    participant Storage as Object Storage
    participant Notifier as Notification Service

    U->>API: Upload video (multipart/form-data)
    API->>SVC: CreateVideoRequest
    SVC->>DB: Save video metadata (status=PENDING)
    SVC->>MQ: Publish VideoProcessEvent
    MQ->>FFmpeg: Consume VideoProcessEvent
    FFmpeg->>Storage: Store processed video (mp4, webm)
    FFmpeg->>DB: Update video metadata (status=PROCESSED, duration=120)
    SVC->>Notifier: SendProcessingCompleteEvent
    Notifier->>API: Notify user via Webhook/Email
    API->>U: Return video ID and status

7. Caching Design

Cache Strategy Selection

For video processing workflow, I'll use cache-aside pattern with a distributed Redis cluster as the primary cache layer. This approach provides:

• Flexibility: Cache can be invalidated or updated independently of the data source
• Scalability: Redis cluster supports horizontal scaling and high availability
• Performance: Fast read operations for frequently accessed video metadata and processing status

Cache Key Design

Cache keys will follow a structured namespace pattern:

video:{videoId}:{type}:{subKey}

Examples:

• video:12345:metadata:
• video:12345:status:processing
• video:12345:thumbnails:small
• video:12345:transcoding:mp4_720p

TTL Configuration

Data Type	TTL	Reason
Video Metadata	1 hour	Frequently accessed, relatively stable
Processing Status	30 minutes	Short-lived, changes frequently during processing
Transcoded Video URLs	24 hours	Cached URLs remain valid for extended periods
Thumbnail Images	1 week	Thumbnails are stable once generated
User Preferences	1 hour	User preferences may change but not immediately

Cache Eviction Policy

LRU (Least Recently Used) with TTL-based expiration as the primary policy. This ensures:

• Hot data stays in cache based on access patterns
• Memory is freed for newer content automatically
• Prevents stale data from bloating cache

Cache Stampede Prevention

Implement probabilistic early expiry with 15% random variance:

// Pseudo-code for cache stampede prevention
public String getCachedValue(String key, Supplier<String> loader) {
    String cached = redis.get(key);
    if (cached != null) {
        // Check if cache is expiring soon (within 15% of TTL)
        long ttl = redis.ttl(key);
        if (ttl < (TTL * 0.15)) {
            // Add random delay to prevent stampede
            Thread.sleep(ThreadLocalRandom.current().nextInt(50, 200));
        }
        return cached;
    }
    
    // Cache miss - load and set with probabilistic early expiry
    String value = loader.get();
    long ttlWithVariation = (long) (TTL * 0.85 + ThreadLocalRandom.current().nextDouble() * 0.15 * TTL);
    redis.setex(key, ttlWithVariation, value);
    return value;
}

Distributed Cache Architecture

Redis Cluster with Sentinel:

• 3 master nodes for data distribution
• 3 replica nodes for high availability
• Automatic failover with Redis Sentinel
• Cross-region replication for disaster recovery

Cache Invalidation Strategy

Event-driven invalidation using the outbox pattern:

1. When video metadata changes, publish VideoMetadataUpdated event
2. When processing completes, publish VideoProcessingComplete event
3. Cache consumers listen to these events and invalidate relevant cache entries

// Example event-driven invalidation
@EventListener
public void handleVideoMetadataUpdate(VideoMetadataUpdatedEvent event) {
    String key = "video:" + event.getVideoId() + ":metadata:";
    redis.del(key);
    
    // Invalidate related processing status
    String statusKey = "video:" + event.getVideoId() + ":status:";
    redis.del(statusKey);
}

Cache Read-Through Pattern

public class VideoMetadataCache {
    private final RedisTemplate<String, VideoMetadata> redisTemplate;
    private final VideoMetadataService videoMetadataService;
    
    public VideoMetadata getVideoMetadata(String videoId) {
        String cacheKey = "video:" + videoId + ":metadata:";
        
        // Try cache first
        VideoMetadata metadata = redisTemplate.opsForValue().get(cacheKey);
        if (metadata != null) {
            return metadata;
        }
        
        // Cache miss - fetch from database and populate cache
        metadata = videoMetadataService.getVideoMetadata(videoId);
        if (metadata != null) {
            redisTemplate.opsForValue().set(
                cacheKey, 
                metadata, 
                Duration.ofHours(1)
            );
        }
        
        return metadata;
    }
}

Cache Performance Considerations

• Warm-up strategy: Pre-populate cache for popular videos during startup
• Cache warming: Use background jobs to warm cache for trending content
• Memory monitoring: Track cache hit ratio and adjust TTLs dynamically based on access patterns

Cache Consistency Trade-offs

• Eventual consistency: Cache updates are asynchronous, which may cause brief inconsistencies
• Read-after-write consistency: For critical operations like status checks, use cache bypass or read-through with immediate refresh
• Cache warming: Pre-warm cache for high-priority content to minimize latency spikes

8. Repository / Data Access Layer

Repository Pattern Implementation

The repository pattern is implemented to abstract data access logic and provide a clean separation between domain logic and persistence concerns. Each entity will have its own repository interface and implementation.

public interface VideoRepository {
    Video save(Video video);
    Optional<Video> findById(UUID id);
    List<Video> findByStatus(VideoStatus status);
    void updateStatus(UUID id, VideoStatus status);
    void delete(UUID id);
}

public class VideoRepositoryImpl implements VideoRepository {
    private final JdbcTemplate jdbcTemplate;
    private final ObjectMapper objectMapper;
    
    @Override
    public Video save(Video video) {
        String sql = """
            INSERT INTO videos (id, file_path, status, metadata, created_at, updated_at) 
            VALUES (?, ?, ?, ?, ?, ?)
            """;
            
        jdbcTemplate.update(sql, video.getId(), video.getFilePath(), 
                           video.getStatus().name(), 
                           objectMapper.writeValueAsString(video.getMetadata()),
                           video.getCreatedAt(), video.getUpdatedAt());
        
        return video;
    }
    
    @Override
    public Optional<Video> findById(UUID id) {
        String sql = "SELECT * FROM videos WHERE id = ?";
        try {
            return Optional.of(jdbcTemplate.queryForObject(sql, new VideoRowMapper(), id));
        } catch (EmptyResultDataAccessException e) {
            return Optional.empty();
        }
    }
    
    @Override
    public List<Video> findByStatus(VideoStatus status) {
        String sql = "SELECT * FROM videos WHERE status = ? ORDER BY created_at ASC";
        return jdbcTemplate.query(sql, new VideoRowMapper(), status.name());
    }
    
    @Override
    public void updateStatus(UUID id, VideoStatus status) {
        String sql = "UPDATE videos SET status = ?, updated_at = ? WHERE id = ?";
        jdbcTemplate.update(sql, status.name(), Instant.now(), id);
    }
    
    @Override
    public void delete(UUID id) {
        String sql = "DELETE FROM videos WHERE id = ?";
        jdbcTemplate.update(sql, id);
    }
}

ORM vs Raw Query Decisions

For this system, we use raw SQL queries with JdbcTemplate for better performance and control over database operations. The decision is based on:

1. Performance: Raw queries avoid ORM overhead and allow for precise query optimization
2. Complexity: Video processing workflows involve complex joins and batch operations that are better handled with raw SQL
3. Database portability: We're targeting PostgreSQL, where raw queries provide better control over features like JSONB handling

Read vs Write Separation

We implement CQRS pattern for read and write operations:

public interface VideoReadRepository {
    VideoView findVideoViewById(UUID id);
    List<VideoSummary> findVideosByStatus(VideoStatus status, Pageable pageable);
}

public class VideoReadRepositoryImpl implements VideoReadRepository {
    private final JdbcTemplate jdbcTemplate;
    
    @Override
    public VideoView findVideoViewById(UUID id) {
        String sql = """
            SELECT v.id, v.file_path, v.status, v.metadata, 
                   v.created_at, v.updated_at,
                   p.processing_steps, p.current_step
            FROM videos v
            LEFT JOIN video_processing_info p ON v.id = p.video_id
            WHERE v.id = ?
            """;
        return jdbcTemplate.queryForObject(sql, new VideoViewRowMapper(), id);
    }
    
    @Override
    public List<VideoSummary> findVideosByStatus(VideoStatus status, Pageable pageable) {
        String sql = """
            SELECT id, file_path, status, created_at, updated_at
            FROM videos 
            WHERE status = ?
            ORDER BY created_at DESC
            LIMIT ? OFFSET ?
            """;
        return jdbcTemplate.query(sql, new VideoSummaryRowMapper(), 
                                status.name(), pageable.getPageSize(), 
                                pageable.getOffset());
    }
}

Connection Pool Configuration

@Configuration
public class DatabaseConfig {
    
    @Bean
    public DataSource dataSource() {
        HikariConfig config = new HikariConfig();
        config.setJdbcUrl("jdbc:postgresql://localhost:5432/video_processing");
        config.setUsername("user");
        config.setPassword("password");
        config.setMaximumPoolSize(20);
        config.setMinimumIdle(5);
        config.setConnectionTimeout(30000);
        config.setIdleTimeout(600000);
        config.setMaxLifetime(1800000);
        config.setLeakDetectionThreshold(60000);
        return new HikariDataSource(config);
    }
}

N+1 Query Prevention Strategy

We use eager loading and batch operations to prevent N+1 queries:

public class VideoBatchProcessor {
    
    public List<VideoView> batchLoadVideos(List<UUID> videoIds) {
        if (videoIds.isEmpty()) return Collections.emptyList();
        
        String sql = """
            SELECT v.id, v.file_path, v.status, v.metadata, 
                   v.created_at, v.updated_at,
                   p.processing_steps, p.current_step
            FROM videos v
            LEFT JOIN video_processing_info p ON v.id = p.video_id
            WHERE v.id IN (%s)
            ORDER BY v.created_at DESC
            """.formatted(String.join(",", Collections.nCopies(videoIds.size(), "?")));
        
        return jdbcTemplate.query(sql, videoIds.toArray(), new VideoViewRowMapper());
    }
}

Batch Operation Design

public class VideoBatchRepository {
    
    public void batchUpdateStatus(List<UUID> videoIds, VideoStatus status) {
        String sql = "UPDATE videos SET status = ?, updated_at = ? WHERE id = ?";
        
        try (Connection conn = dataSource.getConnection();
             PreparedStatement stmt = conn.prepareStatement(sql)) {
            
            for (UUID id : videoIds) {
                stmt.setString(1, status.name());
                stmt.setTimestamp(2, Timestamp.from(Instant.now()));
                stmt.setObject(3, id);
                stmt.addBatch();
            }
            
            stmt.executeBatch();
        } catch (SQLException e) {
            throw new DataAccessException("Failed to batch update video statuses", e);
        }
    }
    
    public void batchSaveProcessingInfo(List<VideoProcessingInfo> processingInfos) {
        String sql = """
            INSERT INTO video_processing_info (video_id, processing_steps, current_step, created_at, updated_at)
            VALUES (?, ?, ?, ?, ?)
            ON CONFLICT (video_id) DO UPDATE SET
                processing_steps = EXCLUDED.processing_steps,
                current_step = EXCLUDED.current_step,
                updated_at = EXCLUDED.updated_at
            """;
            
        try (Connection conn = dataSource.getConnection();
             PreparedStatement stmt = conn.prepareStatement(sql)) {
            
            for (VideoProcessingInfo info : processingInfos) {
                stmt.setObject(1, info.getVideoId());
                stmt.setString(2, info.getProcessingSteps());
                stmt.setString(3, info.getCurrentStep());
                stmt.setTimestamp(4, Timestamp.from(info.getCreatedAt()));
                stmt.setTimestamp(5, Timestamp.from(info.getUpdatedAt()));
                stmt.addBatch();
            }
            
            stmt.executeBatch();
        } catch (SQLException e) {
            throw new DataAccessException("Failed to batch save processing info", e);
        }
    }
}

9. Asynchronous Processing & Messaging

Event-driven boundaries in this feature

The video processing workflow is inherently asynchronous due to the time-intensive nature of transcoding, thumbnail generation, and metadata extraction. The system will use a hybrid synchronous/asynchronous approach:

• Synchronous flows: Video upload, status polling, and metadata retrieval
• Asynchronous flows:
  • Video transcoding (multiple formats)
  • Thumbnail generation
  • Metadata extraction and indexing
  • Notification delivery

The decision to use async processing stems from the need to handle large files efficiently without blocking client requests. Each processing task is independent and can be parallelized.

Message schema design

{
  "version": "1.0",
  "event_type": "video.processing.started",
  "timestamp": "2023-10-15T14:30:00Z",
  "payload": {
    "video_id": "uuid-12345",
    "source_file_path": "/videos/original/12345.mp4",
    "processing_jobs": [
      {
        "job_id": "job-1",
        "type": "transcode",
        "format": "mp4-hd",
        "priority": 1
      },
      {
        "job_id": "job-2",
        "type": "thumbnail",
        "resolution": "1920x1080",
        "count": 3
      }
    ]
  },
  "correlation_id": "upload-req-abc"
}

Dead letter queue strategy and poison message handling

For failed processing jobs, we implement a dead-letter queue (DLQ) pattern:

1. Retry mechanism: Messages are retried up to 3 times with exponential backoff (5s, 25s, 125s)
2. DLQ routing: Messages that fail after max retries are moved to a video.processing.failed topic
3. Manual intervention: Failed messages in DLQ are logged with detailed error context and sent to monitoring dashboard
4. Retry timeout: Messages that have been in DLQ for more than 24 hours are archived

At-least-once vs. exactly-once delivery

At-least-once delivery is selected for this system because:

1. Idempotency: Processing jobs are designed to be idempotent (reprocessing same job doesn't change outcome)
2. Simpler implementation: Easier to implement than exactly-once semantics
3. Performance: Avoids complex state tracking required for exactly-once

Retry policy for consumers

public class VideoProcessingRetryPolicy {
    private static final int MAX_RETRIES = 3;
    private static final long BASE_DELAY_MS = 5000;
    private static final double BACKOFF_MULTIPLIER = 5.0;
    
    public boolean shouldRetry(ProcessingJob job, int attempt) {
        if (attempt >= MAX_RETRIES) return false;
        
        // Don't retry on certain error types
        if (isTerminalError(job.getLastError())) {
            return false;
        }
        
        return true;
    }
    
    public long getDelay(int attempt) {
        return (long) (BASE_DELAY_MS * Math.pow(BACKOFF_MULTIPLIER, attempt));
    }
    
    private boolean isTerminalError(String errorType) {
        return Set.of("INVALID_FORMAT", "INSUFFICIENT_STORAGE").contains(errorType);
    }
}

Outbox pattern for reliable event publishing

Outbox pattern is implemented to ensure message delivery reliability:

1. Transaction boundary: Events are written to an event_outbox table within the same database transaction as business logic
2. Background processor: A separate daemon polls event_outbox and publishes messages to message broker
3. Idempotency: Events are marked as published after successful delivery

@Entity
@Table(name = "event_outbox")
public class EventOutbox {
    @Id
    private UUID id;
    
    @Column(name = "event_type")
    private String eventType;
    
    @Column(name = "payload")
    private String payload;
    
    @Column(name = "status")
    private EventStatus status = EventStatus.PENDING;
    
    @Column(name = "created_at")
    private LocalDateTime createdAt;
}

Sequence diagram for async flows

sequenceDiagram
    participant U as User
    participant API as VideoService
    participant DB as Database
    participant MQ as MessageQueue
    participant Worker as VideoProcessingWorker
    participant Storage as ObjectStorage
    
    U->>API: Upload video
    API->>DB: Save video metadata (status=UPLOADING)
    API->>MQ: Publish VIDEO_UPLOADED event
    MQ->>Worker: Consume VIDEO_UPLOADED
    Worker->>Storage: Store original file
    Worker->>DB: Update video status=PROCESSING
    Worker->>MQ: Publish VIDEO_TRANSCODE_STARTED
    Worker->>Storage: Transcode to multiple formats
    Worker->>MQ: Publish VIDEO_TRANSCODE_COMPLETED
    Worker->>DB: Update video status=READY
    Worker->>MQ: Publish VIDEO_PROCESSING_COMPLETED
    API->>U: Return success response

The async flow ensures that:

1. User gets immediate feedback on upload completion
2. Heavy processing happens in background
3. System can scale processing workers independently
4. Failures are handled gracefully with retry logic and DLQ
5. Events are published in order to maintain consistency

10. Error Handling Design

Error Classification Hierarchy

Errors in the video processing workflow are categorized into three primary levels:

1. Validation Errors - Input data doesn't meet business rules or format requirements
2. Domain Errors - Business logic violations (e.g., unsupported video format, quota exceeded)
3. Infrastructure Errors - System-level issues (e.g., storage failure, processing timeout)

public abstract class VideoProcessingException extends RuntimeException {
    private final ErrorCode errorCode;
    
    public VideoProcessingException(ErrorCode code, String message) {
        super(message);
        this.errorCode = code;
    }
    
    public ErrorCode getErrorCode() {
        return errorCode;
    }
}

public class ValidationException extends VideoProcessingException {
    public ValidationException(ErrorCode code, String message) {
        super(code, message);
    }
}

public class DomainException extends VideoProcessingException {
    public DomainException(ErrorCode code, String message) {
        super(code, message);
    }
}

public class InfrastructureException extends VideoProcessingException {
    public InfrastructureException(ErrorCode code, String message) {
        super(code, message);
    }
}

Error Propagation Strategy

Errors are handled at the appropriate layer:

• Validation errors are caught and transformed into ValidationException at the API boundary
• Domain errors bubble up to the service layer and are re-thrown as DomainException
• Infrastructure errors are wrapped in InfrastructureException and may trigger circuit breaker logic

Retry Strategy with Exponential Backoff and Jitter

For transient infrastructure errors, a retry mechanism is implemented using exponential backoff with jitter:

public class RetryableOperation {
    private static final int MAX_RETRIES = 3;
    private static final long BASE_DELAY_MS = 1000;
    private static final double JITTER_FACTOR = 0.5;
    
    public <T> T executeWithRetry(Supplier<T> operation) {
        int attempt = 0;
        while (true) {
            try {
                return operation.get();
            } catch (InfrastructureException e) {
                if (++attempt >= MAX_RETRIES) {
                    throw e;
                }
                
                long delay = (long) (BASE_DELAY_MS * Math.pow(2, attempt - 1));
                delay += (long) (JITTER_FACTOR * delay * Math.random());
                
                try {
                    Thread.sleep(delay);
                } catch (InterruptedException ie) {
                    Thread.currentThread().interrupt();
                    throw new InfrastructureException(ErrorCode.RETRY_INTERRUPTED, 
                        "Retry interrupted", ie);
                }
            }
        }
    }
}

Circuit Breaker Design

The circuit breaker pattern protects downstream services from cascading failures:

public class CircuitBreaker {
    private final int failureThreshold;
    private final long timeoutMs;
    private final AtomicInteger failureCount = new AtomicInteger(0);
    private volatile CircuitState state = CircuitState.CLOSED;
    private volatile Instant lastFailureTime = null;
    
    public enum CircuitState {
        CLOSED, OPEN, HALF_OPEN
    }
    
    public <T> T execute(Supplier<T> operation) {
        switch (state) {
            case CLOSED:
                try {
                    T result = operation.get();
                    onSuccess();
                    return result;
                } catch (Exception e) {
                    onFailure(e);
                    throw e;
                }
                
            case OPEN:
                if (isTimeoutExpired()) {
                    state = CircuitState.HALF_OPEN;
                    return operation.get();
                }
                throw new InfrastructureException(ErrorCode.CIRCUIT_OPEN, 
                    "Circuit is currently open");
                
            case HALF_OPEN:
                try {
                    T result = operation.get();
                    state = CircuitState.CLOSED;
                    return result;
                } catch (Exception e) {
                    state = CircuitState.OPEN;
                    throw e;
                }
        }
        throw new IllegalStateException("Unexpected circuit state");
    }
    
    private void onSuccess() {
        failureCount.set(0);
        lastFailureTime = null;
    }
    
    private void onFailure(Exception e) {
        if (e instanceof InfrastructureException) {
            failureCount.incrementAndGet();
            lastFailureTime = Instant.now();
            
            if (failureCount.get() >= failureThreshold) {
                state = CircuitState.OPEN;
            }
        }
    }
    
    private boolean isTimeoutExpired() {
        return lastFailureTime != null && 
            Duration.between(lastFailureTime, Instant.now()).toMillis() > timeoutMs;
    }
}

Fallback Behavior Design

For critical dependencies like storage or transcoding services, fallback mechanisms are implemented:

public class VideoProcessingService {
    private final StorageService storageService;
    private final TranscodingService transcodingService;
    
    public VideoProcessingResult processVideo(VideoRequest request) {
        try {
            return transcodingService.transcode(request);
        } catch (InfrastructureException e) {
            // Fallback to basic processing or return error
            return fallbackToBasicProcessing(request, e);
        }
    }
    
    private VideoProcessingResult fallbackToBasicProcessing(
            VideoRequest request, InfrastructureException originalError) {
        // Log the failure and attempt basic processing
        logger.warn("Transcoding failed, falling back to basic processing", originalError);
        
        // Return a default or minimal result
        return new VideoProcessingResult(
            request.getVideoId(), 
            ProcessingStatus.FAILED, 
            "Basic processing fallback: " + originalError.getMessage()
        );
    }
}

Error Logging Format and Alerting Thresholds

Structured logging is implemented with consistent fields:

public class ErrorLogger {
    private static final Logger logger = LoggerFactory.getLogger(ErrorLogger.class);
    
    public void logError(String operation, String userId, String entityId, 
                        Exception exception, Map<String, Object> additionalFields) {
        Map<String, Object> logFields = new HashMap<>();
        logFields.put("timestamp", Instant.now().toString());
        logFields.put("operation", operation);
        logFields.put("userId", userId);
        logFields.put("entityId", entityId);
        logFields.put("errorCode", extractErrorCode(exception));
        logFields.put("errorMessage", exception.getMessage());
        logFields.put("stackTrace", exception.getStackTrace().toString());
        logFields.putAll(additionalFields);
        
        logger.error("Video processing error occurred", 
            new ObjectMapper().valueToTree(logFields));
    }
    
    private String extractErrorCode(Exception exception) {
        if (exception instanceof VideoProcessingException) {
            return ((VideoProcessingException) exception).getErrorCode().name();
        }
        return "UNKNOWN_ERROR";
    }
}

Alerting thresholds:

• Critical (P0): Circuit breaker opens, storage failures → On-call escalation
• Warning (P1): Retry limits exceeded, validation errors → Team alerting
• Info (P2): Non-critical infrastructure warnings → Internal monitoring

11. Security Design

Input Validation and Sanitization Strategy

All external inputs are validated using a combination of:

• Apache Commons Validator for standard format validation (email, URL, phone)
• Custom regex patterns for file name and path sanitization
• Length limits on all string inputs (max 255 chars for filenames, 1024 for descriptions)
• Whitelist-based content filtering using OWASP Java HTML Sanitizer for any user-generated content

SQL Injection Prevention

• Prepared statements exclusively used for all database queries (no string concatenation)
• Parameterized queries with named parameters in all repository methods
• ORM framework (Hibernate) used for complex queries to avoid raw SQL injection vectors

Authentication Middleware Design

• JWT-based authentication with 2-hour expiration and 15-minute refresh window
• OAuth2 integration for third-party video platform authentication (YouTube, Vimeo)
• Authentication middleware intercepts all requests and validates JWT tokens
• Token refresh endpoint (POST /auth/refresh) with rate limiting (10 requests/minute per IP)

Authorization Enforcement

• Role-Based Access Control (RBAC) with roles: USER, MODERATOR, ADMIN
• Permission checking at service layer using @PreAuthorize annotations in Spring Security
• Resource ownership validation: Users can only process videos they own (via owner_id foreign key)
• Admin scope validation for system-level operations like transcoding profile management

Sensitive Data Encryption

• At rest: AES-256 encryption using AWS KMS for video files and metadata
• In transit: TLS 1.3 with HSTS header enforcement for all API endpoints
• Secrets storage: AWS Secrets Manager for API keys and encryption keys with automatic rotation

PII Handling and Masking Strategy

• PII data (user IDs, email addresses) are logged with masking: user@example.com → u***@example.com
• Video metadata containing PII is stripped before storage in public indexes
• Audit logs do not contain raw file paths or sensitive metadata fields

Secrets Management

• AWS Secrets Manager for storing:
  • Third-party API keys (YouTube, Vimeo)
  • Encryption keys and certificates
  • Database credentials (rotated every 90 days)
• Automated rotation using AWS Lambda functions with CloudWatch triggers
• Access control via IAM policies limiting secret access to specific services only

Audit Logging Design

• Events logged:
  • Video upload initiated
  • Transcoding started/completed/failed
  • User authentication events
  • Admin actions (profile updates, system config changes)
• Log format:

{
  "timestamp": "2023-10-15T14:30:45.123Z",
  "trace_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "user_id": "user_12345",
  "event_type": "VIDEO_TRANSCODE_STARTED",
  "video_id": "video_abcde",
  "ip_address": "192.168.1.100",
  "user_agent": "Mozilla/5.0...",
  "status": "SUCCESS"
}

• Retention policy: 90 days for audit logs with daily backups to S3 Glacier
• Compliance: Logs are retained per GDPR and SOC2 requirements with searchable indexes

12. Testing Strategy

Unit Testing Scope and Coverage Targets

Unit tests will cover 85%+ of business logic and service layer code paths. Each service class will have dedicated test classes with:

• 100% branch coverage for all conditional logic paths
• 95%+ statement coverage using JaCoCo or similar tools
• Test cases for all valid and invalid input combinations

Integration Testing Strategy

Database Integration Tests:

• Use Testcontainers with PostgreSQL 13+ for real database testing
• Each test runs in isolated transaction with rollback after completion
• Use embedded H2 for fast unit tests where DB interaction is minimal

External Service Integration Tests:

• WireMock or Mockito for AWS S3, CloudFront, and notification services
• Test end-to-end workflows with mocked external dependencies
• Integration tests run in CI pipeline with dedicated test environments

End-to-End Workflow Tests:

• Test complete video upload → processing → delivery flow
• Validate event-driven orchestration and state transitions
• Test error recovery scenarios with mocked failures

Contract Testing

API Contract Tests:

• Use Pact for consumer-driven contract testing between video service and client applications
• Define contracts for all public endpoints with versioned schemas
• Validate that API responses match expected format and content

Event Contract Tests:

• Define Avro schemas for all event types with version control
• Use Pact or similar tools to test that events published match contract expectations

Performance Testing Tool and Scenarios

Load Testing:

• Use k6 for load testing with realistic concurrent user scenarios
• Test 1000+ concurrent video uploads with varying file sizes (10MB, 100MB, 1GB)
• Simulate burst traffic patterns with 5x concurrent upload spikes

Stress Testing:

• Test system behavior under extreme conditions (10k concurrent uploads)
• Validate queue backpressure and error handling under load
• Monitor memory usage, GC pressure, and thread pool exhaustion

Key Performance Scenarios:

1. Video upload and processing completion time (p95 < 30 seconds)
2. Concurrent upload throughput (1000+ uploads/sec)
3. Error recovery time (retry and fallback handling within 5 seconds)

Test Data Management Strategy

Test Fixtures:

• Use JSON/YAML fixtures for common video metadata and processing configurations
• Include sample video files (MP4, MOV) of various sizes for testing
• Maintain versioned test data sets with clear naming conventions

Test Data Factories:

public class VideoTestDataFactory {
    public static Video createVideoWithStatus(ProcessingStatus status) {
        return Video.builder()
            .id(UUID.randomUUID())
            .name("test-video.mp4")
            .size(1024 * 1024 * 50) // 50MB
            .status(status)
            .createdAt(Instant.now())
            .build();
    }
    
    public static ProcessingJob createProcessingJob() {
        return ProcessingJob.builder()
            .videoId(UUID.randomUUID())
            .status(ProcessingStatus.PENDING)
            .priority(100)
            .createdAt(Instant.now())
            .build();
    }
}

Test Data Seeding:

• Use database migrations for test data setup in CI environment
• Implement test data cleanup strategy to prevent cross-test contamination
• Use database snapshots for faster test environment initialization

Test Data Isolation:

• Each test suite runs with isolated database schema or namespace
• Use UUIDs for all test entities to prevent ID collisions
• Implement database transaction rollback per test method for clean state

Test Environment Management:

• Separate CI/CD environments with dedicated test databases
• Use Docker Compose for local development testing environments
• Implement automated cleanup of test data after CI pipeline completion

13. Observability Design

Structured Logging Schema

All logs follow a structured JSON format with consistent fields to enable programmatic parsing and filtering:

{
  "timestamp": "2023-10-27T14:30:45.123Z",
  "level": "INFO",
  "service": "video-processing-service",
  "trace_id": "12345678-1234-1234-1234-123456789012",
  "span_id": "abcdef12-3456-7890-abcd-ef1234567890",
  "user_id": "user_12345",
  "entity_id": "video_98765",
  "operation": "processVideo",
  "message": "Started video processing",
  "duration_ms": 150,
  "error_code": null,
  "error_message": null,
  "stack_trace": null
}

PII masking rules:

• User IDs are logged as user_XXXXX (masked)
• Email addresses are masked as ***@***.com
• File paths are truncated to last 3 components

Distributed Tracing Instrumentation

Using OpenTelemetry for tracing with custom spans:

Key Operations to Trace:

1. video.upload - Upload endpoint
2. video.processing - Main processing workflow
3. video.transcoding - FFmpeg transcoding step
4. video.thumbnail - Thumbnail generation
5. video.storage - Storage operations

Span Attributes:

• video.id: Video identifier
• video.status: Current status (e.g., "processing", "completed")
• video.format: Input/output format
• video.duration_ms: Processing duration
• error.code: Error code if applicable

Metrics Design

RED Method Metrics:

Metric Name	Type	Description	Labels
video_processing_requests	Counter	Total video processing requests	status, format
video_processing_errors	Counter	Failed video processing operations	error_type, status
video_processing_duration	Histogram	Processing time in milliseconds	status, format
video_upload_size_bytes	Histogram	Uploaded video file sizes	format
video_storage_usage_bytes	Gauge	Current storage usage in bytes	service

Additional Service Metrics:

• active_workers (Gauge): Current number of processing workers
• queue_length (Gauge): Number of videos in processing queue
• failed_transcoding_attempts (Counter): Failed transcoding retries

Health Check Endpoint Design

GET /health
GET /ready
GET /live
GET /startup

/health - Returns overall system health:

{
  "status": "healthy",
  "timestamp": "2023-10-27T14:30:45Z",
  "components": {
    "database": "healthy",
    "storage": "healthy",
    "message_queue": "healthy"
  }
}

/ready - Checks if service can accept traffic:

{
  "status": "ready",
  "timestamp": "2023-10-27T14:30:45Z",
  "ready": true,
  "dependencies": {
    "database": "connected",
    "storage": "available"
  }
}

/live - Basic liveness check:

{
  "status": "alive",
  "timestamp": "2023-10-27T14:30:45Z",
  "uptime": "3600s"
}

Alerting Thresholds

Metric	Warning Level	Critical Level	Escalation Path
video_processing_errors (5m window)	> 10 errors	> 50 errors	On-call engineer
video_processing_duration (p99)	> 30s	> 60s	SRE team
queue_length	> 100 videos	> 500 videos	SRE team
active_workers (5m)	< 2 workers	< 1 worker	SRE team
database_connection_errors	> 5 errors	> 20 errors	DBA team

Alerts are configured with:

• 5-minute rolling window for short-term anomalies
• Slack notifications for critical alerts
• PagerDuty escalation for system-wide issues

14. Open Questions, Risks & Trade-offs

Decisions requiring product / engineering alignment

• Video format support matrix: Which video formats and codecs should be supported natively vs. transcoded? This impacts storage costs, processing complexity, and client compatibility requirements.
• Storage tier strategy: Should we use object storage (S3/GCS) or block storage for intermediate processing files? This affects cost, latency, and data durability trade-offs.
• Processing pipeline orchestration: Should we use a dedicated workflow engine (Apache Airflow, Temporal) or build custom orchestration logic? This impacts operational complexity and scalability.
• Content delivery strategy: Should we pre-generate multiple renditions or use adaptive streaming? This affects bandwidth costs and user experience.
• User notification strategy: Should we send real-time push notifications or rely on polling for processing status updates?

Known trade-offs and justifications

Trade-off	Option A	Option B	Justification
Storage vs. Compute	Store all intermediate files	Store only final outputs	Storing intermediate files enables faster retries and debugging but increases storage costs by 2-3x. Final-only approach reduces storage but requires robust retry logic.
Sync vs. Async processing	Synchronous video processing	Asynchronous processing	Synchronous approach provides immediate feedback but blocks user operations and increases latency. Async enables better scalability and user experience at cost of complexity.
Single vs. Multi-tenant architecture	Single tenant per pipeline	Shared processing pool	Single tenant provides better isolation and security but increases infrastructure costs. Shared pool reduces costs but introduces potential for resource contention and cross-tenant interference.
Event-driven vs. Polling	Event-driven processing	Poll-based status checks	Event-driven provides better real-time experience but increases system complexity and message broker load. Polling is simpler to implement but introduces latency and unnecessary load.

Risks and mitigation strategies

Processing failure risk: Video processing can fail due to corrupted input, encoding errors, or system timeouts.

• Mitigation: Implement comprehensive error handling with retry logic, dead-letter queue processing, and detailed failure notifications to users.

Resource exhaustion risk: Video processing is CPU and memory intensive, leading to system instability.

• Mitigation: Implement rate limiting, resource quotas per user, and automatic scaling policies with proper monitoring.

Storage cost risk: Large video files and multiple renditions can lead to unexpectedly high storage costs.

• Mitigation: Implement automatic file lifecycle management, compression strategies, and cost monitoring dashboards.

Latency risk: Video processing can take several minutes to hours for large files.

• Mitigation: Implement progress tracking, user notifications, and allow users to queue multiple videos for processing.

Security risk: Video content may contain sensitive information or malicious payloads.

• Mitigation: Implement content validation, virus scanning, and proper access controls with role-based permissions.

Technical debt acknowledged

• Hardcoded processing parameters: Current implementation uses hardcoded video quality settings and transcoding profiles that are not easily configurable.
• Tight coupling between services: The current architecture tightly couples the processing service with specific video encoding libraries, making future library migration difficult.
• Limited retry logic: Basic retry mechanisms exist but lack sophisticated error classification and exponential backoff strategies.
• No processing pipeline versioning: Different processing versions are not properly versioned, making it difficult to roll back or maintain backward compatibility.
• Inconsistent error handling: Error responses vary across endpoints, making client-side error handling inconsistent and harder to maintain.
• Lack of A/B testing capability: No built-in support for A/B testing different processing pipelines or quality settings for performance comparison.