URL Shortener Service Design - Scalable Backend Architecture

A scalable URL shortening service requires careful consideration of distributed ID generation, efficient storage strategies, and intelligent caching mechanisms to handle high traffic loads while maintaining low latency and maximum availability.

1. Problem Statement & Requirements

Clarifying questions

1. What scale expectations are there for this URL shortener?

   • How many URLs are expected to be shortened per day/month/year?
   • What is the anticipated peak concurrent request rate?

2. What consistency requirements exist for URL redirection?

   • Should redirects be strongly consistent or eventually consistent?
   • Is there a requirement for sub-second redirect latency?

3. What entities and their relationships are key to this system?

   • Should the system support user accounts, teams, or organization levels?
   • Do URLs need to be categorized or tagged?

4. What is the read/write ratio for shortened URLs?

   • Are most accesses reads (redirects) or writes (creation)?
   • How frequently are URLs updated or deleted?

5. How sensitive is the system to latency?

   • What are the p50/p95/p99 latency targets for redirect responses?
   • Are there any SLA requirements for URL creation?

6. What authentication and authorization model is required?

   • Should users be able to create short URLs without auth?
   • Do we need support for OAuth, API keys, or JWT tokens?

7. Are there any external integrations or dependencies?

   • Should the system integrate with analytics, domain management, or external services?
   • Are there any third-party URL validation or spam detection requirements?

Functional requirements

• URL Shortening: Accept a long URL and generate a unique short code.
• Redirect Service: Redirect users from the short code to the original URL.
• URL Management: Allow users to view, update, or delete their shortened URLs.
• Analytics: Track clicks, referrers, and geographic data for each short URL.
• Custom Short Codes: Allow users to optionally specify custom short codes (e.g., bit.ly/custom).
• Bulk Creation: Support batch creation of short URLs.
• URL Validation: Validate that input URLs are well-formed and accessible.

Non-functional requirements

• Latency Targets:

  • p50: < 50ms for redirect response
  • p95: < 100ms for redirect response
  • p99: < 200ms for redirect response

• Throughput:

  • Minimum 10,000 redirects per second
  • Support for burst traffic up to 50,000 redirects/sec

• Availability:

  • 99.9% uptime for redirect service
  • 99.95% availability for URL creation and management

• Scalability:

  • Support horizontal scaling across multiple regions
  • Handle millions of active short URLs

Constraints and assumptions

• Assumptions:

  • Shortened URLs will be accessed primarily via web browsers or mobile apps.
  • Redirects are the primary user interaction; management features are secondary.
  • The system will be built using a microservices architecture with a focus on high availability and low latency.
  • No requirement for real-time analytics or A/B testing features.

• Constraints:

  • Short codes must be unique and deterministic (same input URL always generates same short code if no custom code is used).
  • Short codes must be at least 3 characters long and not exceed 12 characters.
  • Only HTTP/HTTPS URLs are supported.
  • No requirement for URL expiration or TTL-based deletion.

Out of scope

• User account management (login, signup, password reset)
• Advanced analytics (e.g., user behavior tracking, funnel analysis)
• URL categorization or tagging
• API rate limiting enforcement (handled at gateway level)
• Domain management (custom domains, subdomain support)
• URL preview or screenshot generation
• Internationalization and localization
• Webhook integrations with external services
• Bulk URL deletion or archival features

2. API Design

API Style Selection

gRPC with Protocol Buffers is selected for the URL Shortener service due to:

• Strong typing and automatic code generation for all supported languages
• Efficient binary serialization reducing payload size by ~50% vs JSON
• Built-in support for streaming and bidirectional communication
• First-class support for versioning through proto package management
• Better performance characteristics for high-throughput scenarios

API Endpoints

Method	Path	Description	Auth
POST	/urlshortener.v1.UrlShortenerService/CreateShortUrl	Creates a new short URL with custom or auto-generated slug	Optional
GET	/urlshortener.v1.UrlShortenerService/Redirect	Redirects to the original URL using short slug	None
GET	/urlshortener.v1.UrlShortenerService/GetShortUrlInfo	Retrieves metadata about a short URL	Optional
PUT	/urlshortener.v1.UrlShortenerService/UpdateShortUrl	Updates existing short URL configuration	Optional
DELETE	/urlshortener.v1.UrlShortenerService/DeleteShortUrl	Deletes a short URL	Optional
GET	/urlshortener.v1.UrlShortenerService/GetUsageStats	Retrieves usage statistics for a short URL	Optional

Request and Response Schemas

CreateShortUrlRequest

message CreateShortUrlRequest {
  string original_url = 1;
  string custom_slug = 2;
  int32 ttl_seconds = 3;
  bool is_permanent = 4;
}

CreateShortUrlResponse

message CreateShortUrlResponse {
  string short_url = 1;
  string slug = 2;
  int64 created_at = 3;
  int64 expires_at = 4;
}

RedirectRequest

message RedirectRequest {
  string slug = 1;
}

RedirectResponse

message RedirectResponse {
  string original_url = 1;
}

GetShortUrlInfoRequest

message GetShortUrlInfoRequest {
  string slug = 1;
}

GetShortUrlInfoResponse

message GetShortUrlInfoResponse {
  string slug = 1;
  string original_url = 2;
  int64 created_at = 3;
  int64 expires_at = 4;
  int64 click_count = 5;
  bool is_active = 6;
}

HTTP Status Codes and Error Response Format

Status Code	Description
200	Success
400	Bad Request - Invalid input
404	Not Found - Slug does not exist
429	Too Many Requests - Rate limit exceeded
500	Internal Server Error

Standardized error response:

message ErrorResponse {
  int32 code = 1;
  string message = 2;
  repeated ErrorDetail details = 3;
}

message ErrorDetail {
  string field = 1;
  string description = 2;
}

API Versioning Strategy

Versioned using package namespace: urlshortener.v1
Future versions will use urlshortener.v2 with breaking changes documented in changelog

Pagination Design

Cursor-based pagination is used because:

• Provides consistent results even when data changes during pagination
• Better performance for large datasets with complex filtering
• More predictable behavior in distributed systems

Authentication and Authorization

• Public endpoints (Redirect) - No authentication required
• Protected endpoints (CreateShortUrl, GetShortUrlInfo) - JWT Bearer token required
• Admin endpoints (DeleteShortUrl, UpdateShortUrl) - JWT with admin role required
• Rate limiting per user token (1000 requests/hour) with distributed rate limiter

Rate Limiting Strategy

• Per-user limit: 1000 requests/hour
• Per-IP limit: 5000 requests/hour
• Implementation: Redis-based token bucket algorithm with sliding window
• Enforcement: Middleware interceptor in gRPC service

Idempotency Design

For mutation endpoints:

• Idempotency Key: X-Idempotency-Key header
• Storage: Redis with TTL of 5 minutes
• Behavior: If key exists and operation is same, return cached result
• Use case: Retry scenarios where client might send same request multiple times

Example Request/Response Flow

CreateShortUrlRequest

{
  "original_url": "https://www.example.com/very/long/path/to/resource?param1=value1&param2=value2",
  "custom_slug": "my-custom-slug",
  "ttl_seconds": 3600,
  "is_permanent": false
}

CreateShortUrlResponse

{
  "short_url": "https://short.ly/my-custom-slug",
  "slug": "my-custom-slug",
  "created_at": 1698765432,
  "expires_at": 1698769032
}

RedirectRequest

{
  "slug": "my-custom-slug"
}

RedirectResponse

{
  "original_url": "https://www.example.com/very/long/path/to/resource?param1=value1&param2=value2"
}

3. Data Model Design

Entity-Relationship Diagram

erDiagram
    URL_SHORTENER {
        string id PK
        string short_code PK
        string original_url
        string user_id
        integer click_count
        timestamp created_at
        timestamp expires_at
        boolean is_active
        string metadata
    }

    USER {
        string id PK
        string email
        string name
        timestamp created_at
    }

    URL_SHORTENER ||--o{ USER : "owned_by"

Table Schema Definitions

CREATE TABLE url_shortener (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    short_code VARCHAR(12) NOT NULL UNIQUE,
    original_url TEXT NOT NULL,
    user_id UUID REFERENCES users(id),
    click_count INTEGER DEFAULT 0,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
    expires_at TIMESTAMP WITH TIME ZONE,
    is_active BOOLEAN DEFAULT TRUE,
    metadata JSONB,
    
    -- Indexes
    INDEX idx_short_code (short_code),
    INDEX idx_user_id (user_id),
    INDEX idx_created_at (created_at),
    INDEX idx_expires_at (expires_at),
    INDEX idx_active (is_active),
    INDEX idx_click_count (click_count)
);

CREATE TABLE users (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    email VARCHAR(255) UNIQUE NOT NULL,
    name VARCHAR(255),
    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
    
    INDEX idx_email (email)
);

Primary Key Strategy

UUID over auto-increment:

• UUIDs provide global uniqueness across distributed systems
• No need for coordination between services or shards
• Better security as they don't reveal sequential information
• Support for distributed generation without contention

Index Design Justification

1. idx_short_code:

   • Used for fast lookups by short code (URL redirection)
   • Composite index would be redundant since this is the main lookup field

2. idx_user_id:

   • For user-based URL listing and ownership queries
   • Supports efficient filtering by user

3. idx_created_at:

   • For time-based analytics and reporting
   • Pagination in user dashboards

4. idx_expires_at:

   • For automatic cleanup of expired URLs
   • Efficient batch deletion operations

5. idx_active:

   • For filtering active URLs in read operations
   • Optimizes queries for valid URL lookups

6. idx_click_count:

   • For analytics and popular URL queries
   • Supports ranking by engagement

Data Partitioning Strategy

Sharding by user_id hash:

• Hash(user_id) % 16 shards for initial partitioning
• Allows horizontal scaling of URL storage
• Maintains referential integrity with user data

Soft Delete vs. Hard Delete

Soft delete selected:

• URLs may need to be recovered for analytics or user requests
• Historical tracking of deleted URLs for audit purposes
• Support for URL reclamation by same user within grace period

Audit Trail Design

CREATE TABLE url_audit_log (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    url_id UUID REFERENCES url_shortener(id),
    action VARCHAR(50) NOT NULL, -- 'CREATE', 'UPDATE', 'DELETE'
    user_id UUID REFERENCES users(id),
    old_values JSONB,
    new_values JSONB,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
);

Data Archival Strategy

Tiered archival policy:

• URLs with click_count = 0 and created_at < 30 days → archived to cold storage
• URLs with click_count > 0 and created_at < 90 days → retained in hot storage
• URLs with click_count > 0 and created_at > 90 days → archived to cold storage with metadata retention

Retention policy:

• Hot storage: 30 days (active URLs)
• Warm storage: 1 year (inactive but popular URLs)
• Cold storage: 5 years (all other URLs)

Data Validation Rules

1. short_code:

   • Length: 6-12 characters
   • Characters: alphanumeric + hyphens + underscores only
   • Uniqueness constraint enforced at DB level

2. original_url:

   • Valid URL format (regex validation)
   • Maximum length: 2048 characters
   • Required field

3. expires_at:

   • Must be after created_at if provided
   • Maximum 365 days from creation (configurable)

4. click_count:

   • Non-negative integer
   • Updated atomically with increment operations

Data Migration Strategy

• Initial migration: Bulk load existing URLs with generated short codes
• Schema versioning: Use database migration tools (Flyway/liquibase) with versioned scripts
• Backward compatibility: Support both old and new short code formats during transition period

4. Class Design & Service Layer

UML Class Diagram

classDiagram
    class URLShortenerService {
        +generateShortUrl(String originalUrl, String customAlias) String
        +resolveShortUrl(String shortCode) String
        +getStats(String shortCode) URLStats
        +deleteShortUrl(String shortCode) boolean
    }
    
    class URLRepository {
        +save(ShortUrl shortUrl) void
        +findByShortCode(String shortCode) Optional~ShortUrl~
        +findByOriginalUrl(String originalUrl) List~ShortUrl~
        +deleteByShortCode(String shortCode) boolean
        +getStats(String shortCode) URLStats
    }
    
    class ShortUrl {
        -id: UUID
        -shortCode: String
        -originalUrl: String
        -createdAt: Instant
        -expiresAt: Instant
        -clickCount: Long
        -isActive: Boolean
        +generateShortCode() String
        +incrementClickCount() void
    }
    
    class URLStats {
        -shortCode: String
        -clickCount: Long
        -createdAt: Instant
        -lastAccessedAt: Instant
    }
    
    class ShortCodeGenerator {
        +generate() String
    }
    
    class URLValidator {
        +validate(String url) boolean
        +normalize(String url) String
    }
    
    class URLShortenerEventPublisher {
        +publishUrlCreated(ShortUrl shortUrl) void
        +publishUrlAccessed(ShortUrl shortUrl) void
    }
    
    class CacheService {
        +get(String key) String
        +put(String key, String value, Duration ttl) void
        +invalidate(String key) void
    }
    
    class RetryableOperation {
        +executeWithRetry(Supplier~T~ operation, int maxRetries) T
    }
    
    URLShortenerService --> ShortUrl : creates
    URLShortenerService --> URLRepository : delegates to
    URLShortenerService --> ShortCodeGenerator : generates codes
    URLShortenerService --> URLValidator : validates urls
    URLShortenerService --> URLShortenerEventPublisher : publishes events
    URLShortenerService --> CacheService : caches results
    URLRepository --> ShortUrl : manages instances
    URLShortenerEventPublisher --> EventBus : publishes to
    CacheService --> RedisCache : uses redis backend
    
    class EventBus {
        +publish(Event event) void
    }
    
    class Event {
        -type: String
        -payload: Map~String, Object~
        -timestamp: Instant
    }
    
    class URLCreatedEvent {
        +from(ShortUrl shortUrl) URLCreatedEvent
    }
    
    class URLAccessedEvent {
        +from(ShortUrl shortUrl) URLAccessedEvent
    }

State Machine Diagram for ShortUrl

stateDiagram-v2
    [*] --> Active
    Active --> Inactive : deactivate()
    Active --> Expired : expires_at < now
    Expired --> Inactive : manual_deactivate()
    Inactive --> Active : reactivate()

Service Class Structure and Responsibilities

The URLShortenerService is the primary orchestrator that handles all business logic for URL shortening operations. It encapsulates:

• URL validation and normalization
• Short code generation with collision handling
• Persistence operations through repository
• Statistics tracking and retrieval
• Event publishing for audit and analytics
• Caching strategy for performance optimization

Interface / Abstract Class Design

public interface URLRepository {
    void save(ShortUrl shortUrl);
    Optional<ShortUrl> findByShortCode(String shortCode);
    List<ShortUrl> findByOriginalUrl(String originalUrl);
    boolean deleteByShortCode(String shortCode);
    URLStats getStats(String shortCode);
}

public interface ShortCodeGenerator {
    String generate();
}

public interface URLValidator {
    boolean validate(String url);
    String normalize(String url);
}

Business Logic Encapsulation Patterns

Strategy Pattern for URL Validation:

public interface URLValidationStrategy {
    boolean validate(String url);
    String normalize(String url);
}

@Component
public class HttpUrlValidationStrategy implements URLValidationStrategy {
    
    private static final Pattern URL_PATTERN = 
        Pattern.compile("^(https?://)?([\\da-z\\.-]+)\\.([a-z\\.]{2,6})([\\w\\-\\.\\?\\:\\/\\@\\&\\=\\%\\+\\*\\,\\(\\)\\[\\]\\{\\}\\\\\\|\\^\\~\\`\\'\\\"\\;\\<\\>\\!\\$\\#\\%\\@\\+\\-\\_\\s]*)$", 
            Pattern.CASE_INSENSITIVE);
    
    @Override
    public boolean validate(String url) {
        if (url == null || url.isEmpty()) return false;
        return URL_PATTERN.matcher(url).matches();
    }
    
    @Override
    public String normalize(String url) {
        if (url ==ll || url.isEmpty()) return url;
        // Normalize to https:// prefix and remove trailing slashes
        String normalized = url.trim();
        if (!normalized.startsWith("http://") && !normalized.startsWith("https://")) {
            normalized = "https://" + normalized;
        }
        return normalized.replaceAll("/$", "");
    }
}

Critical-Path Method Implementation

@Transactional
public String generateShortUrl(String originalUrl, String customAlias) {
    // 1. Validate input
    if (!urlValidator.validate(originalUrl)) {
        throw new InvalidUrlException("Invalid URL format: " + originalUrl);
    }
    
    String normalizedUrl = urlValidator.normalize(originalUrl);
    
    // 2. Check for existing short URL with same original URL
    List<ShortUrl> existingUrls = urlRepository.findByOriginalUrl(normalizedUrl);
    if (!existingUrls.isEmpty()) {
        return existingUrls.get(0).getShortCode();
    }
    
    // 3. Handle custom alias if provided
    String shortCode;
    if (customAlias != null && !customAlias.isEmpty()) {
        // Check for conflicts
        Optional<ShortUrl> existing = urlRepository.findByShortCode(customAlias);
        if (existing.isPresent()) {
            throw new ShortCodeAlreadyExistsException("Short code already exists: " + customAlias);
        }
        shortCode = customAlias;
    } else {
        // 4. Generate unique short code
        int maxRetries = 3;
        int attempts = 0;
        String generatedCode;
        
        do {
            generatedCode = shortCodeGenerator.generate();
            attempts++;
        } while (urlRepository.findByShortCode(generatedCode).isPresent() && attempts < maxRetries);
        
        if (attempts >= maxRetries) {
            throw new ShortCodeGenerationException("Failed to generate unique short code after " + maxRetries + " attempts");
        }
        
        shortCode = generatedCode;
    }
    
    // 5. Create and persist new ShortUrl entity
    ShortUrl shortUrl = new ShortUrl();
    shortUrl.setShortCode(shortCode);
    shortUrl.setOriginalUrl(normalizedUrl);
    shortUrl.setCreatedAt(Instant.now());
    shortUrl.setExpiresAt(Instant.now().plus(Duration.ofDays(365))); // 1 year default
    shortUrl.setActive(true);
    
    urlRepository.save(shortUrl);
    
    // 6. Publish event for audit and analytics
    urlShortenerEventPublisher.publishUrlCreated(shortUrl);
    
    // 7. Cache the result for fast retrieval
    cacheService.put(
        "shorturl:" + shortCode, 
        normalizedUrl, 
        Duration.ofHours(24)
    );
    
    return shortCode;
}

Transaction Boundary Design

Transaction boundaries are defined per business operation:

• URL Generation: Single transaction encompassing validation, code generation, persistence, and event publishing
• URL Resolution: Read-only transaction for cache lookup and database read
• Stats Retrieval: Read-only transaction for statistics fetch
• URL Deletion: Single transaction for soft-delete and cache invalidation

Service Interface Contracts

public interface URLShortenerService {
    String generateShortUrl(String originalUrl, String customAlias);
    String resolveShortUrl(String shortCode);
    URLStats getStats(String shortCode);
    boolean deleteShortUrl(String shortCode);
}

Saga / Orchestration Pattern

For distributed operations like URL creation with external analytics tracking, we use a saga pattern:

public class URLCreationSaga {
    private final List<SagaStep> steps = new ArrayList<>();
    
    public void execute(String originalUrl, String customAlias) {
        steps.add(new ValidateUrlStep());
        steps.add(new GenerateShortCodeStep());
        steps.add(new SaveToDatabaseStep());
        steps.add(new PublishAnalyticsEventStep());
        
        // Execute steps with compensation logic
        for (SagaStep step : steps) {
            try {
                step.execute();
            } catch (Exception e) {
                compensate();
                throw new SagaExecutionException("Saga failed at step: " + step.getName(), e);
            }
        }
    }
    
    private void compensate() {
        // Reverse the steps in reverse order
        for (int i = steps.size() - 1; i >= 0; i--) {
            steps.get(i).compensate();
        }
    }
}

Domain Object / Value Object Design

@Entity
@Table(name = "short_urls")
public class ShortUrl {
    @Id
    @GeneratedValue(strategy = GenerationType.UUID)
    private UUID id;
    
    @Column(unique = true, nullable = false)
    private String shortCode;
    
    @Column(name = "original_url", nullable = false)
    private String originalUrl;
    
    @Column(name = "created_at", nullable = false)
    private Instant createdAt;
    
    @Column(name = "expires_at")
    private Instant expiresAt;
    
    @Column(name = "click_count", nullable = false, defaultValue = "0")
    private Long clickCount = 0L;
    
    @Column(name = "is_active", nullable = false, defaultValue = "true")
    private Boolean active = true;
    
    // Getters and setters
    
    public String generateShortCode() {
        return Base62.encode(System.currentTimeMillis());
    }
    
    public void incrementClickCount() {
        this.clickCount++;
    }
}

public class URLStats {
    private String shortCode;
    private Long clickCount;
    private Instant createdAt;
    private Instant lastAccessedAt;
    
    // Getters and setters
}

5. Concurrency & Thread Safety

Concurrency Model

The URL shortener service will use a thread pool model with Java's ExecutorService for handling concurrent requests. Each incoming HTTP request will be processed by a separate thread from the pool, allowing for efficient handling of multiple concurrent URL shortening and redirection operations. The service will leverage Java's java.util.concurrent package for thread-safe operations and synchronization primitives.

Race Condition Prevention Strategy

Race conditions in URL shortener can occur during:

1. URL generation: Multiple concurrent requests trying to generate the same short code
2. Short code collision handling: Concurrent attempts to resolve hash collisions
3. Database updates: Concurrent access to the same URL record during updates

To prevent these, we'll use:

• Atomic operations for generating unique identifiers
• Database-level constraints (unique indexes) to prevent duplicate short codes
• Optimistic locking with version fields for concurrent updates
• Thread-local storage for per-request state management

Optimistic vs. Pessimistic Locking Selection

We'll use optimistic locking for URL record updates because:

1. Read-heavy workload with low write contention
2. Better performance under concurrent access patterns
3. URL shortener typically has more reads than writes (redirects vs. creation)
4. Database constraints provide sufficient data integrity

Implementation approach:

// Using version field in URL entity for optimistic locking
@Entity
@Table(name = "urls")
public class UrlEntity {
    @Id
    private String shortCode;
    
    @Column(name = "original_url")
    private String originalUrl;
    
    @Version
    private Long version;
}

Deadlock Prevention Strategy

Deadlock prevention in URL shortener focuses on:

1. Consistent ordering of database operations - Always acquire locks in the same order (e.g., by shortCode hash)
2. Timeout configuration - Set reasonable timeouts for database transactions (5 seconds max)
3. Avoid nested transaction contexts - Keep operations within single transaction boundaries
4. Use of connection pool with proper timeout settings

Idempotency Implementation for Concurrent Duplicate Requests

For concurrent duplicate URL creation requests, we implement idempotency using:

1. Idempotency key in request header (X-Idempotency-Key)
2. Cache-based lookup for previously processed requests
3. Database-level uniqueness constraints as fallback

public class IdempotencyService {
    private final RedisTemplate<String, String> redisTemplate;
    
    public String getIdempotentShortCode(String originalUrl, String idempotencyKey) {
        if (idempotencyKey != null) {
            String cachedResult = redisTemplate.opsForValue().get("idempotency:" + idempotencyKey);
            if (cachedResult != null) {
                return cachedResult;
            }
        }
        
        // Proceed with URL shortening logic
        String shortCode = generateShortCode(originalUrl);
        
        if (idempotencyKey != null) {
            redisTemplate.opsForValue().set(
                "idempotency:" + idempotencyKey, 
                shortCode, 
                Duration.ofHours(24)
            );
        }
        
        return shortCode;
    }
}

6. Sequence Diagrams for Key Flows

sequenceDiagram
    participant U as User
    participant API as URL Shortener API
    participant DB as Database
    participant Cache as Redis Cache
    participant Queue as Message Queue

    U->>API: POST /shorten with longUrl
    API->>Cache: Check if shortCode exists
    Cache-->>API: Miss
    API->>DB: Check if longUrl already exists
    DB-->>API: Not found
    API->>DB: Generate new shortCode and store
    DB-->>API: Success
    API->>Cache: Store shortCode -> longUrl mapping
    Cache-->>API: Success
    API->>Queue: Publish URL creation event
    Queue-->>API: Acknowledged
    API-->>U: 201 Created with shortCode and shortUrl

sequenceDiagram
    participant U as User
    participant API as URL Shortener API
    participant DB as Database
    participant Cache as Redis Cache
    participant Redirector as Redirect Service

    U->>API: GET /r/{shortCode}
    API->>Cache: Check if shortCode exists
    Cache-->>API: Hit
    API->>Redirector: Redirect to longUrl
    Redirector-->>U: 302 Found with Location header

sequenceDiagram
    participant API as URL Shortener API
    participant DB as Database
    participant Queue as Message Queue
    participant Consumer as Event Consumer

    API->>Queue: Publish URL creation event
    Queue->>Consumer: Deliver event
    Consumer->>DB: Update analytics table
    DB-->>Consumer: Success
    Consumer-->>Queue: Acknowledge event

7. Caching Design

Cache Strategy Selection

For a URL shortener service, I'll use cache-aside pattern with LRU eviction policy. This approach provides optimal control over cache behavior and is well-suited for read-heavy workloads where URLs are frequently accessed but updated infrequently.

Cache Key Design

Cache keys will follow a structured namespace pattern:

url_shortener:{type}:{key}

Where:

• type = "short_url", "long_url", "analytics"
• key = the actual URL hash or short code

Example key structure:

url_shortener:short_url:abc123
url_shortener:long_url:https://example.com/very/long/path
url_shortener:analytics:abc123

TTL Configuration

Data Type	TTL	Reason
Short URL mapping	30 days	URLs are assumed to be permanent
Long URL lookup	7 days	For analytics and fallback scenarios
Analytics data	30 minutes	Recent access patterns are most valuable
User session cache	1 hour	Session-based caching for auth

Cache Eviction Policy

LRU (Least Recently Used) with probabilistic early expiry to prevent cache stampede:

• Primary policy: LRU eviction
• Secondary: 10% probabilistic early expiry to prevent thundering herd on cache misses

Cache Stampede Prevention

Implement distributed lock with TTL:

public class CacheStampedePrevention {
    private static final String CACHE_LOCK_PREFIX = "url_shortener:lock:";
    
    public boolean acquireLock(String key, int ttlSeconds) {
        String lockKey = CACHE_LOCK_PREFIX + key;
        return redisTemplate.opsForValue()
            .setIfAbsent(lockKey, "locked", Duration.ofSeconds(ttlSeconds));
    }
    
    public void releaseLock(String key) {
        String lockKey = CACHE_LOCK_PREFIX + key;
        redisTemplate.delete(lockKey);
    }
}

Distributed Cache Architecture

Using Redis Cluster with:

• 3 master nodes for high availability
• 3 replica nodes per master for read scaling
• Automatic failover with Redis Sentinel monitoring
• Cross-region replication for disaster recovery

Cache Invalidation Strategy

Event-driven invalidation using Redis Pub/Sub:

@Component
public class CacheInvalidator {
    
    @Autowired
    private RedisTemplate<String, Object> redisTemplate;
    
    @EventListener
    public void handleUrlCreatedEvent(UrlCreatedEvent event) {
        // Invalidate short URL cache
        redisTemplate.delete("url_shortener:short_url:" + event.getShortCode());
        
        // Invalidate analytics cache if exists
        redisTemplate.delete("url_shortener:analytics:" + event.getShortCode());
    }
    
    @EventListener
    public void handleUrlUpdatedEvent(UrlUpdatedEvent event) {
        // Invalidate both old and new mappings
        redisTemplate.delete("url_shortener:short_url:" + event.getShortCode());
        redisTemplate.delete("url_shortener:long_url:" + event.getLongUrl());
    }
}

Cache Read-Through Pattern

@Service
public class CachedUrlService {
    
    private static final String SHORT_URL_CACHE_KEY = "url_shortener:short_url:";
    private static final String LONG_URL_CACHE_KEY = "url_shortener:long_url:";
    
    public String getLongUrl(String shortCode) {
        String cacheKey = SHORT_URL_CACHE_KEY + shortCode;
        
        // Try cache first
        String longUrl = (String) redisTemplate.opsForValue().get(cacheKey);
        if (longUrl != null) {
            return longUrl;
        }
        
        // Cache miss - acquire lock to prevent stampede
        if (acquireCacheLock(shortCode)) {
            try {
                // Double-check cache after lock acquisition
                longUrl = (String) redisTemplate.opsForValue().get(cacheKey);
                if (longUrl != null) {
                    return longUrl;
                }
                
                // Fetch from database
                longUrl = urlRepository.findLongUrlByShortCode(shortCode);
                
                // Cache the result with TTL
                redisTemplate.opsForValue()
                    .set(cacheKey, longUrl, Duration.ofDays(30));
                
                return longUrl;
            } finally {
                releaseCacheLock(shortCode);
            }
        } else {
            // Another thread is processing, wait and retry
            Thread.sleep(50);
            return getLongUrl(shortCode);
        }
    }
}

8. Repository / Data Access Layer

Repository Pattern Implementation

The repository pattern is implemented to abstract data access logic and provide a clean separation between domain logic and database operations. Each entity has its own repository interface with standardized CRUD operations.

public interface UrlRepository {
    UrlEntity save(UrlEntity url);
    Optional<UrlEntity> findById(String id);
    Optional<UrlEntity> findByShortCode(String shortCode);
    List<UrlEntity> findAllByUserId(String userId, Pageable pageable);
    void deleteById(String id);
    boolean existsById(String id);
    long countByUserId(String userId);
}

public interface UrlClickRepository {
    void save(UrlClickEntity click);
    List<UrlClickEntity> findByShortCode(String shortCode, Pageable pageable);
    long countByShortCode(String shortCode);
}

ORM vs Raw Query Decisions

For the URL entity, we use JPA/Hibernate as the ORM to leverage type safety and reduce boilerplate code for common operations like findById, save, and delete. For analytics queries (click counting, time-series data), raw SQL is preferred for better performance and query optimization.

@Repository
public class UrlRepositoryImpl implements UrlRepository {
    
    @PersistenceContext
    private EntityManager entityManager;
    
    @Override
    public UrlEntity save(UrlEntity url) {
        if (url.getId() == null) {
            entityManager.persist(url);
        } else {
            entityManager.merge(url);
        }
        return url;
    }
    
    @Override
    public Optional<UrlEntity> findById(String id) {
        return Optional.ofNullable(entityManager.find(UrlEntity.class, id));
    }
    
    @Override
    public Optional<UrlEntity> findByShortCode(String shortCode) {
        TypedQuery<UrlEntity> query = entityManager.createQuery(
            "SELECT u FROM UrlEntity u WHERE u.shortCode = :shortCode", 
            UrlEntity.class);
        query.setParameter("shortCode", shortCode);
        return query.getResultStream().findFirst();
    }
    
    @Override
    public List<UrlEntity> findAllByUserId(String userId, Pageable pageable) {
        TypedQuery<UrlEntity> query = entityManager.createQuery(
            "SELECT u FROM UrlEntity u WHERE u.userId = :userId ORDER BY u.createdAt DESC", 
            UrlEntity.class);
        query.setParameter("userId", userId);
        query.setFirstResult(pageable.getOffset());
        query.setMaxResults(pageable.getPageSize());
        return query.getResultList();
    }
    
    @Override
    public void deleteById(String id) {
        entityManager.createQuery("DELETE FROM UrlEntity u WHERE u.id = :id")
                     .setParameter("id", id)
                     .executeUpdate();
    }
    
    @Override
    public boolean existsById(String id) {
        return entityManager.createQuery(
            "SELECT COUNT(u) FROM UrlEntity u WHERE u.id = :id", 
            Long.class)
            .setParameter("id", id)
            .getSingleResult() > 0;
    }
    
    @Override
    public long countByUserId(String userId) {
        return entityManager.createQuery(
            "SELECT COUNT(u) FROM UrlEntity u WHERE u.userId = :userId", 
            Long.class)
            .setParameter("userId", userId)
            .getSingleResult();
    }
}

Read vs Write Separation

The URL shortener follows a CQRS pattern for read and write operations to optimize performance:

// Write side - primary database with ACID transactions
@Repository
public class UrlWriteRepository {
    // All write operations (save, update, delete)
}

// Read side - denormalized database or cache for fast reads
@Repository
public class UrlReadRepository {
    // Optimized queries for analytics, search, and retrieval
}

Connection Pool Configuration

Connection pool is configured with:

• Max pool size: 30
• Min idle connections: 5
• Connection timeout: 30 seconds
• Validation query: SELECT 1

# application.yml
spring:
  datasource:
    hikari:
      maximum-pool-size: 30
      minimum-idle: 5
      connection-timeout: 30000
      validation-timeout: 5000
      idle-timeout: 600000
      max-lifetime: 1800000

N+1 Query Prevention Strategy

To prevent N+1 query problems, we implement eager loading and batch operations:

// Using JOIN FETCH to prevent N+1 in user's URL list
@Query("SELECT u FROM UrlEntity u JOIN FETCH u.clicks WHERE u.userId = :userId")
List<UrlEntity> findAllWithClicksByUserId(@Param("userId") String userId);

// Batch operations for bulk analytics
@Modifying
@Query("UPDATE UrlEntity u SET u.clickCount = u.clickCount + :count WHERE u.id IN :ids")
void incrementClickCounts(@Param("ids") List<String> ids, @Param("count") int count);

Batch Operation Design

Batch operations are implemented for high-volume analytics:

public class BatchUrlClickProcessor {
    
    public void processBatchClicks(List<UrlClickEntity> clicks) {
        // Batch insert using JDBC for better performance
        String sql = "INSERT INTO url_clicks (id, short_code, timestamp, ip_address) VALUES (?, ?, ?, ?)";
        
        try (Connection conn = dataSource.getConnection();
             PreparedStatement stmt = conn.prepareStatement(sql)) {
            
            for (UrlClickEntity click : clicks) {
                stmt.setString(1, click.getId());
                stmt.setString(2, click.getShortCode());
                stmt.setTimestamp(3, Timestamp.valueOf(click.getTimestamp()));
                stmt.setString(4, click.getIpAddress());
                stmt.addBatch();
            }
            
            stmt.executeBatch();
        } catch (SQLException e) {
            throw new DataAccessException("Failed to batch insert clicks", e);
        }
    }
}

9. Asynchronous Processing & Messaging

Event-driven boundaries in this feature

The URL shortener service will use a hybrid synchronous/asynchronous approach. The core createShortUrl operation will be synchronous to provide immediate feedback to the client, but secondary operations like analytics tracking, URL validation, and cleanup of expired URLs will be handled asynchronously.

Key triggers for async processing:

• URL creation: Publishes UrlCreatedEvent to trigger analytics and cache population
• Short URL access: Triggers UrlAccessedEvent for analytics and usage tracking
• URL expiration: Triggers UrlExpiredEvent to clean up resources and send notifications

Message schema design with sample payload

{
  "version": "1.0",
  "event_type": "url.created",
  "timestamp": "2023-11-15T14:30:00Z",
  "payload": {
    "short_code": "abc123",
    "original_url": "https://www.example.com/very/long/url/path",
    "created_by": "user-123",
    "created_at": "2023-11-15T14:30:00Z",
    "expires_at": "2024-11-15T14:30:00Z"
  }
}

Dead letter queue strategy and poison message handling

Poison messages will be routed to a dead-letter queue (DLQ) after 3 failed delivery attempts. The DLQ will be monitored by an alerting system that triggers manual inspection and potential reprocessing.

@Component
public class MessageErrorHandler {
    private static final int MAX_RETRY_ATTEMPTS = 3;
    
    @EventListener
    public void handleFailedMessage(FailedMessageEvent event) {
        if (event.getRetryCount() >= MAX_RETRY_ATTEMPTS) {
            // Route to DLQ
            sendToDeadLetterQueue(event);
        } else {
            // Retry with exponential backoff
            retryWithBackoff(event);
        }
    }
}

At-least-once vs. exactly-once delivery

At-least-once delivery is selected for this feature because:

1. URL creation is idempotent - duplicate creation doesn't cause harm
2. Analytics tracking can tolerate duplicates with proper deduplication logic
3. Simpler implementation and better performance characteristics

Retry policy for consumers

Retry policy with exponential backoff:

• Max attempts: 5
• Base delay: 100ms
• Backoff factor: 2
• Jitter: ±20% to prevent thundering herd

public class RetryPolicy {
    private static final int MAX_RETRIES = 5;
    private static final long BASE_DELAY_MS = 100;
    private static final double BACKOFF_FACTOR = 2.0;
    
    public boolean shouldRetry(int attempt) {
        return attempt < MAX_RETRIES;
    }
    
    public long getDelay(int attempt) {
        long delay = (long) (BASE_DELAY_MS * Math.pow(BACKOFF_FACTOR, attempt));
        // Add jitter: ±20%
        long jitter = (long) (delay * 0.2 * (Math.random() - 0.5) * 2);
        return delay + jitter;
    }
}

Outbox pattern for reliable event publishing

Outbox pattern is implemented to ensure event delivery reliability:

@Service
public class UrlService {
    
    @Transactional
    public ShortUrl createShortUrl(CreateUrlRequest request) {
        // Business logic to generate short code
        ShortUrl shortUrl = new ShortUrl();
        shortUrl.setShortCode(generateShortCode());
        shortUrl.setOriginalUrl(request.getOriginalUrl());
        shortUrl.setExpiresAt(calculateExpiryTime());
        
        // Save to database
        urlRepository.save(shortUrl);
        
        // Publish to outbox table (within same transaction)
        OutboxEvent event = new OutboxEvent();
        event.setEventType("url.created");
        event.setPayload(buildEventPayload(shortUrl));
        event.setStatus("pending");
        outboxRepository.save(event);
        
        return shortUrl;
    }
}

Sequence diagram for async flows

sequenceDiagram
    participant Client
    participant URLService
    participant DB
    participant OutboxRepo
    participant MessageBroker
    participant AnalyticsConsumer
    participant CacheService
    
    Client->>URLService: POST /shorten
    URLService->>DB: Save ShortUrl
    URLService->>OutboxRepo: Insert OutboxEvent
    URLService-->>Client: 201 Created
    
    loop Process Outbox Events
        OutboxRepo->>MessageBroker: Publish Event
        MessageBroker->>AnalyticsConsumer: Deliver url.created
        MessageBroker->>CacheService: Deliver url.created
    end

10. Error Handling Design

Error Classification Hierarchy

The URL shortener service follows a structured error classification system:

Error
├── ValidationException (400)
│   ├── InvalidUrlException
│   └── InvalidShortCodeException
├── ResourceNotFoundException (404)
│   └── UrlMappingNotFoundException
├── ConflictException (409)
│   └── ShortCodeCollisionException
├── RateLimitExceededException (429)
│   └── RequestRateLimitException
├── InternalServiceException (500)
│   ├── DatabaseException
│   ├── CacheException
│   └── ExternalServiceException
└── UnauthorizedException (401)
    └── InvalidTokenException

Error Propagation Strategy

• Domain-level errors: Caught and converted to appropriate HTTP exceptions at service boundaries
• Infrastructure errors: Bubble up to the global exception handler for consistent formatting
• Validation errors: Handled early in API layer with immediate HTTP 400 responses

Retry Strategy with Exponential Backoff and Jitter

For external service calls (e.g., cache, database):

public class RetryConfig {
    private static final int MAX_RETRIES = 3;
    private static final long BASE_DELAY_MS = 1000;
    private static final double JITTER_FACTOR = 0.2;
    
    public static long calculateDelay(int attempt) {
        long delay = (long) (BASE_DELAY_MS * Math.pow(2, attempt - 1));
        double jitter = ThreadLocalRandom.current().nextDouble(-JITTER_FACTOR, JITTER_FACTOR);
        return (long) (delay * (1 + jitter));
    }
}

Circuit Breaker Design

Using Resilience4j for circuit breaker pattern:

public class CircuitBreakerManager {
    private final CircuitBreaker circuitBreaker = CircuitBreaker.ofDefaults("url-shortener");
    
    public <T> T executeWithCircuitBreaker(Supplier<T> operation) {
        return circuitBreaker.executeSupplier(operation);
    }
    
    public void onCallSuccess() {
        circuitBreaker.onSuccess();
    }
    
    public void onCallFailure() {
        circuitBreaker.onError();
    }
}

Configuration:

• Failure rate threshold: 50%
• Minimum number of calls: 10
• Slow call duration threshold: 500ms
• Slow call rate threshold: 50%
• Timeout duration: 1000ms

Fallback Behavior Design

public class FallbackHandler {
    public String handleUrlLookupFallback(String shortCode, Throwable throwable) {
        if (throwable instanceof CircuitBreakerOpenException) {
            // Return a default page or redirect to homepage
            return "https://short.ly";
        }
        if (throwable instanceof TimeoutException) {
            // Return cached or default response
            return "https://short.ly/error";
        }
        throw new InternalServiceException("Failed to resolve URL");
    }
}

Error Logging Format

Structured logging with consistent fields:

public class ErrorLogger {
    private static final Logger logger = LoggerFactory.getLogger(ErrorLogger.class);
    
    public void logError(String operation, String errorCode, String errorMessage, 
                        Map<String, Object> context) {
        logger.error("URL_SHORTENER_ERROR",
            "operation", operation,
            "errorCode", errorCode,
            "errorMessage", errorMessage,
            "context", context,
            "timestamp", Instant.now().toString(),
            "traceId", MDC.get("trace_id"),
            "userId", MDC.get("user_id"));
    }
}

Alerting Thresholds

Metric	Warning Level	Critical Level	Alerting Channel
5xx Errors	> 10 per minute	> 50 per minute	Slack/Email
Circuit Breaker Open	10% failure rate	50% failure rate	PagerDuty
Cache Miss Rate	> 20%	> 50%	Email
DB Connection Failures	> 5 per minute	> 20 per minute	PagerDuty

Error Response Format

Standardized JSON response:

{
  "error": {
    "code": "INVALID_URL",
    "message": "The provided URL is malformed or invalid.",
    "timestamp": "2023-10-15T14:30:00Z",
    "requestId": "req-123456789"
  }
}

Error Handling in Service Layer

public class UrlShorteningService {
    
    public String shortenUrl(String originalUrl) throws ServiceException {
        try {
            // Validate input
            if (!isValidUrl(originalUrl)) {
                throw new InvalidUrlException("URL format is invalid");
            }
            
            // Generate short code
            String shortCode = generateShortCode(originalUrl);
            
            // Store mapping
            urlMappingRepository.save(new UrlMapping(shortCode, originalUrl));
            
            return shortCode;
        } catch (DatabaseException e) {
            logger.error("Database error during URL shortening", e);
            throw new InternalServiceException("Failed to store URL mapping");
        } catch (CacheException e) {
            logger.warn("Cache error during URL shortening", e);
            // Fallback to database
            return generateShortCode(originalUrl);
        }
    }
}

11. Security Design

Input validation and sanitization strategy

All URL inputs undergo strict validation using regex patterns to ensure they conform to valid URL format (RFC 3986). The validation includes:

• Protocol check (http://, https://)
• Hostname validation with DNS resolution
• Port number range validation (1-65535)
• Path and query parameter sanitization to prevent injection
• Length limits (max 2048 characters for full URL)
• Use of OWASP Java HTML Sanitizer library to prevent XSS in URLs stored as metadata

SQL injection prevention

All database interactions use parameterized prepared statements exclusively. No dynamic SQL string concatenation is performed. Input parameters are strictly typed and validated before binding to queries. The system uses an ORM (Hibernate) with strict type mapping to prevent SQL injection vectors.

Authentication middleware design

The URL shortener service operates as a stateless API with JWT-based authentication for admin operations (URL creation, analytics access). For public read operations, no authentication is required. JWT tokens are validated using HS256 signing with rotating keys and include:

• Issuer (iss)
• Audience (aud)
• Expiration (exp)
• Subject (sub) - user ID or service identifier
• Permissions (permissions) - role-based access control claims

Authorization enforcement

Role-Based Access Control (RBAC) is implemented at the service layer:

• ADMIN role can create, delete, and view all URLs
• USER role can only manage URLs they created
• ANALYTICS role can view analytics but not modify URLs
• Authorization checks occur before any operation on URL entities using a centralized permission checker service that validates JWT claims against resource ownership and roles

Sensitive data encryption at rest and in transit

• In transit: All API communications use TLS 1.3 with HSTS header enforcement (max-age=31536000)
• At rest: URLs are encrypted using AES-256-GCM with a key rotation policy (every 90 days) stored in AWS KMS
• Shortened URLs: The hash portion is encrypted using the same key to prevent reverse-engineering of the original URL
• Metadata: Creation timestamps and user IDs are stored in plaintext but logged with audit trail

PII handling and masking strategy

• User IDs are stored as UUIDs to avoid exposing internal identifiers
• No personal information is stored in URL metadata beyond creation timestamp and owner ID
• Analytics data is aggregated to prevent PII exposure (e.g., only count, not individual click tracking)
• When logging user actions, only the first 4 characters of user IDs are logged for debugging purposes

Secrets management

• API keys and JWT signing keys are stored in AWS Secrets Manager with automatic rotation every 90 days
• Database credentials are managed via AWS RDS IAM authentication or encrypted secrets in Secrets Manager
• Environment-specific configurations (e.g., Redis connection strings) are loaded from AWS Systems Manager Parameter Store with encryption at rest
• Secrets are rotated using a scheduled Lambda function that updates secrets in all services

Audit logging design

All security-relevant operations are logged with structured JSON format containing:

{
  "timestamp": "2023-10-27T14:30:00Z",
  "user_id": "uuid-1234567890",
  "operation": "CREATE_SHORT_URL",
  "resource_id": "abc123",
  "ip_address": "192.168.1.100",
  "user_agent": "Mozilla/5.0...",
  "status": "SUCCESS",
  "request_payload": {},
  "response_payload": {}
}

• Logs are written to CloudWatch with retention policy of 90 days
• Critical events (failed authentication, unauthorized access attempts) trigger immediate alerts to security team
• Audit logs include IP address geolocation data for suspicious activity detection
• Log format includes trace_id for distributed tracing correlation

Security headers and CORS policy

All HTTP responses include security headers:

• X-Content-Type-Options: nosniff
• X-Frame-Options: DENY
• X-XSS-Protection: 1; mode=block
• Strict-Transport-Security: max-age=31536000; includeSubDomains
• Content-Security-Policy: default-src 'self'; script-src 'self' 'unsafe-inline'; object-src 'none'

Rate limiting and DDoS protection

• API rate limits enforced per user (1000 requests/hour) and IP address (5000 requests/hour)
• DDoS protection using AWS WAF with custom rules for suspicious traffic patterns
• Request throttling implemented at the API gateway level before hitting application servers
• Bot detection using behavioral analysis (user agent, request patterns) with automatic IP blocking for malicious actors

Vulnerability scanning and patching

• Weekly automated security scans using OWASP ZAP and SonarQube for code vulnerabilities
• Dependency scanning with Snyk or OWASP Dependency-Check to identify vulnerable libraries
• Patch management policy ensures all dependencies are updated within 7 days of security advisories
• Vulnerability alerts sent to security team via Slack with severity classification and remediation steps

12. Testing Strategy

Unit Testing Scope and Coverage Targets

• Target coverage: 85% code coverage for business logic and service layer classes
• Scope:
  • Service methods with complex business logic (URL shortening, redirection, analytics)
  • Repository layer methods (CRUD operations with validation)
  • Utility classes (URL encoding, hash generation, rate limiting)
  • Exception handling paths and boundary conditions
• Testing approach:
  • Use Mockito for mocking external dependencies (database, cache, messaging)
  • Test edge cases: null inputs, invalid URLs, duplicate URLs, rate limit violations
  • Validate state transitions in domain objects (e.g., URL status changes)
  • Mock time-based operations for testing TTL and expiration logic

Integration Testing Strategy

• Database integration:
  • Use Testcontainers with PostgreSQL to test actual database interactions
  • Validate schema migrations and constraint enforcement (e.g., unique short codes)
  • Test transaction boundaries and concurrent access scenarios
• External service integration:
  • Use WireMock for mocking analytics API or external URL validation services
  • Validate event publishing to message broker (e.g., Kafka) in async flows
• End-to-end flow testing:
  • Test full URL shortening → redirection → analytics pipeline
  • Validate cache integration with real Redis instance (via Testcontainers)
  • Test rate limiting and concurrent request handling

Contract Testing

• Framework: Pact (consumer-driven contract testing)
• Scope:
  • Define contracts between URL shortener and analytics service
  • Define contracts with external URL validation service (if used)
• Implementation:
  • Consumer tests in URL shortener service verify expected payload format
  • Provider tests in analytics service validate contract compliance
  • Contract versioning aligned with API versioning strategy

Performance Testing Tool and Key Scenarios

• Tool: k6 (JavaScript-based load testing)
• Key scenarios:
  • Concurrent URL shortening under high load (1000+ concurrent requests)
  • Redirection throughput with cache hits vs misses
  • Analytics event processing rate (batch size: 1000 events/sec)
  • Cache warming and TTL behavior under stress
• Metrics tracked:
  • p50/p95/p99 response times for all endpoints
  • Error rates and failure scenarios under load
  • Memory usage and GC pressure during high concurrency

Test Data Management Strategy

• Fixtures:
  • Predefined test data in JSON files for URL records, user accounts, and analytics
  • Use factories (e.g., Builder pattern) to generate test URL objects with various properties (valid/invalid, expired, active)
• Seeding:
  • Use database migration scripts for initial test data setup
  • Seed with realistic distribution of URL lengths, access patterns, and expiration dates
• Test data isolation:
  • Use separate test database instances or schema per test suite
  • Clear test data after each test run to prevent cross-contamination
• Data generation:
  • Generate large volume of test URLs for performance testing (e.g., 100K+ entries)
  • Include edge cases: very long URLs, special characters, international domain names

13. Observability Design

Structured Logging Schema

The service will implement structured logging with the following required fields:

{
  "timestamp": "2023-10-27T14:30:45.123Z",
  "level": "INFO",
  "service": "url-shortener-service",
  "trace_id": "12345678-1234-1234-1234-123456789012",
  "span_id": "abcdef12-3456-7890-abcd-ef1234567890",
  "user_id": "user-123",
  "entity_id": "abc123",
  "operation": "generateShortUrl",
  "request_id": "req-456",
  "message": "URL shortening completed successfully",
  "duration_ms": 15,
  "error_code": null,
  "error_message": null,
  "http_status": 201,
  "url_path": "/shorten",
  "method": "POST"
}

PII masking rules:

• User IDs will be masked in logs after first 4 characters (e.g., user-123 → user-***)
• Full URLs will be truncated to first 50 characters + ellipsis for logging
• Query parameters containing sensitive data will be redacted

Distributed Tracing Instrumentation

Using OpenTelemetry for distributed tracing with custom spans:

Key Operations to Trace:

1. url_shortener.generate_short_url - Entry point for URL shortening
2. url_shortener.resolve_short_url - Resolving a short URL to original
3. url_shortener.validate_url - URL format validation
4. url_shortener.check_duplicate - Duplicate URL detection

Span Attributes:

• url.original: Original long URL (redacted for PII)
• url.short_code: Generated short code
• url.user_id: User identifier (if authenticated)
• url.is_duplicate: Boolean indicating if URL was duplicate
• url.attempts: Number of attempts for generation/retry logic

Metrics Design

RED Method Metrics (Rate, Errors, Duration)

Metric Name	Type	Description	Labels
url_shortener_requests_total	Counter	Total requests processed	method, operation, status
url_shortener_errors_total	Counter	Error occurrences by type	method, operation, error_type
url_shortener_request_duration_seconds	Histogram	Request processing duration	method, operation, status
url_shortener_cache_hits_total	Counter	Cache hit count	operation
url_shortener_cache_misses_total	Counter	Cache miss count	operation
url_shortener_active_users	Gauge	Active user count	-
url_shortener_url_count	Gauge	Total URLs stored	-

Additional Metrics:

• url_shortener_duplicate_ratio (Gauge): Ratio of duplicate URLs to total
• url_shortener_generation_retries_total (Counter): Retry attempts for URL generation
• url_shortener_cache_evictions_total (Counter): Cache eviction count

Health Check Endpoint Design

GET /health
Accept: application/json

Response:
{
  "status": "healthy",
  "timestamp": "2023-10-27T14:30:45.123Z",
  "service": "url-shortener-service",
  "database": {
    "status": "healthy",
    "latency_ms": 5
  },
  "cache": {
    "status": "healthy",
    "latency_ms": 2
  },
  "dependencies": {
    "redis": "healthy",
    "postgres": "healthy"
  }
}

Endpoint Variants:

• /ready - Checks if service is ready to serve traffic (DB connection, cache init)
• /live - Basic liveness check (process running)
• /startup - Startup validation (config loaded, dependencies resolved)

Alerting Thresholds

Metric	Warning Level	Critical Level	Alerting Channel
url_shortener_request_duration_seconds (p95)	> 200ms	> 500ms	Slack/On-call
url_shortener_errors_total (per minute)	> 10 errors	> 50 errors	Slack/On-call
url_shortener_cache_miss_rate (per minute)	> 50%	> 70%	Slack
url_shortener_database_connection_errors	> 5 errors	> 20 errors	Slack/On-call
url_shortener_cache_connection_errors	> 5 errors	> 20 errors	Slack/On-call

Escalation Path:

1. First alert → Slack channel
2. Second alert within 5 minutes → On-call engineer
3. Third alert within 10 minutes → Team lead + SRE team

14. Open Questions, Risks & Trade-offs

Decisions requiring product / engineering alignment

• URL encoding strategy: Should we support base62, base32, or base58 for short code generation? Each has different collision probabilities and character set implications for URL safety.
• Short code length policy: Should we enforce a fixed length (e.g., 6 characters) or allow variable-length codes based on usage patterns?
• Redirection behavior: Should we return 301 permanent redirects or 302 temporary redirects for shortened URLs?
• Analytics and tracking: Should we collect click data, user agent information, or geographic data for business intelligence?

Known trade-offs and justifications

• Storage vs. Performance Trade-off: Using a single database table with primary key index offers simplicity but may not scale well for high-throughput scenarios. Alternative approaches like sharding or separate read/write databases trade complexity for performance.
• Consistency vs. Availability Trade-off: Implementing strong consistency for URL creation and retrieval may impact availability during network partitions, while eventual consistency can improve availability but introduces potential race conditions.
• Cache Invalidation vs. Cache Hit Rate Trade-off: Aggressive cache TTLs reduce stale data but increase database load; longer TTLs improve hit rates but may serve outdated content.
• Idempotency vs. Complexity Trade-off: Adding idempotency keys increases API robustness but adds complexity to request handling and storage.

Risks and mitigation strategies

• Rate limiting bypass: Malicious actors could potentially bypass rate limits through IP spoofing or bot networks. Mitigation includes implementing multi-factor rate limiting (IP + user-agent + API key) and monitoring for unusual patterns.
• URL collision: Though rare, hash collisions can occur. Mitigation includes using a sufficiently long random string (e.g., 8+ characters) and implementing collision resolution logic with exponential backoff.
• Cache stampede: High concurrent requests for the same URL can cause cache misses and database overload. Mitigation includes probabilistic early expiry and distributed locking mechanisms.
• Data loss: Single point of failure in database or cache layer can result in complete service unavailability. Mitigation includes implementing read-replicas, distributed caching, and proper backup strategies.

Technical debt acknowledged

• Legacy URL handling: Support for legacy URLs with no metadata or analytics tracking may require additional complexity in the data model and service layer.
• Analytics integration: Early design decisions around analytics collection may not support future requirements for A/B testing or advanced behavioral analysis.
• Multi-tenant support: Initial design assumes single-tenant usage, but future expansion to multi-tenant environments will require additional authentication and authorization logic.
• Internationalization: Current design assumes ASCII URLs, but supporting internationalized domain names (IDN) would require additional validation and encoding logic.
• API versioning: Future API evolution may require breaking changes that could impact existing clients without proper versioning strategy.