A ranking engine is a critical backend system that determines the order of content items based on various factors like relevance, popularity, and user behavior. This document presents a scalable architecture for processing millions of content items with real-time scoring capabilities and efficient sorting mechanisms.
1. Problem Statement & Requirements
Clarifying questions
- What is the scope of ranking? Is this for content (e.g., articles, videos), products, users, or a combination? Are we building a general-purpose ranking engine or one tailored to specific use cases?
- What is the expected scale? What's the anticipated volume of items to rank, and how many concurrent ranking requests per second? Will we need to support real-time ranking or batch processing?
- What consistency requirements are there? Should rankings be strongly consistent, eventually consistent, or can they tolerate some staleness for performance?
- What's the read/write ratio? Is this primarily a read-heavy system where rankings are frequently accessed, or do we expect frequent updates to items and re-ranking?
- Latency sensitivity? What are the latency SLAs for ranking operations (e.g., p50/p95/p99)? Are there different SLAs for real-time vs. batch ranking?
- What authentication model is expected? Will this engine be used internally or exposed to external consumers? What auth/permission model is required?
- Are there any external integrations or dependencies? Will the ranking engine integrate with other systems like search, recommendation engines, or analytics platforms?
Functional requirements
- Rank items based on configurable criteria - Support custom ranking rules and weights for different metrics (e.g., popularity, recency, relevance)
- Support real-time and batch ranking - Handle both immediate ranking requests and scheduled batch recomputations
- Provide ranking context - Return detailed ranking information (score breakdown, rank position, confidence)
- Support multiple ranking modes - Allow configuration of different ranking strategies (e.g., popularity-based, recency-based, custom algorithm)
- Support filtering and faceting - Apply rankings within filtered or faceted contexts (e.g., "top 10 articles in tech category")
- Support ranking persistence - Store and retrieve rankings with versioning capability
- Support ranking history tracking - Maintain historical ranking data for analysis and rollback capability
Non-functional requirements
- Latency targets:
- p50: <50ms for real-time ranking requests
- p95: <200ms for real-time ranking requests
- p99: <500ms for real-time ranking requests
- Throughput:
- 10,000+ real-time ranking requests per second
- 100,000+ batch ranking operations per hour
- Availability:
- 99.9% uptime (28.8 minutes of downtime/month max)
- Scalability:
- Support horizontal scaling across multiple nodes
- Handle 10M+ items in the ranking system
Constraints and assumptions
- Assumption: The engine will primarily serve read-heavy workloads with infrequent updates to items and their ranking criteria
- Constraint: Ranking algorithms must be deterministic to ensure consistency across sessions
- Assumption: Items will have a stable identifier (e.g., UUID) that persists across ranking operations
- Constraint: Ranking engine will not handle item creation or deletion - these are handled by other systems
- Assumption: Ranking weights and configurations can be updated without downtime
Out of scope
- Item creation/deletion: The engine does not handle item lifecycle management
- Search integration: Full-text search capabilities are out of scope (will integrate with existing search systems)
- User personalization: Individual user preferences and personalization are out of scope (will integrate with recommendation systems)
- Real-time analytics: Real-time metrics and dashboards for ranking performance are out of scope
- Machine learning model training: The engine does not train or manage ML models for ranking, only applies pre-trained models
- Cross-system data synchronization: Handling data consistency across multiple systems is out of scope (will rely on event-driven architecture)
2. API Design
API Style Selection
gRPC with Protocol Buffers is selected for the ranking engine due to:
- Strong typing and schema evolution support
- Efficient binary serialization for high-throughput ranking operations
- Built-in support for streaming and bidirectional communication
- Native support for versioning through proto file management
- Better performance characteristics for real-time ranking scenarios
API Endpoints
| Method | Path | Description | Auth |
|---|---|---|---|
| POST | /v1/rank | Rank items based on provided criteria and weights | JWT |
| GET | /v1/rank/{id} | Retrieve ranked items for a specific ranking session | JWT |
| PUT | /v1/rank/{id} | Update existing ranking criteria and weights | JWT |
| DELETE | /v1/rank/{id} | Delete a ranking session | JWT |
| POST | /v1/rank/preview | Preview ranking results without persisting | JWT |
| GET | /v1/rank/history | Retrieve historical ranking sessions | JWT |
Request and Response Schemas
RankRequest (Protocol Buffer):
message RankRequest {
string ranking_id = 1;
repeated Item items = 2;
RankingCriteria criteria = 3;
string user_id = 4;
int32 page_size = 5;
string cursor = 6;
}
message Item {
string id = 1;
map<string, double> features = 2;
string category = 3;
}
message RankingCriteria {
map<string, double> weights = 1;
string sort_by = 2; // "relevance", "popularity", "timestamp"
repeated string excluded_categories = 3;
}RankResponse (Protocol Buffer):
message RankResponse {
string ranking_id = 1;
repeated RankedItem ranked_items = 2;
int32 total_count = 3;
string next_cursor = 4;
double score_threshold = 5;
}
message RankedItem {
string item_id = 1;
double score = 2;
int32 rank = 3;
map<string, double> feature_scores = 4;
}HTTP Status Codes and Error Response
Standardized Error Format:
{
"error": {
"code": "INVALID_ARGUMENT",
"message": "Missing required field: criteria",
"details": [
{
"field": "criteria",
"reason": "Required field is missing"
}
]
}
}Status Codes:
200 OK- Successful ranking201 Created- New ranking created400 Bad Request- Invalid input parameters401 Unauthorized- Missing or invalid auth token403 Forbidden- Insufficient permissions404 Not Found- Ranking session not found429 Too Many Requests- Rate limit exceeded500 Internal Server Error- Server-side failure
API Versioning Strategy
Protocol Buffer Versioning:
- Major version (v1) for breaking changes
- Minor versions within same major version for backward-compatible additions
- Field numbers preserved across versions to maintain compatibility
Pagination Design
Cursor-based Pagination is implemented because:
- More stable and consistent across data changes
- Better performance for large datasets
- Handles concurrent modifications gracefully
Cursor Format:
{
"cursor": "eyJpZCI6IjEyMzQ1Iiwic2NvcmUiOjAuOTUsInJhbmsiOjF9"
}Authentication and Authorization
Authentication:
- JWT tokens with HS256 signing
- Token expiration set to 24 hours
- Refresh token support for long-lived sessions
Authorization:
- RBAC model with roles:
admin,user,service - User-specific ranking sessions with ownership checks
- Service accounts for system-to-system communication
Rate Limiting
Rate Limits:
| Endpoint | Limit | Period |
|---|---|---|
/v1/rank | 1000 requests | 1 minute |
/v1/rank/preview | 5000 requests | 1 minute |
/v1/rank/history | 200 requests | 1 minute |
Enforcement Strategy:
- In-memory token bucket per user
- Redis-based distributed rate limiting for multi-instance deployments
- Burst handling with configurable max burst size
Idempotency Design
Idempotency Key Strategy:
- Client-provided
idempotency_keyheader (UUID format) - Server-side cache with TTL of 5 minutes
- Same operation with same key returns cached result
- Used for
POST /v1/rankandPUT /v1/rank/{id}operations
Example Request/Response
Request:
{
"ranking_id": "rank-12345",
"items": [
{
"id": "item-1",
"features": {
"popularity": 0.8,
"relevance": 0.9
},
"category": "electronics"
}
],
"criteria": {
"weights": {
"popularity": 0.6,
"relevance": 0.4
},
"sort_by": "relevance"
}
}Response:
{
"ranking_id": "rank-12345",
"ranked_items": [
{
"item_id": "item-1",
"score": 0.86,
"rank": 1,
"feature_scores": {
"popularity": 0.48,
"relevance": 0.34
}
}
],
"total_count": 1,
"next_cursor": null
}3. Data Model Design
Entity-Relationship Diagram
erDiagram
USER {
string id PK
string username
string email
timestamp created_at
timestamp updated_at
}
CONTENT {
string id PK
string title
string description
string content_type
timestamp created_at
timestamp updated_at
boolean is_active
}
RANKING {
string id PK
string content_id FK
string user_id FK
integer score
timestamp created_at
timestamp updated_at
}
RANKING_HISTORY {
string id PK
string ranking_id FK
integer old_score
integer new_score
timestamp changed_at
string changed_by
}
USER_CONTENT_SCORE {
string user_id FK
string content_id FK
integer score
timestamp last_updated
primary key (user_id, content_id)
}
USER_CONTENT_SCORE_HISTORY {
string user_id FK
string content_id FK
integer old_score
integer new_score
timestamp changed_at
string changed_by
primary key (user_id, content_id, changed_at)
}
USER --> RANKING : creates
CONTENT --> RANKING : contains
RANKING --> RANKING_HISTORY : logs changes
USER_CONTENT_SCORE --> USER_CONTENT_SCORE_HISTORY : logs changesTable Schema Definitions
users
CREATE TABLE users (
id VARCHAR(36) PRIMARY KEY,
username VARCHAR(255) UNIQUE NOT NULL,
email VARCHAR(255) UNIQUE NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);
CREATE INDEX idx_users_email ON users(email);content
CREATE TABLE content (
id VARCHAR(36) PRIMARY KEY,
title VARCHAR(500) NOT NULL,
description TEXT,
content_type VARCHAR(100) NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
is_active BOOLEAN DEFAULT TRUE
);
CREATE INDEX idx_content_type ON content(content_type);
CREATE INDEX idx_content_active ON content(is_active);rankings
CREATE TABLE rankings (
id VARCHAR(36) PRIMARY KEY,
content_id VARCHAR(36) NOT NULL,
user_id VARCHAR(36) NOT NULL,
score INTEGER NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
FOREIGN KEY (content_id) REFERENCES content(id) ON DELETE CASCADE,
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE,
UNIQUE KEY uk_user_content (user_id, content_id),
INDEX idx_content_score (content_id, score DESC),
INDEX idx_user_score (user_id, score DESC)
);ranking_history
CREATE TABLE ranking_history (
id VARCHAR(36) PRIMARY KEY,
ranking_id VARCHAR(36) NOT NULL,
old_score INTEGER NOT NULL,
new_score INTEGER NOT NULL,
changed_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
changed_by VARCHAR(36),
FOREIGN KEY (ranking_id) REFERENCES rankings(id) ON DELETE CASCADE
);
CREATE INDEX idx_ranking_history_ranking ON ranking_history(ranking_id);
CREATE INDEX idx_ranking_history_changed_at ON ranking_history(changed_at);user_content_scores
CREATE TABLE user_content_scores (
user_id VARCHAR(36) NOT NULL,
content_id VARCHAR(36) NOT NULL,
score INTEGER NOT NULL DEFAULT 0,
last_updated TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (user_id, content_id),
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE,
FOREIGN KEY (content_id) REFERENCES content(id) ON DELETE CASCADE
);
CREATE INDEX idx_user_content_scores_user ON user_content_scores(user_id);
CREATE INDEX idx_user_content_scores_content ON user_content_scores(content_id);user_content_scores_history
CREATE TABLE user_content_scores_history (
user_id VARCHAR(36) NOT NULL,
content_id VARCHAR(36) NOT NULL,
old_score INTEGER NOT NULL,
new_score INTEGER NOT NULL,
changed_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
changed_by VARCHAR(36),
PRIMARY KEY (user_id, content_id, changed_at),
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE,
FOREIGN KEY (content_id) REFERENCES content(id) ON DELETE CASCADE
);
CREATE INDEX idx_user_content_scores_history_user ON user_content_scores_history(user_id);
CREATE INDEX idx_user_content_scores_history_content ON user_content_scores_history(content_id);Primary Key Strategy
UUIDs are used as primary keys for the following reasons:
- Distributed system compatibility: UUIDs can be generated without coordination across services
- Security: Prevents enumeration attacks since IDs are not sequential
- Scalability: No contention on ID generation in high-concurrency environments
- Global uniqueness: Ensures no collisions across distributed systems
Index Design Justification
-
rankings.idx_content_score:
- Covers queries for content rankings by score
- Composite index on (content_id, score DESC) enables efficient top-N queries
-
rankings.idx_user_score:
- Supports user-specific ranking lookups
- Enables efficient queries for a user's content scores
-
users.idx_users_email:
- Supports email-based lookups for authentication and user management
-
content.idx_content_type:
- Enables filtering by content type for different ranking algorithms
-
user_content_scores.idx_user_content_scores_user:
- Optimizes user score lookups for recommendation systems
-
ranking_history.idx_ranking_history_ranking:
- Supports audit trail queries for specific rankings
Data Partitioning Strategy
Partition content by content_type using hash partitioning:
PARTITION BY HASH(content_type) PARTITIONS 4;Partition rankings by content_id hash:
PARTITION BY HASH(content_id) PARTITIONS 8;This ensures even distribution of content and allows for horizontal scaling.
Soft Delete vs. Hard Delete
Soft delete is implemented using is_active boolean flag on content table:
- Content can be marked inactive rather than deleted
- Allows for recovery and audit trail preservation
- Maintains referential integrity in rankings table
Audit Trail Design
All modifications to ranking scores are logged in ranking_history and user_content_scores_history tables:
- Records old vs new score values
- Captures timestamp of change
- Tracks who made the change (user_id)
- Supports rollback and audit capabilities
Data Archival Strategy
Content older than 2 years will be archived to cold storage:
- Move inactive content to
content_archivetable - Keep rankings in active tables for performance
- Archive user scores with content to maintain relationships
- Retention policy: 3 years for active content, indefinite for historical rankings
-- Archival job would move data like:
INSERT INTO content_archive SELECT * FROM content WHERE created_at < DATE_SUB(NOW(), INTERVAL 2 YEAR);
DELETE FROM content WHERE created_at < DATE_SUB(NOW(), INTERVAL 2 YEAR);4. Class Design & Service Layer
UML Class Diagram
classDiagram
class RankingEngine {
+rankItems(items: List~Item~, criteria: List~RankingCriteria~) List~RankedItem~
+updateRanking(item: Item, score: Double) void
}
class Item {
+id: String
+name: String
+score: Double
+lastUpdated: Instant
+metadata: Map~String, Object~
}
class RankingCriteria {
+id: String
+name: String
+weight: Double
+type: CriteriaType
+config: Map~String, Object~
}
class RankedItem {
+item: Item
+rank: Integer
+score: Double
}
class ScoreCalculator {
+calculateScore(item: Item, criteria: List~RankingCriteria~) Double
}
class WeightedScoreCalculator {
+calculateWeightedScore(item: Item, criteria: List~RankingCriteria~) Double
}
class RankingStrategy {
+rank(items: List~Item~, criteria: List~RankingCriteria~) List~RankedItem~
}
class SortBasedRankingStrategy {
+rank(items: List~Item~, criteria: List~RankingCriteria~) List~RankedItem~
}
class ItemRepository {
+save(item: Item) void
+findById(id: String) Optional~Item~
+findAll() List~Item~
+updateScore(itemId: String, newScore: Double) void
}
class EventPublisher {
+publishRankingUpdate(item: Item, oldScore: Double, newScore: Double) void
}
class RankingEvent {
+itemId: String
+oldScore: Double
+newScore: Double
+timestamp: Instant
}
RankingEngine --> Item : uses
RankingEngine --> ScoreCalculator : delegates to
RankingEngine --> RankingStrategy : uses
ScoreCalculator --> Item : reads
ScoreCalculator --> RankingCriteria : reads
RankingStrategy --> SortBasedRankingStrategy : implements
ItemRepository --> Item : persists
EventPublisher --> RankingEvent : publishesState Machine Diagram for Item
stateDiagram-v2
[*] --> Active
Active --> Inactive : score < threshold
Inactive --> Active : score > threshold
Active --> Deleted : softDelete()
Deleted --> [*]Service Class Structure and Responsibilities
The RankingEngine is the primary service responsible for:
- Ranking items based on configured criteria
- Updating item scores and publishing events
- Managing ranking strategies and score calculations
Interface / Abstract Class Design
public interface RankingStrategy {
List<RankedItem> rank(List<Item> items, List<RankingCriteria> criteria);
}
public interface ScoreCalculator {
double calculateScore(Item item, List<RankingCriteria> criteria);
}
public enum CriteriaType {
POPULARITY,
RECENTNESS,
ENGAGEMENT,
CUSTOM
}Business Logic Encapsulation Patterns
The ranking engine uses the Strategy pattern for different ranking approaches and the Factory pattern to create appropriate score calculators based on criteria type.
Critical-Path Method Implementation
public List<RankedItem> rankItems(List<Item> items, List<RankingCriteria> criteria) {
// Transaction boundary - all operations in single DB transaction
return transactionTemplate.execute(status -> {
// 1. Calculate scores for all items
List<RankedItem> rankedItems = new ArrayList<>();
// 2. Apply scoring logic with proper locking to prevent race conditions
for (Item item : items) {
// Read current score with pessimistic locking if needed
Item lockedItem = itemRepository.findByIdForUpdate(item.getId())
.orElseThrow(() -> new ItemNotFoundException("Item not found: " + item.getId()));
double newScore = scoreCalculator.calculateScore(lockedItem, criteria);
// Update score in DB
itemRepository.updateScore(lockedItem.getId(), newScore);
// Publish event for async processing
eventPublisher.publishRankingUpdate(lockedItem, lockedItem.getScore(), newScore);
rankedItems.add(new RankedItem(lockedItem, newScore));
}
// 3. Sort items by calculated score in descending order
rankedItems.sort(Comparator.comparing(RankedItem::getScore).reversed());
// 4. Assign ranks (1-based)
int rank = 1;
for (RankedItem rankedItem : rankedItems) {
rankedItem.setRank(rank++);
}
return rankedItems;
});
}Transaction Boundary Design
A single transaction encompasses:
- Reading items with pessimistic locking to prevent race conditions
- Calculating new scores
- Updating item scores in DB
- Publishing ranking update events
Service Interface Contracts
public interface RankingService {
List<RankedItem> rankItems(List<String> itemIds, String criteriaId);
void updateItemScore(String itemId, double newScore);
List<RankedItem> getTopItems(int limit, String criteriaId);
}Saga / Orchestration Pattern
For distributed transactions involving external systems (e.g., analytics), a saga pattern is used:
- Each ranking operation triggers a saga
- Sagas coordinate updates across multiple services (DB, cache, analytics)
- Compensation logic handles rollback scenarios
Domain Object / Value Object Design
public class Item {
private final String id;
private final String name;
private double score;
private final Instant lastUpdated;
private final Map<String, Object> metadata;
// Constructor, getters, setters
}
public class RankedItem {
private final Item item;
private final int rank;
private final double score;
// Constructor, getters
}
public class RankingCriteria {
private final String id;
private final String name;
private final double weight;
private final CriteriaType type;
private final Map<String, Object> config;
// Constructor, getters
}5. Concurrency & Thread Safety
Concurrency Model
The ranking engine will use a thread-based concurrency model with Java's java.util.concurrent package for managing parallel operations. The system will leverage a thread pool executor with configurable core and maximum pool sizes to handle concurrent ranking calculations and data updates. For I/O-bound operations like database queries and external API calls, the system will use non-blocking I/O patterns with CompletableFuture for asynchronous processing.
Race Condition Prevention Strategy
Race conditions will be prevented through:
- Atomic operations for counter updates using
AtomicLongin ranking calculations - Database-level atomic operations using SQL
UPDATE ... SET ... WHEREwith version checks - Thread-local state management for temporary calculation contexts
- Synchronized access to shared mutable state in ranking algorithms with read-write locks
Optimistic vs. Pessimistic Locking Selection
Optimistic locking will be used for ranking calculations because:
- Ranking updates are infrequent and typically involve small data sets
- High concurrency is expected with multiple simultaneous ranking calculations
- The system can tolerate occasional retries due to conflicts
- Reduces lock contention and improves throughput
Implementation approach:
// Example optimistic locking in ranking update
public boolean updateRankingWithOptimisticLock(String entityId, int newScore) {
int maxRetries = 3;
for (int attempt = 0; attempt < maxRetries; attempt++) {
try {
// Read current state with version
RankingEntity entity = repository.findById(entityId);
int currentVersion = entity.getVersion();
// Perform calculation
entity.setScore(newScore);
entity.setVersion(currentVersion + 1);
// Update with version check
boolean success = repository.updateWithVersionCheck(
entity, currentVersion);
if (success) {
return true;
}
} catch (OptimisticLockException e) {
if (attempt == maxRetries - 1) {
throw new RetryableException("Failed to update ranking after " + maxRetries + " attempts", e);
}
Thread.sleep(100 * (1 << attempt)); // Exponential backoff
}
}
return false;
}Deadlock Prevention Strategy
Deadlock prevention will be achieved through:
- Consistent lock ordering - All ranking operations acquire locks in the same order (by entity ID hash)
- Timeout-based lock acquisition - 500ms timeout for lock acquisition to prevent indefinite blocking
- Hierarchical lock management - Locks acquired in a consistent hierarchy (global -> entity -> attribute)
- Lock monitoring - Periodic checks for long-held locks with alerts
Idempotency Implementation for Concurrent Duplicate Requests
Idempotency will be implemented using a deduplication key strategy:
public class IdempotencyService {
private final RedisTemplate<String, String> redisTemplate;
private static final String IDEMPOTENCY_PREFIX = "idempotency:";
public boolean isIdempotent(String operationKey, String payloadHash) {
String key = IDEMPOTENCY_PREFIX + operationKey;
String existingHash = redisTemplate.opsForValue().get(key);
if (existingHash != null && existingHash.equals(payloadHash)) {
return true; // Already processed
}
// Store the operation hash for 1 hour
redisTemplate.opsForValue().set(key, payloadHash, Duration.ofHours(1));
return false;
}
}Thread Safety in Ranking Calculations
Ranking calculations will be made thread-safe through:
- Immutable calculation objects - All ranking algorithms will use immutable data structures
- Thread-local calculation contexts - Each thread maintains its own calculation state
- Synchronized access to shared configuration - Using
java.util.concurrent.ConcurrentHashMapfor caching algorithm parameters - Atomic counters for performance metrics - Using
AtomicLongfor tracking calculation statistics
Concurrency Control in Batch Operations
For batch ranking updates:
public class BatchRankingProcessor {
private final ExecutorService executorService =
Executors.newFixedThreadPool(10);
public CompletableFuture<Void> processBatch(List<String> entityIds) {
List<CompletableFuture<Void>> futures = new ArrayList<>();
for (String entityId : entityIds) {
CompletableFuture<Void> future = CompletableFuture.runAsync(() -> {
try {
// Process each entity with proper error handling
processEntityRanking(entityId);
} catch (Exception e) {
// Log and continue with other entities
logger.error("Failed to process entity ranking: {}", entityId, e);
}
}, executorService);
futures.add(future);
}
return CompletableFuture.allOf(futures.toArray(new CompletableFuture[0]));
}
}Lock Granularity Strategy
The system will implement entity-level locking with:
- Fine-grained locks for individual ranking entities to minimize contention
- Shared read locks for concurrent read operations
- Exclusive write locks for update operations
- Lock timeout configuration to prevent indefinite waits
Memory Management Considerations
To ensure thread safety in memory:
- Immutable ranking result objects - All calculated rankings are immutable
- Thread-safe collections for intermediate calculation results
- Proper synchronization of shared mutable state
- Avoiding thread-local pollution through proper cleanup mechanisms
Performance Considerations
The concurrency model is designed to:
- Maximize throughput for concurrent ranking calculations
- Minimize lock contention through optimistic locking
- Support horizontal scaling with proper thread pool sizing
- Handle burst traffic gracefully with backpressure mechanisms
The design balances between high throughput, low latency, and system stability while maintaining data consistency through proper concurrency control mechanisms.
6. Sequence Diagrams for Key Flows
sequenceDiagram
participant U as User
participant API as API Gateway
participant Auth as Auth Service
participant Ranker as Ranking Engine
participant DB as Database
participant Cache as Redis Cache
U->>API: GET /rankings?query=tech+news&limit=10
API->>Auth: Validate JWT token
Auth-->>API: Authenticated
API->>Ranker: Process ranking request
Ranker->>Cache: Check if results cached
Cache-->>Ranker: Cache miss
Ranker->>DB: Fetch relevant content items
DB-->>Ranker: Content data
Ranker->>DB: Fetch user preferences
DB-->>Ranker: User preferences
Ranker->>DB: Fetch user engagement data
DB-->>Ranker: Engagement data
Ranker->>Cache: Store ranked results
Cache-->>Ranker: Cached
Ranker-->>API: Return ranked content list
API-->>U: 200 OK with ranked itemssequenceDiagram
participant U as User
participant API as API Gateway
participant Auth as Auth Service
participant Ranker as Ranking Engine
participant DB as Database
participant Cache as Redis Cache
U->>API: POST /rankings/reindex
API->>Auth: Validate admin token
Auth-->>API: Authenticated
API->>Ranker: Trigger reindexing
Ranker->>DB: Fetch all content items
DB-->>Ranker: Content data
Ranker->>DB: Fetch all user engagement
DB-->>Ranker: Engagement data
Ranker->>DB: Update content scores
DB-->>Ranker: Updated scores
Ranker->>Cache: Invalidate all cached rankings
Cache-->>Ranker: Invalidated
Ranker-->>API: Reindexing completed
API-->>U: 200 OKsequenceDiagram
participant U as User
participant API as API Gateway
participant Auth as Auth Service
participant Ranker as Ranking Engine
participant DB as Database
participant Cache as Redis Cache
participant EventBus as Message Broker
U->>API: POST /content/123/rate
API->>Auth: Validate JWT token
Auth-->>API: Authenticated
API->>Ranker: Rate content event
Ranker->>DB: Update content score
DB-->>Ranker: Updated
Ranker->>Cache: Invalidate content cache
Cache-->>Ranker: Invalidated
Ranker->>EventBus: Publish content rated event
EventBus-->>Ranker: Event published
Ranker->>EventBus: Publish user engagement event
EventBus-->>Ranker: Event published
Ranker-->>API: Rate successful
API-->>U: 200 OK7. Caching Design
Cache Strategy Selection
For the ranking engine, we'll use a cache-aside pattern with write-through behavior for critical ranking data. This approach provides the best balance between consistency and performance for a system where freshness of rankings is important but not mission-critical for all reads.
Cache Key Design
Cache keys will follow a namespace strategy to avoid collisions and enable efficient invalidation:
ranking:{ranked_entity_type}:{entity_id}:{timestamp}:{sort_key}
ranking:leaderboard:{leaderboard_id}:{sort_key}:{time_window}
ranking:user_rank:{user_id}:{leaderboard_id}TTL Strategy
| Data Type | TTL | Reason |
|---|---|---|
| Leaderboard rankings | 5 minutes | Frequent updates, acceptable staleness |
| User rank in leaderboard | 1 minute | High user interaction frequency |
| Individual entity scores | 30 seconds | Fast-changing scores |
| Aggregated stats | 10 minutes | Less frequent updates |
Cache Eviction Policy
We'll use LRU (Least Recently Used) eviction policy with a maxmemory policy set to allkeys-lru for Redis clusters. This ensures that the most frequently accessed ranking data stays in cache while maintaining memory efficiency.
Cache Stampede Prevention
To prevent cache stampedes:
- Probabilistic Early Expiry: Cache entries expire 20% earlier than TTL to allow for warm-up
- Distributed Locking: Use Redis distributed locks (
SET key value NX EX ttl) for cache population operations - Cache warming: Pre-warm hot leaderboards during low-traffic periods
Distributed Cache Architecture
We'll implement a Redis Cluster with:
- 3 master nodes for write availability
- 3 replica nodes per master for read scaling
- Automatic failover with Redis Sentinel monitoring
- Cross-zone replication for high availability
Cache Invalidation Strategy
We'll use event-driven invalidation:
- Write-through: When data changes, update cache immediately
- Event-driven invalidation:
- Publish
ranking.updateevents on score changes - Cache consumers listen to these events and invalidate relevant keys
- Publish
- Time-based invalidation: TTLs ensure stale data is eventually cleaned up
@Component
public class RankingCacheManager {
private final RedisTemplate<String, Object> redisTemplate;
private final ObjectMapper objectMapper;
public void invalidateUserRank(String userId, String leaderboardId) {
String key = "ranking:user_rank:" + userId + ":" + leaderboardId;
redisTemplate.delete(key);
}
public void invalidateLeaderboard(String leaderboardId, String sortKey) {
String pattern = "ranking:leaderboard:" + leaderboardId + ":" + sortKey + ":*";
Set<String> keys = redisTemplate.keys(pattern);
redisTemplate.delete(keys);
}
public void invalidateEntityScore(String entityType, String entityId) {
String pattern = "ranking:" + entityType + ":" + entityId + ":*";
Set<String> keys = redisTemplate.keys(pattern);
redisTemplate.delete(keys);
}
}8. Repository / Data Access Layer
Repository Pattern Implementation
The repository pattern is implemented to abstract data access logic and provide a clean separation between domain logic and database operations. Each entity will have its own repository interface and implementation.
public interface RankingRepository {
List<Ranking> findByUserId(String userId);
Optional<Ranking> findById(String id);
List<Ranking> findAll();
Ranking save(Ranking ranking);
void delete(String id);
List<Ranking> findByCriteria(RankingCriteria criteria);
}
public class RankingRepositoryImpl implements RankingRepository {
private final JdbcTemplate jdbcTemplate;
private final ObjectMapper objectMapper;
public RankingRepositoryImpl(JdbcTemplate jdbcTemplate, ObjectMapper objectMapper) {
this.jdbcTemplate = jdbcTemplate;
this.objectMapper = objectMapper;
}
@Override
public List<Ranking> findByUserId(String userId) {
String sql = "SELECT * FROM rankings WHERE user_id = ? ORDER BY created_at DESC";
return jdbcTemplate.query(sql, new Object[]{userId}, new RankingRowMapper());
}
@Override
public Optional<Ranking> findById(String id) {
String sql = "SELECT * FROM rankings WHERE id = ?";
try {
Ranking ranking = jdbcTemplate.queryForObject(sql, new Object[]{id}, new RankingRowMapper());
return Optional.of(ranking);
} catch (EmptyResultDataAccessException e) {
return Optional.empty();
}
}
@Override
public List<Ranking> findAll() {
String sql = "SELECT * FROM rankings ORDER BY created_at DESC";
return jdbcTemplate.query(sql, new RankingRowMapper());
}
@Override
public Ranking save(Ranking ranking) {
if (ranking.getId() == null) {
String id = UUID.randomUUID().toString();
ranking.setId(id);
insertRanking(ranking);
} else {
updateRanking(ranking);
}
return ranking;
}
private void insertRanking(Ranking ranking) {
String sql = """
INSERT INTO rankings (id, user_id, score, metadata, created_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?)
""";
jdbcTemplate.update(sql,
ranking.getId(),
ranking.getUserId(),
ranking.getScore(),
objectMapper.writeValueAsString(ranking.getMetadata()),
ranking.getCreatedAt(),
ranking.getUpdatedAt()
);
}
private void updateRanking(Ranking ranking) {
String sql = """
UPDATE rankings
SET score = ?, metadata = ?, updated_at = ?
WHERE id = ?
""";
jdbcTemplate.update(sql,
ranking.getScore(),
objectMapper.writeValueAsString(ranking.getMetadata()),
ranking.getUpdatedAt(),
ranking.getId()
);
}
@Override
public void delete(String id) {
String sql = "DELETE FROM rankings WHERE id = ?";
jdbcTemplate.update(sql, id);
}
@Override
public List<Ranking> findByCriteria(RankingCriteria criteria) {
StringBuilder sql = new StringBuilder("SELECT * FROM rankings WHERE 1=1");
List<Object> params = new ArrayList<>();
if (criteria.getUserId() != null) {
sql.append(" AND user_id = ?");
params.add(criteria.getUserId());
}
if (criteria.getScoreRange() != null) {
sql.append(" AND score BETWEEN ? AND ?");
params.add(criteria.getScoreRange().getMin());
params.add(criteria.getScoreRange().getMax());
}
sql.append(" ORDER BY created_at DESC");
return jdbcTemplate.query(sql.toString(), params.toArray(), new RankingRowMapper());
}
}ORM vs Raw Query Decisions
For the ranking engine, raw SQL queries are preferred over ORM for performance-critical operations:
- Performance: Raw queries provide better control over query execution and can be optimized for specific use cases
- Complex aggregations: Ranking calculations often require complex SQL operations that are hard to express with ORM
- Database-specific features: We leverage PostgreSQL window functions and materialized views for ranking calculations
However, simple CRUD operations use ORM for developer productivity:
// For simple operations - using JPA/Hibernate
@PersistenceContext
private EntityManager entityManager;
public List<Ranking> findRecentByUserId(String userId) {
return entityManager.createQuery(
"SELECT r FROM Ranking r WHERE r.userId = :userId ORDER BY r.createdAt DESC",
Ranking.class)
.setParameter("userId", userId)
.setMaxResults(100)
.getResultList();
}Read vs Write Separation
The ranking engine follows CQRS pattern for read and write operations:
public class ReadRankingRepository {
private final RedisTemplate<String, String> redisTemplate;
public ReadRankingRepository(RedisTemplate<String, String> redisTemplate) {
this.redisTemplate = redisTemplate;
}
public List<Ranking> getTopRankings(int limit) {
Set<String> rankingKeys = redisTemplate.opsForZSet().range("rankings:leaderboard", 0, limit - 1);
return rankingKeys.stream()
.map(key -> {
String json = redisTemplate.opsForValue().get(key);
return parseRankingJson(json);
})
.collect(Collectors.toList());
}
public void updateLeaderboard(Ranking ranking) {
String key = "ranking:" + ranking.getId();
String value = objectMapper.writeValueAsString(ranking);
// Update in Redis with sorted set for leaderboards
redisTemplate.opsForZSet().add("rankings:leaderboard", key, ranking.getScore());
redisTemplate.opsForValue().set(key, value, 30, TimeUnit.DAYS);
}
}Connection Pool Configuration
Connection pool is configured with optimal settings for high-concurrency ranking engine:
# application.yml
spring:
datasource:
hikari:
maximum-pool-size: 50
minimum-idle: 10
connection-timeout: 30000
idle-timeout: 600000
max-lifetime: 1800000
leak-detection-threshold: 60000N+1 Query Prevention Strategy
To prevent N+1 queries, eager loading and batch operations are implemented:
// Using @BatchSize annotation for batch loading
@OneToMany(mappedBy = "ranking", fetch = FetchType.LAZY)
@BatchSize(size = 20)
private List<RankingEntry> entries;
// Custom batch query for ranking entries
public List<RankingEntry> findEntriesForRankings(List<String> rankingIds) {
String sql = "SELECT * FROM ranking_entries WHERE ranking_id IN (" +
rankingIds.stream().map(id -> "'"+id+"'").collect(Collectors.joining(",")) + ")";
return jdbcTemplate.query(sql, new RankingEntryRowMapper());
}Batch Operation Design
Batch operations are implemented for bulk ranking updates:
public class BatchRankingService {
private final JdbcTemplate jdbcTemplate;
public void batchUpdateRankings(List<RankingUpdateRequest> updates) {
String sql = "UPDATE rankings SET score = ?, updated_at = ? WHERE id = ?";
jdbcTemplate.batchUpdate(sql, new BatchPreparedStatementSetter() {
@Override
public void setValues(PreparedStatement ps, int i) throws SQLException {
RankingUpdateRequest update = updates.get(i);
ps.setDouble(1, update.getScore());
ps.setTimestamp(2, Timestamp.valueOf(LocalDateTime.now()));
ps.setString(3, update.getId());
}
@Override
public int getBatchSize() {
return updates.size();
}
});
}
}Data Access Layer Design Summary
The repository layer is designed with:
- Separation of concerns: Clear distinction between read and write operations
- Performance optimization: Raw SQL for complex queries, ORM for simple CRUD
- Scalability: Connection pooling and batch operations for high throughput
- Maintainability: Clean interfaces with proper error handling and logging
- Flexibility: Support for both relational and NoSQL data stores through abstraction
9. Asynchronous Processing & Messaging
Event-driven boundaries in this feature
The ranking engine will process content items and update their scores asynchronously to avoid blocking user requests. The key async boundaries are:
- Content creation/update → triggers ranking recalculation
- User interaction events (likes, comments, shares) → trigger content score adjustments
- Periodic re-rankings → batch updates for content that hasn't been recently interacted with
Synchronous operations include:
- Real-time score retrieval for a specific content item
- Manual rank adjustment by admin
Message schema design
{
"version": "1.0",
"eventType": "content_ranking_update",
"payload": {
"contentId": "uuid",
"score": 87.5,
"timestamp": "2023-10-27T10:30:00Z",
"reason": "user_interaction",
"interactionType": "like"
},
"metadata": {
"source": "user-service",
"eventId": "uuid",
"correlationId": "uuid"
}
}Dead letter queue strategy and poison message handling
Messages that fail processing more than 3 times will be moved to a dead letter queue (DLQ) with:
- A 24-hour delay before retry
- Metadata retention for debugging (original message, error details, failure count)
- Alerting to on-call team when DLQ threshold is exceeded (5+ messages)
Poison message handling:
- Messages that consistently fail due to malformed data will be flagged for manual review
- Retry logic includes exponential backoff with jitter (1s, 2s, 4s, 8s max)
At-least-once vs. exactly-once delivery
At-least-once delivery is selected because:
- Content scores are idempotent - multiple updates to same content with same score don't change the result
- The system uses a combination of content ID + timestamp to ensure duplicate events don't cause incorrect score adjustments
- The cost of ensuring exactly-once (complex state tracking) outweighs the benefit for this use case
Retry policy for consumers
Retry configuration:
- Max attempts: 3
- Backoff formula: exponential with jitter (1s + random(0,1s), 2s + random(0,1s), 4s + random(0,1s))
- Non-retryable errors: malformed content ID, invalid score format
- Retryable errors: network timeouts, database connection issues
Outbox pattern for reliable event publishing
The outbox pattern is implemented to ensure message delivery reliability:
public class OutboxEvent {
private String id;
private String eventType;
private String payload;
private String status; // PENDING, SENT, FAILED
private LocalDateTime createdAt;
private LocalDateTime updatedAt;
}
public class EventPublisher {
private final EventRepository eventRepository;
private final KafkaTemplate<String, String> kafkaTemplate;
public void publishEvent(String eventType, String payload) {
// 1. Save to outbox table
OutboxEvent event = new OutboxEvent();
event.setId(UUID.randomUUID().toString());
event.setEventType(eventType);
event.setPayload(payload);
event.setStatus("PENDING");
eventRepository.save(event);
// 2. Publish to Kafka
try {
kafkaTemplate.send("ranking-events", event.getId(), payload);
event.setStatus("SENT");
} catch (Exception e) {
event.setStatus("FAILED");
// Log and trigger DLQ handling
}
eventRepository.save(event);
}
}Sequence diagram for async flows
sequenceDiagram
participant User
participant ContentService
participant RankingEngine
participant Kafka
participant EventProcessor
User->>ContentService: Create content
ContentService->>Kafka: Publish content_created event
Kafka->>EventProcessor: Deliver content_created event
EventProcessor->>RankingEngine: Trigger rank calculation
RankingEngine->>RankingEngine: Calculate new score
RankingEngine->>Kafka: Publish content_ranked event
Kafka->>ContentService: Deliver content_ranked event
ContentService->>User: Return updated content with rank10. Error Handling Design
Error Classification Hierarchy
The ranking engine follows a three-tier error classification system:
- Domain Errors - Business logic violations (e.g., invalid ranking criteria, missing required fields)
- Infrastructure Errors - System-level issues (e.g., database connection failures, cache timeouts)
- Validation Errors - Input parameter issues (e.g., malformed query parameters, out-of-range scores)
public enum ErrorType {
VALIDATION_ERROR,
DOMAIN_ERROR,
INFRASTRUCTURE_ERROR
}Error Propagation Strategy
Errors are propagated up the call stack with appropriate wrapping to maintain context:
public class RankingEngineException extends RuntimeException {
private final ErrorType errorType;
private final String errorCode;
public RankingEngineException(ErrorType errorType, String errorCode, String message) {
super(message);
this.errorType = errorType;
this.errorCode = errorCode;
}
public RankingEngineException(ErrorType errorType, String errorCode, String message, Throwable cause) {
super(message, cause);
this.errorType = errorType;
this.errorCode = errorCode;
}
// Getters
}Retry Strategy with Exponential Backoff and Jitter
For infrastructure errors, a retry mechanism is implemented with exponential backoff and jitter:
public class RetryableOperation {
private static final int MAX_RETRIES = 3;
private static final long BASE_DELAY_MS = 1000;
private static final double JITTER_FACTOR = 0.2;
public <T> T executeWithRetry(Supplier<T> operation) {
int attempts = 0;
while (true) {
try {
return operation.get();
} catch (RankingEngineException e) {
if (attempts >= MAX_RETRIES || !isRetryable(e)) {
throw e;
}
long delay = calculateDelay(attempts);
try {
Thread.sleep(delay);
} catch (InterruptedException ie) {
Thread.currentThread().interrupt();
throw new RankingEngineException(
ErrorType.INFRASTRUCTURE_ERROR,
"RETRY_INTERRUPTED",
"Retry interrupted",
ie
);
}
attempts++;
}
}
}
private boolean isRetryable(RankingEngineException e) {
return e.getErrorType() == ErrorType.INFRASTRUCTURE_ERROR;
}
private long calculateDelay(int attempt) {
long baseDelay = (long) Math.pow(2, attempt) * BASE_DELAY_MS;
long jitter = (long) (baseDelay * JITTER_FACTOR * (Math.random() - 0.5) * 2);
return baseDelay + jitter;
}
}Circuit Breaker Design
For external dependencies and high-traffic operations, a circuit breaker is implemented:
public class CircuitBreaker {
private final int failureThreshold;
private final long timeout;
private final int successThreshold;
private volatile CircuitState state = CircuitState.CLOSED;
private int failureCount = 0;
private long lastFailureTime = 0;
public enum CircuitState {
CLOSED, OPEN, HALF_OPEN
}
public <T> T execute(Supplier<T> operation) {
switch (state) {
case CLOSED:
return executeClosed(operation);
case OPEN:
return executeOpen(operation);
case HALF_OPEN:
return executeHalfOpen(operation);
default:
throw new IllegalStateException("Unknown circuit state");
}
}
private <T> T executeClosed(Supplier<T> operation) {
try {
T result = operation.get();
onSuccess();
return result;
} catch (Exception e) {
onFailure();
throw new RankingEngineException(
ErrorType.INFRASTRUCTURE_ERROR,
"CIRCUIT_OPEN",
"Circuit breaker is open"
);
}
}
private <T> T executeOpen(Supplier<T> operation) {
if (System.currentTimeMillis() - lastFailureTime > timeout) {
state = CircuitState.HALF_OPEN;
return executeHalfOpen(operation);
}
throw new RankingEngineException(
ErrorType.INFRASTRUCTURE_ERROR,
"CIRCUIT_OPEN",
"Circuit breaker is open"
);
}
private <T> T executeHalfOpen(Supplier<T> operation) {
try {
T result = operation.get();
state = CircuitState.CLOSED;
failureCount = 0;
return result;
} catch (Exception e) {
state = CircuitState.OPEN;
throw new RankingEngineException(
ErrorType.INFRASTRUCTURE_ERROR,
"CIRCUIT_OPEN",
"Circuit breaker is open"
);
}
}
private void onSuccess() {
failureCount = 0;
}
private void onFailure() {
failureCount++;
lastFailureTime = System.currentTimeMillis();
if (failureCount >= failureThreshold) {
state = CircuitState.OPEN;
}
}
}Fallback Behavior Design
For critical operations, fallback mechanisms are implemented:
public class RankingFallback {
public List<RankedItem> getRankingsWithFallback(
String criteria,
List<String> items,
Supplier<List<RankedItem>> primaryOperation
) {
try {
return primaryOperation.get();
} catch (RankingEngineException e) {
// Log the error for monitoring
logger.warn("Primary ranking operation failed, using fallback", e);
// Return default ranked items or empty list based on error type
if (e.getErrorType() == ErrorType.VALIDATION_ERROR) {
return Collections.emptyList();
}
// For infrastructure errors, attempt to return cached results or default ranking
return fallbackToCachedRankings(criteria);
}
}
private List<RankedItem> fallbackToCachedRankings(String criteria) {
// Implementation depends on cache layer design
return Collections.emptyList();
}
}Error Logging Format
Structured logging follows a consistent schema:
public class ErrorLogFormatter {
public static String formatError(
String operation,
String userId,
String entityId,
String errorCode,
String errorMessage,
Exception exception
) {
return String.format(
"{\"operation\":\"%s\",\"userId\":\"%s\",\"entityId\":\"%s\",\"errorCode\":\"%s\"," +
"\"errorMessage\":\"%s\",\"stackTrace\":\"%s\",\"timestamp\":\"%s\"}",
operation, userId, entityId, errorCode,
escapeJson(errorMessage), escapeJson(exception.toString()),
Instant.now().toString()
);
}
private static String escapeJson(String input) {
return input.replace("\\", "\\\\").replace("\"", "\\\"");
}
}Alerting Thresholds
Critical error thresholds trigger alerts:
| Error Type | Alert Level | Threshold | Alert Channel |
|---|---|---|---|
| Infrastructure errors | Critical | >100 occurrences/minute | On-call |
| Domain errors | Warning | >50 occurrences/minute | Slack |
| Validation errors | Info | >1000 occurrences/hour |
Error Budget Considerations
The ranking engine operates within a defined error budget:
public class ErrorBudgetManager {
private static final double ERROR_BUDGET_PERCENTAGE = 0.05; // 5% of total requests
public boolean isWithinErrorBudget(int totalRequests, int errorCount) {
double errorRate = (double) errorCount / totalRequests;
return errorRate <= ERROR_BUDGET_PERCENTAGE;
}
public void handleErrorBudgetExceeded() {
// Trigger alerting and potentially pause new ranking operations
logger.error("Error budget exceeded - triggering alert");
}
}11. Security Design
Input Validation and Sanitization Strategy
All API inputs undergo strict validation using Apache Commons Validator and custom regex patterns for:
- Numeric ranges (e.g., rank limits 1-1000)
- String length constraints (max 255 chars for titles)
- Allowed character sets (alphanumeric + spaces + hyphens for names)
SQL Injection Prevention
All database queries use parameterized prepared statements exclusively. No dynamic query construction or string concatenation. ORM framework (Hibernate) is configured with strict validation mode to prevent any raw SQL injection vectors.
Authentication Middleware Design
Custom JWT-based authentication middleware with:
- Token expiration handling (1-hour TTL)
- Refresh token rotation (30-day validity)
- Token revocation via Redis blacklist with 1-hour TTL
- Middleware intercepts all requests and validates JWT signature, expiration, and audience claims
Authorization Enforcement
Role-Based Access Control (RBAC) with:
- Admin role: Full CRUD access to all rankings and entities
- Editor role: Read/write access to owned rankings only
- Viewer role: Read-only access to public rankings
- Permission checks performed at service layer using Spring Security annotations (
@PreAuthorize) - User context propagation through
SecurityContextandAuthenticationobjects
Sensitive Data Encryption
At Rest:
- AES-256 GCM encryption for sensitive ranking data (metadata, user preferences)
- Key rotation every 90 days using AWS KMS
- Encrypted field annotations on domain objects (e.g.,
@Encrypted)
In Transit:
- TLS 1.3 with HSTS header enforcement
- All API endpoints require HTTPS (redirect from HTTP)
- Certificate pinning for external service integrations
PII Handling and Masking Strategy
- User identifiers (user IDs, IP addresses) are hashed using SHA-256 with salt
- Personal names are masked in logs and audit trails (first letter + asterisks)
- Audit trail entries contain only hashed user IDs, not raw identifiers
- GDPR-compliant data retention policy (6 months for logs, 1 year for user preferences)
Secrets Management
- AWS Secrets Manager integration for:
- Database credentials (rotated every 30 days)
- JWT signing keys (rotated every 90 days)
- External service API keys
- Secrets are fetched at application startup and cached with automatic refresh on expiration
- No hardcoded secrets in source code or configuration files
Audit Logging Design
Events Logged:
- Ranking creation/deletion (user, timestamp, action)
- Rank modifications (before/after values, change reason)
- User access to rankings (user ID, timestamp, action type)
Format:
{
"timestamp": "2023-10-15T14:30:00Z",
"userId": "hashed_user_id",
"action": "RANKING_CREATE",
"entityId": "uuid-12345",
"details": {
"rankingName": "hashed_name",
"modifiedFields": ["rank", "score"]
}
}Retention:
- 1 year for audit logs
- 30 days for error logs and security events
- Logs stored in secure S3 bucket with encryption at rest
// Example of input validation in service layer
public class RankingService {
private static final Pattern VALID_RANKING_NAME = Pattern.compile("^[a-zA-Z0-9\\s\\-]{1,255}$");
public void createRanking(RankingRequest request) {
// Validate input
if (!VALID_RANKING_NAME.matcher(request.getName()).matches()) {
throw new InvalidInputException("Invalid ranking name format");
}
// Sanitize input
String sanitized = sanitizeInput(request.getName());
// Validate numeric ranges
if (request.getRank() < 1 || request.getRank() > 1000) {
throw new InvalidInputException("Rank must be between 1 and 1000");
}
// Proceed with creation...
}
private String sanitizeInput(String input) {
return input.replaceAll("[^a-zA-Z0-9\\s\\-]", "");
}
}12. Testing Strategy
Unit Testing Scope and Coverage Targets
- Target coverage: 85% line coverage, 90% branch coverage for core business logic
- Scope: All service classes, domain objects, value objects, and utility functions
- Testing approach:
- Mock external dependencies (database, cache, messaging systems)
- Test edge cases and boundary conditions for ranking calculations
- Validate business rule implementations with parameterized test cases
Integration Testing Strategy
- Database integration:
- Use testcontainers for PostgreSQL/MySQL instances to validate actual query execution and schema compliance
- Test transaction boundaries and data consistency across multiple operations
- External service integration:
- WireMock or equivalent for testing interactions with external ranking providers (e.g., Elasticsearch, Solr)
- Validate API contract compliance and error handling for external service failures
- End-to-end flow testing:
- Test complete ranking pipeline from data ingestion to result retrieval
- Validate cache interaction and consistency between database and cache layers
Contract Testing
- Framework: Pact for consumer-driven contract testing
- Scope:
- Define contracts between ranking service and downstream consumers (e.g., search, recommendation services)
- Validate that the API response structure matches consumer expectations
- Versioning: Contract version alignment with API versioning strategy
Performance Testing Tool and Key Scenarios
- Tool: k6 for load testing and performance benchmarking
- Key scenarios:
- Concurrent ranking requests (1000+ concurrent users)
- High-volume data ingestion and recalculation workflows
- Latency-sensitive ranking operations (p95 < 200ms, p99 < 500ms)
- Metrics tracked:
- Request rate, response time distribution, error rates
- Throughput for batch ranking operations
Test Data Management Strategy
- Fixtures:
- Predefined test data sets for common ranking scenarios (e.g., trending, popularity, recency)
- Use JSON/YAML fixtures for consistent data across test environments
- Factories:
- Implement factory pattern for generating realistic test entities (users, items, rankings)
- Support for generating large-scale synthetic data sets for performance testing
- Seeding:
- Automated test data setup using database migration scripts with test-specific seed data
- Ensure test isolation by cleaning up test data after each test run
// Example unit test for ranking calculation logic
@Test
public void testRankingCalculationWithMultipleFactors() {
// Given
List<RankingFactor> factors = Arrays.asList(
new RankingFactor("popularity", 0.4),
new RankingFactor("recency", 0.3),
new RankingFactor("engagement", 0.3)
);
Item item = Item.builder()
.id("item-1")
.popularity(95)
.recency(2)
.engagement(80)
.build();
// When
double score = rankingService.calculateRankingScore(item, factors);
// Then
assertThat(score).isEqualTo(85.0); // Expected weighted score calculation
}13. Observability Design
Structured Logging Schema
All logs must follow a consistent schema to enable effective querying and analysis:
{
"timestamp": "2023-10-27T14:30:45.123Z",
"level": "INFO|WARN|ERROR",
"service": "ranking-engine",
"trace_id": "12345678-1234-1234-1234-123456789012",
"span_id": "abcdef12-3456-7890-abcd-ef1234567890",
"user_id": "user_12345",
"entity_id": "rankable_item_98765",
"operation": "calculateRanking",
"request_id": "req_abc123",
"duration_ms": 45,
"error_code": "RANKING_CALCULATION_FAILED",
"error_message": "Failed to calculate rank for item due to invalid weight",
"payload": {
"item_id": "item_123",
"weights": {"popularity": 0.7, "recency": 0.3}
},
"pii_masked": true
}PII fields (user_id, email, etc.) must be masked in logs using a consistent pattern: *** or hash-based obfuscation. The logging framework should support structured logging with contextual fields and automatic trace propagation.
Distributed Tracing Instrumentation
OpenTelemetry is used for distributed tracing with custom spans:
Key Operations to Trace:
calculateRanking- Main ranking calculation operationfetchItemMetadata- Fetches item metadata for scoringapplyWeights- Applies weight calculations to itemspersistRankings- Saves calculated rankings to database
Span Attributes:
// Example span attributes for calculateRanking operation
span.setAttribute("ranking_engine.operation", "calculateRanking");
span.setAttribute("ranking_engine.item_count", items.size());
span.setAttribute("ranking_engine.algorithm", "weighted_score");
span.setAttribute("ranking_engine.cache_hit", false);Custom Spans:
RankingCalculationSpan- Wraps entire ranking calculation workflowItemScoringSpan- Per-item scoring operationDatabaseQuerySpan- Database query operations with query plan
Metrics Design
RED Method Metrics (Rate, Errors, Duration):
| Metric Name | Type | Labels | Description |
|---|---|---|---|
ranking_calculations_total | Counter | operation, status | Total ranking calculations |
ranking_calculations_errors | Counter | operation, error_type | Failed ranking calculations |
ranking_calculation_duration_ms | Histogram | operation | Duration of ranking calculations |
cache_hits_total | Counter | cache_type, hit_type | Cache hit statistics |
database_query_duration_ms | Histogram | query_type, status | Database query latencies |
Additional Service-Level Metrics:
active_ranking_jobs- Gauge, tracks concurrent ranking jobscache_hit_rate- Gauge, calculated from cache metricsqueue_depth- Gauge, tracks pending ranking calculation jobsitem_processing_rate- Counter, items processed per second
Health Check Endpoint Design
Endpoint Structure:
GET /health
GET /ready
GET /live
GET /startupResponse Format:
{
"status": "healthy|unhealthy",
"timestamp": "2023-10-27T14:30:45.123Z",
"service": "ranking-engine",
"checks": {
"database": {
"status": "healthy",
"latency_ms": 12
},
"cache": {
"status": "healthy",
"latency_ms": 5
},
"dependencies": {
"status": "healthy",
"details": ["auth-service:healthy", "notification-service:healthy"]
}
},
"version": "1.2.3",
"uptime_seconds": 3600
}Check Types:
/health- Basic service health, checks core dependencies/ready- Service ready for traffic, includes database connectivity and cache status/live- Process alive check, basic liveness probe/startup- Startup validation, validates configuration and initialization
Alerting Thresholds
Critical Alerts (Escalation to On-Call):
| Metric | Threshold | Alert Condition | Escalation |
|---|---|---|---|
ranking_calculations_errors | > 5 errors/minute | Error rate exceeds threshold for 5 min | |
ranking_calculation_duration_ms | > 500ms p99 | P99 latency exceeds threshold | |
database_query_duration_ms | > 1000ms p99 | DB query latency exceeds threshold | |
cache_hit_rate | < 80% | Cache hit rate drops below threshold |
Warning Alerts (Monitoring):
| Metric | Threshold | Alert Condition |
|---|---|---|
ranking_calculation_duration_ms | > 200ms p95 | P95 latency exceeds threshold |
cache_hit_rate | < 90% | Cache hit rate drops below threshold |
queue_depth | > 100 | Pending jobs exceed threshold |
On-Call Escalation Path:
- Critical alerts → On-call engineer → Slack alert + email
- Warning alerts → Monitoring team → Slack notification
Log Rotation and Retention
- Log files: Rotate every 24 hours or 100MB
- Retention policy: 30 days for operational logs, 90 days for audit logs
- Compression: GZIP compression for archived logs
- Centralized logging: Send to ELK stack or cloud logging service with search capabilities
Performance Monitoring
- CPU usage: Alert if > 85% for more than 5 minutes
- Memory usage: Alert if > 90% of allocated heap
- Thread pool utilization: Alert if > 95% of max threads in use
- Database connection pool: Alert if > 90% of connections in use
Audit Trail Integration
All ranking-related operations must be logged with:
- User context (if available)
- Timestamps
- Operation type and parameters
- Results and status
- Error details for failed operations
This observability design enables comprehensive monitoring, rapid incident response, and performance optimization for the ranking engine system.
14. Open Questions, Risks & Trade-offs
Decisions requiring product / engineering alignment
- Ranking algorithm selection: Should we use a simple weighted score system or complex machine learning models? This decision will impact both development timeline and operational complexity.
- Real-time vs. batch ranking: How frequently should rankings be updated? Real-time updates increase system complexity and latency, while batch processing introduces staleness.
- Ranking scope: Should rankings be global or scoped to specific user segments, regions, or contexts? This affects data model complexity and caching strategy.
- Personalization depth: How much personalization should be applied to rankings? Deep personalization increases system complexity and requires more training data.
Known trade-offs and justifications
| Trade-off | Option 1 | Option 2 | Justification |
|---|---|---|---|
| Consistency vs. Performance | Strong consistency with distributed locks | Eventual consistency with optimistic concurrency | For a ranking engine, users may tolerate slight staleness for better performance and scalability |
| Complexity vs. Maintainability | Complex ML-based ranking algorithms | Simple weighted scoring system | Simpler systems are easier to debug, maintain, and scale |
| Real-time vs. Batch Updates | Real-time ranking updates | Batch-based updates with hourly granularity | Batch processing reduces system load and improves throughput |
| Storage vs. Compute | Store pre-computed rankings | Compute rankings on-demand | Pre-computed rankings improve read performance but increase storage requirements |
Risks and mitigation strategies
-
Ranking algorithm bias: Risk of systematic favoritism toward certain content categories or user segments
- Mitigation: Regular algorithm audits, A/B testing with diverse user groups, bias detection mechanisms
-
Cache stampede: High concurrent requests causing cache misses and database overload
- Mitigation: Implement probabilistic early expiration, distributed locking for cache warming, request queuing
-
Data inconsistency: Rankings becoming stale due to concurrent updates
- Mitigation: Use optimistic locking with version checks, implement proper transaction boundaries, cache invalidation strategies
-
Scalability bottlenecks: Ranking engine becoming a single point of failure under high load
- Mitigation: Horizontal sharding by content category, read replicas for ranking computation, circuit breaker patterns
Technical debt acknowledged
- Algorithm evolution complexity: Future modifications to ranking algorithms may require significant refactoring of existing systems
- Monitoring gap: Current design lacks comprehensive metrics for ranking quality and user engagement tracking
- Personalization limitations: Initial implementation may not support advanced personalization features that could be added later
- Eventual consistency: The system's eventual consistency model may lead to inconsistent user experiences in high-concurrency scenarios
- Data freshness: Batch processing introduces latency that may impact user experience for time-sensitive content ranking