Payment Gateway Integration Design: Secure Transaction Processing System

Comprehensive low-level design for a scalable payment gateway integration system with fraud detection and multi-provider support.

PublishedNovember 9, 2025
Length9,116 words
Reading time46 min read

Backend LLD

This document presents a robust low-level design for integrating multiple payment gateways with focus on security, reliability, and scalability for handling financial transactions.

1. Problem Statement & Requirements

Clarifying questions

What scale and throughput requirements are expected?
- How many transactions per second (TPS) should the system support?
- What is the expected peak volume and concurrent user load?
What consistency and availability guarantees are required?
- Should all transactions be strongly consistent or eventual consistency acceptable?
- What is the target uptime (e.g., 99.9% SLA)?
What are the key entities and their relationships?
- Are there customers, accounts, payment methods, orders, etc.?
- How do these entities relate to each other (e.g., one-to-many, many-to-many)?
What is the read/write ratio and latency sensitivity?
- Is the system primarily used for processing payments or more for querying transaction status?
- What are the acceptable latencies (e.g., p50/p95/p99) for payment creation vs. status checks?
What authentication and authorization model is in place?
- Will the gateway support OAuth2, JWT tokens, or API keys?
- Are there role-based access controls (RBAC) for different user roles?
What external integrations are required?
- Which payment providers (e.g., Stripe, PayPal, Adyen) will be supported?
- Are there third-party fraud detection or ACH validation services?
What is the out-of-scope boundary for this design?
- Should we handle refunds, disputes, or chargebacks?
- Is the system expected to support recurring payments or subscriptions?

Functional requirements

Accept and process payment requests from merchants
Support multiple payment methods (credit cards, ACH, digital wallets)
Handle payment authorizations and captures
Support refunds and chargebacks (if in scope)
Provide real-time transaction status updates
Enable payment method storage and retrieval for future use
Support recurring payments (if in scope)
Integrate with external payment gateways (e.g., Stripe, Adyen, PayPal)
Support fraud detection and prevention mechanisms
Enable transaction reconciliation with external providers

Non-functional requirements

Latency targets:
- p50: < 100ms
- p95: < 300ms
- p99: < 500ms
Throughput:
- Minimum 10,000 TPS during peak hours
- Support for burst traffic up to 50,000 TPS
Availability:
- 99.9% uptime (43.2 minutes of downtime/month max)
- Zero-downtime deployments
Security:
- PCI-DSS compliance for card data handling
- End-to-end encryption of sensitive payment data

Constraints and assumptions

The system will integrate with one or more external payment providers (e.g., Stripe, Adyen)
All sensitive data must be encrypted at rest and in transit
Transactional integrity is critical; all payment operations must be idempotent
The system will support a multi-tenant architecture for different merchants
Payment methods will be stored securely using tokenization or encryption
Fraud detection is handled by external providers or internal rules engine
All communication with external providers will be over HTTPS

Out of scope

Full fraud detection and prevention engine (handled by external providers or internal rules)
Chargeback management and dispute resolution workflows
Subscription and recurring billing (unless explicitly in scope)
Multi-currency support (unless specified)
Real-time settlement with banks or payment processors
Internal accounting and financial reporting features
Mobile SDKs or web UI for end-users to interact with payments directly

2. API Design

API Style Selection

gRPC with Protocol Buffers is selected for the payment gateway integration due to:

Strong typing and automatic code generation
Efficient binary serialization (lower payload size than JSON)
Built-in support for streaming and bidirectional communication
Native support for versioning through proto files
Better performance characteristics for high-throughput payment operations

Endpoint Definitions

Method	Path	Description	Auth	Rate Limit
CreatePaymentIntent	`/v1/payment/intents`	Creates a new payment intent with amount and currency	JWT + OAuth2	100/minute
ConfirmPaymentIntent	`/v1/payment/intents/{intentId}/confirm`	Confirms and processes a payment intent	JWT + OAuth2	50/minute
CancelPaymentIntent	`/v1/payment/intents/{intentId}/cancel`	Cancels an unconfirmed payment intent	JWT + OAuth2	100/minute
GetPaymentIntent	`/v1/payment/intents/{intentId}`	Retrieves payment intent details	JWT + OAuth2	200/minute
RefundPayment	`/v1/payment/refunds`	Initiates a refund for completed payment	JWT + OAuth2	25/minute
GetRefund	`/v1/payment/refunds/{refundId}`	Retrieves refund details	JWT + OAuth2	200/minute
ListPaymentIntents	`/v1/payment/intents`	Lists payment intents with filters	JWT + OAuth2	50/minute

Request and Response Schemas

CreatePaymentIntentRequest

{
  "amount": 1000,
  "currency": "usd",
  "description": "Payment for order #12345",
  "customer_id": "cus_1234567890",
  "metadata": {
    "order_id": "12345",
    "source": "web"
  }
}

CreatePaymentIntentResponse

{
  "intent_id": "pi_1234567890",
  "client_secret": "pi_1234567890_secret_abcde",
  "amount": 1000,
  "currency": "usd",
  "status": "requires_payment_method",
  "created_at": "2023-10-15T14:30:00Z"
}

ConfirmPaymentIntentRequest

{
  "payment_method_id": "pm_1234567890",
  "return_url": "https://shop.example.com/payment-success"
}

ConfirmPaymentIntentResponse

{
  "intent_id": "pi_1234567890",
  "status": "succeeded",
  "payment_method_id": "pm_1234567890",
  "amount": 1000,
  "currency": "usd",
  "created_at": "2023-10-15T14:30:00Z",
  "updated_at": "2023-10-15T14:32:00Z"
}

HTTP Status Codes and Error Response Format

Status Code	Description
200	Success - GET, PUT, PATCH, DELETE
201	Created - POST
400	Bad Request - Invalid input
401	Unauthorized - Missing or invalid auth
403	Forbidden - Insufficient permissions
404	Not Found - Resource not found
429	Too Many Requests - Rate limit exceeded
500	Internal Server Error - Unexpected error

Standardized error response:

{
  "error": {
    "code": "VALIDATION_ERROR",
    "message": "Invalid amount provided",
    "details": [
      {
        "field": "amount",
        "reason": "must be greater than 0"
      }
    ],
    "timestamp": "2023-10-15T14:30:00Z"
  }
}

API Versioning Strategy

Versioned using path prefix: /v1/ with support for future versions (/v2/, etc.)

Pagination Design

Cursor-based pagination selected because:

More consistent with large datasets
Handles concurrent modifications better than offset-based pagination
Provides stable results even when items are inserted/deleted between queries

Authentication and Authorization

All endpoints require:

JWT Bearer token with OAuth2 flow
API key for server-to-server communication (optional)
RBAC roles: payment_admin, payment_user

Rate Limiting

Rate limiting implemented at:

Per-user (JWT subject) - 100/minute for creation, 50/minute for confirmation
Per-service (API key) - 1000/minute for batch operations

Idempotency Design

All mutation endpoints support idempotency keys:

// Example usage in gRPC service
public PaymentIntent createPaymentIntent(CreatePaymentIntentRequest request, String idempotencyKey) {
    // Check if this operation was already processed
    PaymentIntent existing = paymentIntentRepository.findByIdempotencyKey(idempotencyKey);
    if (existing != null) {
        return existing;
    }
    
    // Process new request
    PaymentIntent intent = paymentService.createPaymentIntent(request);
    paymentIntentRepository.save(intent, idempotencyKey);
    
    return intent;
}

gRPC Service Definition

syntax = "proto3";

package payment;

service PaymentService {
  rpc CreatePaymentIntent(CreatePaymentIntentRequest) returns (CreatePaymentIntentResponse);
  rpc ConfirmPaymentIntent(ConfirmPaymentIntentRequest) returns (ConfirmPaymentIntentResponse);
  rpc CancelPaymentIntent(CancelPaymentIntentRequest) returns (CancelPaymentIntentResponse);
  rpc GetPaymentIntent(GetPaymentIntentRequest) returns (GetPaymentIntentResponse);
  rpc RefundPayment(RefundPaymentRequest) returns (RefundPaymentResponse);
  rpc GetRefund(GetRefundRequest) returns (GetRefundResponse);
  rpc ListPaymentIntents(ListPaymentIntentsRequest) returns (ListPaymentIntentsResponse);
}

message CreatePaymentIntentRequest {
  int64 amount = 1;
  string currency = 2;
  string description = 3;
  string customer_id = 4;
  map<string, string> metadata = 5;
}

message CreatePaymentIntentResponse {
  string intent_id = 1;
  string client_secret = 2;
  int64 amount = 3;
  string currency = 4;
  string status = 5;
  google.protobuf.Timestamp created_at = 6;
}

message ConfirmPaymentIntentRequest {
  string intent_id = 1;
  string payment_method_id = 2;
  string return_url = 3;
}

message ConfirmPaymentIntentResponse {
  string intent_id = 1;
  string status = 2;
  string payment_method_id = 3;
  int64 amount = 4;
  string currency = 5;
  google.protobuf.Timestamp created_at = 6;
  google.protobuf.Timestamp updated_at = 7;
}

message CancelPaymentIntentRequest {
  string intent_id = 1;
}

message CancelPaymentIntentResponse {
  string intent_id = 1;
  string status = 2;
  google.protobuf.Timestamp updated_at = 3;
}

message GetPaymentIntentRequest {
  string intent_id = 1;
}

message GetPaymentIntentResponse {
  string intent_id = 1;
  string status = 2;
  int64 amount = 3;
  string currency = 4;
  google.protobuf.Timestamp created_at = 5;
  google.protobuf.Timestamp updated_at = 6;
}

message RefundPaymentRequest {
  string payment_intent_id = 1;
  int64 amount = 2;
  string reason = 3;
}

message RefundPaymentResponse {
  string refund_id = 1;
  string status = 2;
  int64 amount = 3;
  string currency = 4;
  google.protobuf.Timestamp created_at = 5;
}

message GetRefundRequest {
  string refund_id = 1;
}

message GetRefundResponse {
  string refund_id = 1;
  string status = 2;
  int64 amount = 3;
  string currency = 4;
  google.protobuf.Timestamp created_at = 5;
}

message ListPaymentIntentsRequest {
  string customer_id = 1;
  int32 limit = 2;
  string cursor = 3;
}

message ListPaymentIntentsResponse {
  repeated PaymentIntent items = 1;
  string next_cursor = 2;
}

message PaymentIntent {
  string intent_id = 1;
  string status = 2;
  int64 amount = 3;
  string currency = 4;
  google.protobuf.Timestamp created_at = 5;
  google.protobuf.Timestamp updated_at = 6;
}

3. Data Model Design

erDiagram
    PAYMENT {
        string id PK
        string customer_id FK
        string merchant_id FK
        string amount
        string currency
        string status
        string payment_method_type
        string gateway_transaction_id
        string description
        string failure_reason
        timestamp created_at
        timestamp updated_at
        timestamp settled_at
    }

    CUSTOMER {
        string id PK
        string user_id FK
        string email
        string name
        timestamp created_at
        timestamp updated_at
    }

    MERCHANT {
        string id PK
        string name
        string merchant_account_id
        string webhook_url
        timestamp created_at
        timestamp updated_at
    }

    PAYMENT_METHOD {
        string id PK
        string customer_id FK
        string token
        string type
        string last_four_digits
        string expiry_month
        string expiry_year
        boolean is_default
        timestamp created_at
        timestamp updated_at
    }

    TRANSACTION {
        string id PK
        string payment_id FK
        string transaction_type
        string status
        string amount
        string currency
        string gateway_response_code
        timestamp created_at
        timestamp updated_at
    }

    REFUND {
        string id PK
        string payment_id FK
        string amount
        string currency
        string status
        string reason
        timestamp created_at
        timestamp updated_at
    }

    PAYMENT_METHOD ||--o{ PAYMENT : "uses"
    CUSTOMER ||--o{ PAYMENT : "initiates"
    MERCHANT ||--o{ PAYMENT : "receives"
    CUSTOMER ||--o{ PAYMENT_METHOD : "owns"
    PAYMENT ||--o{ TRANSACTION : "has"
    PAYMENT ||--o{ REFUND : "can be refunded"

Table Schema Definitions

Payment

Field	Type	Constraints	Index	Description
id	UUID	PK, NOT NULL	Yes	Unique identifier
customer_id	UUID	FK, NOT NULL	Yes	Reference to customer
merchant_id	UUID	FK, NOT NULL	Yes	Reference to merchant
amount	DECIMAL(15,2)	NOT NULL	Yes	Payment amount
currency	VARCHAR(3)	NOT NULL	Yes	ISO 4217 currency code
status	VARCHAR(20)	NOT NULL	Yes	Payment lifecycle status
payment_method_type	VARCHAR(50)	NOT NULL	Yes	Type of payment method
gateway_transaction_id	VARCHAR(255)	NULL	Yes	Gateway's transaction reference
description	TEXT	NULL	No	Description of payment
failure_reason	TEXT	NULL	No	Reason for failure if applicable
created_at	TIMESTAMP	NOT NULL	Yes	Creation timestamp
updated_at	TIMESTAMP	NOT NULL	Yes	Last update timestamp
settled_at	TIMESTAMP	NULL	No	Settlement completion timestamp

Customer

Field	Type	Constraints	Index	Description
id	UUID	PK, NOT NULL	Yes	Unique identifier
user_id	UUID	FK, NOT NULL	Yes	Reference to application user
email	VARCHAR(255)	UNIQUE, NOT NULL	Yes	Customer's email
name	VARCHAR(255)	NULL	No	Customer's name
created_at	TIMESTAMP	NOT NULL	Yes	Creation timestamp
updated_at	TIMESTAMP	NOT NULL	Yes	Last update timestamp

Merchant

Field	Type	Constraints	Index	Description
id	UUID	PK, NOT NULL	Yes	Unique identifier
name	VARCHAR(255)	NOT NULL	Yes	Merchant's business name
merchant_account_id	VARCHAR(255)	UNIQUE, NOT NULL	Yes	Gateway-specific account ID
webhook_url	VARCHAR(500)	NULL	No	Webhook endpoint for notifications
created_at	TIMESTAMP	NOT NULL	Yes	Creation timestamp
updated_at	TIMESTAMP	NOT NULL	Yes	Last update timestamp

PaymentMethod

Field	Type	Constraints	Index	Description
id	UUID	PK, NOT NULL	Yes	Unique identifier
customer_id	UUID	FK, NOT NULL	Yes	Reference to customer
token	VARCHAR(255)	UNIQUE, NOT NULL	Yes	Tokenized payment method
type	VARCHAR(50)	NOT NULL	Yes	Payment method type (card, bank, etc.)
last_four_digits	VARCHAR(4)	NULL	No	Last four digits of card
expiry_month	SMALLINT	NULL	No	Expiry month
expiry_year	SMALLINT	NULL	No	Expiry year
is_default	BOOLEAN	NOT NULL	No	Default payment method
created_at	TIMESTAMP	NOT NULL	Yes	Creation timestamp
updated_at	TIMESTAMP	NOT NULL	Yes	Last update timestamp

Transaction

Field	Type	Constraints	Index	Description
id	UUID	PK, NOT NULL	Yes	Unique identifier
payment_id	UUID	FK, NOT NULL	Yes	Reference to payment
transaction_type	VARCHAR(20)	NOT NULL	Yes	Type (authorization, capture, void)
status	VARCHAR(20)	NOT NULL	Yes	Transaction status
amount	DECIMAL(15,2)	NOT NULL	Yes	Transaction amount
currency	VARCHAR(3)	NOT NULL	Yes	ISO 4217 currency code
gateway_response_code	VARCHAR(50)	NULL	No	Gateway's response code
created_at	TIMESTAMP	NOT NULL	Yes	Creation timestamp
updated_at	TIMESTAMP	NOT NULL	Yes	Last update timestamp

Refund

Field	Type	Constraints	Index	Description
id	UUID	PK, NOT NULL	Yes	Unique identifier
payment_id	UUID	FK, NOT NULL	Yes	Reference to original payment
amount	DECIMAL(15,2)	NOT NULL	Yes	Refund amount
currency	VARCHAR(3)	NOT NULL	Yes	ISO 4217 currency code
status	VARCHAR(20)	NOT NULL	Yes	Refund lifecycle status
reason	TEXT	NULL	No	Reason for refund
created_at	TIMESTAMP	NOT NULL	Yes	Creation timestamp
updated_at	TIMESTAMP	NOT NULL	Yes	Last update timestamp

Primary Key Strategy

UUIDs for all tables - Selected over auto-increment due to:

Global uniqueness across distributed systems
Prevention of sequential pattern attacks
Better support for sharding strategies
Easier cross-system integration

Index Design

Composite Indexes:
- Payment(customer_id, created_at) - For customer payment history queries
- Payment(merchant_id, status, created_at) - For merchant dashboard filtering
- PaymentMethod(customer_id, is_default) - For default payment method lookups
Partial Indexes:
- Payment(status = 'failed') - For failed payment monitoring
- Refund(payment_id, status) - For refund status tracking
Covering Indexes:
- Payment(id, customer_id, merchant_id, amount, currency, status) - For payment summary queries

Data Partitioning / Sharding Strategy

Shard by Merchant ID - Primary sharding key:

Each merchant gets their own database shard
Ensures data isolation and compliance
Allows for independent scaling per merchant
Simplifies audit and compliance reporting

Soft Delete vs. Hard Delete Decision

Soft Delete with deleted_at timestamp - Chosen for:

Audit trail preservation
Compliance and regulatory requirements (e.g., PCI DSS)
Data recovery capability for disputes
Historical analytics and reporting

Audit Trail Design

Audit Log Table:

CREATE TABLE audit_log (
    id UUID PRIMARY KEY,
    entity_type VARCHAR(50),
    entity_id UUID,
    operation VARCHAR(20),
    user_id UUID,
    details JSONB,
    created_at TIMESTAMP NOT NULL DEFAULT NOW()
);

Data Archival Strategy

30-day retention policy for completed payments:

Payments with status = 'completed' and settled_at < 30 days ago
Archived to cold storage (e.g., AWS Glacier)
Metadata retained in database for 2 years
Audit logs preserved for 7 years per regulatory requirements

// Example of soft delete pattern in Payment entity
@Entity
@Table(name = "payment")
public class Payment {
    @Id
    @GeneratedValue(generator = "uuid")
    @GenericGenerator(name = "uuid", strategy = "uuid2")
    private UUID id;
    
    @Column(name = "deleted_at")
    private LocalDateTime deletedAt;
    
    public boolean isDeleted() {
        return deletedAt != null;
    }
    
    public void markAsDeleted() {
        this.deletedAt = LocalDateTime.now();
    }
}

4. Class Design & Service Layer

UML Class Diagram

classDiagram
    class PaymentGatewayService {
        +processPayment(PaymentRequest) PaymentResponse
        +handleRefund(String, BigDecimal) RefundResponse
        +getPaymentStatus(String) PaymentStatus
    }

    class PaymentRequest {
        +amount: BigDecimal
        +currency: String
        +paymentMethod: PaymentMethod
        +customerId: String
        +orderId: String
    }

    class PaymentResponse {
        +paymentId: String
        +status: PaymentStatus
        +amount: BigDecimal
        +currency: String
        +transactionId: String
    }

    class RefundRequest {
        +paymentId: String
        +amount: BigDecimal
        +reason: String
    }

    class RefundResponse {
        +refundId: String
        +status: RefundStatus
        +amount: BigDecimal
        +transactionId: String
    }

    class PaymentMethod {
        +type: PaymentMethodType
        +cardNumber: String
        +expiryMonth: int
        +expiryYear: int
        +cvv: String
    }

    class PaymentProcessor {
        +process(PaymentRequest) PaymentResponse
    }

    class PaymentProcessorFactory {
        +createProcessor(PaymentMethodType) PaymentProcessor
    }

    class PaymentEventPublisher {
        +publishPaymentEvent(PaymentEvent)
    }

    class PaymentEvent {
        +eventType: String
        +paymentId: String
        +status: String
        +timestamp: Instant
    }

    class PaymentStatus {
        +PENDING, PROCESSING, SUCCESS, FAILED, REFUNDED
    }

    class RefundStatus {
        +PENDING, PROCESSING, SUCCESS, FAILED
    }

    PaymentGatewayService --> PaymentRequest : uses
    PaymentGatewayService --> RefundRequest : uses
    PaymentGatewayService --> PaymentProcessorFactory : delegates to
    PaymentGatewayService --> PaymentEventPublisher : publishes
    PaymentRequest --> PaymentMethod : contains
    PaymentProcessorFactory --> PaymentProcessor : creates
    PaymentEventPublisher --> PaymentEvent : publishes

State Machine Diagram for Payment Entity

stateDiagram-v2
    [*] --> PENDING
    PENDING --> PROCESSING : Payment initiated
    PROCESSING --> SUCCESS : Payment successful
    PROCESSING --> FAILED : Payment failed
    SUCCESS --> REFUNDED : Refund initiated
    FAILED --> REFUNDED : Refund initiated (for failed payments)
    REFUNDED --> [*]

Service Class Structure and Responsibilities

The PaymentGatewayService is the primary orchestrator that handles all payment-related operations. It encapsulates business logic for processing payments, handling refunds, and managing the lifecycle of payment entities.

Key responsibilities:

Validate incoming payment requests
Route to appropriate payment processor based on payment method type
Manage transaction boundaries for atomic operations
Publish domain events for observability and integration
Handle idempotency for concurrent requests
Implement retry logic for external payment provider calls

Interface and Abstract Class Design

public interface PaymentProcessor {
    PaymentResponse process(PaymentRequest request);
    boolean supports(PaymentMethodType type);
}

public abstract class BasePaymentProcessor implements PaymentProcessor {
    protected final PaymentEventPublisher eventPublisher;
    protected final PaymentRepository paymentRepository;

    public BasePaymentProcessor(PaymentEventPublisher eventPublisher, 
                               PaymentRepository paymentRepository) {
        this.eventPublisher = eventPublisher;
        this.paymentRepository = paymentRepository;
    }

    @Override
    public boolean supports(PaymentMethodType type) {
        return getSupportedType() == type;
    }

    protected abstract PaymentMethodType getSupportedType();
    
    protected void publishPaymentEvent(String paymentId, String status) {
        PaymentEvent event = new PaymentEvent(
            "PAYMENT_" + status.toUpperCase(),
            paymentId,
            status,
            Instant.now()
        );
        eventPublisher.publish(event);
    }
}

public interface PaymentProcessorFactory {
    PaymentProcessor createProcessor(PaymentMethodType type);
}

Business Logic Encapsulation Patterns

Strategy Pattern Implementation:

public class PaymentProcessorFactoryImpl implements PaymentProcessorFactory {
    private final Map<PaymentMethodType, PaymentProcessor> processors;
    
    public PaymentProcessorFactoryImpl(List<PaymentProcessor> processorList) {
        this.processors = processorList.stream()
            .collect(Collectors.toMap(
                PaymentProcessor::getSupportedType,
                p -> p
            ));
    }

    @Override
    public PaymentProcessor createProcessor(PaymentMethodType type) {
        return processors.getOrDefault(type, new DefaultPaymentProcessor());
    }
}

Critical-Path Method Implementation

public PaymentResponse processPayment(PaymentRequest request) {
    // Idempotency check
    String idempotencyKey = generateIdempotencyKey(request);
    String existingPaymentId = paymentRepository.findPaymentIdByExternalId(idempotencyKey);
    
    if (existingPaymentId != null) {
        return paymentRepository.getPaymentResponse(existingPaymentId);
    }

    // Transaction boundary - using Spring @Transactional
    return transactionTemplate.execute(status -> {
        try {
            // Create payment record in PENDING state
            Payment payment = new Payment(
                UUID.randomUUID().toString(),
                request.getCustomerId(),
                request.getOrderId(),
                request.getAmount(),
                request.getCurrency(),
                PaymentStatus.PENDING,
                idempotencyKey
            );
            
            paymentRepository.save(payment);
            
            // Validate and process payment with external provider
            PaymentProcessor processor = processorFactory.createProcessor(
                request.getPaymentMethod().getType()
            );
            
            PaymentResponse response = processor.process(request);
            
            // Update payment status
            payment.updateStatus(response.getStatus());
            payment.setTransactionId(response.getTransactionId());
            
            // Publish event for monitoring and integration
            publishPaymentEvent(payment.getId(), response.getStatus().toString());
            
            return response;
            
        } catch (Exception e) {
            // Handle failure and update status
            payment.updateStatus(PaymentStatus.FAILED);
            throw new PaymentProcessingException("Payment processing failed", e);
        }
    });
}

Transaction Boundary Design

Single transaction encompasses: payment record creation, external provider call, and status update
Uses Spring's declarative transaction management with proper rollback behavior
Transaction scope includes all database operations within the same session
External API calls are handled within transaction boundaries but may require compensation logic for distributed transactions

Service Interface Contracts

public interface PaymentGatewayService {
    PaymentResponse processPayment(PaymentRequest request);
    RefundResponse handleRefund(String paymentId, BigDecimal amount);
    PaymentStatus getPaymentStatus(String paymentId);
    List<Payment> getPaymentsByCustomer(String customerId, Pageable pageable);
}

Saga/Orchestration Pattern for Distributed Transactions

For complex payment flows involving multiple systems (e.g., inventory, shipping), we implement a saga pattern:

public class PaymentSaga {
    private final PaymentEventPublisher eventPublisher;
    
    public void initiatePaymentSaga(PaymentRequest request) {
        // Start saga with payment creation
        PaymentEvent event = new PaymentEvent(
            "PAYMENT_INITIATED",
            request.getOrderId(),
            "INITIATED",
            Instant.now()
        );
        
        eventPublisher.publish(event);
        
        // Orchestrate additional steps through compensating transactions
        // when failures occur in downstream systems
    }
}

Domain Object / Value Object Design

public class Payment {
    private final String id;
    private final String customerId;
    private final String orderId;
    private final BigDecimal amount;
    private final String currency;
    private PaymentStatus status;
    private final String externalId;
    private String transactionId;
    private final Instant createdAt;
    private Instant updatedAt;

    public Payment(String id, String customerId, String orderId, 
                   BigDecimal amount, String currency, PaymentStatus status, 
                   String externalId) {
        this.id = id;
        this.customerId = customerId;
        this.orderId = orderId;
        this.amount = amount;
        this.currency = currency;
        this.status = status;
        this.externalId = externalId;
        this.createdAt = Instant.now();
        this.updatedAt = Instant.now();
    }

    public void updateStatus(PaymentStatus newStatus) {
        this.status = newStatus;
        this.updatedAt = Instant.now();
    }
}

public enum PaymentMethodType {
    CREDIT_CARD, DEBIT_CARD, PAYPAL, BANK_TRANSFER
}

public enum PaymentStatus {
    PENDING, PROCESSING, SUCCESS, FAILED, REFUNDED
}

5. Concurrency & Thread Safety

Concurrency Model

The payment gateway system will utilize a thread pool-based concurrency model with the following characteristics:

Thread Pool Size: Configurable based on expected concurrent payment operations (e.g., 100-500 threads)
Thread Management: Using Java's java.util.concurrent framework with ExecutorService
Async Processing: Non-blocking operations for external payment provider calls using Reactor or CompletableFuture
Event Loop Pattern: For high-throughput scenarios, the system will leverage Reactor's event loop for handling payment operations

Race Condition Prevention Strategy

Race conditions in the payment gateway system will be mitigated through:

Atomic Operations: Critical sections using java.util.concurrent.atomic classes
Locking Strategy: Optimistic locking for payment status updates using version fields
Database-Level Constraints: Unique constraints on payment reference IDs and transaction IDs
Idempotency Keys: For handling duplicate external API calls

Optimistic vs. Pessimistic Locking Selection

Optimistic Locking is selected for the following reasons:

High Concurrency: Payment operations typically have low contention
Performance: Avoids blocking threads and reduces latency
Scalability: Better for distributed systems with eventual consistency requirements
Implementation: Using version fields in database records

Pessimistic Locking was considered but rejected due to:

Potential for deadlocks in complex payment flows
Increased latency from thread blocking
Complexity in managing lock timeouts

Deadlock Prevention Strategy

Deadlock prevention will be implemented through:

Lock Ordering: All database operations follow a consistent order (e.g., by payment ID, then transaction ID)
Timeout Configuration: 5-second timeout for database locks with automatic rollback
Lock Granularity: Fine-grained locking at the payment record level rather than table-level
Retry Logic: Exponential backoff with jitter for retrying failed optimistic lock attempts

Idempotency Implementation for Concurrent Duplicate Requests

Idempotency will be implemented using:

Idempotency Key Strategy:
- Unique identifier provided by client (e.g., idempotencyKey)
- Stored in database with payment reference
- Used to detect and reject duplicate requests
Implementation Pattern:

public class PaymentService {
    private static final String IDEMPOTENCY_KEY_PREFIX = "payment_idempotency:";
    
    public PaymentResponse processPayment(PaymentRequest request) {
        String idempotencyKey = request.getIdempotencyKey();
        
        // Check if this idempotency key already exists
        String existingPaymentId = paymentRepository.findPaymentIdByReference(idempotencyKey);
        if (existingPaymentId != null) {
            return paymentRepository.getPaymentResponse(existingPaymentId);
        }
        
        // Proceed with new payment processing
        return processNewPayment(request, idempotencyKey);
    }
}

Idempotency Key Validation:
- Client-provided key must be UUID or unique string
- System-generated fallback key for missing keys
- TTL of 24 hours for idempotency records

Thread Safety in Critical Components

Payment State Machine:
- Thread-safe state transitions using atomic operations
- Immutable state objects with copy-on-write semantics
Payment Processor:
- Synchronized access to shared resources
- Thread-local storage for operation context
Event Publisher:
- Concurrent queue implementation for event publishing
- Thread-safe event serialization

Concurrency Control in Payment Flow

The system will implement the following concurrency controls:

Payment Creation:
- Atomic creation of payment record with unique ID
- Concurrent-safe reference number generation using UUID
Payment Status Updates:
- Optimistic locking with version field validation
- Retry mechanism for concurrent update conflicts
External API Calls:
- Asynchronous execution with timeout handling
- Thread-safe HTTP client configuration

Performance Considerations

Thread Pool Configuration:
- Core pool size: 50 threads
- Max pool size: 200 threads
- Keep-alive time: 60 seconds
Resource Management:
- Connection pooling for database and external APIs
- Memory-efficient object reuse patterns
- Proper thread cleanup to prevent resource leaks
Monitoring:
- Thread pool metrics collection
- Concurrency bottleneck detection
- Performance degradation alerts

Sample Concurrent Operation Implementation

@Service
public class ConcurrentPaymentProcessor {
    private final ExecutorService executor = Executors.newFixedThreadPool(100);
    private final Map<String, AtomicLong> concurrentCounters = new ConcurrentHashMap<>();
    
    public CompletableFuture<PaymentResult> processPaymentAsync(PaymentRequest request) {
        return CompletableFuture.supplyAsync(() -> {
            try {
                // Thread-safe operation with proper resource management
                return executePaymentWithRetry(request);
            } catch (Exception e) {
                throw new PaymentProcessingException("Concurrent payment processing failed", e);
            }
        }, executor);
    }
    
    private PaymentResult executePaymentWithRetry(PaymentRequest request) {
        int maxRetries = 3;
        int retryCount = 0;
        
        while (retryCount < maxRetries) {
            try {
                return processPaymentRequest(request);
            } catch (OptimisticLockException e) {
                retryCount++;
                if (retryCount >= maxRetries) {
                    throw new PaymentProcessingException("Payment processing failed after retries", e);
                }
                // Exponential backoff with jitter
                Thread.sleep(100 * (1L << retryCount) + ThreadLocalRandom.current().nextInt(100));
            }
        }
        throw new PaymentProcessingException("Payment processing failed");
    }
}

6. Sequence Diagrams for Key Flows

sequenceDiagram
    participant C as Client (Mobile/Web)
    participant PG as Payment Gateway Service
    participant P as Payment Provider (Stripe/Adyen)
    participant DB as Database

    C->>PG: 1. Initiate Payment Request
    PG->>DB: 2. Create Payment Record (status: PENDING)
    PG->>P: 3. Forward to Payment Provider
    P-->>PG: 4. Payment Provider Response (success/failure)
    alt Success
        PG->>DB: 5. Update Payment Status (status: SUCCESS)
        PG-->>C: 6. Return Success Response
    else Failure
        PG->>DB: 5. Update Payment Status (status: FAILED)
        PG-->>C: 6. Return Failure Response
    end

sequenceDiagram
    participant C as Client (Mobile/Web)
    participant PG as Payment Gateway Service
    participant P as Payment Provider (Stripe/Adyen)
    participant DB as Database
    participant E as Event Bus

    C->>PG: 1. Initiate Payment Request
    PG->>DB: 2. Create Payment Record (status: PENDING)
    PG->>P: 3. Forward to Payment Provider
    P-->>PG: 4. Payment Provider Response (success/failure)
    alt Success
        PG->>DB: 5. Update Payment Status (status: SUCCESS)
        PG->>E: 6. Publish Payment Success Event
        E-->>PG: 7. Event Consumer Acknowledges
        PG-->>C: 8. Return Success Response
    else Failure
        PG->>DB: 5. Update Payment Status (status: FAILED)
        PG->>E: 6. Publish Payment Failure Event
        E-->>PG: 7. Event Consumer Acknowledges
        PG-->>C: 8. Return Failure Response
    end

sequenceDiagram
    participant C as Client (Mobile/Web)
    participant PG as Payment Gateway Service
    participant P as Payment Provider (Stripe/Adyen)
    participant DB as Database
    participant E as Event Bus
    participant A as Audit Service

    C->>PG: 1. Initiate Payment Request
    PG->>DB: 2. Create Payment Record (status: PENDING)
    PG->>P: 3. Forward to Payment Provider
    P-->>PG: 4. Payment Provider Response (success/failure)
    alt Success
        PG->>DB: 5. Update Payment Status (status: SUCCESS)
        PG->>E: 6. Publish Payment Success Event
        E-->>PG: 7. Event Consumer Acknowledges
        PG->>A: 8. Log Audit Trail (Payment Completed)
        PG-->>C: 9. Return Success Response
    else Failure
        PG->>DB: 5. Update Payment Status (status: FAILED)
        PG->>E: 6. Publish Payment Failure Event
        E-->>PG: 7. Event Consumer Acknowledges
        PG->>A: 8. Log Audit Trail (Payment Failed)
        PG-->>C: 9. Return Failure Response
    end

7. Caching Design

Cache Strategy Selection

For a payment gateway integration, we'll use cache-aside pattern with LRU eviction policy. This approach provides the best balance between performance and consistency for our use case where we need to cache payment method data, transaction metadata, and merchant configurations.

The cache-aside pattern is preferred because:

Payment method data changes infrequently but must be consistent with database
We can implement cache warming strategies for high-traffic merchants
Cache misses are acceptable for read-heavy operations (payment method lookups)
We can implement cache validation with ETags or version numbers

Cache Key Design

Cache keys will follow a structured namespace pattern:

payment-gateway:{entity-type}:{id}:{version}

Examples:

payment-gateway:merchant:12345:1
payment-gateway:payment-method:abc123:def456
payment-gateway:transaction:txn_7890:1

Cache TTL Strategy

Data Type	TTL	Reason
Payment Methods	24h	Infrequently updated, high read frequency
Merchant Configurations	1h	May change during business hours
Transaction Metadata	5m	Recent transactions need fresh data
Tokenized Payment Data	10m	Sensitive data should not persist long
Rate Limits	1m	Time-sensitive rate limiting

Cache Eviction Policy

We'll implement LRU (Least Recently Used) eviction with a secondary TTL-based cleanup:

// Example LRU cache implementation using Guava
public class PaymentCache {
    private final LoadingCache<String, CachedValue> cache;
    
    public PaymentCache() {
        this.cache = CacheBuilder.newBuilder()
            .maximumSize(10000)
            .expireAfterWrite(Duration.ofHours(24))
            .recordStats()
            .build(new CacheLoader<String, CachedValue>() {
                @Override
                public CachedValue load(String key) throws Exception {
                    return fetchFromDatabase(key);
                }
            });
    }
}

Cache Stampede Prevention

To prevent cache stampedes:

Probabilistic early expiry: Set cache TTL to 80% of actual TTL
Distributed locks: Use Redis distributed locks for cache warming operations

public class CacheStampedePrevention {
    private final RedisTemplate<String, String> redisTemplate;
    
    public boolean acquireCacheLock(String key, String lockValue, int expirySeconds) {
        String lockKey = "cache-lock:" + key;
        return redisTemplate.opsForValue()
            .setIfAbsent(lockKey, lockValue, Duration.ofSeconds(expirySeconds));
    }
    
    public void releaseCacheLock(String key, String lockValue) {
        String lockKey = "cache-lock:" + key;
        String currentLockValue = redisTemplate.opsForValue().get(lockKey);
        if (lockValue.equals(currentLockValue)) {
            redisTemplate.delete(lockKey);
        }
    }
}

Distributed Cache Architecture

We'll use Redis Cluster with:

3 master nodes for data distribution
3 replica nodes for high availability
Redis Sentinel for automatic failover
Cache sharding based on merchant ID prefix

Cache Invalidation Strategy

We'll implement event-driven invalidation:

Write-through pattern: Cache is updated on database writes
Event-driven invalidation: Cache entries invalidated via Redis pub/sub when data changes

@Component
public class PaymentCacheInvalidator {
    private final RedisTemplate<String, String> redisTemplate;
    
    @EventListener
    public void handlePaymentMethodUpdated(PaymentMethodUpdatedEvent event) {
        String cacheKey = "payment-gateway:payment-method:" + 
                         event.getPaymentMethodId() + ":" + 
                         event.getVersion();
        redisTemplate.delete(cacheKey);
        
        // Invalidate related cache entries
        String merchantKey = "payment-gateway:merchant:" + 
                            event.getMerchantId() + ":*";
        Set<String> keys = redisTemplate.keys(merchantKey);
        redisTemplate.delete(keys);
    }
}

Cache Hit/Miss Metrics

We'll track cache performance with:

Cache hit ratio (target: >85%)
Average cache retrieval time
Cache size and memory usage

public class CacheMetrics {
    private final Counter cacheHits = Counter.build()
        .name("payment_gateway_cache_hits_total")
        .help("Total cache hits")
        .register();
        
    private final Counter cacheMisses = Counter.build()
        .name("payment_gateway_cache_misses_total")
        .help("Total cache misses")
        .register();
        
    private final Histogram cacheLatency = Histogram.build()
        .name("payment_gateway_cache_latency_seconds")
        .help("Cache retrieval latency")
        .register();
}

8. Repository / Data Access Layer

Repository Pattern Implementation

The repository pattern is implemented to abstract data access logic and provide a clean separation between domain logic and database operations. Each entity has its own repository interface with standard CRUD operations, allowing for easy mocking and testing.

public interface PaymentRepository {
    Optional<Payment> findById(UUID id);
    List<Payment> findByStatus(PaymentStatus status);
    List<Payment> findByUserId(UUID userId, Pageable pageable);
    Payment save(Payment payment);
    void delete(UUID id);
    long countByUserIdAndStatus(UUID userId, PaymentStatus status);
}

public interface PaymentMethodRepository {
    Optional<PaymentMethod> findByUserIdAndId(UUID userId, UUID id);
    List<PaymentMethod> findByUserId(UUID userId);
    PaymentMethod save(PaymentMethod paymentMethod);
    void delete(UUID id);
}

ORM vs Raw Query Decisions

We use JPA/Hibernate for most operations due to its type safety, query optimization, and integration with Spring Data. For high-volume read operations or complex aggregations, raw SQL queries are used to achieve better performance.

// JPA for standard operations
@Query("SELECT p FROM Payment p WHERE p.userId = :userId AND p.status = :status")
List<Payment> findByUserIdAndStatus(@Param("userId") UUID userId, @Param("status") PaymentStatus status);

// Raw SQL for performance-critical aggregations
@Query(value = "SELECT COUNT(*) FROM payments WHERE user_id = :userId AND status = :status", nativeQuery = true)
long countByUserIdAndStatus(@Param("userId") UUID userId, @Param("status") PaymentStatus status);

Read vs Write Separation

For high-throughput scenarios, we implement CQRS (Command Query Responsibility Segregation) to separate read and write operations:

// Write-side repository (for mutations)
public interface PaymentWriteRepository {
    Payment save(Payment payment);
    void updateStatus(UUID id, PaymentStatus status);
}

// Read-side repository (for queries)
public interface PaymentReadRepository {
    @Query("SELECT new com.example.dto.PaymentSummary(p.id, p.amount, p.status) FROM Payment p WHERE p.userId = :userId")
    List<PaymentSummary> findSummariesByUserId(@Param("userId") UUID userId);
}

Connection Pool Configuration

Connection pool is configured with HikariCP for optimal performance:

spring:
  datasource:
    hikari:
      maximum-pool-size: 20
      minimum-idle: 5
      connection-timeout: 30000
      idle-timeout: 600000
      max-lifetime: 1800000
      validation-timeout: 5000

N+1 Query Prevention Strategy

Eager loading is used for related entities to prevent N+1 queries:

@Query("SELECT p FROM Payment p JOIN FETCH p.paymentMethod WHERE p.userId = :userId")
List<Payment> findByUserIdWithPaymentMethod(@Param("userId") UUID userId);

// Using @EntityGraph for explicit fetch plan
@EntityGraph(attributePaths = {"paymentMethod", "user"})
@Query("SELECT p FROM Payment p WHERE p.id = :id")
Optional<Payment> findByIdWithRelations(@Param("id") UUID id);

Batch Operation Design

Batch operations are implemented for bulk processing:

public class PaymentBatchRepository {
    
    @Modifying
    public void updateStatusBatch(List<UUID> paymentIds, PaymentStatus newStatus) {
        String sql = "UPDATE payments SET status = :status WHERE id IN :ids";
        Query query = entityManager.createNativeQuery(sql);
        query.setParameter("status", newStatus.name());
        query.setParameter("ids", paymentIds);
        query.executeUpdate();
    }
    
    public void saveBatch(List<Payment> payments) {
        for (int i = 0; i < payments.size(); i++) {
            entityManager.persist(payments.get(i));
            if (i % 50 == 0) {
                entityManager.flush();
                entityManager.clear();
            }
        }
    }
}

Transaction Management

Repository methods are wrapped in appropriate transaction boundaries:

@Repository
@Transactional(readOnly = true)
public class PaymentRepositoryImpl {
    
    @PersistenceContext
    private EntityManager entityManager;
    
    @Transactional
    public Payment save(Payment payment) {
        entityManager.persist(payment);
        return payment;
    }
    
    @Transactional
    public void updateStatus(UUID id, PaymentStatus status) {
        Payment payment = entityManager.find(Payment.class, id);
        if (payment != null) {
            payment.setStatus(status);
        }
    }
}

Repository Implementation Examples

@Repository
public class PaymentRepositoryImpl implements PaymentRepository {
    
    @PersistenceContext
    private EntityManager entityManager;
    
    @Override
    public Optional<Payment> findById(UUID id) {
        return Optional.ofNullable(entityManager.find(Payment.class, id));
    }
    
    @Override
    public List<Payment> findByUserId(UUID userId, Pageable pageable) {
        CriteriaBuilder cb = entityManager.getCriteriaBuilder();
        CriteriaQuery<Payment> query = cb.createQuery(Payment.class);
        Root<Payment> root = query.from(Payment.class);
        
        query.select(root)
             .where(cb.equal(root.get("userId"), userId))
             .orderBy(cb.desc(root.get("createdAt")));
        
        TypedQuery<Payment> typedQuery = entityManager.createQuery(query);
        typedQuery.setFirstResult(pageable.getOffset());
        typedQuery.setMaxResults(pageable.getPageSize());
        
        return typedQuery.getResultList();
    }
    
    @Override
    @Transactional
    public Payment save(Payment payment) {
        if (payment.getId() == null) {
            entityManager.persist(payment);
        } else {
            entityManager.merge(payment);
        }
        return payment;
    }
}

Data Access Layer Architecture

The data access layer follows a layered architecture pattern:

Repository Layer: Handles direct database interactions
Service Layer: Coordinates business logic and transaction boundaries
DTO Layer: Transforms data for external consumption
Mapper Layer: Converts between domain and database representations

This design ensures clean separation of concerns, supports easy testing, and provides flexibility for future database migrations or schema changes.

9. Asynchronous Processing & Messaging

Event-driven boundaries in this feature

The payment gateway integration introduces several asynchronous flows that decouple core business logic from external dependencies:

Payment processing completion - When a payment is processed, we need to notify downstream systems (e.g., inventory management, order fulfillment) without blocking the main transaction.
Payment status updates - External payment providers may send webhook events indicating payment success/failure, requiring asynchronous handling.
Refund processing - Refunds may involve multiple steps and can be long-running operations.
Dispute handling - When disputes arise, they require asynchronous monitoring and escalation.

We separate synchronous flows (like initiating a payment) from asynchronous ones (like handling webhooks or processing refunds) to maintain transactional consistency while allowing for scalability and resilience.

Message Schema Design

We use Protocol Buffers for message schema definition to ensure type safety and version compatibility:

// PaymentEvent.proto
syntax = "proto3";

package payment.events;

message PaymentProcessedEvent {
  string payment_id = 1;
  string order_id = 2;
  string provider_transaction_id = 3;
  double amount = 4;
  string currency = 5;
  string status = 6; // "SUCCESS", "FAILED", "PENDING"
  string provider = 7;
  int64 timestamp = 8;
  optional string failure_reason = 9;
}

message RefundProcessedEvent {
  string refund_id = 1;
  string payment_id = 2;
  double amount = 3;
  string status = 4; // "SUCCESS", "FAILED"
  int64 timestamp = 5;
  optional string failure_reason = 6;
}

message PaymentDisputeEvent {
  string dispute_id = 1;
  string payment_id = 2;
  string reason = 3;
  int64 timestamp = 4;
  string status = 5; // "OPEN", "RESOLVED"
}

Versioning strategy uses semantic versioning with major version changes requiring migration. Minor versions are backward compatible.

Dead Letter Queue Strategy

We implement a dead letter queue (DLQ) system with:

Poison message handling: Messages that fail more than 3 times are moved to DLQ
Retry logic: Exponential backoff with jitter (1s, 2s, 4s, 8s) up to 5 retries
DLQ monitoring: Alert on DLQ queue size exceeding threshold (100 messages)
Manual reprocessing: Admin dashboard to inspect and retry failed messages

At-least-once vs. Exactly-once Delivery

We choose at-least-once delivery for payment events because:

Payment status changes are idempotent operations
Duplicate processing is safer than missing events
The downstream systems can handle duplicate events gracefully

We implement idempotency keys in message headers to prevent duplicate processing.

Retry Policy for Consumers

Retry policy configuration:

Max attempts: 5 retries
Backoff formula: Exponential with jitter (1s, 2s, 4s, 8s, 16s)
Non-retryable errors:
- Invalid payment state transitions (e.g., refund on failed payment)
- Insufficient funds
- Invalid merchant credentials

Outbox Pattern for Reliable Event Publishing

We implement the outbox pattern to ensure event publishing is atomic with database writes:

// PaymentService.java
public class PaymentService {
    private final PaymentRepository paymentRepository;
    private final EventPublisher eventPublisher;
    private final OutboxRepository outboxRepository;

    public PaymentResult processPayment(PaymentRequest request) {
        // Begin transaction
        return paymentRepository.withTransaction(() -> {
            Payment payment = createPayment(request);
            
            // Save payment to DB
            paymentRepository.save(payment);
            
            // Create outbox event (within same transaction)
            OutboxEvent outboxEvent = new OutboxEvent(
                "payment.processed",
                payment.getId(),
                PaymentProcessedEvent.newBuilder()
                    .setPaymentId(payment.getId())
                    .setOrderId(request.getOrderId())
                    .setStatus("PENDING")
                    .build()
            );
            
            outboxRepository.save(outboxEvent);
            
            // Publish to message broker (this will be handled by a background task)
            return new PaymentResult(payment.getId(), "PENDING");
        });
    }
}

Sequence Diagram for Async Flows

sequenceDiagram
    participant Client
    participant PaymentService
    participant PaymentRepository
    participant OutboxRepository
    participant MessageBroker
    participant OrderService
    participant InventoryService

    Client->>PaymentService: Process Payment Request
    PaymentService->>PaymentRepository: Save Payment (DB Transaction)
    PaymentService->>OutboxRepository: Save Outbox Event
    PaymentService-->>Client: Return Payment ID
    OutboxRepository->>MessageBroker: Publish Event to Queue
    MessageBroker->>OrderService: Deliver PaymentProcessedEvent
    OrderService->>OrderService: Update Order Status
    MessageBroker->>InventoryService: Deliver PaymentProcessedEvent
    InventoryService->>InventoryService: Reserve Stock

    Note over PaymentService,MessageBroker: Outbox pattern ensures atomicity

The outbox pattern ensures that if a payment is successfully created and saved to the database, the corresponding event will eventually be published to the message broker. This prevents data inconsistency where a payment exists but no downstream systems are notified.

For refund processing, we have a similar pattern with additional state management:

sequenceDiagram
    participant Client
    participant PaymentService
    participant RefundRepository
    participant OutboxRepository
    participant MessageBroker
    participant RefundService

    Client->>PaymentService: Initiate Refund Request
    PaymentService->>RefundRepository: Save Refund Record (DB Transaction)
    PaymentService->>OutboxRepository: Save Outbox Event
    PaymentService-->>Client: Return Refund ID
    OutboxRepository->>MessageBroker: Publish RefundProcessedEvent
    MessageBroker->>RefundService: Deliver RefundProcessedEvent
    RefundService->>RefundService: Process Refund with Provider

10. Error Handling Design

Error Classification Hierarchy

The payment gateway system implements a structured error handling hierarchy to distinguish between different types of failures:

public abstract class PaymentGatewayException extends RuntimeException {
    private final String errorCode;
    private final int httpStatus;

    public PaymentGatewayException(String errorCode, int httpStatus, String message) {
        super(message);
        this.errorCode = errorCode;
        this.httpStatus = httpStatus;
    }

    public String getErrorCode() { return errorCode; }
    public int getHttpStatus() { return httpStatus; }
}

public class ValidationException extends PaymentGatewayException {
    public ValidationException(String message) {
        super("VALIDATION_ERROR", 400, message);
    }
}

public class PaymentProcessingException extends PaymentGatewayException {
    public PaymentProcessingException(String message) {
        super("PAYMENT_PROCESSING_ERROR", 500, message);
    }
}

public class GatewayTimeoutException extends PaymentGatewayException {
    public GatewayTimeoutException(String message) {
        super("GATEWAY_TIMEOUT", 504, message);
    }
}

public class InsufficientFundsException extends PaymentGatewayException {
    public InsufficientFundsException(String message) {
        super("INSUFFICIENT_FUNDS", 402, message);
    }
}

public class AuthenticationException extends PaymentGatewayException {
    public AuthenticationException(String message) {
        super("AUTHENTICATION_ERROR", 401, message);
    }
}

Error Propagation Strategy

Domain-level errors (validation, insufficient funds) are caught and converted to specific payment gateway exceptions
Infrastructure errors (network timeout, database connection failure) are wrapped in appropriate payment gateway exceptions
Service layer handles domain exceptions and rethrows them as appropriate HTTP errors
Controller layer catches all payment gateway exceptions and returns standardized JSON responses

Retry Strategy with Exponential Backoff and Jitter

For transient infrastructure failures, the system implements a retry strategy:

public class RetryablePaymentProcessor {
    private static final int MAX_RETRIES = 3;
    private static final long BASE_DELAY_MS = 1000;
    private static final double JITTER_FACTOR = 0.5;

    public PaymentResult processPaymentWithRetry(PaymentRequest request) {
        int attempt = 0;
        Exception lastException = null;

        while (attempt < MAX_RETRIES) {
            try {
                return processPayment(request);
            } catch (GatewayTimeoutException | PaymentProcessingException e) {
                lastException = e;
                attempt++;
                
                if (attempt < MAX_RETRIES) {
                    long delay = calculateBackoffDelay(attempt);
                    try {
                        Thread.sleep(delay);
                    } catch (InterruptedException ie) {
                        Thread.currentThread().interrupt();
                        throw new RuntimeException("Retry interrupted", ie);
                    }
                }
            }
        }

        throw new PaymentProcessingException(
            "Payment processing failed after " + MAX_RETRIES + " attempts: " + lastException.getMessage());
    }

    private long calculateBackoffDelay(int attempt) {
        long delay = (long) (BASE_DELAY_MS * Math.pow(2, attempt - 1));
        long jitter = (long) (delay * JITTER_FACTOR * (Math.random() - 0.5) * 2);
        return delay + jitter;
    }
}

Circuit Breaker Design

The payment gateway uses a circuit breaker pattern to prevent cascading failures:

public class CircuitBreaker {
    private final int failureThreshold = 5;
    private final long timeoutMs = 60000;
    private final int successThreshold = 1;

    private volatile CircuitState state = CircuitState.CLOSED;
    private int failureCount = 0;
    private long lastFailureTime = 0;

    public enum CircuitState {
        CLOSED, OPEN, HALF_OPEN
    }

    public <T> T execute(Supplier<T> operation) {
        switch (state) {
            case CLOSED:
                return executeClosed(operation);
            case OPEN:
                return executeOpen(operation);
            case HALF_OPEN:
                return executeHalfOpen(operation);
            default:
                throw new IllegalStateException("Unknown circuit state");
        }
    }

    private <T> T executeClosed(Supplier<T> operation) {
        try {
            T result = operation.get();
            onSuccess();
            return result;
        } catch (Exception e) {
            onFailure(e);
            throw e;
        }
    }

    private <T> T executeOpen(Supplier<T> operation) {
        if (System.currentTimeMillis() - lastFailureTime > timeoutMs) {
            state = CircuitState.HALF_OPEN;
            throw new PaymentProcessingException("Circuit breaker is half-open");
        } else {
            throw new PaymentProcessingException("Circuit breaker is open");
        }
    }

    private <T> T executeHalfOpen(Supplier<T> operation) {
        try {
            T result = operation.get();
            state = CircuitState.CLOSED;
            failureCount = 0;
            return result;
        } catch (Exception e) {
            state = CircuitState.OPEN;
            failureCount = 0;
            throw e;
        }
    }

    private void onSuccess() {
        if (state == CircuitState.HALF_OPEN) {
            failureCount = 0;
        }
    }

    private void onFailure(Exception e) {
        if (state == CircuitState.CLOSED) {
            failureCount++;
            lastFailureTime = System.currentTimeMillis();
            if (failureCount >= failureThreshold) {
                state = CircuitState.OPEN;
            }
        }
    }
}

Fallback Behavior Design

For critical operations, the system provides fallback mechanisms:

public class PaymentService {
    private final CircuitBreaker circuitBreaker = new CircuitBreaker();
    
    public PaymentResult processPayment(PaymentRequest request) {
        return circuitBreaker.execute(() -> {
            try {
                return processPaymentInternal(request);
            } catch (Exception e) {
                // Log the failure for monitoring
                log.error("Payment processing failed: {}", e.getMessage(), e);
                
                // Fallback to a safer operation or return error
                if (isRecoverable(e)) {
                    return handleRecovery(request);
                }
                
                throw e;
            }
        });
    }

    private boolean isRecoverable(Exception e) {
        return e instanceof GatewayTimeoutException || 
               e instanceof PaymentProcessingException;
    }

    private PaymentResult handleRecovery(PaymentRequest request) {
        // Implement recovery logic - e.g., queue for retry later
        return new PaymentResult("RECOVERY", "Payment queued for retry");
    }
}

Error Logging Format and Alerting Thresholds

Structured logging includes trace context and error details:

public class PaymentErrorLogger {
    private static final Logger logger = LoggerFactory.getLogger(PaymentErrorLogger.class);

    public void logPaymentError(String traceId, String userId, String paymentId, 
                               Exception error, Map<String, Object> additionalContext) {
        Map<String, Object> logContext = new HashMap<>();
        logContext.put("trace_id", traceId);
        logContext.put("user_id", userId);
        logContext.put("payment_id", paymentId);
        logContext.put("error_code", extractErrorCode(error));
        logContext.put("error_message", error.getMessage());
        logContext.put("timestamp", Instant.now().toString());
        logContext.putAll(additionalContext);

        logger.error("Payment processing error: {}", logContext);
    }

    private String extractErrorCode(Exception error) {
        if (error instanceof PaymentGatewayException) {
            return ((PaymentGatewayException) error).getErrorCode();
        }
        return "UNKNOWN_ERROR";
    }
}

Alerting thresholds:

Critical: > 50 concurrent payment processing errors per minute
Warning: > 10 concurrent payment processing errors per minute
Info: Any authentication or validation error (logged but not alerted)

Error Response Format

Standardized JSON error responses:

{
  "error": {
    "code": "INSUFFICIENT_FUNDS",
    "message": "Insufficient funds in the payment method",
    "httpStatus": 402,
    "timestamp": "2023-12-01T10:30:00Z",
    "traceId": "abc123def456"
  }
}

11. Security Design

Input Validation and Sanitization Strategy

All payment data is validated using a combination of:

Library: Apache Commons Validator for standard validations (email, phone, credit card format)
Rules:
- Credit card numbers must pass Luhn algorithm check
- Amounts must be positive decimals with max 2 decimal places
- Currency codes must match ISO 4217 standard
- Card expiration dates must be in MM/YY format and not expired
Sanitization: Strip all whitespace and control characters from sensitive fields

SQL Injection Prevention

ORM Usage: All database operations use JPA/Hibernate with parameterized queries
Raw SQL: When required, all user inputs are properly escaped using prepared statements with parameter binding
Validation Layer: Input validation occurs before any database interaction

Authentication Middleware Design

JWT-based Authentication:
- JWT tokens issued by Identity Provider (Auth0/Okta) with 15-minute expiration
- Token validation performed via JWK Set endpoint for public key verification
- Middleware intercepts all requests and validates token signature and claims
Token Refresh: Refresh tokens with 7-day TTL stored in secure HTTP-only cookies

Authorization Enforcement

RBAC Implementation:
- Roles: CUSTOMER, MERCHANT, ADMIN
- Permissions mapped to roles via Spring Security annotations (@PreAuthorize)
- Merchant role can only access their own payment records
Resource-based Access Control:
- Payment entities are scoped to merchant ID in database
- Service layer checks merchant ownership before processing any operation

Sensitive Data Encryption at Rest and in Transit

At Rest:
- AES-256-GCM encryption using AWS KMS for payment data (card details, CVV)
- Keys rotated quarterly with audit trail
In Transit:
- TLS 1.3 enforced for all external communications
- Payment gateway integration uses mutual TLS (mTLS) with certificate validation

PII Handling and Masking Strategy

Masking:
- Credit card numbers masked to XXXX-XXXX-XXXX-1234 in logs and responses
- CVV, CVC fields never logged or returned in API responses
Storage:
- Card numbers are never stored; only tokenized identifiers are persisted
- Tokenization handled by external payment processor (Stripe/Adyen)

Secrets Management

Vault:
- Payment gateway API keys and secrets stored in HashiCorp Vault
- Secrets rotated every 90 days with automated rotation pipeline
Access Control:
- Least privilege principle enforced via Vault policies
- Service accounts have read-only access to secrets

Audit Logging Design

Events Logged:
- Payment creation, authorization, capture, refund operations
- Failed payment attempts and system errors

Format:

{
  "timestamp": "2023-10-15T14:30:45.123Z",
  "userId": "user_123",
  "merchantId": "merchant_456",
  "paymentId": "pay_789",
  "action": "PAYMENT_AUTHORIZE",
  "status": "SUCCESS",
  "amount": 100.00,
  "currency": "USD"
}

Retention:
- Audit logs retained for 2 years in encrypted format
- Logs archived to S3 with 10-year retention policy

12. Testing Strategy

Unit Testing Scope and Coverage Targets (%)

Service Layer: 85% code coverage with focus on business logic and edge cases
Repository Layer: 90% coverage for all data access methods with mock database interactions
Domain Objects: 100% coverage for value objects and immutable entities
Error Handling Paths: 100% of error conditions must be unit tested
Integration Points: 95% coverage for external service interactions using mocks

Integration Testing Strategy

Database Integration: Test with embedded H2/PostgreSQL in-memory databases for schema validation and basic CRUD operations
External Service Mocks: Use WireMock or Mockito to simulate payment processor responses with various success/failure scenarios
Cross-Service Communication: Test with testcontainers for Redis and Kafka integration
End-to-End Flows: Validate complete payment lifecycle with real database connections and mocked external services

Contract Testing

Pact Framework: Implement consumer-driven contract testing between payment gateway service and downstream consumers (e.g., billing, order management)
Versioned Contracts: Maintain contract versions with semantic versioning to support backward compatibility
Contract Validation: Run contract tests in CI pipeline before each service deployment

Performance Testing Tool and Key Scenarios

Tool: k6 (JavaScript-based load testing) for realistic concurrent user simulation
Key Scenarios:
- Concurrent payment processing (1000+ concurrent requests)
- Payment failure retry simulation (50% failure rate)
- High-volume batch processing (10,000 payments in 5 minutes)
- Latency-sensitive operations under load (p95 < 200ms)

Test Data Management Strategy

Fixtures: Define standardized test data in YAML/JSON format for consistent test environments
Factories: Implement factory classes for generating realistic domain objects (Payment, Transaction, Customer)
Seeding: Use database migration scripts to seed test data with realistic distribution patterns
Isolation: Ensure each test runs in isolated database transaction with automatic rollback

// Example: Unit test for PaymentService with Mockito
@Test
public void processPayment_WithValidCard_ShouldCreateTransaction() {
    // Given
    PaymentRequest request = createValidPaymentRequest();
    PaymentMethod paymentMethod = mock(PaymentMethod.class);
    when(paymentMethod.validate()).thenReturn(true);
    
    // When
    PaymentResult result = paymentService.processPayment(request, paymentMethod);
    
    // Then
    assertThat(result.getStatus()).isEqualTo(PaymentStatus.SUCCESS);
    verify(transactionRepository).save(any(Transaction.class));
}

// Example: Integration test with embedded database
@Test
public void processPayment_WithDatabasePersistence_ShouldPersistTransaction() {
    // Given
    PaymentRequest request = createPaymentRequest();
    
    // When
    PaymentResult result = paymentService.processPayment(request);
    
    // Then
    Transaction savedTransaction = transactionRepository.findById(result.getTransactionId());
    assertThat(savedTransaction.getStatus()).isEqualTo(TransactionStatus.COMPLETED);
}

13. Observability Design

Structured Logging Schema

All logs must follow a consistent schema to enable effective correlation and analysis:

{
  "timestamp": "2023-10-27T14:30:45.123Z",
  "level": "INFO|WARN|ERROR",
  "service": "payment-gateway",
  "trace_id": "12345678-1234-1234-1234-123456789012",
  "span_id": "abcdef12-3456-7890-abcd-ef1234567890",
  "user_id": "user_12345",
  "payment_id": "pay_abcde12345",
  "request_id": "req_fghij67890",
  "event_type": "payment_processed",
  "message": "Payment processed successfully",
  "error_code": "PAYMENT_SUCCESS",
  "duration_ms": 150,
  "payload": {
    "amount": 100.00,
    "currency": "USD",
    "status": "SUCCESS"
  },
  "error_details": {
    "exception_type": "PaymentProcessingException",
    "stack_trace": "...",
    "retry_count": 2
  }
}

PII masking rules:

Credit card numbers: mask all but last 4 digits
User emails: mask domain part (user@***.com)
Bank account numbers: mask all but last 4 digits

Distributed Tracing Instrumentation

Using OpenTelemetry for end-to-end tracing:

Custom Spans:

payment_gateway.process_payment - Main payment processing span
payment_gateway.validate_request - Input validation
payment_gateway.call_payment_provider - External provider calls
payment_gateway.handle_response - Response processing

Attributes to Capture:

Span currentSpan = Span.current();
currentSpan.setAttribute("payment.amount", payment.getAmount());
currentSpan.setAttribute("payment.currency", payment.getCurrency());
currentSpan.setAttribute("provider.name", provider.getName());
currentSpan.setAttribute("payment.status", status.toString());
currentSpan.setAttribute("retry.count", retryCount);

Metrics Design

RED Method Metrics:

Metric Name	Type	Labels	Description
`payment_gateway.requests_total`	Counter	`status`, `provider`, `method`	Total payment requests
`payment_gateway.errors_total`	Counter	`error_type`, `provider`	Payment processing errors
`payment_gateway.duration_seconds`	Histogram	`status`, `provider`	Payment processing latency
`payment_gateway.active_requests`	Gauge	`provider`	Concurrent active requests
`payment_gateway.provider_errors_total`	Counter	`provider`, `error_code`	Provider-specific errors

Key Labels:

status: SUCCESS, FAILURE, RETRY
provider: STRIPE, PAYPAL, BRAINTREE
error_type: VALIDATION_ERROR, PROVIDER_ERROR, TIMEOUT

Health Check Endpoint Design

@RestController
@RequestMapping("/health")
public class HealthController {
    
    @GetMapping("/live")
    public ResponseEntity<Map<String, Object>> live() {
        Map<String, Object> response = new HashMap<>();
        response.put("status", "LIVE");
        response.put("timestamp", Instant.now());
        return ResponseEntity.ok(response);
    }
    
    @GetMapping("/ready")
    public ResponseEntity<Map<String, Object>> ready() {
        Map<String, Object> response = new HashMap<>();
        response.put("status", "READY");
        response.put("timestamp", Instant.now());
        response.put("database", databaseHealth());
        response.put("payment_providers", providerHealth());
        return ResponseEntity.ok(response);
    }
    
    @GetMapping("/startup")
    public ResponseEntity<Map<String, Object>> startup() {
        Map<String, Object> response = new HashMap<>();
        response.put("status", "STARTUP_COMPLETE");
        response.put("timestamp", Instant.now());
        return ResponseEntity.ok(response);
    }
}

Alerting Thresholds

Metric	Warning Level	Critical Level	Escalation
`payment_gateway.errors_total` (rate)	> 5 errors/minute	> 20 errors/minute	On-call engineer
`payment_gateway.duration_seconds` (p99)	> 500ms	> 2000ms	On-call engineer
`payment_gateway.requests_total` (rate)	> 1000 req/sec	> 5000 req/sec	On-call engineer
`payment_gateway.provider_errors_total` (rate)	> 2 errors/minute	> 10 errors/minute	On-call engineer
`payment_gateway.active_requests` (count)	> 500 concurrent	> 1000 concurrent	On-call engineer

Alerting Rules:

Warning: 5-minute sliding window with 3 consecutive violations
Critical: 1-minute sliding window with 5 consecutive violations
Escalation Path: Slack channel → On-call engineer → Manager if unresolved after 30 minutes

14. Open Questions, Risks & Trade-offs

Decisions requiring product / engineering alignment

Payment processor selection: Which payment providers to integrate with (Stripe, PayPal, Adyen) and whether to support multiple providers simultaneously or focus on one primary provider. This decision impacts the complexity of our service layer and the need for abstraction layers.
Currency support strategy: Whether to support multi-currency transactions natively or restrict to a single currency with conversion handled by the payment provider. This affects data model complexity and internationalization requirements.
Refund policy enforcement: How strictly to enforce refund policies (e.g., time windows, partial refunds) and whether to allow custom business logic for different payment types.
Cross-border transaction handling: Whether to support international payments and how to handle compliance with different countries' regulations, which may require additional service layer complexity.

Known trade-offs and justifications

Trade-off	Option 1	Option 2	Justification
Transaction consistency vs. performance	Use distributed transactions with 2PC	Use eventual consistency with compensating actions	2PC provides strong consistency but introduces latency and complexity; eventual consistency offers better performance but requires careful handling of edge cases
Single payment provider vs. multi-provider	Single provider integration	Multi-provider support	Single provider simplifies implementation and reduces maintenance overhead but creates vendor lock-in risk
Real-time vs. async processing	All payment operations synchronous	Async processing for long-running operations	Real-time processing provides better UX but can impact system availability under high load
Full audit trail vs. minimal logging	Store complete transaction history with all metadata	Store only essential audit fields	Complete audit trail provides better compliance but increases storage and query complexity

Risks and mitigation strategies

Risk	Impact	Mitigation
Payment fraud and security breaches	High financial and reputational damage	Implement comprehensive fraud detection, PCI DSS compliance, end-to-end encryption, and regular security audits
Payment provider outages	System-wide payment processing failure	Implement circuit breaker patterns, fallback strategies, and monitoring with alerting
Data loss in transaction processing	Financial loss and customer trust erosion	Implement robust transaction management, outbox pattern, and regular data backups
Compliance violations	Regulatory fines and legal penalties	Establish compliance monitoring, regular audits, and clear audit trail design
High latency under load	Poor user experience and revenue loss	Implement caching, async processing, read replicas, and horizontal scaling strategies

Technical debt acknowledged

Limited support for complex payment flows: Current design assumes simple one-time payments, with no built-in support for recurring payments or subscription-based flows that may require additional service layer complexity.
Tight coupling to specific payment providers: The current abstraction may not scale well if we need to support more complex provider-specific features or require extensive customization for different providers.
Limited retry and compensation logic: The current design assumes simple success/failure scenarios, with no built-in support for complex retry strategies or compensation patterns needed for distributed payment processing.
Audit trail limitations: Current audit design may not support all compliance requirements (e.g., GDPR data retention policies) without significant refactoring.
No support for payment method switching: The system doesn't currently support changing payment methods after initial transaction creation, which may be required for certain business flows.