Promotion Gateway & Double-Layer Cache

Problem

The Shopee homepage served flash-sale traffic with high request volume and strict latency requirements. Backend fan-out to multiple promotion, voucher, and recommendation services caused latency spikes during traffic surges.

My Role

Worked on the gateway API serving homepage promotion traffic, designed and optimized the double-layer cache strategy, and participated in performance and reliability optimization for high-QPS scenarios.

Architecture / Approach

Implemented a cache-first serving strategy with two cache layers: a local in-memory cache with short expiration for hot data, and a distributed Redis/Memcached cache with mechanisms for cache invalidation. Data was serialized before cache storage for performance and cross-service compatibility.

• Double-layer cache: local in-memory (short TTL) + distributed Redis/Memcached
• Cache-first serving strategy optimizing for latency over strict freshness
• Data serialization before cache storage for performance and compatibility
• Cache invalidation mechanism for the distributed layer

Architecture Diagrams

flowchart LR
    Client[Shopee App Homepage]
    Client --> Gateway[Gateway API\nAggregation Layer]
    Gateway --> L1[Local Memory Cache]
    L1 -->|cache miss| L2[Distributed Cache\nRedis / Memcached]
    L2 -->|cache miss| P[Promotion Service]
    P --> PDB[(Promotion DB)]
    Gateway --> V[Voucher Service]
    V --> ADB[(Account DB)]
    Gateway --> R[Other Services]

Key Decisions

• Optimized for latency and resilience over strict data freshness
• Used local memory cache to absorb traffic spikes before hitting distributed cache
• Reduced backend fan-out during peak traffic to protect downstream services

Result

Reduced backend dependency during traffic spikes, improved homepage serving resilience, and supported high-volume promotion traffic during flash sales.

What I Learned

Learned the effectiveness of multi-layer caching for high-traffic scenarios — local memory cache absorbs burst traffic that would otherwise overwhelm even distributed caches.