Part 038 — Redis Performance Engineering

Redis cepat, tetapi bukan berarti semua penggunaan Redis cepat.

Redis performance ditentukan oleh kombinasi:

• command complexity
• payload size
• number of round trips
• network latency
• serialization/deserialization cost
• connection pool behavior
• client event loop pressure
• hot key concentration
• big key operations
• Lua script duration
• blocking commands
• memory pressure
• persistence/replication overhead
• cluster redirection

Untuk Java/JAX-RS backend, Redis performance problem sering muncul sebagai latency API. Namun root cause-nya bisa berada di Redis server, network, client library, serialization, PostgreSQL fallback, Kafka/RabbitMQ invalidation lag, atau Kubernetes resource pressure.

Senior engineer tidak cukup mengatakan:

Redis lambat.

Pertanyaan yang benar:

Redis command mana, dari service mana, untuk key pattern apa, dengan payload sebesar apa, pada connection/client path yang mana, dan di bawah traffic/failure mode apa?

1. Core Mental Model

Redis request path dari Java service:

JAX-RS Resource
  -> Service Layer
  -> Redis Wrapper/Repository
  -> Serializer
  -> Redis Client
  -> Connection Pool / Event Loop
  -> Network
  -> Redis Server Event Loop
  -> Command Execution
  -> Response Serialization
  -> Network
  -> Client Deserialization
  -> Service Response

Latency dapat muncul di setiap titik.

Redis server mungkin cepat, tetapi Java service tetap lambat karena:

• terlalu banyak round trip
• serializer mahal
• connection pool wait
• event loop blocked
• response terlalu besar
• fallback ke PostgreSQL mahal
• retry memperpanjang tail latency

2. Performance Is Not Only Server CPU

Redis performance diagnosis harus memisahkan:

Jangan hanya melihat Redis CPU. Tail latency bisa tinggi walaupun CPU rendah.

3. Command Complexity

Setiap Redis command punya time complexity.

Contoh konseptual:

Command murah pada key kecil bisa menjadi command mahal pada key besar.

4. Single-Threaded Command Execution Implication

Redis mengeksekusi command dengan model event-loop command execution yang sangat cepat, tetapi command panjang tetap mengganggu command lain.

Contoh:

Client A runs huge SMEMBERS
        ↓
Redis spends time preparing huge response
        ↓
Client B small GET waits behind it
        ↓
API latency spikes globally

Karena itu command yang menghasilkan response besar adalah risiko shared latency.

Redis cepat karena command biasanya pendek. Jika Anda membuat command panjang, Anda merusak model itu.

5. Network Round Trip Cost

Banyak Redis latency berasal dari round trip, bukan command execution.

Contoh buruk:

for each quoteId:
    GET quote:{id}

Jika ada 100 quote, ini 100 round trips.

Lebih baik:

MGET quote:1 quote:2 quote:3 ... quote:100

Atau pipelining:

pipeline GET quote:1
pipeline GET quote:2
pipeline GET quote:3
execute pipeline

Namun batching juga punya risiko:

• response terlalu besar
• timeout lebih besar
• satu batch lambat menghambat semua item
• memory buffer client naik
• partial failure lebih sulit ditangani

6. Pipelining

Pipelining mengirim beberapa command tanpa menunggu response satu per satu.

Mental model:

Without pipeline:
  send GET a -> wait
  send GET b -> wait
  send GET c -> wait

With pipeline:
  send GET a
  send GET b
  send GET c
  wait for all responses

Benefit:

• mengurangi round trip overhead
• meningkatkan throughput
• cocok untuk batch independent commands

Risiko:

• response buffer besar
• error mapping per command harus benar
• pipeline terlalu besar bisa meningkatkan latency
• command tetap dieksekusi Redis satu per satu
• bukan transaction

Pipelining meningkatkan throughput, bukan mengubah atomicity.

7. Batching Strategy

Batching harus bounded.

Contoh aman:

batch size: 50-200 keys depending payload size and latency target

Contoh berbahaya:

MGET 50,000 keys in one request

Trade-off:

Review question:

Apakah batch size ditentukan secara eksplisit dan diuji, atau tumbuh mengikuti input user?

Batch size yang mengikuti input user tanpa limit adalah production risk.

8. MGET/MSET

MGET/MSET dapat mengurangi round trip untuk multiple string keys.

Use case:

• loading multiple cache entries
• warming small set of keys
• batch lookup from service layer

Caution:

• all values dikembalikan sekaligus
• missing keys harus dipetakan benar
• large payload dapat memperbesar response
• pada Cluster, multi-key command harus memperhatikan slot limitation

Untuk Redis Cluster, MGET multi-slot bisa bermasalah tergantung client dan command routing. Client mungkin memecah command per slot, atau command gagal jika tidak didukung. Verifikasi client behavior internal.

9. Serialization Cost

Redis command bisa cepat tetapi serialization Java lambat.

Contoh:

Redis GET latency: 1 ms
JSON deserialize: 15 ms
business mapping: 5 ms
API response p95: 80 ms

Jika hanya melihat Redis latency, root cause hilang.

Serialization cost dipengaruhi oleh:

• JSON depth
• collection size
• polymorphic types
• date/time parsing
• BigDecimal parsing
• compression/decompression
• object allocation
• GC pressure
• class metadata/cache behavior

Cache payload harus diukur sebagai end-to-end access path, bukan hanya Redis server time.

10. Compression Trade-Off

Compression dapat menghemat memory dan network, tetapi menambah CPU.

Cocok jika:

• payload besar
• network bandwidth menjadi bottleneck
• memory mahal
• access frequency moderate

Tidak cocok jika:

• payload kecil
• request latency sangat sensitif
• CPU Java service sudah tinggi
• cache hit sangat sering pada hot path

Trade-off:

less memory + less network
        vs
more CPU + more latency variance

Untuk CPQ/order systems, quote/catalog payload bisa besar. Compression harus diuji dengan p50/p95/p99 latency dan CPU, bukan diasumsikan baik.

11. Connection Pooling

Redis client connection pool salah konfigurasi dapat menyebabkan latency tinggi.

Symptoms:

• Redis server latency rendah
• application Redis call latency tinggi
• thread menunggu connection
• timeout meningkat saat traffic naik
• pool exhausted

Penyebab:

• pool terlalu kecil
• pool terlalu besar dan membebani Redis
• per-pod pool tidak memperhitungkan replica count
• connection leak
• blocking command memakai pool request path
• slow command menahan connection

Formula konseptual:

total_connections = pods × pool_size_per_pod × redis_clients_per_pod

Kubernetes scaling dapat menggandakan connection count tanpa perubahan config.

12. Lettuce/Jedis/Redisson Performance Awareness

Setiap client punya model berbeda.

Hal yang harus diverifikasi:

• sync vs async usage
• thread safety
• Netty event loop behavior
• connection sharing
• pooling requirement
• command timeout
• topology refresh
• reconnect behavior
• retry behavior
• metrics exposure

Kesalahan umum:

Using async client but blocking on future in request thread without timeout discipline.

Atau:

Using blocking Redis operation on shared connection used by latency-sensitive commands.

Client choice bukan hanya dependency preference. Client choice adalah concurrency model.

13. Timeout Engineering

Timeout Redis harus lebih kecil dari HTTP request timeout.

Contoh buruk:

HTTP timeout: 2 seconds
Redis command timeout: 5 seconds

Akibat:

• request sudah tidak berguna tetapi Redis call masih menunggu
• thread/pool tertahan
• retry storm
• tail latency memburuk

Contoh lebih masuk akal:

HTTP timeout: 2 seconds
Redis read timeout for cache: 50-100 ms depending SLA/network
Redis write timeout for correctness-critical path: explicitly defined
fallback/circuit breaker: enabled where appropriate

Angka harus disesuaikan environment dan SLA internal. Jangan copy paste angka tanpa measurement.

14. Retry Can Hurt Performance

Retry Redis command tidak selalu aman.

Risiko retry:

• memperbesar load saat Redis sedang degraded
• duplicate side effect untuk command tertentu
• memperpanjang request latency
• menyebabkan connection storm
• membuat failure terlihat sebagai slowness

Untuk read cache miss, retry mungkin tidak perlu. Untuk idempotency/lock/rate limiter, retry harus didesain hati-hati.

Pertanyaan review:

• Command ini idempotent atau tidak?
• Retry terjadi di client library, wrapper, atau circuit breaker?
• Ada jitter/backoff?
• Apa max attempt?
• Apa behavior saat timeout ambiguity?

15. Hot Key Performance

Hot key terjadi saat satu key menerima traffic tidak proporsional.

Symptoms:

• Redis CPU tinggi pada satu node
• cluster node tertentu overload
• p99 latency naik
• command stats didominasi key/prefix tertentu
• cache hit tinggi tetapi latency tetap buruk

Mitigation:

• local cache untuk hot read-only data
• key sharding/manual split
• request coalescing
• stale-while-revalidate
• reduce payload size
• move source to more appropriate system

Hot key sering lebih buruk di Redis Cluster karena satu key hanya berada di satu slot/node. Cluster tidak memecah satu key panas otomatis.

16. Big Key Performance

Big key menyebabkan latency karena:

• command memproses banyak element
• response besar
• network transfer besar
• client deserialization besar
• deletion/trimming mahal
• replication/persistence overhead

Command yang tampak aman bisa mahal:

HGETALL large hash
SMEMBERS large set
LRANGE large list 0 -1
ZRANGE large zset 0 -1
XRANGE huge stream range

Gunakan pattern bounded:

HSCAN with count
SSCAN with count
ZRANGE with small range
XRANGE with COUNT
stream trimming

Namun SCAN family juga bukan magic. Tetap ada total work.

17. Slow Commands

Redis SLOWLOG membantu melihat command yang memakan waktu di server.

Namun slowlog hanya menangkap waktu execution di Redis server, bukan:

• network latency
• client wait for connection
• serialization
• deserialization
• application mapping
• thread scheduling

Karena itu Redis slowlog kosong tidak berarti Redis path cepat end-to-end.

Butuh dua sisi metric:

server-side Redis latency
client-side Redis command latency
application endpoint latency

Jika client latency tinggi tetapi slowlog rendah, cari bottleneck di network/client/pool/serialization.

18. Blocking Commands

Blocking commands seperti blocking pop valid untuk worker pattern.

Namun jangan campurkan blocking workload dengan latency-sensitive request path tanpa desain.

Risiko:

• connection tertahan
• pool habis
• thread request menunggu
• graceful shutdown sulit
• backpressure tidak jelas

Worker Redis queue sebaiknya punya:

• dedicated client/connection pool
• timeout untuk unblock shutdown
• metrics worker idle/busy
• retry/poison handling
• separation dari request Redis client

19. Lua Script Performance

Lua script atomic tetapi berjalan di Redis execution path.

Script buruk dapat memblokir Redis.

Risiko:

• loop besar
• scan dalam script
• operasi pada big key
• input tidak bounded
• script dipanggil pada hot path
• script return payload besar

Script rate limiter/unlock/idempotency biasanya kecil dan cocok. Script analytics/aggregation besar biasanya red flag.

Review question:

Apakah script ini O(1), O(log N), atau O(N) terhadap data yang bounded?

Jika N tidak bounded, script adalah incident candidate.

20. Cache Hit Ratio Is Not Enough

Hit ratio tinggi tidak selalu berarti performa baik.

Contoh:

hit ratio: 95%
but top 5% misses trigger expensive PostgreSQL queries
or hits deserialize huge payloads

Metric yang lebih lengkap:

• hit ratio by endpoint/use case
• miss cost
• load time from PostgreSQL
• cache fill latency
• payload size
• Redis command latency
• deserialization latency
• stale fallback count

Cache performance harus diukur dari user/request impact, bukan global Redis hit ratio saja.

21. Cache Miss Amplification

Cache miss dapat memicu efek domino:

cache miss
  -> PostgreSQL query
  -> MyBatis mapping
  -> cache fill
  -> serialization
  -> Redis write
  -> response

Jika miss terjadi massal:

cache expiry burst
  -> many services reload same data
  -> DB overload
  -> Redis fill storm
  -> latency spike

Performance engineering cache harus mencakup stampede prevention:

• TTL jitter
• single-flight
• request coalescing
• stale-while-revalidate
• reload rate limiting
• negative caching

22. PostgreSQL/MyBatis Performance Interaction

Redis optimization bisa menyembunyikan PostgreSQL query buruk.

Saat Redis miss spike, query buruk muncul.

Review question:

• Apakah fallback query indexed?
• Apakah MyBatis mapping efisien?
• Apakah cache fill melakukan N+1 query?
• Apakah cache key terlalu granular sehingga hit ratio rendah?
• Apakah invalidation terlalu agresif sehingga DB terus reload?

Redis bukan pengganti query optimization. Redis hanya mengurangi frekuensi query jika cache benar.

23. Kafka/RabbitMQ Performance Interaction

Redis sering diperbarui oleh consumer Kafka/RabbitMQ.

Performance risk:

• consumer menulis Redis terlalu cepat
• Redis latency membuat consumer lag
• retry storm dari broker ke Redis
• cache projection rebuild membanjiri Redis
• invalidation event massal memicu delete storm

Mitigation:

• bounded consumer concurrency
• batching/pipelining with limits
• backpressure
• DLQ for repeated Redis failures
• replay throttle
• metrics for Redis write latency per consumer

Broker dapat menyimpan backlog, Redis tidak otomatis menyerap write storm tanpa batas.

24. Kubernetes Performance Factors

Kubernetes dapat memengaruhi Redis performance dari sisi client dan server.

Faktor umum:

• CPU throttling pada Java pod
• CPU throttling pada Redis pod
• memory pressure
• noisy neighbor
• cross-node network latency
• DNS resolution delay
• rolling update connection storm
• HPA scaling menambah Redis connections
• network policy overhead

Jika Redis latency naik hanya saat deployment atau scaling event, cek connection storm dan topology refresh.

25. AWS/Azure/On-Prem Latency Factors

Cloud/on-prem topology memengaruhi Redis latency.

Cek:

• same AZ vs cross-AZ
• region mismatch
• private endpoint routing
• firewall/proxy path
• TLS overhead
• managed service maintenance
• failover event
• cluster redirection
• network saturation

Redis dekat secara logical belum tentu dekat secara network.

Untuk latency-sensitive path, topology matter.

26. TLS Performance Awareness

TLS penting untuk security, tetapi dapat menambah CPU/latency.

Biasanya acceptable dan wajib di banyak environment. Namun tetap perlu diukur:

• connection reuse
• handshake frequency
• CPU overhead
• certificate rotation behavior
• client config

Masalah umum:

connection churn + TLS handshake + pod scaling
        ↓
latency spike and CPU increase

Connection pooling dan keepalive menjadi makin penting.

27. Benchmarking Redis Path

Benchmark yang berguna harus mirip workload nyata.

Jangan hanya benchmark:

redis-benchmark GET small-key

Ukur juga:

• payload nyata
• serializer nyata
• network path nyata
• Java client nyata
• concurrency nyata
• TTL/invalidation behavior
• cache miss path
• cluster/sentinel topology
• TLS enabled jika production enabled

Benchmark harus menjawab:

Apakah Redis path memenuhi latency budget endpoint di bawah traffic realistis?

28. Load Testing

Load test Redis-based feature harus mencakup:

• steady state
• cold cache
• warm cache
• mass expiry
• Redis slow response
• Redis unavailable
• high cardinality tenant
• hot key
• big payload
• rolling deployment
• connection storm
• broker replay
• database fallback under miss spike

Tanpa scenario failure, load test hanya membuktikan happy path.

29. Performance Metrics

Server-side Redis metrics:

• ops/sec
• CPU
• commandstats
• slowlog
• latency history
• connected clients
• blocked clients
• used memory
• evictions
• keyspace hit/miss
• replication lag
• cluster node latency

Client-side metrics:

• command latency p50/p95/p99
• pool wait time
• timeout count
• retry count
• connection count
• reconnect count
• serialization time
• deserialization time
• payload size

Application metrics:

• endpoint latency
• cache hit/miss by use case
• fallback count
• stale fallback count
• DB query latency on miss
• invalidation lag

30. Latency Diagnosis Flow

Practical flow:

1. Which endpoint is slow?
2. Which Redis operation is on that endpoint?
3. Is it hit path, miss path, fill path, limiter, lock, idempotency, stream, or session?
4. Is client-side Redis latency high?
5. Is Redis server slowlog showing command latency?
6. Is pool wait time high?
7. Is payload size large?
8. Is serialization/deserialization high?
9. Is Redis memory/eviction/CPU under pressure?
10. Is network/topology/failover involved?
11. Did deployment or traffic change coincide?

This prevents blind tuning.

31. Common Performance Failure Modes

31.1 N+1 Redis Calls

Symptoms:

• many Redis commands per request
• latency grows with item count
• Redis ops/sec high but each command small

Mitigation:

• batch with bounds
• MGET/pipeline
• redesign response aggregation
• local request-level cache

31.2 Pool Exhaustion

Symptoms:

• application Redis latency high
• server Redis latency low
• thread waiting for connection
• timeouts during traffic spike

Mitigation:

• tune pool
• separate blocking workload
• reduce slow commands
• bound concurrency
• check total connections across pods

31.3 Large Payload Hit

Symptoms:

• cache hit but endpoint slow
• Redis GET not too slow
• deserialization/GC high
• network egress high

Mitigation:

• reduce payload
• split cache object
• cache smaller read model
• evaluate compression carefully
• avoid caching full aggregate when not needed

31.4 Hot Key Saturation

Symptoms:

• one Redis node hot
• p99 latency high
• cache hit ratio high
• traffic concentrated on one key/prefix

Mitigation:

• local cache
• split/shard key
• request coalescing
• stale-while-revalidate
• reduce update frequency

31.5 Expiry Burst

Symptoms:

• miss spike at predictable interval
• DB load spike
• Redis write spike after reload
• latency wave

Mitigation:

• TTL jitter
• refresh ahead
• stale fallback
• single-flight
• stagger cache warming

31.6 Lua Blocking

Symptoms:

• Redis latency spike
• slowlog shows EVAL/EVALSHA
• unrelated commands slow

Mitigation:

• bound script work
• remove scans/loops over large data
• split operation
• test script under worst-case size

32. Design Patterns for Performance

Useful patterns:

• cache-aside with bounded reload
• MGET/pipeline for batch read
• TTL jitter
• local + distributed cache for hot read-only data
• stale-while-revalidate
• request coalescing
• negative cache for known absent data
• bounded sorted set cleanup
• stream trimming
• dedicated Redis client for worker/blocking workload
• separate Redis topology for cache vs correctness-critical state

Anti-patterns:

• unbounded key cardinality
• unbounded list/set/hash/zset/stream
• huge JSON aggregate cache
• KEYS in application path
• full collection command in request path
• retry without backoff
• same Redis pool for blocking worker and API request
• using Redis to hide bad DB query indefinitely

33. Java/JAX-RS Review Example

Problematic flow:

// conceptual anti-pattern
for (String quoteId : quoteIds) {
    String json = redis.get("quote:" + quoteId);
    Quote quote = objectMapper.readValue(json, Quote.class);
    result.add(quote);
}

Problems:

• N Redis round trips
• no batch limit
• no null/miss handling shown
• deserialization per item
• no timeout/fallback boundary
• no payload size control

Better conceptual direction:

// conceptual direction only
List<String> boundedIds = limit(quoteIds, MAX_BATCH_SIZE);
List<String> keys = boundedIds.stream()
    .map(id -> keyFactory.quoteCacheKey(tenantId, id))
    .toList();

Map<String, CachedPayload> payloads = redisCache.mget(keys, redisTimeoutBudget);

// miss handling separated and bounded

Performance starts with API input bounds.

34. PR Review Checklist

Saat review PR Redis performance:

• Berapa Redis command per HTTP request?
• Apakah command count bounded?
• Apakah ada N+1 Redis pattern?
• Apakah batch size bounded?
• Apakah payload size measured?
• Apakah serializer/deserializer measured?
• Apakah command complexity aman untuk key size?
• Apakah full collection command dipakai?
• Apakah pipelining/MGET cocok?
• Apakah Redis Cluster multi-key limitation diperhatikan?
• Apakah connection pool cukup tetapi tidak berlebihan?
• Apakah timeout lebih kecil dari request timeout?
• Apakah retry policy aman?
• Apakah hot key/big key risk ada?
• Apakah cache miss path membebani PostgreSQL?
• Apakah consumer Kafka/RabbitMQ menulis Redis dengan backpressure?
• Apakah metrics client-side tersedia?

35. Internal Verification Checklist

Cek di codebase, config, dashboard, dan runbook:

• Redis command latency p50/p95/p99 dari Java client.
• Redis server slowlog.
• commandstats per command.
• jumlah Redis command per endpoint.
• pool wait time dan pool exhaustion.
• timeout/retry/reconnect metrics.
• payload size distribution.
• serialization/deserialization time.
• cache hit/miss per use case.
• DB query latency pada cache miss.
• hot key dan big key report.
• TTL expiry burst pattern.
• Lua script runtime.
• stream consumer lag dan pending entries.
• Kafka/RabbitMQ consumer Redis write latency.
• Kubernetes CPU throttling/memory pressure.
• AWS/Azure/on-prem network/failover events.
• load test result untuk Redis path.

36. Senior Engineer Takeaway

Redis performance bukan sekadar Redis cepat atau lambat.

Redis performance adalah hasil dari:

command complexity
× payload size
× round trip count
× client concurrency model
× serialization cost
× memory condition
× topology
× failure behavior

Mental model yang harus dipegang:

A fast Redis command can still produce a slow API if it is called too many times, carries too much data, waits for a connection, deserializes too much object graph, or triggers an expensive miss path.

Senior engineer tidak hanya mengoptimalkan command. Senior engineer mengoptimalkan lifecycle penuh dari request sampai fallback dan observability.