Problem Analysis

• While reviewing legacy code, we discovered Redis operations without TTL settings. This likely occurred because:

  • Hash structures can’t set expiration time during creation
  • Subsequent developers forgot to add expiration

• Common problematic patterns:

  • Date-based Hash keys
    HSET xxx:ad_id:20220315 userid value
  • Orphaned ad keys
    HSET xxx:ad_id userid value (after ad campaign ended)

• Initial cluster status: 8-node cluster with 8GB/node, but actual usage reached 51G+

• Memory overconsumption forced continuous resource scaling

Optimization Strategies

1. Key Distribution Analysis

• Used Alibaba Cloud’s offline analysis platform:
  https://help.aliyun.com/document_detail/102093.htm
• Alternative method (using redis-cli):
  redis-cli --bigkeys -i 0.1 (remove -i during off-peak)

2. Code Refactoring

• Added TTL for date-based keys:
  EXPIRE xxx:ad_id:20220315 86400
• Implemented cleanup for expired ads:
  UNLINK xxx:ad_id

3. Legacy Data Cleanup

• Modern Redis (≥4.0) solution:
  UNLINK xxx:ad_id:20220315
• Bulk deletion pattern:
  redis-cli --scan --pattern xxx:ad_id:* | xargs redis-cli unlink

4. Memory Fragmentation Management

• After massive deletions, monitor fragmentation ratio:
  mem_fragmentation_ratio = used_memory_rss / used_memory

• Recommended configuration for auto-defrag:

activedefrag yes
active-defrag-ignore-bytes 100mb
active-defrag-threshold-lower 10
active-defrag-threshold-upper 100
active-defrag-cycle-min 25
active-defrag-cycle-max 75

• Fragmentation reduction progress after 12 hours:

Important Notes

1. Data Backup Protocol

• Check last backup timestamp:
  LASTSAVE
• Initiate backup:
  BGSAVE
• Verify completion:
  Compare timestamps via LASTSAVE

2. Redis Expiration Mechanism

Redis expires keys in two ways: passive and active.

Passive expiration occurs when clients access keys and find them expired.

Active expiration runs 10 times/second:

1. Randomly test 20 keys from expiration set
2. Delete expired keys
3. If >25% keys expired, repeat step 1

• Warning: The default File cache driver in Laravel uses passive expiration. Combined with ThrottleRequests middleware, this can cause massive file accumulation.