Resolving Redis maxmemory OOM Errors in Docker Compose for Cache vs BullMQ Queues
Fix ReplyError: OOM command not allowed when used memory > 'maxmemory' in Dockerized Redis. Configure maxmemory-policy (allkeys-lru vs volatile-lru vs noeviction), BullMQ removeOnComplete retention, and Linux vm.overcommit_memory.
Redis throws `OOM command not allowed when used memory > 'maxmemory'` on any write command (`SET`, `LPUSH`, `XADD`, `HSET`) when `used_memory` reaches the configured `maxmemory` ceiling and the active `maxmemory-policy` either refuses eviction (`noeviction`, the default) or cannot find any keys with a TTL to evict (`volatile-lru` when all keys lack an `EXPIRE` timestamp). To fix this in production without losing critical queue jobs: (1) split cache and job-queue workloads or choose the right `maxmemory-policy` (`allkeys-lru` for caches; `noeviction` + `removeOnComplete` pruning for BullMQ/n8n queues), and (2) set `maxmemory` to 80% of your Docker container's cgroup memory limit.
Verified Root Causes in Production Environments
- Default maxmemory-policy is noeviction: Out of the box, Redis sets `maxmemory-policy noeviction`, meaning once `maxmemory` is reached, every write command fails with an OOM error instead of evicting old keys.
- Using volatile-lru With Unexpired Keys: Setting `maxmemory-policy volatile-lru` only evicts keys that have a TTL (`EXPIRE` / `PEXPIRE`) set. If your application writes keys without a TTL—or if BullMQ stores completed job hashes indefinitely—Redis still hits OOM.
- Unbounded BullMQ / Celery / Sidekiq Completed Job History: By default, Bull and BullMQ keep every completed and failed job payload in Redis forever unless `removeOnComplete` and `removeOnFail` are configured.
1. Diagnose What Is Consuming Redis Memory With INFO MEMORY & --bigkeys
Before blindly raising `maxmemory` or switching to `allkeys-lru` (which can accidentally delete active user sessions or queued payment webhooks!), inspect your live Redis instance using `redis-cli INFO memory` and `redis-cli --bigkeys`.
Look specifically at three metrics in `INFO memory`: `used_memory_human` (actual data stored), `used_memory_rss_human` (physical OS RAM allocated including fragmentation), and `maxmemory_policy`. Then check the `Keyspace` section (`INFO keyspace`) to see how many keys actually have an expiration (`expires`) versus total `keys`.
# 1. Inspect current memory usage, fragmentation ratio, and active eviction policy
docker compose exec redis redis-cli INFO memory | grep -E 'used_memory_human|used_memory_rss_human|maxmemory_human|maxmemory_policy|mem_fragmentation_ratio'
# 2. Check how many keys have a TTL (expires) vs permanent keys
docker compose exec redis redis-cli INFO keyspace
# Example output showing only 120 out of 450,000 keys have a TTL:
# db0:keys=450120,expires=120,avg_ttl=341200
# 3. Scan keyspace Safely (using SCAN, not KEYS *) to find giant prefixes
docker compose exec redis redis-cli --bigkeys2. Choose the Correct Eviction Policy: Cache (`allkeys-lru`) vs Job Queue (`noeviction`)
If your Redis container is used strictly as an application cache or rate-limiter (where any key can be regenerated from PostgreSQL), apply `maxmemory-policy allkeys-lru` or `allkeys-lfu` instantaneously using `CONFIG SET` and persist it in your `docker-compose.yml` command.
However, if your Redis instance backs BullMQ, n8n Queue Mode, or Sidekiq background workers, NEVER use `allkeys-lru`—doing so will randomly evict active queue lock keys and stall workers. Instead, keep `noeviction` and configure BullMQ's `defaultJobOptions` (`removeOnComplete: { count: 500, age: 3600 }`) to automatically prune finished job payloads.
import { Queue } from 'bullmq';
// Prevent Redis OOM in BullMQ by auto-pruning completed and failed job hashes
export const deploymentQueue = new Queue('container-deployments', {
connection: {
host: process.env.REDIS_HOST || 'redis',
port: 6379,
},
defaultJobOptions: {
attempts: 3,
backoff: { type: 'exponential', delay: 2000 },
// Keep at most 500 completed jobs or 1 hour of history
removeOnComplete: {
age: 3600,
count: 500,
},
// Keep at most 1000 failed jobs for 24 hours for debugging
removeOnFail: {
age: 86400,
count: 1000,
},
},
});3. Production Redis 7 Docker Compose Template With Cgroup Headroom
Never set Redis `maxmemory` equal to 100% of your Docker container's `deploy.resources.limits.memory`. Redis requires 20% to 25% memory headroom above `maxmemory` for client output buffers, replica synchronization buffers, and copy-on-write (`fork()`) pages during RDB/AOF background snapshots. If your container limit is `1024M`, set `--maxmemory 768mb`.
services:
redis-cache:
image: redis:7.4-alpine
container_name: vexlo-redis-cache
restart: unless-stopped
# Set maxmemory to ~75% of the 1024M container limit and enable active defrag
command: >
redis-server
--maxmemory 768mb
--maxmemory-policy allkeys-lru
--activedefrag yes
--appendonly no
--save ""
deploy:
resources:
limits:
memory: 1024M
volumes:
- redis_cache_data:/data
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 10s
timeout: 3s
retries: 5
volumes:
redis_cache_data:Pre-Deployment Verification Checklist
- Run `redis-cli INFO keyspace` to check if keys are missing TTLs (`expires=0`).
- Size `--maxmemory` to 75–80% of the Docker container memory limit to leave room for copy-on-write snapshot buffers.
- Use `allkeys-lru` for pure caching instances, and `noeviction` + `removeOnComplete` for BullMQ / Sidekiq queues.
- Set `sysctl vm.overcommit_memory=1` on the Linux host so Redis background saves (`BGSAVE`) do not fail under memory pressure.
Deploy on Vexlo Cloud in 60s (₹149/mo UPI)
Automated Docker & Nixpacks builds, managed PostgreSQL/Redis, and AI Deployment Doctor. Instant INR UPI for India or USD/EUR/GBP across 17 global currencies.
Frequently Asked Questions
Why am I getting Redis OOM errors even though maxmemory-policy is set to volatile-lru?
`volatile-lru` only evicts keys that have an explicit TTL (`EXPIRE`) set. If your application writes keys without an expiration time, `volatile-lru` behaves identically to `noeviction` once non-expiring keys fill `maxmemory`. Switch to `allkeys-lru` or add TTLs to all `SET` calls.
How do I increase Redis maxmemory on a running container without restarting it?
Run `docker compose exec redis redis-cli CONFIG SET maxmemory 1536mb` and `docker compose exec redis redis-cli CONFIG SET maxmemory-policy allkeys-lru` for an instant zero-downtime update, then update your `docker-compose.yml` so the change persists across container recreations.
Why does Redis memory usage exceed maxmemory in docker stats?
`maxmemory` caps the dataset heap allocated by Redis keys, while `docker stats` measures total Resident Set Size (RSS), which includes allocator fragmentation (`jemalloc`), client pub/sub output buffers, and AOF rewrite buffers.