Incident Response

PostgreSQL "could not resize shared memory segment": No Space Left on Device

A big query fails with "No space left on device", but the data disk is half empty. The disk in question is /dev/shm, and in Docker it is only 64 MB by default. Here is why parallel queries need it, how to confirm the cause and how to size it properly in Docker and Kubernetes.

· 13 min read · AG Data Team

The PostgreSQL error "could not resize shared memory segment ... No space left on device" confuses people because the data volume usually has plenty of room. It is not about that disk. PostgreSQL is trying to allocate dynamic shared memory for a parallel query, that memory lives in /dev/shm, and /dev/shm is full. In Docker it is only 64 MB unless you say otherwise, which is why this error is so common in containers and so rare on bare metal.

The full message looks like this (the segment name and size will differ):

ERROR:  could not resize shared memory segment "/PostgreSQL.2051486325" to 67145056 bytes: No space left on device

Typical pattern: the database works fine in development, then one report, an analytics query or a big join in production fails with this error, sometimes only when several such queries run at the same time.

What is actually being resized

When PostgreSQL starts a parallel operation, the backend that starts it first creates a dynamic shared memory segment that lives for the duration of that operation. The workers attach to it to exchange state, errors and data. That is described in the source tree's README.parallel. A parallel hash join goes further: instead of each process building its own copy, the parallel plans documentation explains that the processes build one shared hash table together, and that table also lives in shared memory.

How that memory is obtained is controlled by dynamic_shared_memory_type. The resource configuration docs list posix (shm_open), sysv, windows and mmap, and say the first supported option is usually the platform default, so on Linux you will normally see posix. On Linux, POSIX shared memory objects are files in a tmpfs that is normally mounted at /dev/shm.

The error text comes from dsm_impl.c. On Linux PostgreSQL deliberately calls posix_fallocate() on the new segment. The comment in the code explains why: if it simply extended the file, tmpfs would allocate pages only when they are touched, and running out of space then would crash the process with SIGBUS. Allocating up front means PostgreSQL gets a clean ENOSPC error and cancels just the query. So the message is PostgreSQL protecting itself, not a sign of corruption.

Why containers hit it first

The official postgres Docker image documentation says it plainly: the default /dev/shm size for containers is 64 MB, and when shared memory is exhausted you get exactly this error. It suggests passing --shm-size to docker run (the example uses 256MB), or the equivalent shm_size in docker compose.

A rough feel for why 64 MB is so little: work_mem defaults to 4 MB and hash operations may use work_mem times hash_mem_multiplier (2.0 by default). The docs also warn that limits such as work_mem apply to each parallel worker separately, and that a complex query may run several sort and hash operations at once. Raise work_mem to something realistic for reporting, let a few parallel queries run concurrently, and a 64 MB tmpfs is gone. Outside containers, /dev/shm is typically a tmpfs without an explicit size, and the kernel's tmpfs default is half of physical RAM, so the same query rarely complains there.

Confirm the cause in two minutes

Check the size and usage of /dev/shm where PostgreSQL actually runs:

# inside the container or on the host
df -h /dev/shm
ls -la /dev/shm | head        # PostgreSQL.NNNN files are live DSM segments

docker inspect --format '{{.HostConfig.ShmSize}}' my-postgres

Then look at the settings that decide how much dynamic shared memory queries ask for:

SHOW dynamic_shared_memory_type;
SHOW min_dynamic_shared_memory;
SHOW max_parallel_workers_per_gather;
SHOW work_mem;
SHOW hash_mem_multiplier;

If /dev/shm is 64M, dynamic_shared_memory_type is posix and the failing query shows Gather or Parallel Hash in its EXPLAIN plan, you have your answer. If /dev/shm is large and still fills up, look at what else writes there: other processes in the same container or pod share it.

Not the same as a full data disk. This error does not mean your tables or WAL are out of space, and deleting files from the data directory will not help. If the data volume itself is full, the symptoms and the fix are different; see our guide to PostgreSQL disk full and pg_wal growth.

Fix 1: give /dev/shm a sensible size

This is the real fix. In Docker and docker compose:

docker run -d --name my-postgres --shm-size=1g postgres:18

# docker compose
services:
  db:
    image: postgres:18
    shm_size: 1gb

Kubernetes has no direct equivalent of --shm-size; the size of /dev/shm comes from the container runtime, so check it with df inside the pod. The common approach is to mount a memory-backed emptyDir over it:

spec:
  containers:
    - name: postgres
      volumeMounts:
        - name: dshm
          mountPath: /dev/shm
  volumes:
    - name: dshm
      emptyDir:
        medium: Memory
        sizeLimit: 1Gi

Two details from the Kubernetes volumes documentation matter here. With medium: Memory the volume is a tmpfs, and files written there count against the memory limit of the container that wrote them. And if you do not specify a size, a memory-backed volume is sized to the node's allocatable memory. So set sizeLimit and include the shared memory you expect in the pod's memory limit, or the fix for this error becomes an OOM kill instead. If you run an operator, check its documentation first: some PostgreSQL operators already mount shared memory this way or offer a setting for it.

How big? There is no formula in the docs. A reasonable approach is to start at a few hundred megabytes to a gigabyte for an analytics-capable instance, watch df -h /dev/shm during your heaviest reports, and grow it if needed. It is RAM, so it has to fit next to shared_buffers and everything else.

Fix 2: reduce what queries ask for

If you cannot change the container right now, you can make queries ask for less. Parallel query is disabled for a session when max_parallel_workers_per_gather is 0:

-- quick relief for one report or session
SET max_parallel_workers_per_gather = 0;

-- or for a specific reporting role
ALTER ROLE reporting SET max_parallel_workers_per_gather = 1;

That makes the error go away for those queries at the cost of losing parallelism, so it is a stopgap, not a configuration to keep. Lowering an aggressive work_mem or hash_mem_multiplier for reporting roles also reduces the size of shared hash tables. And the query itself is often the real problem: a hash join over tens of millions of rows that could have used an index. Our slow query tuning checklist covers how to find and fix those.

What about min_dynamic_shared_memory?

min_dynamic_shared_memory reserves memory at server start for parallel queries; the default is 0. When that region is insufficient or used up, new parallel queries fall back to allocating from the operating system with dynamic_shared_memory_type. Because the reserved region is part of the main shared memory allocated at startup rather than a file in /dev/shm, it can help, and the docs note it can benefit from huge pages. But it is a fixed reservation that is taken from RAM whether used or not, and anything beyond it still goes to /dev/shm. Treat it as tuning on top of a correctly sized /dev/shm, not a replacement. Changing it requires a restart.

How to avoid the next one

Key takeaways

PostgreSQL Docker Kubernetes Troubleshooting
Work with us

PostgreSQL in containers misbehaving?

We run PostgreSQL in Docker and Kubernetes for clients and know where the defaults bite. Tell us what is failing and we will help you fix it properly.

Get in touch More from the blog