PostgreSQL "invalid page in block": How to Handle Data Corruption

The PostgreSQL error "invalid page in block" means the server read a page (8 kB by default) from a data file and found it damaged: either its header is not a valid page header, or, with data checksums enabled, the checksum does not match. This is real data corruption, not a query problem. The good news is that it is often limited to one object and sometimes to one page. The bad news is that hasty fixes can make it permanent.

A typical log with checksums enabled:

WARNING:  page verification failed, calculated checksum 21574 but expected 2207
ERROR:  invalid page in block 48213 of relation "base/16384/16529"

The messages come from bufpage.c, which verifies each page when it is read, and bufmgr.c, which raises the error. Before PostgreSQL 18 the path is printed without quotes. PostgreSQL 18 can also report several bad pages at once, as "N invalid pages among blocks X..Y of relation".

Step 1: stop making it worse

Corruption almost always comes from below PostgreSQL: a failing disk, a controller or RAID problem, a storage layer that lied about a completed write, bad RAM, or a bug in a kernel or file system. Whatever wrote the bad page may still be writing. So before any repair:

# before ANY repair attempt: stop the server and copy the data directory
sudo systemctl stop postgresql
sudo rsync -a /var/lib/postgresql/18/main/ /backup/main-before-repair/
dmesg -T | grep -iE 'i/o error|ext4|xfs|nvme|ata' | tail -50

• Copy the whole data directory while the server is stopped, to different storage. Every repair step below changes data. With a copy, a failed attempt is a lesson, not a loss.
• Look at the kernel log and the disk's health. I/O errors or file system warnings around the time of the first error tell you whether this host can be trusted at all.
• If you have a healthy replica, consider failing over to it. Damage caused by the primary's disk is often not present on a physical replica, because the replica applies WAL to its own files rather than copying the primary's data files, but check the replica with the tools below before trusting it.

Step 2: find out what is damaged

The path base/16384/16529 means database OID 16384, relation file 16529, in the default tablespace. The pg_filenode_relation function maps it back to a name:

-- 16384 is the database OID, 16529 the relation's filenode
SELECT datname FROM pg_database WHERE oid = 16384;

-- connect to that database, then (0 = default tablespace):
SELECT pg_filenode_relation(0, 16529) AS relation,
       c.relkind
FROM pg_class c
WHERE c.oid = pg_filenode_relation(0, 16529);

The answer decides the rest of the incident:

• An index (relkind = 'i') is the best case. Index contents can be rebuilt from the table, so no data is lost.
• A table (relkind = 'r') or its TOAST table means rows may be gone.
• A system catalog is the hardest case. Do not experiment; restore, or get help.

Check whether checksums are on and whether other databases are already affected. The pg_stat_database view counts checksum failures since PostgreSQL 12:

SHOW data_checksums;

SELECT datname, checksum_failures, checksum_last_failure
FROM pg_stat_database
WHERE checksum_failures IS NOT NULL;

Step 3: measure the damage

One bad page is rarely alone. Before choosing a fix, check how far it goes. The amcheck extension verifies tables (verify_heapam, PostgreSQL 14+) and B-tree indexes (bt_index_check):

CREATE EXTENSION IF NOT EXISTS amcheck;

-- table pages (PostgreSQL 14+)
SELECT blkno, offnum, msg FROM verify_heapam('public.orders') LIMIT 20;

-- B-tree index structure
SELECT bt_index_check('public.orders_pkey');

For the whole cluster, pg_amcheck (also 14+) runs the same checks across databases in parallel. If checksums are enabled, pg_checksums --check scans every file of a cleanly shut down cluster and exits non-zero if any checksum fails:

pg_amcheck --all --jobs=4 --heapallindexed --install-missing
# offline, whole cluster, only if checksums are on:
pg_checksums --check -D /var/lib/postgresql/18/main --progress

Run these on the copy if the original host is suspect. A long scan on a failing disk can finish it off.

Step 4: choose the fix

Damaged index: rebuild it

Drop and recreate or REINDEX the index. On a busy system, REINDEX ... CONCURRENTLY avoids blocking writes; see our guide to REINDEX CONCURRENTLY for how it behaves and what to do if it fails halfway. Then run bt_index_check again.

Damaged table: restore if you can

If you have a base backup and WAL archive, restoring is the only way to get the lost rows back. You do not have to roll back the whole production database: restore to a separate server at a point just before the damage, then copy the affected rows or the whole table back. Our point-in-time recovery guide walks through it. Remember that the corruption may have existed for a while before a query touched the page, so choose the recovery point carefully, and check the restored table with amcheck too.

No usable backup: salvage what is left

When there is nothing to restore from, PostgreSQL has two developer options for getting past bad pages. Both are dangerous, and the documentation says so plainly:

• zero_damaged_pages makes PostgreSQL report a warning, zero out the damaged page in memory and carry on. The docs say this will destroy data, namely all the rows on the damaged page, and that you should generally not set it until you have given up hope of recovering data from the damaged pages. Zeroed pages are not forced to disk, so the docs recommend recreating the table or index before turning the option off.
• ignore_checksum_failure only applies with checksums enabled. It skips the checksum error (still with a warning) and may let you read undamaged rows from a page whose header is still sane. The docs warn it may cause crashes, propagate or hide corruption.

Both default to off and need superuser or the appropriate SET privilege. A salvage usually looks like this, run against the copy or a server you can afford to lose:

-- LAST RESORT, on a copy if you can: rows on the damaged pages are lost
BEGIN;
SET LOCAL zero_damaged_pages = on;
CREATE TABLE public.orders_salvaged AS SELECT * FROM public.orders;
COMMIT;

SELECT count(*) FROM public.orders_salvaged;   -- compare with expectations

Then compare counts and key ranges, find out which rows are missing (from application logs, exports or another system), rebuild indexes and constraints, and swap the salvaged table in. Never leave either option on in the server configuration.

Step 5: make the next incident smaller

• Enable data checksums. Since PostgreSQL 18, initdb enables them by default. On older clusters (PostgreSQL 12+), pg_checksums --enable can turn them on offline. It rewrites relation blocks in place, so plan downtime that grows with the size of the cluster. Note for upgrades: pg_upgrade requires matching checksum settings, which is why PostgreSQL 18 added initdb --no-data-checksums for upgrading old clusters without checksums.
• Alert on checksum_failures in pg_stat_database. Any value above zero deserves a ticket that day.
• Run pg_amcheck regularly, for example weekly on a replica or a restored backup, so corruption is found by you and not by a customer query.
• Test restores. The table-level restore in step 4 only works if backups and WAL archiving actually work.
• Fix the cause. Replace the disk, update firmware, check that the storage honours flushes. Otherwise you will be back.