Incident Response

PostgreSQL "canceling statement due to statement timeout": Causes and Fixes

The error means a timeout you or someone else configured just did its job. The real work is finding out which limit fired, where it was set and why the query ran that long.

· 14 min read · AG Data Team

ERROR: canceling statement due to statement timeout is PostgreSQL telling you that a query ran longer than statement_timeout allows and was cancelled. Nothing is broken in the database. A limit worked as designed. The useful questions are which limit fired, who set it and why the query needed more time than it used to.

Read the exact message first

Several different mechanisms cancel queries, and their messages look alike at a glance. Telling them apart saves a lot of guessing:

ERROR:  canceling statement due to statement timeout   -- SQLSTATE 57014
ERROR:  canceling statement due to lock timeout        -- SQLSTATE 55P03
ERROR:  canceling statement due to user request        -- SQLSTATE 57014
FATAL:  terminating connection due to transaction timeout   -- SQLSTATE 25P04

The SQLSTATE codes matter for applications: statement timeout and user cancel share 57014 (query_canceled), while lock timeout is 55P03 (lock_not_available). If your retry logic only looks at one code, it may treat a lock timeout as a fatal error, or the reverse.

On a hot standby there is a fourth look-alike, "canceling statement due to conflict with recovery". It has nothing to do with timeouts you set. We covered it in canceling statement due to conflict with recovery.

Find where the timeout comes from

The default statement_timeout is 0, which means no limit. So if you see this error, somebody set a value. It can come from postgresql.conf, ALTER DATABASE ... SET, ALTER ROLE ... SET, a SET in the session, the connection string or PGOPTIONS, a connection pool's init query, or the ORM. Ask the database instead of guessing:

-- which value does this session see, and where did it come from?
SELECT name, setting, unit, source
FROM pg_settings
WHERE name IN ('statement_timeout', 'lock_timeout', 'transaction_timeout');

-- per-role and per-database overrides (psql shortcut: \drds)
SELECT r.rolname, d.datname, s.setconfig
FROM pg_db_role_setting s
LEFT JOIN pg_roles r ON r.oid = s.setrole
LEFT JOIN pg_database d ON d.oid = s.setdatabase;

The source column shows whether the value came from the configuration file, the database, the user or the session. Remember that role and database settings apply only to new sessions. A long-lived pool keeps the old value until it reconnects. When role and database settings conflict, the ALTER ROLE documentation defines the order: a role-in-database setting beats a role setting, which beats a database setting.

One warning from the PostgreSQL docs is worth repeating: setting statement_timeout in postgresql.conf is not recommended because it affects every session, including your maintenance and backup jobs.

Is it a slow query or a lock wait?

Because waiting on locks counts toward statement_timeout, a perfectly fast UPDATE can time out if it sits behind another transaction. Look at what is running and what it waits on:

SELECT pid, usename, state, wait_event_type, wait_event,
       now() - query_start AS running_for,
       pg_blocking_pids(pid) AS blocked_by,
       left(query, 80) AS query
FROM pg_stat_activity
WHERE state <> 'idle' AND pid <> pg_backend_pid()
ORDER BY query_start
LIMIT 20;

To catch these over time, make sure log_min_error_statement is at its default ERROR or lower, so the text of the cancelled statement lands in the log. Add log_min_duration_statement to see queries that are getting close to the limit, and log_lock_waits = on to log lock waits longer than deadlock_timeout (1s by default).

Fixes, in the order we try them

  1. Fix the query, not the limit. A query that used to take 2 seconds and now takes 40 usually has a reason: a missing index after a schema change, stale statistics, a plan change as the table grew, or table bloat. Raising the timeout only hides that until the next incident.
  2. Remove the blocker. If the statement was waiting on a lock, fix the transaction that holds it. Shorten transactions in the application and add idle_in_transaction_session_timeout for the role, so abandoned sessions do not hold locks forever.
  3. Set timeouts per workload, not globally. Web requests, reports and batch jobs need different limits. Put them on roles and raise them for a single transaction with SET LOCAL, which reverts at the end of the transaction (example below).
  4. Keep lock_timeout lower than statement_timeout. The documentation points out that a lock_timeout equal to or larger than a non-zero statement_timeout is pointless, because the statement timeout always fires first. A low lock timeout makes lock waits fail fast with their own clear message.
  5. Handle the error in the application. A cancelled statement aborts the current transaction. Roll back and retry the whole transaction when that makes sense, and do not retry immediately in a tight loop when the database is already overloaded.
-- web traffic: short leash
ALTER ROLE app_web IN DATABASE shop SET statement_timeout = '15s';
ALTER ROLE app_web IN DATABASE shop SET lock_timeout = '3s';

-- reporting role: longer, but still bounded
ALTER ROLE reporting SET statement_timeout = '20min';

-- one heavy job that legitimately needs more time
BEGIN;
SET LOCAL statement_timeout = '3h';
-- ... the long statement ...
COMMIT;   -- the session value is back after COMMIT or ROLLBACK
Do not "fix" it with statement_timeout = 0. Turning the limit off globally means the next runaway query can run for hours, hold locks and block vacuum from cleaning up. Raise the limit only for the role or the transaction that needs it.

Timeouts on large databases

On databases with hundreds of gigabytes or terabytes, timeouts behave differently in practice:

SET lock_timeout = '3s';
ALTER TABLE orders ADD COLUMN note text;
-- ERROR: canceling statement due to lock timeout  -> wait, retry

When you need to stop a query yourself

pg_cancel_backend(pid) cancels the current query and keeps the session. pg_terminate_backend(pid) ends the whole session. Try cancel first. A cancelled write is rolled back and its changes are discarded, but the dead row versions it already wrote stay in the table until vacuum removes them, so a large cancelled backfill can leave noticeable bloat behind.

Key takeaways

PostgreSQL statement_timeout lock_timeout Performance
Work with us

Queries hitting timeouts in production?

We find the slow queries and lock chains behind PostgreSQL timeouts, set sane per-role limits and make migrations on large tables safe to run.

Get in touch More from the blog