PostgreSQL WAL Growth and Archive Failures

Troubleshooting / Guide 45 of 47 / Advanced

• OSUbuntu 26.04 LTS (resolute)
• PostgreSQL18.6-0ubuntu0.26.04.1
• PackagingDebian/Ubuntu (pg_ctlcluster, /etc/postgresql)
• TimeAbout 19 min
• Reviewed27 August 2026

Written against the versions above. `archive_library` is an alternative to `archive_command` from PostgreSQL 15. The failure mode and everything in `pg_stat_archiver` are the same either way.

Before you start

• A cluster with archive_mode = on and a working archive_command.
• Enough free disk to hold a few hundred MB of WAL.

1. 1

   A healthy archiver, for comparison

   pg_stat_archiver is one row for the whole cluster: 30 archived, failed_count 0, last_failed_wal empty.

   archive_status is the queue. PostgreSQL writes a .ready file there when a WAL segment is complete, the archiver picks it up, runs archive_command, and on success renames it .done. An empty-ish directory means the archiver is keeping up.

   pg_wal is 129 MB. Note the archive_command itself: the test ! -f guard makes it refuse to overwrite an existing file, which is the documented shape and the reason a re-run cannot silently destroy an archived segment.

   $sudo -u postgres psql -x -c "SELECT * FROM pg_stat_archiver"-[ RECORD 1 ]------+------------------------------archived_count     | 30last_archived_wal  | 00000001000000000000001Clast_archived_time | 2026-08-27 16:21:01.793125+00failed_count       | 0last_failed_wal    |last_failed_time   |stats_reset        | 2026-08-27 13:49:10.358943+00$sudo -u postgres psql -c "SHOW archive_command"                                     archive_command------------------------------------------------------------------------------------------ test ! -f /var/lib/postgresql/wal_archive/%f && cp %p /var/lib/postgresql/wal_archive/%f(1 row)$sudo bash -c 'ls /var/lib/postgresql/18/main/pg_wal/archive_status | wc -l'1$sudo du -sh /var/lib/postgresql/18/main/pg_wal129M	/var/lib/postgresql/18/main/pg_wal

   Expected result30 archived, 0 failed, pg_wal at 129 MB.

   Success conditionYou know what a working archiver looks like.

2. 2

   Break the destination and generate some WAL

   The archive_command now points at a directory that does not exist. A reload applies it - and nothing complains. No error, no warning to any client.

   300,000 rows, two full-table updates and two pg_switch_wal() calls produce the WAL. pg_switch_wal() closes the current segment early so it becomes eligible for archiving immediately, which is how you make this reproducible rather than waiting.

   Every one of those statements succeeded. From the application's point of view the database is completely healthy, and it will stay that way right up until the filesystem fills.

   $sudo -u postgres psql -c "ALTER SYSTEM SET archive_command = 'test ! -f /var/lib/postgresql/no_such_dir/%f && cp %p /var/lib/postgresql/no_such_dir/%f'"ALTER SYSTEM$sudo -u postgres psql -c "SELECT pg_reload_conf()" pg_reload_conf---------------- t(1 row)$psql -d appdb -c "CREATE TABLE wal_churn AS SELECT g, repeat('x', 200) AS pad FROM generate_series(1, 300000) g"SELECT 300000$psql -d appdb -c "UPDATE wal_churn SET pad = repeat('y', 200)"UPDATE 300000$sudo -u postgres psql -c "SELECT pg_switch_wal()" pg_switch_wal--------------- 0/27EA4AB8(1 row)$psql -d appdb -c "UPDATE wal_churn SET pad = repeat('z', 200)"UPDATE 300000$sudo -u postgres psql -c "SELECT pg_switch_wal()" pg_switch_wal--------------- 0/2E3CDB48(1 row)

   Expected resultEvery statement succeeds. No error anywhere.

   Success conditionYou have a silent archiving failure to diagnose.

3. 3

   Find it, and read the counter carefully

   pg_stat_archiver has the truth: failed_count 4, last_failed_wal 00000001000000000000001D, and last_archived_wal still stuck at ...1C.

   The log confirms it with archive command failed with exit code 1.

   But failed_count is 4 while 18 .ready files are queued. The counter records attempts, not backlog - the archiver retries the *oldest* segment, backs off when it keeps failing, and never gets to the other 17. Alerting on failed_count alone tells you something is wrong but not how far behind you are.

   The .ready count is the backlog, and it is the number to graph.

   pg_wal has gone from 129 MB to 289 MB, and a manual CHECKPOINT pushes it to 305 MB rather than reducing it. A checkpoint cannot recycle a segment that has not been archived - archive_mode makes archiving a precondition for reuse, so the directory can only grow. max_wal_size is a target the server is no longer able to honour.

   Left alone this ends with a full filesystem and a cluster that will not start.

   $sudo -u postgres psql -x -c "SELECT archived_count, last_archived_wal, failed_count, last_failed_wal, last_failed_time FROM pg_stat_archiver"-[ RECORD 1 ]-----+-----------------------------archived_count    | 30last_archived_wal | 00000001000000000000001Cfailed_count      | 4last_failed_wal   | 00000001000000000000001Dlast_failed_time  | 2026-08-27 16:35:37.42456+00$sudo grep "archive command failed" /var/log/postgresql/postgresql-18-main.log | tail -22026-08-27 16:35:37.423 UTC [51084] LOG:  archive command failed with exit code 12026-08-27 16:35:37.424 UTC [51084] LOG:  archive command failed with exit code 1$sudo bash -c 'ls /var/lib/postgresql/18/main/pg_wal/archive_status/*.ready | wc -l'18$sudo du -sh /var/lib/postgresql/18/main/pg_wal289M	/var/lib/postgresql/18/main/pg_wal$sudo -u postgres psql -c "CHECKPOINT"CHECKPOINT$sudo du -sh /var/lib/postgresql/18/main/pg_wal305M	/var/lib/postgresql/18/main/pg_wal$df -h / | tail -1/dev/mapper/ubuntu--vg-ubuntu--lv   48G  8.7G   37G  20% /

   Expected result4 failures, 18 queued, pg_wal growing through a CHECKPOINT.

   Success conditionYou can distinguish the failure count from the size of the backlog.

4. 4

   Repair it and watch the queue drain

   Restoring archive_command and reloading is the entire fix - no restart, and the backlog is not lost. After twenty seconds:

   archived_count 48, up from 30. last_archived_wal has jumped to ...2E. The .ready count is 0. The archiver worked straight through the queue on its own.

   failed_count is 6, not 4, and it stays at 6 - it is cumulative until reset. Rising is the signal; a non-zero value is only history.

   pg_wal is still 305 MB after another CHECKPOINT. Nothing shrank. PostgreSQL *recycles* WAL segments - it renames them for future use rather than deleting them - so the directory stays at its high-water mark and the space is reused rather than returned. It will settle back toward max_wal_size over later checkpoints. Do not expect free space to reappear the moment archiving recovers, and do not go looking for a second fault when it does not.

   48 files in the archive directory, matching archived_count exactly.

   $sudo -u postgres psql -c "ALTER SYSTEM SET archive_command = 'test ! -f /var/lib/postgresql/wal_archive/%f && cp %p /var/lib/postgresql/wal_archive/%f'"ALTER SYSTEM$sudo -u postgres psql -c "SELECT pg_reload_conf()" pg_reload_conf---------------- t(1 row)$bash -c 'sleep 20; echo waited'waited$sudo -u postgres psql -x -c "SELECT archived_count, last_archived_wal, failed_count, last_failed_wal FROM pg_stat_archiver"-[ RECORD 1 ]-----+-------------------------archived_count    | 48last_archived_wal | 00000001000000000000002Efailed_count      | 6last_failed_wal   | 00000001000000000000001D$sudo bash -c 'ls /var/lib/postgresql/18/main/pg_wal/archive_status/*.ready 2>/dev/null | wc -l'0$sudo -u postgres psql -c "CHECKPOINT"CHECKPOINT$sudo du -sh /var/lib/postgresql/18/main/pg_wal305M	/var/lib/postgresql/18/main/pg_wal$sudo bash -c 'ls /var/lib/postgresql/wal_archive | wc -l'48

   Expected result48 archived, zero queued, pg_wal unchanged at 305 MB.

   Success conditionYou know recovery drains the queue without returning the disk space.

5. 5

   Reset the counter so the next incident is visible

   pg_stat_reset_shared('archiver') zeroes the archiver counters and stamps stats_reset.

   Worth doing after any incident you have finished with. A permanently non-zero failed_count is a broken alert - nobody can tell yesterday's resolved problem from this morning's new one, and after a while nobody looks.

   For monitoring, the three things worth watching are the .ready count, the age of last_archived_time, and free space on the pg_wal filesystem. Any one of them catches this before the disk does.

   $psql -d appdb -c "DROP TABLE wal_churn"DROP TABLE$sudo -u postgres psql -c "SELECT pg_stat_reset_shared('archiver')" pg_stat_reset_shared---------------------- (1 row)$sudo -u postgres psql -x -c "SELECT archived_count, failed_count, stats_reset FROM pg_stat_archiver"-[ RECORD 1 ]--+------------------------------archived_count | 0failed_count   | 0stats_reset    | 2026-08-27 16:36:02.400335+00

   Expected resultCounters at zero with a fresh stats_reset.

   Success conditionYour archiver alert will fire on the next failure, not the last one.

Troubleshooting

• pg_wal is growing and queries are all fine.

  Why: Archiving is failing, or a replication slot is holding segments.

  Fix:pg_stat_archiver and pg_replication_slots. Both stall recycling silently.

• failed_count is small but the disk is filling fast.

  Why: The archiver backs off; the counter is attempts, not backlog.

  Fix:Count .ready files in pg_wal/archive_status.

• Archiving recovered and pg_wal did not shrink.

  Why: Segments are recycled for reuse, not deleted.

  Fix:Nothing. It settles toward max_wal_size over subsequent checkpoints.

• The filesystem filled and the server will not start.

  Why: WAL cannot be written, so recovery cannot proceed either.

  Fix:Free space elsewhere first. Never delete files from pg_wal by hand.

Official sources

• PostgreSQL 18 Documentation - Client Authentication
• PostgreSQL 18 Documentation - Server Start-up Failures
• PostgreSQL 18 Documentation - Continuous Archiving
• PostgreSQL 18 Documentation - Preventing Transaction ID Wraparound Failures
• PostgreSQL 18 Documentation - Reliability and the Write-Ahead Log
• PostgreSQL 18 Documentation - pg_checksums