All Your GUCs in a Row: archive_command

Last post’s deferred party has arrived. archive_command is how WAL segments leave the primary and travel somewhere durable — the substrate on top of which point-in-time recovery, warm standbys, and every serious backup tool are built. It is also, in the classic formulation, a shell command written on one line of postgresql.conf. This is as terrifying as it sounds.

The mechanic is straightforward. When PostgreSQL fills a WAL segment (default 16MB), the archiver process invokes the string in archive_command with two placeholders: %p expands to the relative path of the file to archive, and %f to the bare filename. A zero exit status tells the server “archived, you may recycle it.” A nonzero status tells the server “not archived, I’ll ask again later.” Context is sighup. Default is empty.

The canonical cp example you will find in a hundred tutorials:

1
archive_command = 'test ! -f /mnt/archive/%f && cp %p /mnt/archive/%f'

The test ! -f half is not optional. It is the difference between “safe” and “silently overwrites an existing archive file on the next timeline and destroys your ability to restore across it.” Stop.

Here is the trap. That one-line shell command is a database durability guarantee. If it lies about success, you lose data. If it lies about failure, pg_wal/ grows until the primary runs out of disk. If it hangs, WAL archiving stalls and pg_wal/ grows until the primary runs out of disk. Writing a correct one is harder than it looks:

• Handle partial writes. cp to a network filesystem can return success with a truncated file on the other end. Use a temp-file-plus-rename pattern, or a tool that fsyncs.
• Return nonzero on failure, always. Pipelines with | can mask errors; missing set -o pipefail is a classic footgun.
• Do not overwrite. Treat every destination filename as write-once. Timelines diverge; identical filenames can carry different contents.
• Be fast enough. If archiving takes longer than WAL generation, you have unbounded disk growth in your future.

The PostgreSQL docs acknowledge all of this in passing, then provide the cp example anyway, on the theory that you will replace it with something real. Most people never do. This is why we have backup tools.

Recommendation: Do not write your own. Use pgBackRest, Barman, or WAL-G — they handle the partial-write, idempotency, parallelism, and retry cases that shell scripts get wrong, and on PostgreSQL 15+ they can register as an archive_library and bypass archive_command entirely. archive_command stays empty in your postgresql.conf; the tool owns archiving end-to-end. And, per last time, let the same tool handle retention too.