2026.10 / backup tooling 066
Coolify PostgreSQL pg_dump server version mismatch
A scheduled Coolify PostgreSQL backup can fail immediately with an error similar to server version mismatch and aborting because of server version mismatch. The database can remain healthy, applications can keep reading and writing, and yesterday’s backup can still exist. The failed run means the backup client refused to create today’s archive.
The usual cause is precise: the pg_dump executable used by the backup job is from an older PostgreSQL major release than the server it is trying to dump. Renaming the output, retrying the same job, or restarting PostgreSQL does not repair that tool boundary.
Do not downgrade the live database, delete its volume, or treat an old archive as a new recovery point. Preserve the last known-good backup, identify both versions, then align the dump client and prove a fresh restore.
Why PostgreSQL stops the dump
PostgreSQL documents an intentionally asymmetric compatibility rule. A newer pg_dump can dump supported older servers. An older pg_dump will not dump a newer server major version; it refuses rather than risk producing an invalid archive. Dump output is also not guaranteed to load into an older server major version.
| Client and server | Expected result | Safe direction |
|---|---|---|
pg_dump 15 → PostgreSQL 17 | Refused as a newer server | Use a PostgreSQL 17 or newer supported client |
pg_dump 17 → PostgreSQL 15 | Normally supported | Still restore-test the produced archive |
| PostgreSQL 17 dump → PostgreSQL 15 target | Not a supported downgrade promise | Restore to the same or a newer tested major version |
The major number matters for this diagnosis. Patch releases still matter for security and bug fixes, but a client at 16.x talking to a server at 17.x is the mismatch that triggers this refusal.
1. Capture the exact failed execution
Open the PostgreSQL resource in Coolify, select Backups, then open the failed execution. Record the execution time, selected database, backup scope, destination, terminal status, and the two version lines from the error. Keep credentials, connection strings, signed object URLs, internal hostnames, and real addresses out of tickets or screenshots.
Do not infer versions from a resource label such as “Postgres latest.” Tags can move, a service definition can override an image, and the backup process may use a client outside the database container. The useful evidence is the actual running server version and the actual executable used by the failed job.
2. Ask the server for its real version
Use Coolify’s database terminal or another authorised private connection. The numeric setting is easier to compare programmatically than a decorated version string:
SHOW server_version;
SHOW server_version_num;
SELECT current_setting('server_version'),
current_setting('server_version_num');
A value such as 170006 belongs to major version 17. Record the complete value for the incident, but compare the correct major-version component. Do not expose PostgreSQL publicly merely to run this query.
3. Identify the pg_dump that actually failed
The execution log may print the client version or command environment. If you have authorised host access, inspect the relevant backup context rather than whichever pg_dump happens to be first in your interactive shell:
pg_dump --version
# When testing a known PostgreSQL client image:
docker run --rm postgres:SERVER_MAJOR-alpine pg_dump --version
Replace SERVER_MAJOR with the verified major release, not an unpinned latest tag. Never place the password directly on the command line. Use Coolify’s managed backup configuration or an ephemeral secret mechanism appropriate to the controlled environment.
If your shell reports the same major as the server but Coolify still reports an older one, you have measured the wrong client. Follow the failed execution’s container, image, or command path until the version output describes the binary Coolify invoked.
4. Check how the database was upgraded
A mismatch often appears just after a database image change. Separate three different events:
- Client-only drift: the database server stayed on its existing major, but the backup helper or host client is older.
- Completed major upgrade: data was migrated using a supported method, the server is healthy on the new major, but backup tooling was not advanced with it.
- Unsafe image-tag edit: the container tag was changed across PostgreSQL majors without a valid data migration. A running container or a version-mismatch message does not prove the data upgrade is correct.
Do not solve the third case by focusing only on pg_dump. Preserve the volume and logs, stop risky writes if data integrity is uncertain, and follow a PostgreSQL major-upgrade or restore procedure appropriate to the source and target versions.
5. Align the backup client without changing production data
For a normal Coolify standalone PostgreSQL resource, first confirm the resource image and Coolify version are supported and current. Save the backup schedule after correcting its database settings, then run one manual execution. If the failure belongs to a service-defined PostgreSQL component, make sure the service exposes the database variables Coolify’s backup workflow reads and that its effective image supplies or is paired with a compatible client.
For a controlled manual fallback, run pg_dump from a client image whose major version is equal to or newer than the verified server major. Keep it on the same private Docker network, write the archive to an authorised protected path, and avoid printing secrets:
docker run --rm \
--network PRIVATE_DATABASE_NETWORK \
-e PGPASSWORD \
-v /protected/backup-output:/backup \
postgres:SERVER_MAJOR-alpine \
pg_dump --format=custom --no-acl --no-owner \
--host DATABASE_SERVICE --username DATABASE_USER \
--file /backup/database.dump DATABASE_NAME
Every capitalised value above is a placeholder. Supplying PGPASSWORD securely is environment-specific; do not paste it into shell history, source control, job logs, or public documentation. A manual archive is an incident fallback, not a substitute for repairing Coolify’s scheduled path.
6. Match the archive format to the restore tool
Coolify uses a custom-format pg_dump archive for selected PostgreSQL databases. Custom archives are read by pg_restore, not piped into psql. When Backup All Databases is enabled, Coolify uses pg_dumpall and gzip instead, so confirm the scope before choosing a restore command.
pg_restore --version
pg_restore --list /protected/backup-output/database.dump
An “unsupported version in file header” message is a different client/archive compatibility symptom: the available pg_restore is too old to understand the archive. Use a compatible newer restore client. Changing the filename extension does not change the archive format.
7. Prove the replacement backup
- Trigger one Coolify backup manually and wait for its terminal result.
- Confirm the archive is new, non-empty, and associated with the intended database and execution.
- Confirm the expected local or S3-compatible copy exists under the configured retention policy.
- List the custom archive with a compatible
pg_restore. - Restore it into a private, disposable PostgreSQL target—never over production.
- Check expected schemas, extensions, constraints, representative counts, sequence positions, and record recency.
- Point an isolated application instance at the restored target with mail, payments, webhooks, workers, and scheduled side effects disabled.
- Verify representative database-backed reads and one disposable write/read-back where appropriate.
- Delete temporary database copies after retaining non-sensitive pass/fail evidence.
A green retry only proves that the new backup command exited successfully. The separate Coolify PostgreSQL restore-drill guide covers the stronger evidence needed before relying on the archive.
Common wrong fixes
- Restarting PostgreSQL: a restart does not upgrade the dump executable.
- Using the host’s default client blindly: it may not be the binary used by Coolify and may introduce another mismatch.
- Installing an arbitrary “latest” client: unpinned tooling makes the next incident harder to reproduce.
- Downgrading the server image: PostgreSQL data directories are not safely downgraded by changing an image tag.
- Deleting the persistent volume: backup tooling failure is not evidence that the database files should be discarded.
- Feeding a custom archive to
psql: usepg_restorefor Coolify’s selected-database custom format. - Calling an old archive “today’s backup”: preserve it, but report the new recovery-point gap honestly.
- Stopping at file creation: only an isolated restore proves useful recoverability.
Compact incident order
- Preserve the last known-good archive and the failed execution log.
- Read
server_version_numfrom the actual database. - Read
pg_dump --versioninside the actual failed backup context. - Confirm whether a recent server upgrade or image-tag change caused the drift.
- Align Coolify’s backup client with the server major without modifying the data volume.
- Run one fresh backup and inspect its terminal execution.
- Retrieve, list, restore, and application-test the new archive in isolation.
- Record the client/server pairing in the upgrade and recovery runbooks.
The durable fix is not merely “install a newer pg_dump.” It is to make PostgreSQL server upgrades, backup-client versions, archive formats, restore clients, and restore drills one controlled compatibility chain. When the database major changes, backup and recovery tooling must move—and be tested—with it.
Authoritative references: PostgreSQL documents the pg_dump version rules and archive formats and the pg_restore workflow. Coolify documents its database backup scope, variables, retention, and S3 flow, its PostgreSQL backup formats, and its database restore workflow.