2026.09 / Docker storage 058
Coolify “no space left on device”? Clean Docker disk safely
A Coolify build can fail while cloning, unpacking an image layer, installing packages, exporting a BuildKit result, or starting the replacement container with no space left on device. The message identifies a storage failure, but it does not tell you whether the server ran out of filesystem bytes, inodes, Docker’s allocated storage, or space on a different mounted filesystem.
The safe response is not to paste docker system prune -a --volumes into production. First locate the exhausted filesystem, identify the objects using it, protect persistent data and rollback options, and remove the narrowest disposable category. Then retry one deployment and verify its actual release.
Never prune volumes as an emergency reflex. A volume can contain the only durable copy of a database, uploads, mail, or application state.
Confirm which limit was exhausted
Preserve the deployment ID, failing command, timestamp, and exact error first. On the deployment server, start with read-only measurements:
df -h
df -i
docker info --format '{{json .DockerRootDir}}'
docker system df
docker system df -v
df -h reports space by bytes. df -i reports inode consumption: a filesystem containing huge numbers of tiny cache or log files can reject new files even when the byte summary appears to have room. DockerRootDir tells you where Docker stores managed data; do not assume it is on the root filesystem. docker system df divides Docker usage into images, containers, local volumes, and build cache, while the verbose form helps attribute that usage.
| Evidence | Likely constraint | Next action |
|---|---|---|
| Filesystem at or near 100% bytes | Images, layers, logs, backups, artifacts, or application data | Attribute large categories on that filesystem |
| Inodes at or near 100% | Very many small files | Find the responsible directory or workload; deleting one large image may not help |
| Large reclaimable build cache | Old build intermediates | Prune build cache with a retention policy |
| Large stopped-container footprint | Exited replacements or one-off jobs | Confirm they are not needed, then remove those containers |
| Large volume footprint | Potentially durable application data | Map every volume to an owner; do not bulk-prune |
| Docker usage is modest | Host logs, backups, package caches, or data outside Docker | Inspect that filesystem outside Docker |
Map storage to running applications before deleting anything
List containers, their images and mounts. The commands below expose names and paths, so keep the output private if it contains internal project details:
docker ps -a --format 'table {{.Names}}\t{{.Image}}\t{{.Status}}\t{{.Size}}'
docker volume ls
docker inspect APP_CONTAINER --format '{{json .Mounts}}'
docker inspect APP_CONTAINER --format '{{.Image}}'
For each candidate, answer four questions: Is a running container using it? Does it hold persistent state? Is it required for rollback? Is there a tested backup? An old image may be unused by a running container yet still be your fastest rollback. An unattached volume may be a deliberately retained database. “Unused” in Docker’s graph is not the same as “worthless” to the operator.
If a named volume might contain state, inspect its labels and owner, compare it with the application’s Coolify storage configuration, and verify a restorable backup before removal. Do not enter Docker’s data directory and delete overlay files manually; bypassing Docker’s metadata can corrupt the storage graph.
Reclaim the narrowest disposable category
Docker provides separate prune commands so cleanup can match the diagnosis. Review each command’s candidate set on your installed Docker version before confirming it.
1. Build cache
When failed and historical builds dominate usage, cache is usually a narrower target than all unused images:
# Preview attribution first
docker system df -v
# Example retention policy: remove eligible cache older than 168 hours
docker builder prune --filter 'until=168h'
The confirmation prompt shows the operation is destructive. Choose a retention window that keeps recent cache useful; do not copy 168h blindly if your release or rollback cadence needs longer.
2. Stopped containers
Identify exited containers, verify that no one-off job or forensic evidence is still needed, and remove specific names first:
docker ps -a --filter status=exited
docker rm VERIFIED_STOPPED_CONTAINER
3. Images
Prefer removing a verified obsolete image by ID. Docker’s image prune without -a limits itself to dangling images; adding -a broadens it to images with no associated container. That broader set can include useful rollback releases.
docker image prune
docker image rm VERIFIED_OBSOLETE_IMAGE_ID
4. Volumes
Do not use volume prune during first-response cleanup. If a specific orphaned volume is later proven disposable, back up any uncertain contents, record why it is orphaned, and remove that one volume explicitly in a maintenance window. Database recovery is a separate operation from freeing build space.
What docker system prune actually changes
Docker documents that docker system prune removes stopped containers, unused networks, dangling images, and unused build cache. With -a, it removes all images not used by at least one container. Volumes are not removed by default; a volume flag expands the blast radius.
On a single-purpose disposable build host, broad cleanup may be acceptable after review. On a Coolify server running several applications, databases, and retained releases, category-specific cleanup is easier to reason about and verify. Capture before-and-after docker system df and df -h output so you know what reclaimed space and whether enough headroom exists for the next build.
If logs—not images—filled the disk
Large container logs require a different fix. First attribute them through Docker’s effective log configuration and container metadata rather than deleting random files under Docker’s root:
docker info --format '{{json .LoggingDriver}}'
docker inspect APP_CONTAINER --format '{{json .HostConfig.LogConfig}}'
journalctl --disk-usage
If the json-file driver is unbounded, add an intentional rotation policy for future containers, then recreate them through the normal Coolify deployment path. Existing containers do not inherit daemon defaults retroactively. If systemd journal usage is responsible, configure retention appropriate to incident investigation and audit needs. Truncating active log files by path can race the writer and destroys evidence; treat it as a last-resort incident action, not routine maintenance.
Retry the deployment only after restoring headroom
- Re-run
df -h,df -i, anddocker system df. - Confirm Docker and Coolify are healthy and existing public applications still respond.
- Confirm important stateful containers and their volumes remain attached.
- Trigger one deployment; do not stack duplicate retries.
- Follow its deployment identifier until it reaches a terminal state.
- Match the deployed source or image to the intended Git commit.
- Assert unique changed content or an application-specific release marker, not only HTTP 200.
If the build still fails after space is available, return to its exact new log. The original full disk may have left a partial package cache, interrupted Git checkout, or damaged temporary build context. A force rebuild can be justified once you have enough headroom, but it should not substitute for finding why storage keeps growing.
Prevent the next full disk
Coolify provides automated Docker cleanup settings at server level. Its documentation describes scheduled cleanup, a disk-usage threshold, and retention controls. Configure these in the Coolify dashboard for the actual server rather than adding an unmanaged cron job. Run cleanup during quieter periods if the server is resource-constrained, and review cleanup logs.
- Alert before the filesystem is critical; leave room for one normal image build and rollback.
- Monitor both byte and inode utilisation on the Docker filesystem.
- Set a deliberate cleanup frequency and minimum age for retained resources.
- Rotate container and host logs with enough retention for diagnosis.
- Expire obsolete local backups only after confirming off-host, restorable copies.
- Move builds to a separate build server or CI if build layers routinely crowd application data.
- Keep databases, uploads, and other state on explicit persistent storage with tested backups.
Compact recovery checklist
- The exact failed deployment and error line are preserved.
- The exhausted filesystem and Docker root are identified.
- Both bytes and inodes have been measured.
- Docker usage has been split into cache, images, containers, and volumes.
- State, backups, and rollback images are mapped before cleanup.
- The narrowest disposable category is removed; volumes are not bulk-pruned.
- Before-and-after free space proves sufficient headroom.
- Existing applications and state are intact.
- One retry reaches a terminal state and unique live content proves the release.
- Coolify-managed cleanup, monitoring, and log rotation reduce recurrence.
Authoritative references: Coolify’s official automated Docker cleanup guide covers server-level schedules, thresholds, retention, and cleanup logs. Docker’s official documentation explains docker system df, object-specific prune commands, and the scope of docker system prune.