Hands-on Lab·Linux Foundation Certified System Administrator
When a filesystem fills, or refuses
No space left on device has three quite different causes, and only one of them is solved by deleting files. This guide produces all three deliberately - the root reserve, exhausted inodes, and a deleted file still held open - and shows the command that finds each one.
Storage Guide 36 of 38 Advanced
- OSUbuntu 26.04 LTS
- LVM2.03.31 (Ubuntu) / 2.03.36 (AlmaLinux)
- nftables1.1.6 (Ubuntu) / 1.1.5 (AlmaLinux)
- TimeAbout 20 min
ext4 reserves 5% of blocks for root by default and fixes its inode count at mkfs time; xfs allocates inodes dynamically and does not have the second failure mode. lsof, df -i and tune2fs are the same on both families.
- Firewallufw 0.36.2 enabled but reporting inactive / firewalld active
- Network confignetplan + systemd-networkd / NetworkManager 1.56.0
| Server Name | IP Address | OS | Roles | CPU | RAM | HDD |
|---|---|---|---|---|---|---|
| LFCS-A01 | 192.168.0.70 | Ubuntu 26.04 LTS | Primary host - most guides run only here | 2 Core | 4 GB | 50 GB |
This guide includes
Use this because "no space left on device" has three quite different causes. This matters because only one of them is a full disk - and df will never show you the third.
- filling a filesystem, and reading what df means by full
- meeting the 5% you cannot use, and getting it back on a mounted filesystem
- finding what is using the space, with two tools
- finding the deleted file still held open, and reclaiming it without restarting anything
- meeting the third cause, full with 99% of the space free
Before you start
-
Full, and what full means
-
The 5% you cannot use
-
Space back without deleting anything
-
Finding what is using the space
-
The space that df and du disagree about
-
Finding the process holding it
-
Reclaiming it with the process still running
-
Full, with 99% of the space free