Blog → Post


(TIP) Recovering Space
Things to check when trying to free-up disk space
by @admin, may 12, 2021, 10:15am utc

(at least from our point-of-view) Linux is better than Windows and MacOS in nearly every way.
That said, Linux does share one problem of those other operating systems.

Linux can run out of space too...

Using the following 97%  full SSD as an example, we will show a few places to look for disk space. We will add other suggestions over time and this post will become a sticky so that you can easily find it again.

# df -h Filesystem Size Used Avail Use% Mounted on /dev/sda2 423G 387G 15G  97% / /dev/sdc2 4.6T 4.3T 335G 93% /media/fred/5TB-Set1 /dev/sdd2 4.6T 3.9T 712G 85% /media/fred/5TB-Set2

1) Clean up the main log file back-ups:
As a sudo user, take a look in the /var/log folder.

You can remove:

*.1 files, which are the immediately prior versions of log files.
For example when the mail.log file [holds info from the mail system and info about individual sent & received emails] fills up; the file is rolled over to mail.log.1 and a new "mail.log" is created.

You'll see a lot of these /var/log/*.1 files; and they can be removed .

*.gz (GNU Zip) files. In a rotation similar to what is detailed above, *.gz files are compressed versions of *.1 files. For example, if a mail.log.1 was already in existence above, it first would of been compressed to a mail.log.2.gz file. Only then would the mail.log be rolled over to mail.log.1.

Similarly, if there had been a pre-existing mail.log.2.gz file, it would of first been moved to mail.log.3.gz, and so on.

All of these log files and back-ups are there to provide an administrator a good deal of info to have on hand, in the case of a problem. But they do take up a lot of space.

You can also delete  all of the /var/log/*.gz files too.

2) Other email logs:
If you are running a POP3/IMAP email tools, like Dovecot, they can generate pretty big log files as well.

On our system above the dovecot.log was 549MB.
Back the file up and type # echo . > dovecot.log to empty it out, while still retaining all other file permissions and ownership.

3) fail2ban's database:
If you are using fail2ban (and we think that you should be) to limit hacking, spamming, log-in, etc attacks; then monitoring the size of it's file should be on your check list.

/var/lib/fail2ban/fail2ban.sqlite3 3.1 GiB
A ".sqlite3" file is a database file. As as byproduct of cutting down or removing fail2ban.sqlite3, and recovering its' space, you will also find that subsequent $ ufw inserts ... will run much faster; as will $ ufw status numbered, etc.

"Much faster" in the case of this 3.1GB example meant "a few seconds" to run a UFW command, instead of a minute+. When you are trying to manually add a bunch of IPs to block, minutes instead of seconds really matters.

Since Fail2Ban works in part by catching evil IPs based on how quickly attacks come from the same IP; cutting down the database file size will allow Fail2Ban to process faster and thereby catch more too.

Restart Fail2Ban to recreate the .db file:
# sudo service fail2ban restart

If you completely remove the file; you will see it recreated in a second or so; as the next new attacks are caught by Fail2Ban.

4) Archive files:
Whenever you update Linux, new versions of applications, docs and tools most often arrive in the form of packages (.deb files).

The prior version of these packages are retained in the /var/cache/apt/archives folder in case you need to revert back — or simply for historical purposes, so that you know what has been installed in the past.

There are ways to automatically clear these out, but you can do so manually as well.

As you can see from this partial list of packages for just the Linux kernel, these files can really add up over time.

   
   46.1 MiB [######### ]  linux-image-4.19.0-16-amd64_4.19.181-1_amd64.deb   
   46.1 MiB [######### ]  linux-image-4.19.0-14-amd64_4.19.171-2_amd64.deb   
   46.1 MiB [######### ]  linux-image-4.19.0-13-amd64_4.19.160-2_amd64.deb   
   46.0 MiB [######### ]  linux-image-4.19.0-12-amd64_4.19.152-1_amd64.deb   
   46.0 MiB [######### ]  linux-image-4.19.0-11-amd64_4.19.146-1_amd64.deb   
   46.0 MiB [######### ]  linux-image-4.19.0-10-amd64_4.19.132-1_amd64.deb   
   46.0 MiB [######### ]  linux-image-4.19.0-9-amd64_4.19.118-2+deb10u1_amd64.deb   
   45.9 MiB [######### ]  linux-image-4.19.0-9-amd64_4.19.118-2_amd64.deb   
   45.9 MiB [######### ]  linux-image-4.19.0-8-amd64_4.19.98-1+deb10u1_amd64.deb   
   45.8 MiB [######### ]  linux-image-4.19.0-6-amd64_4.19.67-2+deb10u2_amd64.deb   
   45.4 MiB [######### ]  linux-image-4.19.0-5-amd64_4.19.37-5+deb10u2_amd64.deb   
   37.3 MiB [########  ]  linux-image-4.9.0-8-amd64_4.9.130-2_amd64.deb   
   32.9 MiB [#######   ]  linux-image-3.16.0-7-amd64_3.16.59-1_amd64.deb   
   32.4 MiB [#######   ]  linux-image-3.16.0-4-amd64_3.16.51-3_amd64.deb           
    

5) Spelunk!
Using QDirStat, ncdu or related tools, take a look at your home folder to see what sort of large, potentially deletable, files show up.

By default, ncdu will sort folder (and files) into largest to smallest order; which often makes the spelunking approach profitable as far as saving space goes.


Additional References

Scripts to reduce the Fail2Ban db (Advanced Users!)

https://gist.github.com/tschifftner/ac09d17e8878ec89d930387050b4224b

#!/usr/bin/env bash   
# Tobias Schifftner, @tschiffnter   
#   
# Usage:   
#    bash fail2ban-cleanup.php <fail2ban.sqlite3>   
   
FILE=${1:-"/var/lib/fail2ban/fail2ban.sqlite3"}   
   
[ -f "$FILE" ] || { echo "$FILE not found"; exit 1; }   
   
function sql() {   
    $(which sqlite3) "$FILE" "$@"   
}   
   
sql "DELETE FROM bans WHERE timeofban < strftime('%s', 'now', '-7 days');"   
sql "VACUUM;"      


This script flat out deletes the .db and restarts fail2ban.
https://gist.github.com/mitchellkrogza/bfcb5c14b4d9d2d2856f85f50b030186

Notes:
  By the way, you should NEVER let a SSD get to 97% full. Try to maintain 20-25% of free space, so that the same physical area is not continually being overwritten -- which promotes SSD failures.

  As with ANY other changes to your system, we strongly recommend that you back up your system and these files, before happily deleting swaths of them. Trust us -- sooner or later, you will be very happy that you did.

tags: All users, Tips
Footer done in Inkscape