cd /
du -x | sort -nr
But probably somewhere in /var/log is something you can delete to make space.
Some older Ubuntu/Debian installs had strange problems deleting old kernels so you might have gigabytes in /lib/modules too.
This is a 32bit OS. So i can delete anything with an appended number?
Do i just delete the middle then update grub?
I tried to purge old kernels but the “no space” killed it.
I keep trying to remove kernels and it keeps telling me “no space left on the device”
Yep, those are old logs that have been “rotated”, feel free to delete anything in /var/log with a .gz extension.
I’d remove all of the 4.15.0-X with the exception of the -117 and -118 (as they’re the newest) and keep the 4.4.0-169 just in case there’s a problem with the 4.15 branch.
Did all that and it still says 100% full?
Please post df -h and lsblk -f
I set this up a few years ago so. The arrangement might be questional.
user@computer:~$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 3.8G 0 3.8G 0% /dev
tmpfs 784M 30M 754M 4% /run
/dev/mapper/computer--vg-root 54G 54G 0 100% /
tmpfs 3.9G 4.0K 3.9G 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup
/dev/sda1 720M 133M 550M 20% /boot
tmpfs 3.9G 4.0K 3.9G 1% /tmp
/dev/sdb 1.4T 260G 1.2T 19% /mnt/scratch
/dev/sde 1.9T 1.5T 328G 83% /mnt/data
tmpfs 784M 0 784M 0% /run/user/1000
/home/*user*/.Private 54G 54G 0 100% /home/*user*
user@computer:~$ lsblk -f
NAME FSTYPE LABEL MOUNTPOINT
fd0
sda
├─sda1 ext2 /boot
├─sda2
└─sda5 LVM2_member
├─computer--vg-root ext4 /
└─computer--vg-swap_1 swap
└─cryptswap1 swap [SWAP]
sdb btrfs /mnt/scratch
sdc btrfs
sdd btrfs
sde btrfs /mnt/data
sdf btrfs
All the parts are out of warranty, so technically it doubles as a waste bin.
/dev/mapper/computer–vg-root ---- is a 60gb vertex2 SSD
VERY NICE!!!
Anything in /var/tmp? I’d troll through /var for things you can get rid of or boot into a live usb and copy the whole var to another partition temporarily mount that at /var while you clean it up and the copy back over.
Did you do the
cd /
du -x | sort -n
Here’s a sample of the tail end of what I get on my Fedora NAS:
1724972 ./var/cache/dnf
1726236 ./var/cache
2803984 ./usr/lib
4218884 ./var/log/journal
4218884 ./var/log/journal/69d27b356a94476da859461d3a3bc6fd
4229048 ./var/local
4229048 ./var/local/sfdb
4483164 ./var/log
4624404 ./usr/lib64
4716308 ./usr/share
12129440 ./var
14070212 ./usr
26247852 .
You can see the big targets are /usr and /var, and /usr is usually important stuff you can’t delete. And by “delete” I mean rm -f or rm -rf not apt remove
You can see the logs and cache take up a lot of room on mine. Anything in cache is likely to be fair game to delete.
You don’t need a ton of space. Just free up a few hundred megabytes and then you can use apt again.
2 Likes
It did run it the first time, it errored. But i cleared the old kernels and it managed to run
18624 ./sbin
19564 ./usr/src/linux-headers-4.15.0-117/include/linux
19564 ./usr/src/linux-headers-4.15.0-118/include/linux
19952 ./usr/lib/i386-linux-gnu/samba
21032 ./usr/share/perl/5.26.1
21036 ./usr/share/perl
21332 ./usr/lib/expressvpn
21920 ./usr/lib/python2.7/dist-packages
22004 ./usr/lib/python3.6
22076 ./lib/firmware/intel
23924 ./lib/firmware/netronome
24312 ./lib/firmware/liquidio
24424 ./usr/lib/python3.5
26216 ./usr/lib/git-core
27572 ./usr/lib/python3/dist-packages/twisted
27756 ./lib/modules/4.15.0-117-generic/kernel/drivers/net
27756 ./lib/modules/4.15.0-118-generic/kernel/drivers/net
28568 ./lib/i386-linux-gnu
29004 ./usr/share/man
29992 ./usr/share/locale
31516 ./lib/firmware/amdgpu
31880 ./usr/share/vim/vim80
31900 ./usr/share/vim
35184 ./usr/lib/i386-linux-gnu/dri
39816 ./usr/src/linux-headers-4.15.0-117/include
39816 ./usr/src/linux-headers-4.15.0-118/include
41552 ./usr/sbin
41896 ./usr/share/icons
42716 ./usr/lib/python2.7
48392 ./usr/src/linux-headers-4.15.0-117/arch
48392 ./usr/src/linux-headers-4.15.0-118/arch
48924 ./var/lib/dpkg/info
49800 ./usr/lib/python3/dist-packages
49804 ./usr/lib/python3
50540 ./usr/share/doc
51496 ./var/lib/dpkg
54768 ./usr/lib/snapd
84640 ./usr/lib/gcc/i686-linux-gnu/7
84656 ./usr/lib/gcc/i686-linux-gnu
84660 ./usr/lib/gcc
115644 ./usr/src/linux-headers-4.15.0-117
115644 ./usr/src/linux-headers-4.15.0-118
121452 ./lib/modules/4.15.0-117-generic/kernel/drivers
121456 ./lib/modules/4.15.0-118-generic/kernel/drivers
129148 ./var/lib/apt/lists
129232 ./var/lib/apt
141904 ./usr/include/boost
163608 ./lib/modules/4.15.0-117-generic/kernel
163616 ./lib/modules/4.15.0-118-generic/kernel
169000 ./lib/modules/4.15.0-117-generic
169008 ./lib/modules/4.15.0-118-generic
186456 ./usr/include
204740 ./var/lib
264428 ./usr/src
335540 ./lib/firmware
338848 ./lib/modules
374692 ./usr/bin
393832 ./usr/share
507300 ./var/log/journal/aa42336a97f7d0d122d9442a5dd1eb2d
507304 ./var/log/journal
526764 ./var/log
531104 ./usr/lib/i386-linux-gnu
729312 ./lib
740484 ./var
917300 ./usr/libAnd if it is systemd journald files the right way to clean them is like:
journalctl --vacuum-size=2G
On my system that cut the logs down from 4G (which is my setting in /etc/systemd/journald.conf) to 2G.
To control journald sizes have a line in journald.conf like:
SystemMaxUse=4G
If you don’t use journald at all then you might change its Storage option from “persistent” to “volatile” which keeps it in RAM only. It will still copy it to rsyslogd or whatever logger you actually use.
1 Like
I don’t set any loggers. It is whatever ubuntu has configed by default
Same as last time?
Oh wow, pop some champagne everyone…
6 Likes
That was the intention since the project change of hands. I am surprised they were able to do it this fast though.
1 Like
It’s common to keep personal dot files in a github repository, which I already do and am happy with.
It is increasingly looking like I need something like that for system config as well. I find myself writing scripts to generate other scripts, edit config files, generate unit files, etc which look something like this:
printf '%s\n' \
'[Unit]' \
'Description=First Boot Configuration' \
'After=sysinit.target' \
'' \
'[Service]' \
'Type=oneshot' \
'ExecStart=/usr/local/bin/first-boot' \
'TimeoutSec=600' \
'' \
'[Install]' \
'WantedBy=network.target' >\
'/mnt/etc/systemd/system/first-boot.service'
or this:
sed --in-place \
--expression '/^Example/ s/^/#/' \
--expression '/^#LogSyslog/ s/^#//' \
--expression '/^#LocalSocket[[:space:]]/ s/^#//' \
--expression '/^#LocalSocketMode/ s/^#//' \
--expression '/^#ExcludePath/ s/^#//' \
--expression '/^# Default: scan all/ a\
ExcludePath ^/dev/' \
--expression 's/^#\(MaxDirectoryRecursion\).*/\1 0/' \
--expression 's@^#\(VirusEvent\).*$@\1 '"${ALERT_CMD}"'@' \
--expression '/^#ExitonOOM/ s/^#//' \
'/mnt/etc/clamav/clamd.conf'
Which is fine for a couple things, but it’s getting to the point where I should have these in a repo and just pull them.
Anyway, does anyone do this? What’s the best approach? I’m thinking I might make one big repo for all of it, pull it somewhere into /usr/local/share/ and then either copy, hardlink or symlink files from there ad hoc.
hello, sysadmins, I was wondering something about docker and wanted to ask you guys
if I run mysql on docker, can I connect to it through workbench?
I imagine yes if you bind it to a interface with network access.
Oof
encrypted about 1,200 servers, stole 100 GB of unencrypted files, and deleted 20-30 TB Of backups
3 Likes
Have to migrate a bunch of Google Drive files from a personal drive to a shared drive. The --drive-server-side-across-configs flag in rclone is working well. Large file transfers are going at over 200MB/s (yes, big B). Granted, I’m sure the data isn’t actually moving and it’s just updating metadata, but still…
1 Like