Hi,
I am running a recent urbackup server on NAS in docker.
Client is ubuntu 22.04, urbackup is configured to use LVM snapshots (not dattobd),
and I tried (unsuccessfully) to do image backups yesterday.
Today, client has booted into a snapshot, and my /boot has no space left because of a large 5Gb /boot/.overlay* file.
=> I am missing the big picture what happened (a snapshot was created, so that the filesystem could be imaged, but due to interruption/failure it was not cleaned up so I am stuck in the snapshot ?)
=> I need to 1) revert grub back to booting from the plain LVM volume, 2) discard the snapshot and 3) delete the .overlay file. I hope I won’t have to do that manually, but what is the correct way ?
=> How to prevent that from happening in the future ? Not go for image backups at all ?
Thanks in advance,
yours,
Steffen
Hi, issue remains open. I’ll happily accept recommendation in what order I need to manually remove the snapshots in both / and /boot on the client.
The / snapshot is mounted: /dev/mapper/root-98d1f8b1f435 on / type xfs
Although I can’t find it in lvdisplay (see below). Or did I look the wrong way?
And the snapshot(s) appears under /dev/mapper/root* (see below)
and I see /.overlay_* files (see below).
The /boot snapshot is different, it appears not mounted: /dev/mapper/ssd-boot on /boot type ext4
It is not shown in lvdisplay (see below), and no snapshots appear under /dev/mapper/*boot*,
yet I do see /boot/.overlay_* files (see below).
Thanks in advance,
yours,
Steffen
root@happy:/# lvdisplay /dev/mapper/ssd-root
--- Logical volume ---
LV Path /dev/ssd/root
LV Name root
VG Name ssd
LV UUID 1ckH2u-JPs5-wOD6-8p4O-gHfB-AJS5-HrzooX
LV Write Access read/write
LV Creation host, time ubuntu, 2019-03-08 21:04:53 +0100
LV Status available
# open 2
LV Size 36,00 GiB
Current LE 9216
Segments 2
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:0
root@happy:/# ls -la /dev/mapper/root*
lrwxrwxrwx 1 root root 7 Jun 30 14:40 /dev/mapper/root-98d1f8b1f435 -> ../dm-8
lrwxrwxrwx 1 root root 8 Jun 30 14:44 /dev/mapper/root-98d1f8b1f435-a31725acca86421d-clone -> ../dm-20
lrwxrwxrwx 1 root root 8 Jun 30 14:44 /dev/mapper/root-98d1f8b1f435-a31725acca86421d-clone-era -> ../dm-17
lrwxrwxrwx 1 root root 8 Jun 30 14:44 /dev/mapper/root-98d1f8b1f435-a31725acca86421d-era-metadata -> ../dm-18
lrwxrwxrwx 1 root root 8 Jun 30 14:44 /dev/mapper/root-98d1f8b1f435-a31725acca86421d-era-metadata-access -> ../dm-19
root@happy:/# ls -la /.overlay_2fefd007-3e48-4162-b2c6-45ccdda22f37_*
-rw-r--r-- 1 root root 0 Jun 24 18:42 /.overlay_2fefd007-3e48-4162-b2c6-45ccdda22f37_4c90e6bd9b05f65a288e69575981efebf52d33041ea28427
-rw-r--r-- 1 root root 0 Jun 23 08:41 /.overlay_2fefd007-3e48-4162-b2c6-45ccdda22f37_cbe98c116d0387f513be218bb311231a31d265054ab00959
-rw-r--r-- 1 root root 5368709120 Jun 22 19:49 /.overlay_2fefd007-3e48-4162-b2c6-45ccdda22f37_cf47acbf27899af308e33be25fbfd54f929476a4ffe5e120
-rw-r--r-- 1 root root 104857600 Jun 22 22:22 /.overlay_2fefd007-3e48-4162-b2c6-45ccdda22f37_cf47acbf27899af308e33be25fbfd54f929476a4ffe5e120-wsnap
-rw-r--r-- 1 root root 0 Jun 30 14:44 /.overlay_2fefd007-3e48-4162-b2c6-45ccdda22f37_eca2bad1b692ede4f5585f4c1c8a4b04f6896ded872e192c
And output for /boot:
root@happy:/# lvdisplay /dev/mapper/ssd-boot
--- Logical volume ---
LV Path /dev/ssd/boot
LV Name boot
VG Name ssd
LV UUID 8KrJbF-e4KI-1pOL-HYvu-Z4ki-Uqqr-0KkCNl
LV Write Access read/write
LV Creation host, time ubuntu, 2019-03-08 21:09:56 +0100
LV Status available
# open 1
LV Size 5,00 GiB
Current LE 1280
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:3
root@happy:/# ls -l /dev/mapper/*boot*
lrwxrwxrwx 1 root root 7 Jun 30 14:40 /dev/mapper/ssd-boot -> ../dm-3
root@happy:/# ls -la /boot/.overlay_*
-rw-r--r-- 1 root root 89726976 Jun 23 20:07 /boot/.overlay_2fefd007-3e48-4162-b2c6-45ccdda22f37_6ea1e9df3cb55e34bc282f317e9d39e29f9b14ed6ed66ec9
-rw-r--r-- 1 root root 4980699136 Jun 22 19:46 /boot/.overlay_2fefd007-3e48-4162-b2c6-45ccdda22f37_cf2796d29caa4b25473a72107d2d2391b159a1d1c8b88892
-rw-r--r-- 1 root root 0 Jun 24 18:54 /boot/.overlay_2fefd007-3e48-4162-b2c6-45ccdda22f37_f1b1640af29319834158c96a97c88a80e77d3c0dbb475319
Hi, still need help how to proceed to remove this snapshot in the right order.
Any suggestions or links to docs I might’ve missed appreciated.
Yours, Steffen