No Space left, but there is plenty of space

t_b · March 28, 2018, 6:49pm

I installed UrBackup Server on a Raspberry Pi 3 Model B+ (from source, as described).

The client is a Windows 10 PC, it was backing up fine before (older version of the UrBackup Server, older Raspberry Pi). I updated the client, now it is backing up to the new server as expected.

However, the server status page displays this error message:

There is currently not enough free space in the backup folder. UrBackup tried to delete old image and file backups but is now not allowed to delete more. Please change the settings to store less backups or increase the storage amount to allow UrBackup to continue to perform backups

When I click Ok. Reset this error, the error message disapears. I noticed that the backup process (full image backup) is still (or again) running (according to the activity tab).

The error message reappears after a while.

I am backing up a 335.21 GB Windows partition to an 4 TB external usb drive. Currently, around 70 GB are already backed up. The windows 10 machine is the only PC backing up to the UrBackup server.

The live log (accessed via the activity tab) contains several errors:
|28.03.18 19:15 |INFO |Not enough free space. Cleaning up.|
|—|---|—|
|28.03.18 19:15 |ERROR |Could not free space for image. NOT ENOUGH FREE SPACE.|
|28.03.18 19:17 |INFO |Not enough free space. Cleaning up.|
|28.03.18 19:17 |ERROR |Could not free space for image. NOT ENOUGH FREE SPACE.|
|28.03.18 19:33 |INFO |Not enough free space. Cleaning up.|
|28.03.18 19:33 |ERROR |Could not free space for image. NOT ENOUGH FREE SPACE.|
|28.03.18 19:35 |INFO |Not enough free space. Cleaning up.|
|28.03.18 19:35 |ERROR |Could not free space for image. NOT ENOUGH FREE SPACE.|
|28.03.18 19:53 |INFO |Not enough free space. Cleaning up.|
|28.03.18 19:53 |ERROR |Could not free space for image. NOT ENOUGH FREE SPACE.|
|28.03.18 19:55 |INFO |Not enough free space. Cleaning up.|
|28.03.18 19:55 |ERROR |Could not free space for image. NOT ENOUGH FREE SPACE.|
|28.03.18 20:11 |INFO |Not enough free space. Cleaning up.|
|28.03.18 20:11 |ERROR |Could not free space for image. NOT ENOUGH FREE SPACE.|
|28.03.18 20:13 |INFO |Not enough free space. Cleaning up.|
|28.03.18 20:13 |ERROR |Could not free space for image. NOT ENOUGH FREE SPACE.|
|28.03.18 20:29 |INFO |Not enough free space. Cleaning up.|
|28.03.18 20:29 |ERROR |Could not free space for image. NOT ENOUGH FREE SPACE.|
|28.03.18 20:31 |INFO |Not enough free space. Cleaning up.|
|28.03.18 20:31 |ERROR |Could not free space for image. NOT ENOUGH FREE SPACE.|
|28.03.18 20:47 |INFO |Not enough free space. Cleaning up.|
|28.03.18 20:47 |ERROR |Could not free space for image. NOT ENOUGH FREE SPACE.|

Do you have any ideas? Do you need more logs?
Thanks in advance!

orogor · March 28, 2018, 7:06pm

There s a soft quota option, so maybe it would go over it?
The default is maybe 5 or 10% So it would need 200-400Gb free.

Maybe double check the backup path/partition size/free space?

Or maybe lower the min backup for incr/full. Or delete archives/edit archive settings, theses dont count against the min/max values.

t_b · March 28, 2018, 7:27pm

The soft quota option shouldn’t be an issue, the global soft quota option is set to the default value of 95%.

I also double checked the partion using df -h, there is around 3.6 TB free space, only 2% are in use.

So I guess that space on the USB hard disk is not an issue here. Also, the backup process is continuesly running/resumes after a while.

Any other ideas?

orogor · March 28, 2018, 7:51pm

Missmount issue.
Try df -h /your/storage/folder (as set in urbackup) even if the actual mountpoint is /your/

t_b · March 28, 2018, 8:12pm

df -h /mnt/elements/ as well as df -h /mnt/elements report the same values:

Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1       3.7T   83G  3.6T   3% /mnt/elements

Backup storage path is /mnt/elements/urbackup

orogor · March 28, 2018, 9:41pm

“Live log from the activity side” … i think it s the client logs, is there any free space on the client?

t_b · March 29, 2018, 8:29am

That might be an idea …
The client has two partitions, labelled C: and E:

C: is the system partition that is currently beeing imaged. Windows 10 reports 100 GB free space while the total volume is 464 GB. So I guess, there should be enough space.

E: is a data partition, that was also set to be imaged. On this partition Windows reports 20 GB free space, total volume 465 GB. So there might be an issue. However, this parition is currently not beeing imaged!
Just to be sure, I changed the settings, so that this partition will not be imaged.

Update: The errors keep coming, both in the status tab and the log from the activity tab.

However, the log in the log tab shows only two errors (from yesterday, right after starting the UrBackup Server):

Errors
28.03.18 15:16
Backing up System Reserved (SYSVOL) partition failed. Image backup failed

Errors
28.03.18 15:16
Backup failed

orogor · March 29, 2018, 9:44am

Maybe the snapshot thing of windows. (shadowcopy/vss/system protection)

If you right click on drive then property, maybe it has the tab "snapshot/shadow copy
Or , you right click on the drive and there s a “configure snapshot/configure shadow copy” option
Both are the same thing, it depends on the windows version.

And in the parameter are, there should be a maximum size. Its either on the tab itself or there s a configure button.

t_b · March 29, 2018, 11:34am

Found it under Control Panel -> system -> system protection tab.

Protection is turned on for drive C, turned of for drive E.

Current Usage (drive C): 15 GB
Max Usage (drive C): 10% (46 GB)

Do I need to change something there?

orogor · March 29, 2018, 1:50pm

It should show the current used space for the vss somewhere around that.

If it s full you can disable/enable it and it will be 100% free , but you ll lose the snapshot that exists (won t be able to use windows built-in recovery).
You can also increase max size, but you ll get less usable storage.

t_b · March 29, 2018, 1:58pm

Two points:

If I understand you correctly, UrBackup somehow uses the free space of the protection (in my case 46 GB - 15 GB = 31 GB).
So is this 31 GB to small for UrBackup?
The 15 GB current usage of the system protection did not change during the last hours. When UrBackup is using that space, this should be changing, shouldn’t it?

orogor · March 29, 2018, 2:32pm

The space needed by urbackup should is the shadow copy one.
Typically, it’s configured by the setting that you access from the drive letter, but maybe on windows 10 you can t do that , and the setting is only availlable in server versions.
.
But as i understand it, the recovery space use the same type of mechanism and disk space that the shadow copy space.
I am not expert enought to be 100% sureof that and it’s kinda hard for me to double check that on windows 10, i use mostly linux servers/clients, and windows servers.
Yes, urbackup need some of that place, because thats how snapshots works.
All the file modifications made during the backups are written to this space.
Then when the backup is finished this space should be freed (kinda hard to explain).

t_b · March 29, 2018, 7:44pm

I think it has nothing to do with the shadow copy/snapshot thing, because of two points:

I adjusted the maximum allowed usage to 20% (92 GB). This is double the size it was before, however the same errors reappear.
The client finished the image backup and is now doing the file backup. The error on the status page reappears, no more errors in the log.

uroni · March 30, 2018, 3:14pm

Problem is definitely on the server and not the client. For some reason it gets wrong free space.

Linux kernel version?
Os version (debian)?
I assume this is still a 32bit kernel/user space even though Pi3 B+ has an 64bit processor?
File system on external usb drive?
I assume you made really sure that the files are really stored on the usb drive?

uroni · March 30, 2018, 3:19pm

Could you try if this patch fixes it:

diff --git a/urbackupcommon/os_functions_lin.cpp b/urbackupcommon/os_functions_lin.cpp
index 082f1ff8..a1cdfca4 100644
--- a/urbackupcommon/os_functions_lin.cpp
+++ b/urbackupcommon/os_functions_lin.cpp
@@ -352,8 +352,8 @@ int64 os_free_space(const std::string &path)
     int rc=statvfs64((path).c_str(), &buf);
 	if(rc==0)
 	{
-		fsblkcnt_t blocksize = buf.f_frsize ? buf.f_frsize : buf.f_bsize;
-		fsblkcnt_t free = blocksize*buf.f_bavail;
+		fsblkcnt64_t blocksize = buf.f_frsize ? buf.f_frsize : buf.f_bsize;
+		fsblkcnt64_t free = blocksize*buf.f_bavail;
 		if(free>LLONG_MAX)
 		{
 			return LLONG_MAX;

t_b · March 30, 2018, 7:17pm

Here the answers to your question:

Linux kernel version: Linux 4.9.80-v7+
OS version: Raspbian GNU/Linux 9 (stretch), release 9.4, I think it’s based on Debian stretch.
Architecture: armv7
File system on the external ubs drive:
– Partition table: gpt
– file system: ntfs
the files are really stored on the USB drive, the free space is getting lower, in the beginning it was 1% in use, now it is 13% in use.

I am not sure if the patch would help, as it looks like that would fix a 32/64 bit issue, however, the architecture is ARM. What do you think?

uroni · March 30, 2018, 8:33pm

Depending on the fsblkcnt_t type and the free space available it might be a 32bit overflow. Your system is 32bit, which isn’t that usual anymore.

t_b · March 31, 2018, 8:29pm

Just one thing I noticed, while looking at /etc/default/urbackupsrv:

#Temporary file directory
# – this may get very large depending on the advanced settings
DAEMON_TMPDIR="/tmp"

So my tmp dir is on the SD card with only 7 GB of free space. Might this be an issue? Or is the error message specific on the backup dir?

t_b · April 8, 2018, 7:16pm

This issue is no longer important … I needed to do a fresh install of that Raspberry Pi, now everything is working as expected.

I think the error might have been caused by compiling from sources and not using the deb file. I think that the temp might not be set correctly. I noticed on the fresh install, that there is a temp dir inside urbackups backupfolder - I am not sure if it was there before.

Thank you all for your efforts!

threiran · April 22, 2018, 1:29pm

Hi,

I compiled urbackup from source for a Sheevaplug computer (armv5tel) running Debian (marvel). I believe this is a 32-bit system. I encountered the error message in the web-gui stating that there was little disc space with 1.9TB remaining.

Applying the patch @uroni posted above seems to have solved the problem for me Thank you!

CPL