Differential backup suddenly growing to almost full size

[OldNavyGuy]

See the post above on differential vs. incremental -

https://www.terabyteunlimited.com/ucf/v ... 371#p22371

If an event such as indexes on the drive being rebuilt frequently, that would cause the differential to grow even faster.

You might look into a tool to monitor file and folder changes -

https://www.raymond.cc/blog/3-portable- ... s-changes/

Or, switch to incremental.

You would use the Consolidate function for incrementals, as well as differentials (see page 69 in the IFW User's Manual).

[OldNavyGuy]

What do you mean by the speedup option?

Faster Changes Only Backups – (page 58 of the IFW User's manual)

Select this option to have Image for Windows
create a hash file to speed up creating a Changes Only (differential or incremental)
backup. This option is only available when creating an image that is not being
saved to disc media (CD/DVD/BD). By default, this option is also ignored if the
Backwards Compatible option is enabled. The hash file will be limited to the max file
size and have the same file name as the backup with an extension starting at .#0
followed by .#1, .#2, etc. as needed. The actual speed increase realized when
creating an image will vary depending on the system. If the hash file is deleted the
backup will proceed as normal without it. To create a hash for an existing image
use the /hash operation command line parameter

[mythos]

No I don't use that option. I don't see how this is relevant to the file size.

[TeraByte Support]

It's not file based so if there is any movement of files it will cause a lot of changes. Also a SSD will still have defrag run on it at certain times based on a MS formula, just not often as a spinning disk. You could try creating the hash file for your current full version, then run another diff to see if its still large or the size became smaller.

[mythos]

I tried it but once the size grows it's never reduced again. I need to start over with a new full.
Do you know where I can find this defrag task that you mention? There is no such scheduled task and frankly I don't understand why this would exist. Defragging an SSD is utterly pointless and causes unnecessary wear.

[TeraByte Support]

It's built-in now. Here's an older post by someone quoting the storage team: https://www.hanselman.com/blog/the-real ... t-your-ssd

How often it does it now? shouldn't be every few days. But if both the hash and non-hash have same large gains something did move the files data around.

[mythos]

According to that windows does indeed defragment SSDs and the only way to disable this is to essentially disable VSC and system restore!

If this is indeed the case, I really don't understand why this isn't a well known issue since this applies to all win 10 installations so it should be a problem for anyone who takes differential backups of his windows partition.

This has already happened 3 times in less than two months on a clean win 10 installation.

[OldNavyGuy]

Optimize is the only user option for SSD's in Windows 10.

Control Panel > Administrative Tools > Defrag and Optimize Drives.

You can set a schedule there.

As mentioned above, download one of the folder and file monitoring tools to see what's being changed.

My go to is Nirsoft's FolderChangesView.

It can log results to a CSV file, so let it run for the duration between backups.

Also, as mentioned before, check that you aren't rebuilding indexes frequently...that would cause the issue as well.

As a test, turn off indexing and see if it makes a difference.

I personally don't use Windows indexing...a tool called Everything from voidtools works well for us.

[OldNavyGuy]

I tried it but once the size grows it's never reduced again. I need to start over with a new full.

Which is exactly how differentials work...

Differential examples

1. Full image on Sunday...Differential on Tuesday...contains all changes since Sunday.

2. Full image on Sunday...another Differential on Thursday...contains all changes since Sunday.


Incremental examples

1. Full image on Sunday...Incremenal on Tuesday...contains all changes since Sunday.

2. Full image on Sunday...another Incremental on Thursday...contains all changes since Tuesday (the last Incremental).

3. Another Incremental on Saturday...contains all changes since Thursday (the last Incremental).

Net result is smaller backups, but more backup files...which it sounds like what you think should happen with a Differential...it doesn't.


A Differential always contains changes since the "original" full image, no matter when it's done...and it will always grow.

If you think it's growing too fast, you need to monitor changes on your system.

[TAC109]

I believe that the defrag issue is a red herring in this discussion. I regularly defrag my SSDs before an incremental backup and it doesn’t materially increase the backup size.

Cheers