Differential Backup Times

[ronk]

Have been using IFW for several years after switching from Acronis. Until now, have always done full partition backups, especially since I have my data separated by category, and also separated from the 'C' OS drive. This works very well, and restores have worked w/o a hitch when needed, as does mounting and restoring individual files. I have one partition which is video files, and is quite large. So rather than backing up a full image, I thought this might be a time to try differential. In my tests , all to USB external drives from a laptop, the differential runs nearly as long as the full. Is this because of the sector comparisons to the last full backup which must be done to discern changes? If so, I likely will need to resort to continuing full backups, since space isn't an issue, but was trying to save time. Alternatively, I sometimes do something like a teracopy of just changed files rather than IFW, but it is a bit more cumbersome to manage. Am I missing anything?

Using Windows 7, Terabyte 2.77, Thinkpad T520 (no USB 3.0 ports), USB 2.0 externals.

[TeraByte Support]

Yes, you'd use the "speed up changes only backup" option when creating the
full if you want to use a hash file instead.

"ronk" wrote in message news:[email protected]...

Have been using IFW for several years after switching from Acronis. Until
now, have always done full partition backups, especially since I have my
data separated by category, and also separated from the 'C' OS drive. This
works very well, and restores have worked w/o a hitch when needed, as does
mounting and restoring individual files. I have one partition which is
video files, and is quite large. So rather than backing up a full image, I
thought this might be a time to try differential. In my tests , all to USB
external drives from a laptop, the differential runs nearly as long as the
full.


Is this because of the sector comparisons to the last full backup which must
be done to discern changes? If so, I likely will need to resort to
continuing full backups, since space isn't an issue, but was trying to save
time. Alternatively, I sometimes do something like a teracopy of just
changed files rather than IFW, but it is a bit more cumbersome to manage.
Am I missing anything?


Using Windows 7, Terabyte 2.77, Thinkpad T520 (no USB 3.0 ports), USB 2.0
externals.

[TAC109]

Yes, the "speed up changes only backup" option will make a big
difference.

Also, setting the compression to "Enhanced speed - A" (or B) will
speed up all backups (full and diff) on slower processors.

On Tue, 1 Jan 2013 09:41:26 PST, "TeraByte Support"
wrote:

>Yes, you'd use the "speed up changes only backup" option when creating the
>full if you want to use a hash file instead.
>
>"ronk" wrote in message news:[email protected]...
>
>Have been using IFW for several years after switching from Acronis. Until
>now, have always done full partition backups, especially since I have my
>data separated by category, and also separated from the 'C' OS drive. This
>works very well, and restores have worked w/o a hitch when needed, as does
>mounting and restoring individual files. I have one partition which is
>video files, and is quite large. So rather than backing up a full image, I
>thought this might be a time to try differential. In my tests , all to USB
>external drives from a laptop, the differential runs nearly as long as the
>full.
>
>
>
>Is this because of the sector comparisons to the last full backup which must
>be done to discern changes? If so, I likely will need to resort to
>continuing full backups, since space isn't an issue, but was trying to save
>time. Alternatively, I sometimes do something like a teracopy of just
>changed files rather than IFW, but it is a bit more cumbersome to manage.
>Am I missing anything?
>
>
>
>Using Windows 7, Terabyte 2.77, Thinkpad T520 (no USB 3.0 ports), USB 2.0
>externals.
>

[ronk]

Thanks for revealing the option. So I went to a full from last month, then used command line to create a hash file for it (from reading the manual options) and it created the hash. However, after that process, the full .tbi file was altered (modify date) and when attempting to open it, get error 13 not a valid backup.

So, created another full backup in Windows (no command line), then, did same thing, creating a hash file in command line. Then, made a few changes to the partition, and did the IFW changes only. This time it worked fine, and speed was about 1/3 of full backup, and I didn't receive an error 13 messages. What concerns me, is, why did creating the hash on the 1 month old file corrupt the full backup file??? This is a big concern and lessens my feeling of security of using this differential process.

Yes, I realize I could have used the command line for the 'full' and the 'chg' , but I prefer to work within the Windows dialogue box when possible. Why is the hash option not there?

Anyway will test a few more things, but may endup sticking with full backups unless it can be explained to me how the older file was corrupted.

[TAC109]

On Tue, 1 Jan 2013 14:07:37 PST, ronk wrote:

>Thanks for revealing the option. So I went to a full from last month, then used command line to create a hash file for it (from reading the manual options) and it created the hash. However, after that process, the full .tbi file was altered (modify date) and when attempting to open it, get error 13 not a valid backup.
>
>So, created another full backup in Windows (no command line), then, did same thing, creating a hash file in command line. Then, made a few changes to the partition, and did the IFW changes only. This time it worked fine, and speed was about 1/3 of full backup, and I didn't receive an error 13 messages. What concerns me, is, why did creating the hash on the 1 month old file corrupt the full backup file??? This is a big concern and lessens my feeling of security of using this differential process.
>
>Yes, I realize I could have used the command line for the 'full' and the 'chg' , but I prefer to work within the Windows dialogue box when possible. Why is the hash option not there?
>
>Anyway will test a few more things, but may endup sticking with full backups unless it can be explained to me how the older file was corrupted.
>
I can't help you with the error 13 message, but I *can* help you with
creating the hash file at the same time as you take a full backup.

In the GUI, after selecting the backup and supplying a file name and
location for the backup, there is a 'Backup options' screen where you
can select 'Speed up changes only backup' (near the bottom of the
list). This is the equivalent of using the /hash option on the command
line.

You should always validate your backups (on the same 'Backup options'
screen) but 'validate byte-for-byte' will slow down your backups.

After installing any new hardware, I do a 'validate byte-for-byte'
run initially to check that everything is ok, then switch to just
'validate' for subsequent backups.

[ronk]

Sorry, I missed the 'speed up' option on that page, was looking for 'create hash file' (duh). However, as to the corrupted backup, just before created the hash file separately, I could mount the 1 month old file, but after creating it, and it's date was modified, I got the error 13. In fact, the only such error I've ever encountered from IFW (or IFL, for that matter).

Thanks.

[DrTeeth]

On Tue, 1 Jan 2013 17:10:47 PST, just as I was about to take a herb,
Tom Cole disturbed my reverie and
wrote:

>You should always validate your backups (on the same 'Backup options'
>screen) but 'validate byte-for-byte' will slow down your backups.

PMFJI, but I always validate byte-for-byte. Do you find not doing so
saves much time? Do you have any figures?
Thanks.
--

Cheers

DrT
______________________________
We may not be able to prevent the stormy times in
our lives; but we can always choose whether or not
to dance in the puddles (Jewish proverb).

[TAC109]

On Thu, 3 Jan 2013 14:39:06 PST, DrTeeth wrote:

>On Tue, 1 Jan 2013 17:10:47 PST, just as I was about to take a herb,
>Tom Cole
>
> disturbed my reverie and
>wrote:
>
>>You should always validate your backups (on the same 'Backup options'
>>screen) but 'validate byte-for-byte' will slow down your backups.
>
>PMFJI, but I always validate byte-for-byte. Do you find not doing so
>saves much time? Do you have any figures?
>Thanks.

Just did a couple of tests.

The times for the backup part of the jobs are the same (as expected).

The times for the verify part of the jobs are significantly different.

On an old Acer netbook backing up to an external USB2 HD with
compression set to 'Enhanced speed A':-
Verify b-f-b takes 60% more time than straight verify.

On a 3 month old Dell Vostro 3560 notebook backing up to an external
USB3 HD with standard compression:-
Verify b-f-b takes 93% more time than straight verify.

What sort of figures do you get on your rig(s)?

[DrTeeth]

On Fri, 4 Jan 2013 14:47:02 PST, just as I was about to take a herb,
Tom Cole disturbed my reverie and
wrote:

>What sort of figures do you get on your rig(s)?

I haven't done any figures, but will do soon on one of my smaller
partitions. I have always done byte-for=byte assuming it to be much
safer.

I have some figures for one of my Linux partitions (Ext4):-
byte-for-byte 5m 26s; regular validation 2m 20s.

Guess we need to know how much 'better' byte-for-byte is now.
--

Cheers

DrT
______________________________
We may not be able to prevent the stormy times in
our lives; but we can always choose whether or not
to dance in the puddles (Jewish proverb).

[TAC109]

On Sat, 5 Jan 2013 13:32:16 PST, DrTeeth wrote:

>Guess we need to know how much 'better' byte-for-byte is now.

Briefly, normal verify checks the outward path from the 'processor'
out to the HD where the image is written, whereas b-f-b verify also
checks the inward path from the HD being imaged to the 'processor'.

With normal verify, IFW calculates some kind of checksum for the image
block it's about to write, includes it in the block, and writes the
block to the image file. When verifying, IFW reads the image block,
re-calculates the checksum, and checks for a match with the checksum
that is in the block.

The most likely reason for a checksum mismatch would be a write or
read error to/from the hard disk where the image file resides. Much
less likely is a failure in the path between the 'processor' and the
HD containing the image.

B-f-b verify is the same as above, but in addition when verifying,
re-reads the data from the source disk again and matches it against
the image block.

Thus b-f-b is additionally checking that the source disk block has
been read the same as before, and that the electrical path that the
data takes yields the same result as when the block was read when the
image was taken.

In my opinion, if b-f-b verification picks up anything additional to
normal verify, then it is either a bug in IFW, or the hardware is
flaky (or it is something like a Trim issue, etc).

I am happy to only use b-f-b verify when I get a new computer to check
that everything is OK, then just rely on normal verify for subsequent
backups.