Try hash

User discussion and information resource forum for Image products.

Try hash

Postby rseiler » Wed May 16, 2012 2:26 pm

/hash, that is, in v2.71.

GREAT feature.

We have a full backup that's about 125GB, so it's massive. Differentials against that invariably take 60-75 mins, even the day after making the full backup. But with /hash, which added essentially no time to the full backup, differentials are taking a very consistent 37 mins so far, which is a huge reduction.

I'm guessing this is because the differential process no longer needs to reference the mega-sized full backup file but just the hash file, which is 1/100 the size. The disks must be working a lot less.
rseiler
 
Posts: 47
Joined: Sat May 05, 2012 3:29 pm

Re: Try hash

Postby TAC109 » Wed May 16, 2012 3:25 pm

On searching the Terabyte site I can only find brief information on
/hash in the imaging manuals.

Can Terabyte make available more technical details of how this feature
works? I'm concerned about the probability that different hard disk
block contents could yield the same hash, causing a changed block to
be excluded from a diff backup.

If this happened, the subsequent restore from this backup would be
incorrect (corrupt).



On Wed, 16 May 2012 14:26:41 PDT, rseiler wrote:

>/hash, that is, in v2.71.
>
>GREAT feature.
>
>We have a full backup that's about 125GB, so it's massive. Differentials against that invariably take 60-75 mins, even the day after making the full backup. But with /hash, which added essentially no time to the full backup, differentials are taking a very consistent 37 mins so far, which is a huge reduction.
>
>I'm guessing this is because the differential process no longer needs to reference the mega-sized full backup file but just the hash file, which is 1/100 the size. The disks must be working a lot less.
>
TAC109
 
Posts: 216
Joined: Tue Sep 06, 2011 3:41 pm

Re: Try hash

Postby TeraByte Support » Wed May 16, 2012 5:37 pm

for me about 4x (10min vs 40min) 180G base.

there are some problems that can occur, asking for #1 file when it doesn't
exist, hangs, access violation. these will all be fixed in the upcoming
2.72. if it gives you a problem you can simply delete the #* files and it
reverts back to using normal differential.

The odds of a duplicate sector hash is over 1 in 4 billion (and data has to
be quite different). Same type of hash used on many RAID systems and other
sector consistency checks.

"rseiler" wrote in message news:2329@public.image...

/hash, that is, in v2.71.

GREAT feature.

We have a full backup that's about 125GB, so it's massive. Differentials
against that invariably take 60-75 mins, even the day after making the full
backup. But with /hash, which added essentially no time to the full backup,
differentials are taking a very consistent 37 mins so far, which is a huge
reduction.

I'm guessing this is because the differential process no longer needs to
reference the mega-sized full backup file but just the hash file, which is
1/100 the size. The disks must be working a lot less.

TeraByte Support
 
Posts: 2335
Joined: Thu May 05, 2011 3:37 pm

Re: Try hash

Postby DrTeeth » Thu May 17, 2012 3:20 pm

Is the attached an example of the issues that will be fixed in 2.72?

This feature is a real time saver and greatly reduces traffic when backing up over a network.

DrTeeth
Attachments
2012-05-17_22h15_44.png
2012-05-17_22h15_44.png (17.59 KiB) Viewed 2895 times
DrTeeth
 
Posts: 970
Joined: Fri Aug 12, 2011 11:58 am

Re: Try hash

Postby DrTeeth » Sun May 20, 2012 3:19 am

On Thu, 17 May 2012 15:20:00 PDT, just as I was about to take a herb,
DrTeeth disturbed my reverie and wrote:

>Is the attached an example of the issues that will be fixed in 2.72?
>
>This feature is a real time saver and greatly reduces traffic when backing up over a network.
>
>DrTeeth

Ping TBU support please.
--

Cheers

DrT
______________________________
We may not be able to prevent the stormy times in
our lives; but we can always choose to dance
in the puddles (Jewish proverb).
DrTeeth
 
Posts: 970
Joined: Fri Aug 12, 2011 11:58 am

Re: Try hash

Postby TeraByte Support » Sun May 20, 2012 9:20 am

"...these will all be fixed in the upcoming 2.72..."

"DrTeeth" wrote in message news:2359@public.image...

On Thu, 17 May 2012 15:20:00 PDT, just as I was about to take a herb,
DrTeeth disturbed my reverie and wrote:

>Is the attached an example of the issues that will be fixed in 2.72?
>
>This feature is a real time saver and greatly reduces traffic when backing
>up over a network.
>
>DrTeeth

Ping TBU support please.
--

Cheers

DrT
______________________________
We may not be able to prevent the stormy times in
our lives; but we can always choose to dance
in the puddles (Jewish proverb).

TeraByte Support
 
Posts: 2335
Joined: Thu May 05, 2011 3:37 pm

Re: Try hash

Postby DrTeeth » Mon May 21, 2012 12:53 am

Thanks for the confirmation, I just wasn't 110% sure.

It is a brilliant addition to IfW and really cuts down on differential backup time.

Any idea on an ETA for v2.72?

UPDATE:- Could this technology be used in a file copy utility? For example, I backup my network to a hard disk in my main PC, I them copy/sync that folder to an external hard disk. I did notice that network traffic was massively reduced when saving a diff. backup.

DrT
DrTeeth
 
Posts: 970
Joined: Fri Aug 12, 2011 11:58 am

Re: Try hash

Postby rseiler » Fri May 25, 2012 2:42 pm

There might be a glitch in paradise with this function: twice now I've seen the scheduled job not complete. Imagew.exe just stays in memory indefinitely, taking one core of the CPU, until I have to kill it with TaskMan (nothing is written to IFW.log). This is the same machine that's been running the very same job successfully for a couple years now, so this is new behavior. Is this one of the known problems?
rseiler
 
Posts: 47
Joined: Sat May 05, 2012 3:29 pm

Re: Try hash

Postby TeraByte Support(PP) » Fri May 25, 2012 3:33 pm

Yes, that's one of the known problems.
Paul Purviance
TeraByte Support
TeraByte Support(PP)
 
Posts: 1110
Joined: Thu Aug 11, 2011 5:51 pm

Re: Try hash

Postby DrTeeth » Sat May 26, 2012 2:42 am

On Fri, 25 May 2012 15:33:17 PDT, just as I was about to take a herb,
TeraByte Support(PP) disturbed my reverie and wrote:

>Yes, that's one of the known problems.

Do you have an ETA for 2.72 please?
--

Cheers

DrT
______________________________
We may not be able to prevent the stormy times in
our lives; but we can always choose to dance
in the puddles (Jewish proverb).
DrTeeth
 
Posts: 970
Joined: Fri Aug 12, 2011 11:58 am


Return to Image for DOS/Linux/Windows

cron