Try hash

User discussion and information resource forum for Image products.
Post Reply
rseiler
Posts: 70
Joined: Sat May 05, 2012 10:29 pm

Try hash

Post by rseiler »

/hash, that is, in v2.71.

GREAT feature.

We have a full backup that's about 125GB, so it's massive. Differentials against that invariably take 60-75 mins, even the day after making the full backup. But with /hash, which added essentially no time to the full backup, differentials are taking a very consistent 37 mins so far, which is a huge reduction.

I'm guessing this is because the differential process no longer needs to reference the mega-sized full backup file but just the hash file, which is 1/100 the size. The disks must be working a lot less.
TAC109
Posts: 273
Joined: Tue Sep 06, 2011 10:41 pm

Re: Try hash

Post by TAC109 »

On searching the Terabyte site I can only find brief information on
/hash in the imaging manuals.

Can Terabyte make available more technical details of how this feature
works? I'm concerned about the probability that different hard disk
block contents could yield the same hash, causing a changed block to
be excluded from a diff backup.

If this happened, the subsequent restore from this backup would be
incorrect (corrupt).



On Wed, 16 May 2012 14:26:41 PDT, rseiler wrote:

>/hash, that is, in v2.71.
>
>GREAT feature.
>
>We have a full backup that's about 125GB, so it's massive. Differentials against that invariably take 60-75 mins, even the day after making the full backup. But with /hash, which added essentially no time to the full backup, differentials are taking a very consistent 37 mins so far, which is a huge reduction.
>
>I'm guessing this is because the differential process no longer needs to reference the mega-sized full backup file but just the hash file, which is 1/100 the size. The disks must be working a lot less.
>
TeraByte Support
Posts: 3598
Joined: Thu May 05, 2011 10:37 pm

Re: Try hash

Post by TeraByte Support »

for me about 4x (10min vs 40min) 180G base.

there are some problems that can occur, asking for #1 file when it doesn't
exist, hangs, access violation. these will all be fixed in the upcoming
2.72. if it gives you a problem you can simply delete the #* files and it
reverts back to using normal differential.

The odds of a duplicate sector hash is over 1 in 4 billion (and data has to
be quite different). Same type of hash used on many RAID systems and other
sector consistency checks.

"rseiler" wrote in message news:2329@public.image...

/hash, that is, in v2.71.

GREAT feature.

We have a full backup that's about 125GB, so it's massive. Differentials
against that invariably take 60-75 mins, even the day after making the full
backup. But with /hash, which added essentially no time to the full backup,
differentials are taking a very consistent 37 mins so far, which is a huge
reduction.

I'm guessing this is because the differential process no longer needs to
reference the mega-sized full backup file but just the hash file, which is
1/100 the size. The disks must be working a lot less.

DrTeeth
Posts: 1289
Joined: Fri Aug 12, 2011 6:58 pm

Re: Try hash

Post by DrTeeth »

Is the attached an example of the issues that will be fixed in 2.72?

This feature is a real time saver and greatly reduces traffic when backing up over a network.

DrTeeth
Attachments
2012-05-17_22h15_44.png
2012-05-17_22h15_44.png (17.59 KiB) Viewed 8030 times
DrTeeth
Posts: 1289
Joined: Fri Aug 12, 2011 6:58 pm

Re: Try hash

Post by DrTeeth »

On Thu, 17 May 2012 15:20:00 PDT, just as I was about to take a herb,
DrTeeth disturbed my reverie and wrote:

>Is the attached an example of the issues that will be fixed in 2.72?
>
>This feature is a real time saver and greatly reduces traffic when backing up over a network.
>
>DrTeeth

Ping TBU support please.
--

Cheers

DrT
______________________________
We may not be able to prevent the stormy times in
our lives; but we can always choose to dance
in the puddles (Jewish proverb).
TeraByte Support
Posts: 3598
Joined: Thu May 05, 2011 10:37 pm

Re: Try hash

Post by TeraByte Support »

"...these will all be fixed in the upcoming 2.72..."

"DrTeeth" wrote in message news:2359@public.image...

On Thu, 17 May 2012 15:20:00 PDT, just as I was about to take a herb,
DrTeeth disturbed my reverie and wrote:

>Is the attached an example of the issues that will be fixed in 2.72?
>
>This feature is a real time saver and greatly reduces traffic when backing
>up over a network.
>
>DrTeeth

Ping TBU support please.
--

Cheers

DrT
______________________________
We may not be able to prevent the stormy times in
our lives; but we can always choose to dance
in the puddles (Jewish proverb).

DrTeeth
Posts: 1289
Joined: Fri Aug 12, 2011 6:58 pm

Re: Try hash

Post by DrTeeth »

Thanks for the confirmation, I just wasn't 110% sure.

It is a brilliant addition to IfW and really cuts down on differential backup time.

Any idea on an ETA for v2.72?

UPDATE:- Could this technology be used in a file copy utility? For example, I backup my network to a hard disk in my main PC, I them copy/sync that folder to an external hard disk. I did notice that network traffic was massively reduced when saving a diff. backup.

DrT
rseiler
Posts: 70
Joined: Sat May 05, 2012 10:29 pm

Re: Try hash

Post by rseiler »

There might be a glitch in paradise with this function: twice now I've seen the scheduled job not complete. Imagew.exe just stays in memory indefinitely, taking one core of the CPU, until I have to kill it with TaskMan (nothing is written to IFW.log). This is the same machine that's been running the very same job successfully for a couple years now, so this is new behavior. Is this one of the known problems?
TeraByte Support(PP)
Posts: 1644
Joined: Fri Aug 12, 2011 12:51 am

Re: Try hash

Post by TeraByte Support(PP) »

Yes, that's one of the known problems.
DrTeeth
Posts: 1289
Joined: Fri Aug 12, 2011 6:58 pm

Re: Try hash

Post by DrTeeth »

On Fri, 25 May 2012 15:33:17 PDT, just as I was about to take a herb,
TeraByte Support(PP) disturbed my reverie and wrote:

>Yes, that's one of the known problems.

Do you have an ETA for 2.72 please?
--

Cheers

DrT
______________________________
We may not be able to prevent the stormy times in
our lives; but we can always choose to dance
in the puddles (Jewish proverb).
Post Reply