Why Did a 'Simple" Disk Copy Process ~75% More Data?

User discussion and information resource forum for Image products.
Post Reply
Muad'Dib
Posts: 110
Joined: Mon Nov 21, 2011 12:23 pm

Why Did a 'Simple" Disk Copy Process ~75% More Data?

Post by Muad'Dib »

A few days ago I used IFW to copy two drives (all on the same system). The second copy was done using Simple Operations mode, and that's where I saw something unusual. Here are the details:

Both source drives were 2 TB (1.82 TB available) and so were both destination drives. I don't copy drives that frequently, so I was a bit unfamiliar with the steps. Both drives were data drives, so I ran it from IFW installed on the system (no need to run it from an external USB device).

The first source drive was partitioned as MBR, the second source drive was partitioned as GPT. I kept this partitioning as-is with the copies (if it ain't broke...).

The first copy was from a WD Black to an identical drive. I used the Normal Operations mode that I've been using for years for backups (including byte-for-byte validation) and all went fine. The 'Data Processed' was a little over 4 TB (I don't remember the exact amount, maybe 4.1 or 4.2 TB?), I expected double the amount of data due to the byte-for-byte validation, and I figured the extra amount above that was for some sort of additional overhead when verifying, so it didn't bother me. The copy went smoothly. It took around 5.5 hours and so far the copy is working fine.

The second copy was from a Seagate ST2000DM001 (Barracuda?) to a WD Black. The destination drive was SLIGHTLY bigger (if I had done a straight copy there would have been an extra partition at the end of about 1 MB). Because I didn't want to deal with the 'Scale to Fit' and/or the 'Scale to Target' settings (it was late and I rarely deal with those options), I followed TeraByte Support's suggestion in this forum and changed the settings to "Simple Operations and let the program handle it all for you".

This copy also went fine, however it took 7 hours and 6.99TB of data were processed. I want to find out why it needed to process about 75% more data than the first copy (and therefore take an extra 1.5 hours). I have two theories:
  • Was this because of the resizing due to the 1 MB discrepancy that Simple Operations mode handled automatically?
  • Would it have processed the same amount of data had I used Normal Operations mode and used the proper 'Scale to" settings?
  • Or was there something else in Simple Operations that increased the amount of data and time for the copy (if so, what was the 'something else')?
Or..
  • Was the time/data discrepancy due to the first set of drives being MBR and the second set of drives being GPT?
  • If this was the reason, would changing the destination drive to an MBR partition reduced the amount of data processed (and also reduced the time)?
When I started the second copy, even though the data to be processed showed 6.99 TB, early estimates of time to completion were around 5.5 hours, so I decided to let it run its course. However, as the copy progressed, the total amount of time increased and ended up just a few seconds shy of 7 hours.

Again, this second copy went fine, so now I'm just curious for possible future copies. If I had known ahead of time that this second copy would take an extra 1.5 hours, and this was due to some sort of rescaling, I probably would have stayed in Normal Operations mode, not done any 'Scale to' option and accepted the trivial 1 MB partition at the end of the drive (saving 1.5 hours would have been worth it).

I have several screen shots and copies of the command line syntax for both copies, if that would help diagnosing this.
TeraByte Support
Posts: 3627
Joined: Thu May 05, 2011 10:37 pm

Re: Why Did a 'Simple" Disk Copy Process ~75% More Data?

Post by TeraByte Support »

what's the logs show?
Muad'Dib
Posts: 110
Joined: Mon Nov 21, 2011 12:23 pm

Re: Why Did a 'Simple" Disk Copy Process ~75% More Data?

Post by Muad'Dib »

What specifically should I be looking for in the logs?
Bob Coleman
Posts: 788
Joined: Fri Aug 12, 2011 5:58 pm

Re: Why Did a 'Simple" Disk Copy Process ~75% More Data?

Post by Bob Coleman »

Muad'Dib wrote: Sun Feb 26, 2023 5:11 pm What specifically should I be looking for in the logs?
Differences between thr two logs that aren't accounted for by things already understood maybe.
Muad'Dib
Posts: 110
Joined: Mon Nov 21, 2011 12:23 pm

Re: Why Did a 'Simple" Disk Copy Process ~75% More Data?

Post by Muad'Dib »

I looked at the logs, and the only difference I see that would have made some difference was that the first copy had a disk that was less full than the disk from the second copy - 1.37 TB vs 1.75 TB (I had forgotten that I didn't have IFW copy unused sectors). But that alone shouldn't have resulted in the second copy processing 6.99 TB (four times the amount of actual data copied, twice of what I was expecting with a byte-for-byte validation).

In case it helps, here are the command lines from the two copies:
  1. Normal Operations Mode for WD to WD: imagew64.exe /copy /sd:w3 /td:w2 /vb /rft
  2. Simple Operations Mode for Seagate to WD: imagew64.exe /copy /sd:w1 /td:w0 /vb /csig /ohd /rft /fts:0 /x /att /ubp
TeraByte Support
Posts: 3627
Joined: Thu May 05, 2011 10:37 pm

Re: Why Did a 'Simple" Disk Copy Process ~75% More Data?

Post by TeraByte Support »

You can just send the logs to be looked at. But a copy would be the source/target/and verify so it will be larger.
Muad'Dib
Posts: 110
Joined: Mon Nov 21, 2011 12:23 pm

Re: Why Did a 'Simple" Disk Copy Process ~75% More Data?

Post by Muad'Dib »

I'll post the logs when I get back to work (the system is currently in use, so I can't access it). But could you please answer the following:

When I backup 1.75 TB of data with a byte-for-byte validation, the amount of data processed is approximately 3.5 TB (1.75 for copy, 1.75 for validation). So why would copying 1.75 TB with the same validation result in 7 TB of data being processed (twice as much as a backup)?

In other words, is there something fundamentally different between backing up and copying that results in so much more data being processed?
Bob Coleman
Posts: 788
Joined: Fri Aug 12, 2011 5:58 pm

Re: Why Did a 'Simple" Disk Copy Process ~75% More Data?

Post by Bob Coleman »

Muad'Dib wrote: Sun Feb 26, 2023 11:59 pm
In other words, is there something fundamentally different between backing up and copying that results in so much more data being processed?
Possibly compression when backing up and not when copying?
Muad'Dib
Posts: 110
Joined: Mon Nov 21, 2011 12:23 pm

Re: Why Did a 'Simple" Disk Copy Process ~75% More Data?

Post by Muad'Dib »

Not compression - I never use it for either.
Post Reply