Old IFW V2.x slows down dramatically during backup

User discussion and information resource forum for Image products.
Montclair
Posts: 50
Joined: Wed Oct 26, 2011 2:48 pm

Re: Old IFW V2.x slows down dramatically during backup

Post by Montclair »

I absolutely will. It's about 3.5 hours later, and the ETA has barely changed. The backup has slowed to about 20Mb/s at this point. It's at 53%.
TeraByte Support
Posts: 3598
Joined: Thu May 05, 2011 10:37 pm

Re: Old IFW V2.x slows down dramatically during backup

Post by TeraByte Support »

The program itself doesn't slow down, in addition to the other potential fragmentation or OS resource/driver type issue, antivirus scanning the data and the type of data (such as data that won't compress - especially if not using the Enhanced Speed A option) as well as the hardware itself due to heat built up (ensure system fans working and system clean) or the drive accessing the data. You can also enable VSS or adjust PHYLock throttling (you should leave priority at normal).
Montclair
Posts: 50
Joined: Wed Oct 26, 2011 2:48 pm

Re: Old IFW V2.x slows down dramatically during backup

Post by Montclair »

TeraByte Support wrote: Sun Jul 03, 2022 3:16 pm The program itself doesn't slow down, in addition to the other potential fragmentation or OS resource/driver type issue, antivirus scanning the data and the type of data (such as data that won't compress - especially if not using the Enhanced Speed A option) as well as the hardware itself due to heat built up (ensure system fans working and system clean) or the drive accessing the data. You can also enable VSS or adjust PHYLock throttling (you should leave priority at normal).
SSD getting backed up. The last I knew, we're not supposed to defrag.

Negative heat build up. The program uses at max 8% CPU. Trifan liquid cooling radiator on this box. Heat never goes above 45c when running this

AV has realtime disabled. That's not a factor here.

Backup is using VSS. It's always been faster than Phylock in the past for me.

Priority settings make zero difference on this machine for this application. It goes no faster or slower at any of them. Nevertheless it's set to normal.

It is not a hardware or driver issue that I can find. Everything else is very fast and doesn't experience this slow down.

I will report back when the backup completes as it still isn't finished, coming up on 12 hours for 1.3TB. It's definitely faster than v2 though.

You really should examine that completion estimate formula. Use the last 60 seconds as an average and make your estimate. I'm a coder since the mid 80s so I do have some knowledge on this. Just my 2 cents - no offense intended.
TeraByte Support
Posts: 3598
Joined: Thu May 05, 2011 10:37 pm

Re: Old IFW V2.x slows down dramatically during backup

Post by TeraByte Support »

Defraging SSD doesn't give as big a performance boots as a normal drive, but still helpful, especially if there are lots of small chunks, the main concern people had is with the wear (so don't do it every day). For imaging the smaller chunks have some affect but not so much in V3.

The program itself doesn't slow down, it doesn't matter how big the data set is. Compressible data can slow things down, especially not using Enhanced Speed A, but again not so much in V3.

If you want to see if it's local system resources / data vs the remote path, you could attach a USB3, eSATA or internal SATA or NVMe drive to backup to as a test (or even another GiB network adapter connected to a dedicated high quality switch to NAS or dedicated server). Looking at this system I'm on now, which alternates between a local traditional SATA hard drive and NAS device, the last full to local SATA traditional drive (with creating hash files) takes about 29 minutes (actual data size backed up was 542GiB), the last full to NAS took 1hr 1min (with creating hash files and actual data size backed up was 532GiB) - the NAS is on a 1Gbs network. So if you were backing up to 1Gbs network it should take approx 3hrs +/- unknowns but if you're on 100Mbs network, you're looking at 20+ hours. (assuming about 30% compression and about either 100MBs / 10MBs depending on network type).

We already experimented on calculating time remaining using several methods, the method in use was most stable, but you can reset it if you can get a prompt up, such as cancel but then don't.
TeraByte Support
Posts: 3598
Joined: Thu May 05, 2011 10:37 pm

Re: Old IFW V2.x slows down dramatically during backup

Post by TeraByte Support »

As a test I hooked up a 10TiB USB drive with 3.61TiB of data and backed it up (with creating hash files) to the internal SATA traditional spinning drive, after 5 minutes it said it would take 5hr 36min, it ended up taking 5hr 43min. The compression was only about 11% since much of the data already compressed final file size 3.19TiB and hash file created was 28.8GiB.
Montclair
Posts: 50
Joined: Wed Oct 26, 2011 2:48 pm

Re: Old IFW V2.x slows down dramatically during backup

Post by Montclair »

Maybe the LAN switch is garbage. I really don't know. It would explain (I guess) why all the machines with larger drives experience this. If you're backing up 500GB in an hour over your gigabit LAN, and mine takes ~4 hours to do that, getting progressively slower -- there must be a reason.

Anyway, the results are in for this backup with V3.53. I'm using maximum compression since CPU usage is so minimal. It doesn't seem to make much of a difference on this PC. T: is my NAS. The backup took ~15.5 hours. Started off with highish transfer rates, then gradually got slower. The floor seemed to be about 14MBp/s
ifw_slow2.jpg
ifw_slow2.jpg (60.27 KiB) Viewed 1582 times
Here's the log:

[07/03/2022 02:27:53 AM] C:\Program Files (x86)\TeraByte Drive Image Backup and Restore Suite\imagew64.exe /b /d:3 /uy /ui /hash /err /purge:14 /plvolf /plmem:0 /plcs:16384 /pldisk:0 /po:771 /plmwt:1 /pltr:0 /pltw:0 /comp:10 /usevss /logfile:C:\Users\Owner\Desktop\ifw.log /savename:c_lastfullbackup /f:t:\Fullback\David2\C\$~MM$-$~DD$-$~YYYY$
[07/03/2022 02:27:53 AM] Image for Windows (x64) 3.53
[07/03/2022 02:27:53 AM] Starting ...
[07/03/2022 02:27:53 AM] imagew64.exe /b /uy /ui /d:w3 /f:"t:\Fullback\David2\C\$~MM$-$~DD$-$~YYYY$" /comp:10 /err /purge:14 /hash
[07/03/2022 02:27:53 AM] Windows 10 Pro 21H2 64-bit (10.0.19044.1.100)
[07/03/2022 02:27:53 AM] Attempting to Create VSS Snapshot...
[07/03/2022 02:27:53 AM] Added \\?\Volume{17012123-0000-0000-0000-100000000000} (E:) to snapshot.
[07/03/2022 02:27:53 AM] Added \\?\Volume{d9927197-d274-11e4-a601-806e6f6e6963} (C:) to snapshot.
[07/03/2022 02:28:00 AM] VSS Snapshot Created Successfully.
[07/03/2022 02:28:00 AM] Warning: Data excluded from files based on list in Registry.
[removed]
[07/03/2022 02:28:00 AM] [VSS END]
[07/03/2022 02:28:00 AM] [SYSTEM\CurrentControlSet\Control\BackupRestore\FilesNotToSnapshot]
[removed]
[07/03/2022 02:28:00 AM] [VSS END]
[07/03/2022 02:28:06 AM] Backup: Drive 3 (E:) System Reserved - SSD Partition (01) 203.72 MiB NTFS
[07/03/2022 02:28:06 AM] Target File: t:\Fullback\David2\C\07-03-2022.TBI
[07/03/2022 02:28:06 AM] Total Sectors:417,213 (203.72 MiB) Sectors to Process:56,488 (27.58 MiB)
[07/03/2022 02:28:06 AM] 51,792 Sector(s) (25.29 MiB) backed up
[07/03/2022 02:28:06 AM] Backup: Drive 3 (C:) SSD_PNY_2TB Partition (02) 1.82 TiB NTFS
[07/03/2022 02:28:06 AM] Target File: t:\Fullback\David2\C\07-03-2022.TBI
[07/03/2022 02:28:06 AM] Total Sectors:3,906,598,399 (1.82 TiB) Sectors to Process:2,882,675,184 (1.34 TiB)
[07/03/2022 06:02:54 PM] 2,796,998,456 Sector(s) (1.3 TiB) backed up
[07/03/2022 06:03:28 PM] Notifying VSS that backup completed...
[07/03/2022 06:03:36 PM] Elapsed 0:15:35:43
[07/03/2022 06:03:36 PM] Operation Completed with Error Code 0 (Succeeded)
[07/03/2022 06:03:36 PM] Stop
Brian K
Posts: 2214
Joined: Fri Aug 12, 2011 1:11 am
Location: NSW, Australia

Re: Old IFW V2.x slows down dramatically during backup

Post by Brian K »

I don't have a NAS to run a test.

I did create an IFW backup of a data partition. With hashes. M.2 NVMe to M.2 NVMe. The image size is 356 GiB and backup time was 6 minutes.

Edit.... I repeated the backup using IFL. No hashes. Image size is the same and backup time was 3:32. That is fast.
TeraByte Support
Posts: 3598
Joined: Thu May 05, 2011 10:37 pm

Re: Old IFW V2.x slows down dramatically during backup

Post by TeraByte Support »

I'm using maximum compression since CPU usage is so minimal.
Have you tried the default comp of Enhanced Speed A ?
I repeated the backup using IFL. No hashes. Image size is the same and backup time was 3:32. That is fast.
How about with the hash file. Do you use "direct" or "os" to save the file?
Montclair
Posts: 50
Joined: Wed Oct 26, 2011 2:48 pm

Re: Old IFW V2.x slows down dramatically during backup

Post by Montclair »

TeraByte Support wrote: Mon Jul 04, 2022 4:05 pm Have you tried the default comp of Enhanced Speed A ?
On the machine with the 2TB and 1TB M.2 NVMe drives (the main one we're discussing here), when using V2 I found no speed difference between /comp:14 and /comp:10. I have not tested this yet in V3.

My other slower machines use standard compression and I haven't upgraded them yet from V2, and most of their drives are much smaller.
How about with the hash file. Do you use "direct" or "os" to save the file?
That's Brian K, not me. I don't use Linux here.

Anyway, I examined my LAN setup to make sure the NAS and source machine were connected to the same switch, instead of perhaps one to my dual WAN router and one to the switch, and they're both plugged in right next to each other on the switch.

So, it's either the IFW software, the switch hardware, or the NAS. I'm just not seeing hardware run anywhere near full capacity though, but I cannot verify that on the switch since it's unmanaged (TrendNet TEG-S80g https://www.trendnet.com/products/produ ... 5_teg-s80g)
Montclair
Posts: 50
Joined: Wed Oct 26, 2011 2:48 pm

Re: Old IFW V2.x slows down dramatically during backup

Post by Montclair »

I think I'm prepared to say at this point it's probably the switch. I did a 52GB single file transfer to the NAS on the same Win10 machine. I used a program called "FastCopy". The transfer started out near the 1Gbp/s limit. It was moving about 98MB/s. As the copy progressed, the speed gradually decreased.
ifw_slow3.jpg
ifw_slow3.jpg (27.58 KiB) Viewed 1533 times
Here are the final numbers from the copy:
TotalRead = 53,198 MiB
TotalWrite = 53,198 MiB
TotalTime = 10:45
TransRate = 86.4 MB/s

I also had task manager opened and could see the ethernet speed start off strong, then gradually bounce around from as low as 300gbps to maybe as high as 800Gbp/s.

Edit: I ran the test again and this time it started off slower, and got faster as time went on. The second transfer, though, was slower than the first, taking 15:37 at 83.4 MB/s. Confusing!

This is most likely getting beyond the scope of this forum now, but ask two final questions.

a) Does anyone else see this kind performance doing a transfer on their 1Gb LAN?
b) If not, and your NAS backups run at the same consistent speed, what kind of switch are you using?
Post Reply