I was reviewing different modern backup software lately, and IFW appears to be one of the best ones.
However, I couldn't help but notice that it seem to have poor multi-threading support - I hardly ever seen more than 1-1.5 cores load.
On high-performance laptop with quad core Intel Ivy Bridge CPU I've tested, to achieve full backup speed with external HDD (about 80Mb/sec), I had to switch to lower compression "Enhanced Speed - A". With default compression setting, backup was clearly bottlenecked by CPU with write speeds about 40Mb/sec (but still, about 1 core was actually loaded, so lot of spare CPU power was available, but unused).
Using multiple cores on multi-core CPU could provide a lot more power for compression - this could allow using for more aggressive compression without sacrificing backup performance, allowing potentially to store more backups in given storage space and even further increasing backup performance (due to less data need to be actually written for amount of data backed up).
It would be great if Terabyte could add such feature to future versions of Image.
Feature suggestion - better support for multi-core CPUs
Re: Feature suggestion - better support for multi-core CPUs
On Sat, 15 Dec 2012 04:24:58 PST, just as I was about to take a herb,
ifan20 disturbed my reverie and wrote:
>I was reviewing different modern backup software lately, and IFW appears to be one of the best ones.
>
>However, I couldn't help but notice that it seem to have poor multi-threading support - I hardly ever seen more than 1-1.5 cores load.
>On high-performance laptop with quad core Intel Ivy Bridge CPU I've tested, to achieve full backup speed with external HDD (about 80Mb/sec), I had to switch to lower compression "Enhanced Speed - A". With default compression setting, backup was clearly bottlenecked by CPU with write speeds about 40Mb/sec (but still, about 1 core was actually loaded, so lot of spare CPU power was available, but unused).
>
>Using multiple cores on multi-core CPU could provide a lot more power for compression - this could allow using for more aggressive compression without sacrificing backup performance, allowing potentially to store more backups in given storage space and even further increasing backup performance (due to less data need to be actually written for amount of data backed up).
>It would be great if Terabyte could add such feature to future versions of Image.
>
Cannot see much point in this. The data on most computers is not that
compressible or is already compressed. One can only use multithreading
if two (or more) processes can run in parallel. AIUI backing up does
not lend itself to multithreaded use. That is an advantage as it
leaves the CPU available for things to do whilst backing up.
--
Cheers
DrT
______________________________
We may not be able to prevent the stormy times in
our lives; but we can always choose whether or not
to dance in the puddles (Jewish proverb).
ifan20 disturbed my reverie and wrote:
>I was reviewing different modern backup software lately, and IFW appears to be one of the best ones.
>
>However, I couldn't help but notice that it seem to have poor multi-threading support - I hardly ever seen more than 1-1.5 cores load.
>On high-performance laptop with quad core Intel Ivy Bridge CPU I've tested, to achieve full backup speed with external HDD (about 80Mb/sec), I had to switch to lower compression "Enhanced Speed - A". With default compression setting, backup was clearly bottlenecked by CPU with write speeds about 40Mb/sec (but still, about 1 core was actually loaded, so lot of spare CPU power was available, but unused).
>
>Using multiple cores on multi-core CPU could provide a lot more power for compression - this could allow using for more aggressive compression without sacrificing backup performance, allowing potentially to store more backups in given storage space and even further increasing backup performance (due to less data need to be actually written for amount of data backed up).
>It would be great if Terabyte could add such feature to future versions of Image.
>
Cannot see much point in this. The data on most computers is not that
compressible or is already compressed. One can only use multithreading
if two (or more) processes can run in parallel. AIUI backing up does
not lend itself to multithreaded use. That is an advantage as it
leaves the CPU available for things to do whilst backing up.
--
Cheers
DrT
______________________________
We may not be able to prevent the stormy times in
our lives; but we can always choose whether or not
to dance in the puddles (Jewish proverb).
Re: Feature suggestion - better support for multi-core CPUs
DrTeeth wrote:
> On Sat, 15 Dec 2012 04:24:58 PST,
>
> Cannot see much point in this. The data on most computers is not that
> compressible or is already compressed.
This depends of what you back up. If you backing up HDD filled with music or movies, than yes, this data won't be compressible.
The dataset I've tested on was a partition filled mainly with software installation & databases - this data achieved compression of about 20% with "Enhanced Speed A", but about 70% with "Enhanced Size A", which translated to difference in hundredth of gigabytes in total. This quite significant advantage (I'd estimate with more aggressive compression techniques even more savings could be achieved).
> One can only use multithreading
> if two (or more) processes can run in parallel. AIUI backing up does
> not lend itself to multithreaded use
This is not correct, and several of competitor backup/imaging apps already disprove this, so its not like nobody done this yet.
Most common approach is buffering read data in several relatively large buffers, which then compressed in parallel on several cores (e.g. 4 buffers for 4 cores). Then this compressed data collated back and written.
> On Sat, 15 Dec 2012 04:24:58 PST,
>
> Cannot see much point in this. The data on most computers is not that
> compressible or is already compressed.
This depends of what you back up. If you backing up HDD filled with music or movies, than yes, this data won't be compressible.
The dataset I've tested on was a partition filled mainly with software installation & databases - this data achieved compression of about 20% with "Enhanced Speed A", but about 70% with "Enhanced Size A", which translated to difference in hundredth of gigabytes in total. This quite significant advantage (I'd estimate with more aggressive compression techniques even more savings could be achieved).
> One can only use multithreading
> if two (or more) processes can run in parallel. AIUI backing up does
> not lend itself to multithreaded use
This is not correct, and several of competitor backup/imaging apps already disprove this, so its not like nobody done this yet.
Most common approach is buffering read data in several relatively large buffers, which then compressed in parallel on several cores (e.g. 4 buffers for 4 cores). Then this compressed data collated back and written.