Shaun... question regarding DC5.5

[r***@invalid.com]

Shaun,
I recently upgraded to DC5.5 from 5.4. I couldn't find any formal release
notes, but the marketing notes indicated that the speed for the encryption
routines had been increased. I never benchmarked 5.4, but just casual
observation makes me think that it is actually slower. Could you comment
please?

[Shaun Hollingworth]

Post by r***@invalid.com
Shaun,
I recently upgraded to DC5.5 from 5.4. I couldn't find any formal release
notes, but the marketing notes indicated that the speed for the encryption
routines had been increased. I never benchmarked 5.4, but just casual
observation makes me think that it is actually slower. Could you comment
please?

Hi,
Do you mean the time to create an encrypted partition or container, or
the time to write files to the encrypted target device ?

We did some optimisations on the driver for the latter, but the former
on DC5 is under the control of the operating system. However I will
certainly check it out.

Regards,
Shaun.

[r***@invalid.com]

On Wed, 22 Jun 2011 10:58:30 +0100, Shaun Hollingworth

Post by Shaun Hollingworth

Post by r***@invalid.com
Shaun,
I recently upgraded to DC5.5 from 5.4. I couldn't find any formal release
notes, but the marketing notes indicated that the speed for the encryption
routines had been increased. I never benchmarked 5.4, but just casual
observation makes me think that it is actually slower. Could you comment
please?

Hi,
Do you mean the time to create an encrypted partition or container, or
the time to write files to the encrypted target device ?
We did some optimisations on the driver for the latter, but the former
on DC5 is under the control of the operating system. However I will
certainly check it out.
Regards,
Shaun.

I was alluding to the time it takes to read data off the encrypted volume
(partition, in my case). TBH, the more I use the new version, the less
sure I am of a speed decrease... especially since I had done no formal
benchmark.

Out of curiosity though, do you have any idea what the performance penalty
*should* be for reading data from an encrypted drive (partition) as
opposed to simply reading plain-text data from the system (C:) drive? Is
there any way to express the difference as a percentage? I.E. reading
from an encrypted volume should be X% slower than reading from a
non-encrypted drive on the same system.

[Shaun]

Post by r***@invalid.com
Out of curiosity though, do you have any idea what the performance penalty
*should* be for reading data from an encrypted drive (partition) as
opposed to simply reading plain-text data from the system (C:) drive? Is
there any way to express the difference as a percentage? I.E. reading
from an encrypted volume should be X% slower than reading from a
non-encrypted drive on the same system.

Hi,

This is a difficult one to quantify and is related to the time it
takes to load from the disk V the time needed to decrypt the data
loaded. Fast processor, slow drive, fast drive slow processor etc.

If one has enough CPU time, then minimal slowdown should result. If
one is doing heavy calculations then a slowdown might result, though
even then the disk reads should have priority. The question might be,
how much does the encryption slow down the rest of the machine, if it
is busy with other taks.

Containers, rather than dedicated partitions, will be slower, as the
system has to read the data from the original file, and then supply
the data to the caller. IE often a recursive operation back through
the filing system.

When tome permits I will try and do some tests on various hardware and
evalute the mods which were made.

Regards,
Shaun.

[Buzz Murdoch]

Post by Shaun

Post by r***@invalid.com
Out of curiosity though, do you have any idea what the performance penalty
*should* be for reading data from an encrypted drive (partition) as
opposed to simply reading plain-text data from the system (C:) drive? Is
there any way to express the difference as a percentage? I.E. reading
from an encrypted volume should be X% slower than reading from a
non-encrypted drive on the same system.

Hi,
This is a difficult one to quantify and is related to the time it
takes to load from the disk V the time needed to decrypt the data
loaded. Fast processor, slow drive, fast drive slow processor etc.
If one has enough CPU time, then minimal slowdown should result. If
one is doing heavy calculations then a slowdown might result, though
even then the disk reads should have priority. The question might be,
how much does the encryption slow down the rest of the machine, if it
is busy with other taks.
Containers, rather than dedicated partitions, will be slower, as the
system has to read the data from the original file, and then supply
the data to the caller. IE often a recursive operation back through
the filing system.
When tome permits I will try and do some tests on various hardware and
evalute the mods which were made.
Regards,
Shaun.

Thanks Shaun. I may run a few tests myself this weekend by copying some
of my encrypted data to a non-encrypted drive and then perform some timed
copy tests. I'll report the results here.

[Buzz Murdoch]

Post by Buzz Murdoch

Post by Shaun

Post by r***@invalid.com
Out of curiosity though, do you have any idea what the performance penalty
*should* be for reading data from an encrypted drive (partition) as
opposed to simply reading plain-text data from the system (C:) drive? Is
there any way to express the difference as a percentage? I.E. reading
from an encrypted volume should be X% slower than reading from a
non-encrypted drive on the same system.

Hi,
This is a difficult one to quantify and is related to the time it
takes to load from the disk V the time needed to decrypt the data
loaded. Fast processor, slow drive, fast drive slow processor etc.
If one has enough CPU time, then minimal slowdown should result. If
one is doing heavy calculations then a slowdown might result, though
even then the disk reads should have priority. The question might be,
how much does the encryption slow down the rest of the machine, if it
is busy with other taks.
Containers, rather than dedicated partitions, will be slower, as the
system has to read the data from the original file, and then supply
the data to the caller. IE often a recursive operation back through
the filing system.
When tome permits I will try and do some tests on various hardware and
evalute the mods which were made.
Regards,
Shaun.

Thanks Shaun. I may run a few tests myself this weekend by copying some
of my encrypted data to a non-encrypted drive and then perform some timed
copy tests. I'll report the results here.

Well, it was a slow afternoon so I decided to go ahead and run some tests.
I used the file copy benchmarking utility from
http://www.xbitlabs.com/articles/storage/display/fc-test.html

I ran two sets of tests. One with large files and one with smaller files.
Each portion of the test was run five times. The highest and lowest were
discarded and an average was taken of the remaining three. Both the
plaintext and the encrypted partitions are on the same physical drive.
Both are formatted as NTFS.

C: is plaintext
Z: is encrypted

Test one:
66 large files, totaling 7,444,504,576 bytes

Read from C: 55.240 x
57.439 x
55.707
56.831
55.458
55.998 avg 132,942,330 bytes/second

Read from Z: 137.749 x
130.198
126.875 x
133.974
132.944
132.372 avg 56,239,269 bytes/second

Reading from Z: is 136.39% slower than C:


Write to C: 80.231
79.686
80.481
80.684 x
78.967 x
80.132 avg 92,903,017 bytes/second

Write to Z: 96.939
96.393
92.212
90.512 x
98.281 x
92.602
94.536 avg 78,747,827 bytes/second

Writing to Z: is 17.98% slower than C:

Writing to C: is 43.09% slower than reading from C:
Writing to Z: is 40.02% faster than reading from Z: ??


Test two:
21,755 smaller files, totaling 4,893,790,208 bytes

Read from C: 56.083 x
56.113
56.098
56.083
55.864 x
56.098 avg 87,236,447 bytes/second

Read from Z: 115.394
114.833 x
114.894
116.767 x
114.988
115.092 avg 42,520,681

Reading from Z: is 105.16% slower


Write to C: 86.658 x
95.55 x
93.461
91.915
89.154
91.51 avg 53,478,201 bytes/second

Write to Z: 120.245
112.898 x
155.409 x
136.361
130.168
128.924 avg 37,958,721 bytes/second

Writing to Z: is 40.89% slower

Writing to C: is 63.13% slower than reading from C:
Writing to Z: is 12.01% slower than reading from Z:


Well, that's some weird data. What really jumps out is that the writes
are faster than the reads on the encrypted partition for large files. It
looks to me like there's a problem with read (decrypt) performance.

[Shaun]

Some of this data isn't as bizzare as it might first appear.

When writing you encrypt the block, and the lower disk driver will
then write the data asynchronously. Whist you wait for the DMA
hardware to do that, then the process is free to do other things for
example encrypting the next block.

When reading, you have no choice but to wait for the block to become
available, from the DMA, before you can decrypt it. The CPU is not
idle during the wait, on the ASYNC read (IE it blocks out to the next
task to do OTHER things) but it cannot be decrypting the next incoming
block of data whilst waiting on the DMA, as the driver would not yet
have the data at that point so there's no alternative but to block and
wait for the DMA.

On write you have the data immediately. You encrypt it, and then pass
down the stack where the DMA will asynchronously commit the data to
disk. Once the DMA is set up, the driver will have finished. the task
will then be able to encrypt the next block, perhaps even BEFORE the
DMA has written out the previous one. If (unlikely) the DMA still
hasn't finished, only then will the computer have to wait on it.

As said, when time permits I will review the code.

What algorithm are you using ? Some are faster than others.


Regards.
Shaun.

Post by Buzz Murdoch

Post by Buzz Murdoch

Post by Shaun

Post by r***@invalid.com
Out of curiosity though, do you have any idea what the performance penalty
*should* be for reading data from an encrypted drive (partition) as
opposed to simply reading plain-text data from the system (C:) drive? Is
there any way to express the difference as a percentage? I.E. reading
from an encrypted volume should be X% slower than reading from a
non-encrypted drive on the same system.

Hi,
This is a difficult one to quantify and is related to the time it
takes to load from the disk V the time needed to decrypt the data
loaded. Fast processor, slow drive, fast drive slow processor etc.
If one has enough CPU time, then minimal slowdown should result. If
one is doing heavy calculations then a slowdown might result, though
even then the disk reads should have priority. The question might be,
how much does the encryption slow down the rest of the machine, if it
is busy with other taks.
Containers, rather than dedicated partitions, will be slower, as the
system has to read the data from the original file, and then supply
the data to the caller. IE often a recursive operation back through
the filing system.
When tome permits I will try and do some tests on various hardware and
evalute the mods which were made.
Regards,
Shaun.

Thanks Shaun. I may run a few tests myself this weekend by copying some
of my encrypted data to a non-encrypted drive and then perform some timed
copy tests. I'll report the results here.

Well, it was a slow afternoon so I decided to go ahead and run some tests.
I used the file copy benchmarking utility from
http://www.xbitlabs.com/articles/storage/display/fc-test.html
I ran two sets of tests. One with large files and one with smaller files.
Each portion of the test was run five times. The highest and lowest were
discarded and an average was taken of the remaining three. Both the
plaintext and the encrypted partitions are on the same physical drive.
Both are formatted as NTFS.
C: is plaintext
Z: is encrypted
66 large files, totaling 7,444,504,576 bytes
Read from C: 55.240 x
57.439 x
55.707
56.831
55.458
55.998 avg 132,942,330 bytes/second
Read from Z: 137.749 x
130.198
126.875 x
133.974
132.944
132.372 avg 56,239,269 bytes/second
Write to C: 80.231
79.686
80.481
80.684 x
78.967 x
80.132 avg 92,903,017 bytes/second
Write to Z: 96.939
96.393
92.212
90.512 x
98.281 x
92.602
94.536 avg 78,747,827 bytes/second
Writing to Z: is 40.02% faster than reading from Z: ??
21,755 smaller files, totaling 4,893,790,208 bytes
Read from C: 56.083 x
56.113
56.098
56.083
55.864 x
56.098 avg 87,236,447 bytes/second
Read from Z: 115.394
114.833 x
114.894
116.767 x
114.988
115.092 avg 42,520,681
Reading from Z: is 105.16% slower
Write to C: 86.658 x
95.55 x
93.461
91.915
89.154
91.51 avg 53,478,201 bytes/second
Write to Z: 120.245
112.898 x
155.409 x
136.361
130.168
128.924 avg 37,958,721 bytes/second
Writing to Z: is 40.89% slower
Well, that's some weird data. What really jumps out is that the writes
are faster than the reads on the encrypted partition for large files. It
looks to me like there's a problem with read (decrypt) performance.

[Buzz Murdoch]

Post by Shaun
Some of this data isn't as bizzare as it might first appear.
When writing you encrypt the block, and the lower disk driver will
then write the data asynchronously. Whist you wait for the DMA
hardware to do that, then the process is free to do other things for
example encrypting the next block.
When reading, you have no choice but to wait for the block to become
available, from the DMA, before you can decrypt it. The CPU is not
idle during the wait, on the ASYNC read (IE it blocks out to the next
task to do OTHER things) but it cannot be decrypting the next incoming
block of data whilst waiting on the DMA, as the driver would not yet
have the data at that point so there's no alternative but to block and
wait for the DMA.
On write you have the data immediately. You encrypt it, and then pass
down the stack where the DMA will asynchronously commit the data to
disk. Once the DMA is set up, the driver will have finished. the task
will then be able to encrypt the next block, perhaps even BEFORE the
DMA has written out the previous one. If (unlikely) the DMA still
hasn't finished, only then will the computer have to wait on it.
As said, when time permits I will review the code.
What algorithm are you using ? Some are faster than others.

Thanks for the explanation of the differences between how reads and writes
are handled.

I'm using the (default) Rijndael AES 256-bit (Gladman code).

[Buzz Murdoch]

On Fri, 24 Jun 2011 14:48:24 -0400, Buzz Murdoch <***@invalid.com>
wrote:

Meh. Forgot the system specs.

Windows 7 32 bit.
4GB memory
I7 860 (quad core)
WDC WD1001FALS 1TB SATA drive.
DC 5.5.0