<- previous index next ->
Take a look inside the hard drive being passed around.
Mine is bigger than yours.
How fast can you read a block of data?
There are four time components that must be known to answer
this question.
1) The time for the read head to get to the required track.
This is seek time.
2) The time for the disk to rotate to start reading the first byte.
This is the rotational delay time.
3) The time to transfer the data from the disk to your RAM.
This is the transfer time.
4) Overhead that can be from software, application, OS or drivers.
This is overhead time.
Seek time
The head may be on any track, thus there is seek time
before any data can be read. The manufacturers published
average seek time is standardized at the time to go from
track 0 to the middle track, measured in milliseconds.
In the 1990's the size of disk had become large enough
such that the measured average seek time was 1/4 the
published average seek time. We use 1/4 the published average
seek time for our homework and exams. For your computer,
having a hard drive with capacity over 120GB, I suggest using
1/8 the published average seek time for your estimates. The reason
is that the files you are working with tend to cluster, thus
you rarely will have a seek traveling 1/4 the tracks on the disk.
For my example below, the published average seek time was 5.4 ms
and thus 5.4/4 = 1.4 ms is used.
Rotational delay time
The disk is spinning at a known Revolutions Per Minute, RPM.
We deal in seconds, thus divide the RPM by 60 to get
Revolutions Per Second, RPS.
How long, on average, does it take for the read head to reach
data? This is the rotational delay time and only depends on
the RPS. On average the time will be the time for 1/2 of a
revolution, thus 1/2 * 1/RPS . Typically expressed in
milliseconds, ms. Some values are:
RPM RPS 1/4 * 1/RPS
seconds milliseconds
3600 60 0.00417 4.17
5400 90 0.00277 2.77
7200 120 0.00208 2.08
10,025 167 0.00155 1.55
15,000 250 0.00100 1.00
Transfer time
The time to transfer data depends on the bandwidth, typically
given in Megabytes per second. The disk drive has internal RAM
and usually can deliver a continuous stream of bytes at near
the maximum transfer rate. The transfer may be slowed by your
computers system bus or your RAM or other contention for the
system bus to RAM path. The example below uses an 80MB/s
transfer rate. Thus 80MB can be transferred in one second.
Overhead time
The overhead time is estimated. 0.6ms
Example
How long does it take to read
a file from disk? (example calculation)
time = average seek time +
average rotational delay +
transfer time +
overhead
published average seek = 5.4 ms
"average" seek = 5.4/4 = 1.4ms
10,025 RPM or 167 RPS
1/2 * 1/167 = .00299 sec = 3.0ms
Overhead assumed = 0.6ms
Size independent delay, sum= 5.0ms
At 80 MB/sec transfer rate:
10KB 100KB 1MB 10MB
0.125 1.25 12.5 125. transfer time in ms
5.0 5.0 5.0 5.0
_____ ____ ____ _____
5.125 6.25 17.5 130.0 ms
This is a one block "first read"
The next read could be buffered
Notice that on small files, the latency (times 1) 2) and 3) dominate.
On large files the transfer time dominates. Today, files in the
tens of megabytes are common. Many years ago most files were around
10 kilobytes. Today 1 to 10 megabyte is typical.
A benchmark I ran on reading 1KB, 10KB, 100KB, and 1MB of data
from a 10MB file.
/* time_io.c check how much is cached in ram */
/* assumed pre-existing data file time_io.dat */
/* created by running time_io_init */
#include <stdio.h>
#include <time.h>
int main()
{
FILE * handle;
int i;
int j;
double cpu;
char buf[1000000]; /* 1MB */
int check;
int n = 10000; /* number of reads on 10MB file for buf1*/
int k = 1000; /* number of bytes read per read */
printf("time_io.c 10MB file, read 1KB, 10KB, 100KB, 1MB \n");
handle = fopen("time_io.dat","rb");
printf("On rebooted machine, first read \n");
cpu = (double)clock()/(double)CLOCKS_PER_SEC;
for(i=0; i<n; i++)
{
check = fread(buf, k, 1, handle);
if(check != buf[1]) printf("check failed \n");
}
cpu = (double)clock()/(double)CLOCKS_PER_SEC - cpu;
fclose(handle);
printf("first read time %g seconds \n", cpu);
for(n=10000; n>=10; n=n/10)
{
printf("more reads, cached? consistent? \n");
for(j=2; j<10; j++)
{
handle = fopen("time_io.dat","rb");
cpu = (double)clock()/(double)CLOCKS_PER_SEC;
for(i=0; i<n; i++)
{
check = fread(buf, k, 1, handle);
if(check != buf[1]) printf("check failed \n");
}
cpu = (double)clock()/(double)CLOCKS_PER_SEC - cpu;
fclose(handle);
printf("%d read time %g seconds for %dKB block \n", j, cpu, k/1000);
}
k = k*10;
}
return 0;
} /* end time_io.c */
One computers output:
time_io.c 10MB file, read 1KB, 10KB, 100Kb, 1MB
On rebooted machine, first read
first read time 0.12 seconds
more reads, cached? consistent?
2 read time 0.06 seconds for 1KB block
3 read time 0.06 seconds for 1KB block
4 read time 0.06 seconds for 1KB block
5 read time 0.06 seconds for 1KB block
6 read time 0.06 seconds for 1KB block
7 read time 0.06 seconds for 1KB block
8 read time 0.06 seconds for 1KB block
9 read time 0.05 seconds for 1KB block
more reads, cached? consistent?
2 read time 0.05 seconds for 10KB block
3 read time 0.05 seconds for 10KB block
4 read time 0.04 seconds for 10KB block
5 read time 0.05 seconds for 10KB block
6 read time 0.05 seconds for 10KB block
7 read time 0.05 seconds for 10KB block
8 read time 0.05 seconds for 10KB block
9 read time 0.05 seconds for 10KB block
more reads, cached? consistent?
2 read time 0.08 seconds for 100KB block
3 read time 0.07 seconds for 100KB block
4 read time 0.09 seconds for 100KB block
5 read time 0.07 seconds for 100KB block
6 read time 0.07 seconds for 100KB block
7 read time 0.06 seconds for 100KB block
8 read time 0.08 seconds for 100KB block
9 read time 0.08 seconds for 100KB block
more reads, cached? consistent?
2 read time 0.09 seconds for 1000KB block
3 read time 0.09 seconds for 1000KB block
4 read time 0.09 seconds for 1000KB block
5 read time 0.09 seconds for 1000KB block
6 read time 0.11 seconds for 1000KB block
7 read time 0.10 seconds for 1000KB block
8 read time 0.10 seconds for 1000KB block
9 read time 0.10 seconds for 1000KB block
Why did I reboot to run a file read test?
On a computer that is not shut down, a file could
remain in RAM and even partially in cache for days
to weeks, if you were not using the computer.
By now you should know that I do a lot of benchmarking.
I ran the above program on two computers each with two
operating systems with three disk types.
Block 2.5GHz 2.5GHz 1GHz 1GHz
Size P4 ATA 100 P4 ATA 100 ATA 66 SCSI 160
Windows XP Linux Windows 98 Linux
1KB 0.0000015 0.000001 0.000016 0.000004
10KB 0.000015 0.000010 0.000060 0.000035
100KB 0.000150 0.000100 0.000500 0.000300
1MB 0.003100 0.002000 0.005000 0.004000
Fine print: CPU time in seconds, most frequent value of eight
measurements after first read. Using fopen, fread, binary
block read. Each measurement read 10MB. e,g 10 blocks read
at 1MB, 100 blocks read at 100KB, 10,000 blocks read at 1KB.
Other than the first number that is 1.5 microseconds, the
numbers can be read as integer microseconds.
As expected the SCSI disk was faster than the ATA disk.
Note that the faster system clock can allow the actual
transfer rate to be near the maximum while a slower clock
speed can limit the transfer rate. The operating system,
drivers and libraries have some impact on total time. This
is lumped into "overhead."
Where do you find the disk specifications? Both the manufacturer and
some retailers publish the disk specifications, and some prices.
e.g.
evolution specs
2007 hard drives, note cache, RPM, transfer rate
Then SATA replaced ATA
Serial ATA changed the wiring and protocol. ATA had wide flat cables.
Driven by PC manufactures Dell, Gateway, HP, etc, they needed thinner
cables. Thus higher speed transfer over fewer wires.
Typical SATA bus maximum transfer rate is 3GB/s, 3 gigabytes per second.
Similar latency, similar seek, faster transfer rate.
A single drive with 500GB of storage became available at reasonable cost.
A terabyte of disk storage became practical for a desktop PC.
Now multiple terabyte 6Gb/s disks are available.
Still too slow!
Now, SSD, Solid State Disks
Replace the rotating disk drive with NAND Flash digital logic storage.
Technology explanation
Performance comparisons
One technical specification
Transcend 128GB $229.99 TS128GSSD25S-M
enclosure was needed for desktops, initially
Check for latest size, speed, cost
computer-SSD-search SSD
Reworking the example above for time to read a file:
Transfer time
The time to transfer data depends on the bandwidth, typically
given in Megabytes per second. The example below uses an 80MB/s
transfer rate. Thus 80MB can be transferred in one second.
Overhead time
The overhead time is estimated. 0.6ms
No seek time, no rotational delay time, for SSD
Example
How long does it take to read
a file from disk? (example calculation)
time = transfer time +
overhead
At 80 MB/sec transfer rate:
10KB 100KB 1MB 10MB
0.125 1.25 12.5 125. trans
0.6 0.6 0.6 0.6
_____ ____ ____ _____
0.725 1.85 13.1 125.6 ms
This is a one block "first read"
The next read could be buffered
Notice that on very small files, the overhead time dominates.
On large files the transfer time dominates. Today, files in the
tens of megabytes are common. Many year ago most files were around
10 kilobytes.
The SSD has a speedup of 7.07 for a 10KB file.
The SSD has a speedup of 1.03 for a 10MB file.
Your mileage may vary.
A typical desktop is executing 4,000,000 instructions per ms, millisecond.
Homework 11
<- previous index next ->