Bandwidth testing

Units

For these measurements:

• 1 kB/s = 1,000 bytes per second
• 1 mB/s = 1,000,000 bytes per second
• 1 gB/s = 1,000,000,000 bytes per second

Memory:

• 1 Kb = 1024^2 bytes = 1,048,576 bytes
• 1 Gb = 1024^3 bytes = 1,073,741,824 bytes
• 1 Tb = 1024^4 bytes = 1,099,511,627,776 bytes

Machines tested

machine name	processor speed	memSize	10 files of size	dd /dev/zero to /dev/shm	cp /dev/shm to /tmp	cp /dev/shm to hive	rsync /dev/shm to /dev/shm	rsync /dev/shm via rsh	UDT via UPD
hgwdev	Opteron 1150 Mhz	32 Gb	1 gB	5278 mB/s 10X dd	287 mB/s	58 mB/s	1638 mB/s 20X rsync	n/a	4070 mB/s 10X send
kolossus	Intel 2261 Mhz	1 Tb	1 gB	6700 mB/s 10X dd	192 mB/s	58 mB/s	120 mB/s 4X rsync	115 mB/s	1135 mB/s
okazaki	Intel 1600 Mhz	2 Gb	75 mB	700 mB/s	90 mB/s	11.3 mB/s	11.5 mB/s	11.6 mB/s	10.7 mB/s 4X send
mitzie	Intel 400 Mhz	512 Kb	18 mB	113 mB/s	25 mB/s	n/a	54.4 kB/s up 316 kB/s down	n/a
cloud0	Opteron 2000 Mhz	7.5 Gb	288 mB	940 mB/s 4X dd	78 mB/s	n/a	16 mB/s 2X rsync	43 mB/s 3X rsync	46 mB/s 3X send

Procedure

Sizing files based on available /dev/shm space. Largest file size limited to 1 gB.

The /dev/zero to /dev/shm write is done ten times with a dd command:

 dd if=/dev/zero of=/dev/shm/memToMem.$I.zero bs=200000 count=$ddCount

The /dev/shm copy to /tmp is done ten times with the cp command:

cp -p /dev/shm/memToMem.$I.zero ./memToFile.$I.zero

These 10X copies were repeated three times. The slowest measurement of the three times was recorded.

The rsync /dev/shm to /dev/shm was to and from hgwdev. Multiple instances of rsync from kolossus to hgwdev seemed to add together for a total bandwidth higher than a single instance of rsync.

The machine mitzie is at the end of a Santa Cruz DSL line and over a WiFi network connection across the driveway to the garage.