Magma performance benchmarks
Last updated at 4:08 am UTC on 12 January 2007
Benchmarking is a tricky business because there are so many variables. Not just the hardware configuration, particularly with software as complex and configurable as Magma, it is easy to draw incorrect conclusions about exactly how fast it will perform.
Therefore, Magma includes a class that benchmarks key operations under certain conditions that you may understand how fast it goes. There are three configurations under which to run the benchmarks:
- local
- remote with serialization with no network latency
- remote over a network
Running all of these is much more informative than running just one because you can discern how much "penalty" there is for various things like serialization or network latency.
To actually run the either of the first two benchmarks, simply print the following in a workspace:
MagmaBenchmarker runLocalBaseline: false "or true"
or
MagmaBenchmarker runRemoteBaseline: false "or true"
The boolean argument indicates whether to use WriteBarrier, so you can see how much that improves things.
The third way requires separate server and client machines (or, at least, separate images but it was intended to run on different computers so that the network latency will factor into the results).
On the server machine, execute the following:
MagmaBenchmarker startBenchmarkHost
This does nothing more than create a benchmark repository of sample objects and begins listening on a port for a remote client to begin the benchmark. You can figure out the IP address of this server by:
NetworkNameResolver localHostAddress
On another computer, start up a Squeak image with Magma installed and print the following:
MagmaBenchmarker runAgainstBenchmarkHost: #("the ip address") asByteArray withWriteBarrier: false "or true"
After several minutes to run, it will print out statistics similar to below.
Here is the output from recent runs of MagmaBenchmarker, now available in the "Magma tester" package.
Local, without WriteBarrier
'The date is 11 January 2007 9:52:52 pm
Hardware Details:
computer : IBM R40 laptop
cpu : Pentium-M
speed : 1.3GHz
memory : 768MB
disk : IDE
OS Details:
osVersion : NT
platformName : Win32
platformSubtype : IX86
vmVersion : Squeak3.8 of ''5 May 2005'' [latest update: #6665]
imageName : C:\Development\Chris\Development\Squeak\currentMagma.image
Image Details:
version : Squeak3.9
lastUpdate : 7067
Code Package Details (from Monticello):
** modified **
Name: Magma tester-cmm.239
Author: cmm
Time: 15 December 2006, 9:53:50 pm
UUID: 46661681-fbb0-c04e-973c-ee12e421ae7d
Ancestors: Magma tester-cmm.238
Magma server details:
flushAggressively : false
MagmaSession Details:
isLocal : true
allowWriteBarrier : false
refreshPersistentObjectsEvenWhenChangedOnlyByMe : false
Benchmarker Details:
thousands : 1000
Benchmarks:
---
connectionTests
timeToDisconnect : 0:00:00:00.294
timeToConnect : 0:00:00:00.456
---
readTests
peakRefreshRate : 4168.37121212121 per second.
singleObjectRead : 2537.892421515697 per second.
oneThousandElementArrayRead : 204.3591281743651 per second.
oneThousandElementArrayOfObjectsRead : 6.10116118874237 per second.
oneMillionObjectPointersRead : 5 seconds.
oneThousandLevelsDeepRead : 4.901960784313726 per second.
---
writeTests
peakCommitRate : 24.94014365522746 per second.
oneThousandElementArrayCommit : 17.853600476096 per second.
a1001BufferCommit : 10.27464927879866 per second.
aOneMillionBufferCommit : 17 seconds.
---
magmaArrayTests
getMagmaArraySize : 7019.396120775845 per second.
updateThousandsOfMagmaArrayElements : 0.1889287738522577 per second.
---
magmaCollectionTests
addThousandsOfObjectsTenAtATime : 8.54 seconds.
addThousandsOfObjectsOneHundredAtATime : 4.38 seconds.
addThousandsOfObjectsAtOnce : 4.32 seconds.'
Local, with WriteBarrier
'The date is 16 August 2006 11:31:54 pm
Hardware Details:
computer : IBM R40 laptop
cpu : Pentium-M
speed : 1.3GHz
memory : 768MB
disk : internal IDE
OS Details:
osVersion : NT
platformName : Win32
platformSubtype : IX86
vmVersion : Squeak3.7 of ''4 September 2004'' [latest update: #5989]
imageName : C:\Development\Chris\Development\Squeak\lava.image
Image Details:
version : Squeak3.8
lastUpdate : 6665
Code Package Details (from Monticello):
Name: Magma tester-cmm.227
Author: cmm
Time: 8 August 2006, 11:59:49 pm
UUID: 55655f77-16cb-f549-a716-8d7aada60584
Ancestors: Magma tester-cmm.226
Magma server details:
flushAggressively : false
MagmaSession Details:
isLocal : true
allowWriteBarrier : true
refreshPersistentObjectsEvenWhenChangedOnlyByMe : false
Benchmarker Details:
thousands : 1000
Benchmarks:
---
readTests
peakRefreshRate : 3608.71302957634 per second.
singleObjectRead : 3249.750049990002 per second.
oneThousandElementArrayRead : 208.991008991009 per second.
oneThousandElementArrayOfObjectsRead : 6.98742263924935 per second.
oneMillionObjectPointersRead : 6 seconds.
oneThousandLevelsDeepRead : 5.272407732864675 per second.
---
writeTests
peakCommitRate : 83.2667731629393 per second.
oneThousandElementArrayCommit : 37.8863409770688 per second.
a1001BufferCommit : 10.74199323652278 per second.
aOneMillionBufferCommit : 28 seconds.
---
magmaArrayTests
getMagmaArraySize : 7694.461107778444 per second.
updateThousandsOfMagmaArrayElements : 13
---
magmaCollectionTests
addThousandsOfObjectsTenAtATime : 5.769 seconds.
addThousandsOfObjectsOneHundredAtATime : 4.025 seconds.
addThousandsOfObjectsAtOnce : 8.783 seconds.'
Remote baseline (loopback address to same image = no network latency)
'The date is 11 January 2007 10:08:48 pm
Hardware Details:
computer : IBM R40 laptop
cpu : Pentium-M
speed : 1.3GHz
memory : 768MB
disk : IDE
OS Details:
osVersion : NT
platformName : Win32
platformSubtype : IX86
vmVersion : Squeak3.8 of ''5 May 2005'' [latest update: #6665]
imageName : C:\Development\Chris\Development\Squeak\currentMagma.image
Image Details:
version : Squeak3.9
lastUpdate : 7067
Code Package Details (from Monticello):
** modified **
Name: Magma tester-cmm.239
Author: cmm
Time: 15 December 2006, 9:53:50 pm
UUID: 46661681-fbb0-c04e-973c-ee12e421ae7d
Ancestors: Magma tester-cmm.238
MagmaSession Details:
isLocal : false
allowWriteBarrier : false
refreshPersistentObjectsEvenWhenChangedOnlyByMe : false
Benchmarker Details:
thousands : 1000
Benchmarks:
---
connectionTests
timeToDisconnect : 0:00:00:00.358
timeToConnect : 0:00:00:03.782
---
readTests
peakRefreshRate : 210.3896103896104 per second.
singleObjectRead : 230.0159872102318 per second.
oneThousandElementArrayRead : 103.7170263788969 per second.
oneThousandElementArrayOfObjectsRead : 5.950019833399445 per second.
oneMillionObjectPointersRead : 7 seconds.
oneThousandLevelsDeepRead : 4.712350284704496 per second.
---
writeTests
peakCommitRate : 16.08481082069092 per second.
oneThousandElementArrayCommit : 13.57827476038339 per second.
a1001BufferCommit : 7.75656324582339 per second.
aOneMillionBufferCommit : 22 seconds.
---
magmaArrayTests
getMagmaArraySize : 127.1186440677966 per second.
updateThousandsOfMagmaArrayElements : 0.2551996937603675 per second.
---
magmaCollectionTests
addThousandsOfObjectsTenAtATime : 12.035 seconds.
addThousandsOfObjectsOneHundredAtATime : 5.638 seconds.
addThousandsOfObjectsAtOnce : 6.022 seconds.'
Remote over 54Mbps wireless 802.11g LAN
The date is 14 November 2006 9:39:30 pm
Hardware Details:
computer : IBM R40 laptop
cpu : Pentium-M
speed : 1.3GHz
memory : 768MB
disk : internal IDE
OS Details:
osVersion : NT
platformName : Win32
platformSubtype : IX86
vmVersion : Squeak3.7 of ''4 September 2004'' [latest update: #5989]
imageName : C:\Development\Chris\Development\Squeak\currentMagma.image
Image Details:
version : Squeak3.8
lastUpdate : 6665
Code Package Details (from Monticello):
** modified **
Name: Magma tester-cmm.233
Author: cmm
Time: 9 November 2006, 6:14:01 pm
UUID: 45cd9191-142f-ed45-86d6-de1212293992
Ancestors: Magma tester-cmm.232
MagmaSession Details:
isLocal : false
allowWriteBarrier : false
refreshPersistentObjectsEvenWhenChangedOnlyByMe : false
Benchmarker Details:
thousands : 1000
Benchmarks:
---
readTests
peakRefreshRate : 112.9322406556066 per second.
singleObjectRead : 122.5059856344773 per second.
oneThousandElementArrayRead : 60.04388589666866 per second.
oneThousandElementArrayOfObjectsRead : 2.986857825567503 per second.
oneMillionObjectPointersRead : 11 seconds.
oneThousandLevelsDeepRead : 2.869166029074216 per second.
---
writeTests
peakCommitRate : 11.76705225368967 per second.
oneThousandElementArrayCommit : 7.72889417360285 per second.
a1001BufferCommit : 4.332414336352895 per second.
aOneMillionBufferCommit : 26 seconds.
---
magmaArrayTests
getMagmaArraySize : 135.8641358641359 per second.
updateThousandsOfMagmaArrayElements : 0.2605184316790413 per second.
---
magmaCollectionTests
addThousandsOfObjectsTenAtATime : 16.604 seconds.
addThousandsOfObjectsOneHundredAtATime : 7.824 seconds.
addThousandsOfObjectsAtOnce : 12.48 seconds.
Remote over the Internet