Squeak
  links to this page:    
View this PageEdit this PageUploads to this PageHistory of this PageTop of the SwikiRecent ChangesSearch the SwikiHelp Guide
Magma performance benchmarks
Last updated at 4:08 am UTC on 12 January 2007
Benchmarking is a tricky business because there are so many variables. Not just the hardware configuration, particularly with software as complex and configurable as Magma, it is easy to draw incorrect conclusions about exactly how fast it will perform.

Therefore, Magma includes a class that benchmarks key operations under certain conditions that you may understand how fast it goes. There are three configurations under which to run the benchmarks:
Running all of these is much more informative than running just one because you can discern how much "penalty" there is for various things like serialization or network latency.

To actually run the either of the first two benchmarks, simply print the following in a workspace:
  MagmaBenchmarker runLocalBaseline: false "or true"
or
  MagmaBenchmarker runRemoteBaseline: false "or true"

The boolean argument indicates whether to use WriteBarrier, so you can see how much that improves things.

The third way requires separate server and client machines (or, at least, separate images but it was intended to run on different computers so that the network latency will factor into the results).

On the server machine, execute the following:
  MagmaBenchmarker startBenchmarkHost

This does nothing more than create a benchmark repository of sample objects and begins listening on a port for a remote client to begin the benchmark. You can figure out the IP address of this server by:
  NetworkNameResolver localHostAddress

On another computer, start up a Squeak image with Magma installed and print the following:
  MagmaBenchmarker runAgainstBenchmarkHost: #("the ip address") asByteArray withWriteBarrier: false "or true"

After several minutes to run, it will print out statistics similar to below.

Here is the output from recent runs of MagmaBenchmarker, now available in the "Magma tester" package.

Local, without WriteBarrier


 'The date is 11 January 2007 9:52:52 pm

Hardware Details:
	computer : IBM R40 laptop
	cpu :  Pentium-M
	speed :  1.3GHz
	memory :  768MB
	disk :  IDE

OS Details:
	osVersion : NT
	platformName : Win32
	platformSubtype : IX86
	vmVersion : Squeak3.8 of ''5 May 2005'' [latest update: #6665]
	imageName : C:\Development\Chris\Development\Squeak\currentMagma.image

Image Details:
	version : Squeak3.9
	lastUpdate : 7067

Code Package Details (from Monticello):
	** modified **

Name: Magma tester-cmm.239
Author: cmm
Time: 15 December 2006, 9:53:50 pm
UUID: 46661681-fbb0-c04e-973c-ee12e421ae7d
Ancestors: Magma tester-cmm.238

Magma server details:
	flushAggressively : false

MagmaSession Details:
	isLocal : true
	allowWriteBarrier : false
	refreshPersistentObjectsEvenWhenChangedOnlyByMe : false

Benchmarker Details:
	thousands : 1000

Benchmarks:
---

connectionTests
	timeToDisconnect : 0:00:00:00.294
	timeToConnect : 0:00:00:00.456
---

readTests
	peakRefreshRate : 4168.37121212121 per second.
	singleObjectRead : 2537.892421515697 per second.
	oneThousandElementArrayRead : 204.3591281743651 per second.
	oneThousandElementArrayOfObjectsRead : 6.10116118874237 per second.
	oneMillionObjectPointersRead : 5 seconds.
	oneThousandLevelsDeepRead : 4.901960784313726 per second.
---

writeTests
	peakCommitRate : 24.94014365522746 per second.
	oneThousandElementArrayCommit : 17.853600476096 per second.
	a1001BufferCommit : 10.27464927879866 per second.
	aOneMillionBufferCommit : 17 seconds.
---

magmaArrayTests
	getMagmaArraySize : 7019.396120775845 per second.
	updateThousandsOfMagmaArrayElements : 0.1889287738522577 per second.
---

magmaCollectionTests
	addThousandsOfObjectsTenAtATime : 8.54 seconds.
	addThousandsOfObjectsOneHundredAtATime : 4.38 seconds.
	addThousandsOfObjectsAtOnce : 4.32 seconds.'



Local, with WriteBarrier


'The date is 16 August 2006 11:31:54 pm

Hardware Details:
	computer : IBM R40 laptop
	cpu :  Pentium-M
	speed :  1.3GHz
	memory :  768MB
	disk :  internal IDE

OS Details:
	osVersion : NT
	platformName : Win32
	platformSubtype : IX86
	vmVersion : Squeak3.7 of ''4 September 2004'' [latest update: #5989]
	imageName : C:\Development\Chris\Development\Squeak\lava.image

Image Details:
	version : Squeak3.8
	lastUpdate : 6665

Code Package Details (from Monticello):

Name: Magma tester-cmm.227
Author: cmm
Time: 8 August 2006, 11:59:49 pm
UUID: 55655f77-16cb-f549-a716-8d7aada60584
Ancestors: Magma tester-cmm.226

Magma server details:
	flushAggressively : false

MagmaSession Details:
	isLocal : true
	allowWriteBarrier : true
	refreshPersistentObjectsEvenWhenChangedOnlyByMe : false

Benchmarker Details:
	thousands : 1000

Benchmarks:
---

readTests
	peakRefreshRate : 3608.71302957634 per second.
	singleObjectRead : 3249.750049990002 per second.
	oneThousandElementArrayRead : 208.991008991009 per second.
	oneThousandElementArrayOfObjectsRead : 6.98742263924935 per second.
	oneMillionObjectPointersRead : 6 seconds.
	oneThousandLevelsDeepRead : 5.272407732864675 per second.
---

writeTests
	peakCommitRate : 83.2667731629393 per second.
	oneThousandElementArrayCommit : 37.8863409770688 per second.
	a1001BufferCommit : 10.74199323652278 per second.
	aOneMillionBufferCommit : 28 seconds.
---

magmaArrayTests
	getMagmaArraySize : 7694.461107778444 per second.
	updateThousandsOfMagmaArrayElements : 13
---

magmaCollectionTests
	addThousandsOfObjectsTenAtATime : 5.769 seconds.
	addThousandsOfObjectsOneHundredAtATime : 4.025 seconds.
	addThousandsOfObjectsAtOnce : 8.783 seconds.'



Remote baseline (loopback address to same image = no network latency)

 'The date is 11 January 2007 10:08:48 pm

Hardware Details:
	computer : IBM R40 laptop
	cpu :  Pentium-M
	speed :  1.3GHz
	memory :  768MB
	disk :  IDE

OS Details:
	osVersion : NT
	platformName : Win32
	platformSubtype : IX86
	vmVersion : Squeak3.8 of ''5 May 2005'' [latest update: #6665]
	imageName : C:\Development\Chris\Development\Squeak\currentMagma.image

Image Details:
	version : Squeak3.9
	lastUpdate : 7067

Code Package Details (from Monticello):
	** modified **

Name: Magma tester-cmm.239
Author: cmm
Time: 15 December 2006, 9:53:50 pm
UUID: 46661681-fbb0-c04e-973c-ee12e421ae7d
Ancestors: Magma tester-cmm.238

MagmaSession Details:
	isLocal : false
	allowWriteBarrier : false
	refreshPersistentObjectsEvenWhenChangedOnlyByMe : false

Benchmarker Details:
	thousands : 1000

Benchmarks:
---

connectionTests
	timeToDisconnect : 0:00:00:00.358
	timeToConnect : 0:00:00:03.782
---

readTests
	peakRefreshRate : 210.3896103896104 per second.
	singleObjectRead : 230.0159872102318 per second.
	oneThousandElementArrayRead : 103.7170263788969 per second.
	oneThousandElementArrayOfObjectsRead : 5.950019833399445 per second.
	oneMillionObjectPointersRead : 7 seconds.
	oneThousandLevelsDeepRead : 4.712350284704496 per second.
---

writeTests
	peakCommitRate : 16.08481082069092 per second.
	oneThousandElementArrayCommit : 13.57827476038339 per second.
	a1001BufferCommit : 7.75656324582339 per second.
	aOneMillionBufferCommit : 22 seconds.
---

magmaArrayTests
	getMagmaArraySize : 127.1186440677966 per second.
	updateThousandsOfMagmaArrayElements : 0.2551996937603675 per second.
---

magmaCollectionTests
	addThousandsOfObjectsTenAtATime : 12.035 seconds.
	addThousandsOfObjectsOneHundredAtATime : 5.638 seconds.
	addThousandsOfObjectsAtOnce : 6.022 seconds.'


Remote over 54Mbps wireless 802.11g LAN

The date is 14 November 2006 9:39:30 pm

Hardware Details:
	computer : IBM R40 laptop
	cpu :  Pentium-M
	speed :  1.3GHz
	memory :  768MB
	disk :  internal IDE

OS Details:
	osVersion : NT
	platformName : Win32
	platformSubtype : IX86
	vmVersion : Squeak3.7 of ''4 September 2004'' [latest update: #5989]
	imageName : C:\Development\Chris\Development\Squeak\currentMagma.image

Image Details:
	version : Squeak3.8
	lastUpdate : 6665

Code Package Details (from Monticello):
	** modified **

Name: Magma tester-cmm.233
Author: cmm
Time: 9 November 2006, 6:14:01 pm
UUID: 45cd9191-142f-ed45-86d6-de1212293992
Ancestors: Magma tester-cmm.232

MagmaSession Details:
	isLocal : false
	allowWriteBarrier : false
	refreshPersistentObjectsEvenWhenChangedOnlyByMe : false

Benchmarker Details:
	thousands : 1000

Benchmarks:
---

readTests
	peakRefreshRate : 112.9322406556066 per second.
	singleObjectRead : 122.5059856344773 per second.
	oneThousandElementArrayRead : 60.04388589666866 per second.
	oneThousandElementArrayOfObjectsRead : 2.986857825567503 per second.
	oneMillionObjectPointersRead : 11 seconds.
	oneThousandLevelsDeepRead : 2.869166029074216 per second.
---

writeTests
	peakCommitRate : 11.76705225368967 per second.
	oneThousandElementArrayCommit : 7.72889417360285 per second.
	a1001BufferCommit : 4.332414336352895 per second.
	aOneMillionBufferCommit : 26 seconds.
---

magmaArrayTests
	getMagmaArraySize : 135.8641358641359 per second.
	updateThousandsOfMagmaArrayElements : 0.2605184316790413 per second.
---

magmaCollectionTests
	addThousandsOfObjectsTenAtATime : 16.604 seconds.
	addThousandsOfObjectsOneHundredAtATime : 7.824 seconds.
	addThousandsOfObjectsAtOnce : 12.48 seconds.


Remote over the Internet