osgjs : benchmarking matrix inverse 4×4

The matrix inversion function which is at the heart of the JavaScript OpenSceneGraph library has been benchmarked using the benchmark.js and the performance results archived using a custom method.

downloading

The repository was extracted using

hg clone https://bitbucket.org/cedricpinson/osgjs

at revision

root@osgjs:/var/www/osgjs# hg head
changeset:   34:04464429e4fe
tag:         tip
user:        Cedric Pinson 
date:        Sat Apr 02 16:40:43 2011 +0200
summary:     add osgDB.js file, we should put futur io code in this file

tests and coverage

The tests must be run with the webgl extension enabled otherwise some of them will fail.
jslint was installed by doing the following:

apt-get source jscoverage
cd jscoverage*
./configure
make
make install
ldconfig

and run against osg.js. The errors were fixed.
jscoverage was run to process the checked out tree:

root@osgjs:/var/www# jscoverage osgjs osgjs-coverage

and it can be used to check how much of osg.js is covered by the tests (64% at this time).

virtual machine

In order to keep the development context for further work, a dedicated virtual machine was created.

firefox-4

The OpenGL extensions required by osgjs are only available on firefox version 4. It was installed using the experimental 4.0-3 package for Debian GNU/Linux.
However, it turned out to not work despite running the instructions to activate it ( Type “about:config” in your browser’s URL bar – Do a search for “webgl” in the Filter field – Double-click the “enabled_for_all_sites” ). The 3D accelearation is available as shown by :

$ glxinfo | grep 'renderer string'
OpenGL renderer string: Mesa DRI Mobile Intel® GM45 Express Chipset GEM 20090712 2009Q2 RC3

An attempt was made to recompile iceweasel from sources.

apt-get source iceweasel=4.0-3
cd iceweasel*
debuild -uc -us
dpkg-source: info: using source format `3.0 (quilt)'
dpkg-source: info: building iceweasel using existing ./iceweasel_4.0.orig.tar.bz2
dpkg-source: error: unwanted binary file: ../about-base.png
dpkg-source: error: unwanted binary file: ../aboutCredits-base.png
dpkg-source: error: detected 2 unwanted binary file(s) (add them in debian/source/include-binaries to allow their inclusion).
dpkg-buildpackage: error: dpkg-source -b iceweasel-4.0 gave error exit status 2
debuild: fatal error at line 1324:
dpkg-buildpackage -rfakeroot -D -us -uc failed

The firefox tarbal for x86_64 was downloaded instead. When running it from the command line it says

loic@bet:/home/tmp/firefox$ ./firefox
[GLX] your GL driver is currently blocked. If you would like to bypass this, define the MOZ_GLX_IGNORE_BLACKLIST environment variable.

It turns out that it works for iceweasel too and firefox was de-installed.

 MOZ_GLX_IGNORE_BLACKLIST=yes iceweasel

benchmarking

Since the goal is to measure the performances of javascript functions using floating point numbers and doing only computation (no DOM or Ajax), the following methodology was implemented. A function is used to sample the performances of the JavaScript VM by running a loop doing 1 million sqrt:

function compute_unit() {
    var t = (new Date()).getTime();
    for(var i = 0; i < 1000000; i++) {
        Math.sqrt(4);
    }
    return (new Date()).getTime() - t;
}

var unit = compute_unit();

and the unit variable will be used to compute a ratio that should be insensitive to the CPU power. A function is defined to compute such a ratio and expect a predefined result that is bound to a given JavaScript VM. For instance, the osg.Matrix.inverse4x4 function is expected to produce the following ratios:

    var expected = {
        'mozilla': {
            '2.0': 210,
            '1.9.2.16': 110
        },
        'webkit': {
            '534.16': 27
        }
    };

The smaller the ratio, the faster the function is. Based on this methodology, a set of tests were written to compare the matrix inversion functions of osgjs and CubicVR and can be run.
An article explains the concepts implemented at Benchmark.js and references an informative article from John Resig.
The user interface of is not properly published (no hint in the FAQ although there is an example extracted from the jsperf.com site.
A benchmark.js comparison test compares the same four matrix inversion functions as matrix.js:

var m = osg.Matrix.makeIdentity();
suite.add('osg.Matrix.inverse4x4', function() {
        osg.Matrix.inverse4x4(m);
    }).add('CubicVR.inverse$1', function() {
        CubicVR.inverse$1(m);
    }).add('CubicVR.inverse$2', function() {
        CubicVR.inverse$2(m);
    }).add('CubicVR.inverse', function() {
        CubicVR.inverse(m);
    }).on('cycle', function(bench) {
        console.log(String(bench));
    });

It must be run with the debugger disabled and a console available to display the test results (Control-Shift-K under firefox/iceweasel). On Firefox/Iceweasel-4.0 the results are:

[18:26:41.987] "osg.Matrix.inverse4x4 x 1,272,299 ops/sec ±2.26% (75 runs sampled)"
[18:26:47.033] "CubicVR.inverse$1 x 221,986 ops/sec ±2.05% (78 runs sampled)"
[18:26:52.090] "CubicVR.inverse$2 x 848,696 ops/sec ±1.85% (80 runs sampled)"
[18:26:57.105] "CubicVR.inverse x 1,353,549 ops/sec ±2.82% (77 runs sampled)"
[18:26:57.108] "Fastest is CubicVR.inverse"

And on Chromium-10.0.648.204 the results are:

osg.Matrix.inverse4x4 x 2,495,218 ops/sec ±2.18% (73 runs sampled)
CubicVR.inverse$1 x 537,786 ops/sec ±1.03% (77 runs sampled)
CubicVR.inverse$2 x 1,122,268 ops/sec ±1.60% (78 runs sampled)
CubicVR.inverse x 1,644,006 ops/sec ±0.53% (81 runs sampled)
Fastest is osg.Matrix.inverse4x4

Which seems to indicate that the way osg.Matrix.inverse4x4 is written is slightly slower on Firefox but much faster on Chromium. The reasons for such a large difference have not been determined.