Simple Sphere Benchmark

Introduction

The VTK Sphere Benchmark is a simple platform-independent Tcl/Tk graphics performance test using VTK library. It is small with no (or little) version requirements. It is designed to get a rough idea of the capabilities and efficiency of your system by displaying a 3D object and computing an estimation of the average throughput in polygons/s. Different rendering options might be used, and automation is provided to try the same bench at varying 3D complexity. A final report may be sent to us and all results are saved to a database for further comparisons.

What's new

12 Sep. 2001 : version 2.4

fixed compatibility problems with VTK 3.2.

6 Jan. 2001 : version 2.3

fixed compatibility problems.

27 July 2000 : version 2.2

fixed a "jitter" bug,
you do not need to interact anymore with the sphere before lauching the bench.

28 Nov. 1999 : version 2.1

fixed compatibility problem between 2.x and 3.0 (default clipping range was computed differently),
displays VTK version when available,
removed 512x512 resolution (which was of no interest in a 400x400 window),
some slight GUI changes,
new command "Get system info".

7 Apr. 1999 : version 2.0

a Tcl/Tk GUI (Graphical User Interface),
new options (texture, stripper, wireframe, transparency),
quick ways to change the size of the rendering window,
report facilities.

GUI

Sphere

Results

Current results are hosted by Robert Rivière at this page.

Download

Shift-click to download the script sphere-bench.tcl (around 22 Ko). Of course, you might download Tcl/Tk if you haven't installed it yet (it's straightforward and painless), and VTK. Beware of any Tcl/Tk plugin that might launch the script automatically.

Use

Run it, either by double-clicking on the script (Windows), or running VTK and sourcing the script (UNIX). Basically, the script will display some information at startup regarding the authors, the system, and some default values and tests. It will also display notes, warnings, and important features in different colors.

Follow the instructions. Find the best sphere resolutions for your card, launch "Bench combinations" and send us your results (copy the session with Control-/) along with a complete description of your system (VTK/OS version, hardware description of your CPU, RAM and graphic board). The whole stuff might be posted to the VTK mailing-list and email'ed to Robert Rivière and/or me ( Sébastien Barré).

The following sections will detail important components of the GUI, and their meaning.

Commands

GUI

"Bench current" : you might play with this script by choosing a set of sphere resolutions and options, then starting the bench for these current settings with this button. This will rotate the sphere for every resolution, according to the options, and display the results in term of k-polygons per second. Do not send us these results, use the next button instead :)
"Bench combinations" : every option might be activated separately, thus leading to a huge number of different combinations. We had to normalize this and choose among these combinations what would appear to be relevant for our database : after all, this bench was meant to compare different platforms. Hence, this button will launch "Bench current" for different options combinations (shown at startup) and display a summary of these results (best performances first). Do not forget that it will use the sphere resolutions activated at that moment : this might take some time and might just be useless for the resolutions not suited to your graphic card.
"Get system info" : displays some informations regarding your system (also shown at startup).
"Quit" : guess what :)

Sphere resolutions

The sphere resolution is the parameter influencing the complexity of the test, by increasing the number of polygons displayed in each frame. Our sphere is not an "exact" sphere, it's a polygon approximation of its surface, based on subdivisions made along its meridians and parallels. Increasing the number of subdivisions will therefore increase the total number of polygons used to represent the 3D object (indeed, for a given resolution R, the number of triangles is 2 * R * (R - 1)).

GUI	16x16	32x32

	64x64	128x128

The GUI allows you to select/unselect a set of predefined sphere resolutions. The bench will then be performed for every selected resolution. You might tell us that this has no importance if the result is computed in polygons/s : simple spheres will be displayed faster, complex spheres slower, but the ratio should be the same. Well, that's not exactly true, that's why we provided this set of resolutions :

the basic reason is that your graphic card might just be too powerful for a given resolution : your board will spend most of this time waiting for the CPU, the bus, or something like that, and it won't be "stressed" in any way.
on the contrary, your rendering solution might just be too slow for anything larger than a simple resolution (typically pure-software rendering),
your computer might not have enough resources to handle all polygons. Be aware that 130 kpolygons will be generated for R = 256 : you will need a lot of memory.

In either cases, you shall determine your preferred resolution by selecting some of them simultaneously and launching the bench. You will soon notice that your card will reach a maximum rate for a given resolution R, and won't perform any better for higher resolutions : this resolution R might be the one good for you, and you might uncheck the others while playing with the options, or using "Bench combinations".

Options

We introduced options in the v2.0 release of this script. Previous versions were just rotating a flat-shaded sphere. Some users requested us to add features often used in scientific visualization. This is also a way to test if these are handled efficiently by your graphic board. All options might be combined.

GUI	texture	wireframe

	transparency	small_sphere

texture : apply a texture to the sphere. Be aware that some older versions of VTK do not allow removing a texture from a 3D object : you will have to restart the script to get back to flat-shaded.
stripper : add a filter that will transform the polygons to triangle strips, a more efficient way to store geometry, and thus often much faster displayed.
wireframe : do not shade the sphere, use lines instead.
transparency : modify the opacity of the sphere.
small_sphere : an interesting option, which will reduce the size of the sphere. The number of polygons remains the same. This allow you to check the influence of the rendering engine, which is responsible of shading/filling the polygons, compared to the geometry engine, which is responsible of handling transformation from 3D to 2D coordinates, calculating the lighting, and so on. This option will reduce the surface of the polygons, thus modifying the load of the rendering engine, while leaving the geometry engine doing the same job. Modifying the window size is another way to stress the rendering engine, but it's not portable regarding our database.

Window sizes

This part of the GUI has been provided to offer shortcuts to common sizes for the rendering window. As the size of the window will influence the surface of the polygons while leaving the geometry unchanged, it's another good way to compare the rendering engine and the geometry engine (see option small_sphere).

Note : Needless to say that rendering in a 700x700 window might be slower than in a 400x400 window, that's the reason why we decided that our database would not accept results made for size different from 400x400 (a warning will be emitted).

GUI

History

After compiling VTK under Linux with Mesa, Robert Rivière once had the feeling that the new version was slower than the previous one. Of course, he really needed more than a feeling to investigate this problem, but a piece of code actually showing it. Hence he wrote a small Tcl/Tk script, using a vtkSphereSource, rendering it, and timing it turn on itself, thus providing an estimate of the number of polygons displayed every second. It was enough to notice an almost x2 factor between some versions of VTK.

This script happened to be a quick and easy way to get a rough idea of the graphic performances of a platform. But after a few tests, made with me (Sébastien Barré), we noticed that in its early form, the graphic scene wasn't complex enough : it only had about 500 polygons. If it could reveal the speed of a pure-software rendering solution like Mesa (not so "pure" now), it was much too simple to stress any hardware-accelerated renderer. A simple way to bring complexity to the graphic scene was then to increase the "Sphere Resolution", i.e. the number of faces in a meridian and a parallel of the sphere. We improve the script in such way, providing additional features to automate the tests for different resolutions : v1.0 was born.