A Small Mathematica Vs C Benchmark

Sometimes I get frowned upon because I do expensive computations in Mathematica. Some expensive computations will be very fast in Mathematica, of course. Try computing (10^4)//Factorial.

But what about the cheap computations that you could as well do in C? The Euler Project gave me the change to try it out with an easy problem. I was solving problem 9 from project Euler, which in short searches for natural numbers a<b<c, such that a^2+b^2=c^2, and a+b+c=1000.

The C-program that finds these numbers is

int main(void) {
 for(int a = 1; a < 10000; a++) {
  for(int b = a + 1; b <= 10000; b++) {
   int c = 1000 - a - b;
   if(a * a + b * b == c * c) {
    printf("%d\n",a*b*c);
    return 0;
   }
  }
 }
}

Note that I de-optimized a little bit by bounding a and b in [1,10,000], which is ten times greater than it should be. The result is still correct, but now it's slow enough so that we can measure something.

The run-time in C is 0.016 seconds on my machine

Bringing this to Mathematica is more than straightforward:

Module[{a, b, c},
  For[a = 1, a <= 10000, a++,
   For[b = a + 1, b <= 10000, b++,
    c = 1000 - a - b;
    If[a a + b b == c c, Return[a b c]]
    ]
   ];
  0
  ][]

Now the run-time is at 8.5 seconds. But! Mathematica can compile! To something faster, that is. It's as easy as wrapping a Compile around the computation. We get:

Compile[{},
 Module[{a, b, c},
  For[a = 1, a <= 10000, a++,
   For[b = a + 1, b <= 10000, b++,
    c = 1000 - a - b;
    If[a a + b b == c c, Return[a b c]]
    ]
   ];
  0
  ]
 ]

And now we're down to 0.38 seconds. Just as a reminder:

Table 1. Run times of the Euler project 9 algorithm in seconds. M stands for Mathematica

C	uncompiled M	compiled M
0.16	8.53	0.38

Which brings me to the following conclusion: For basic arithmetic computation, Mathematica will slow you down by a factor 40, if you use it only casually, or by a factor of 2 if you have some experience with it (Not everything is nicely compilable, so this is a bit optimistic).

Mathematica taketh away, but it also giveth: The library functions are fast, and there's plenty of them. You can treat Mathematica just like any scripting language: If the operations in your inner loop are costly, Mathematica is good for you! Otherwise, you pay a price for a wide range of library-functions. And it is worth delving into that library.

How fast is Comanche?

The other day I came across this guy who wants to serve webpages in less than a millisecond. Without parallelization. So, his web page is really simple, it looks like this: <h1>Hello TWUG!</h1>.

His point is that things like Twitter are difficult to scale across several machines. The slowness of dynamic pages, he claims, is tried to be made up by advanced caching strategies. He goes on to check how fast certain languages are at serving dynamic content. He gets these numbers:


Clearly, his machine is faster than mine, because serving the same file statically, my Apache (single-threaded using httpd -X) can do only 1850 requests per second, that is 0.5 ms per request. (I use this benchmark: httperf --hog --num-conn 10000 --uri http://localhost:9090/?name=Niko). Here are the numbers I get:


Anyway, I find interesting to mention that Comanche is not far from Apache. Comanche can serve the same page dynamically (the name is a GET parameter) in 1.1 ms, for which Apache takes statically 0.5 ms. As for Apache-php vs. Comanche, I'd call it a tie.

The Squeak code for the Http server would be this:

ma := ModuleAssembly core.
ma addPlug: [:request | HttpResponse fromString: 
   (String streamContents: [:s| 
   s nextPutAll:'<h1>Hello! '; 
   nextPutAll:( request  fields at: 'name'); 
   nextPutAll: '</h1>']).].
(HttpService startOn: 9090 named: 'Example') module: ma rootModule

By this way, as the above snippet suggests, a standard Seaside installation for Squeak is quite able to make RESTful applications. Just avoid the Seaside framework and plug right into the Comanche web server.

For completeness, the PHP-code:

<h1>Hello <?=$_REQUEST['name']?></h1>

Update

By the way, this seems to be about 10 % faster:

s:=(HttpService on: 8080 named: 'Example Http Service').
s onRequestDo: [ :request |
 HttpResponse fromString: 
  (String streamContents: [:s| s nextPutAll:'<h1>Hello! '; 
  nextPutAll:( request  fields at: 'name'); 
  nextPutAll: '</h1>']) ].
s start