Thursday, November 19, 2009

Phexample's lolcats syntax

In this post I'll clear my throat on what there is to come. The number one criticism of Phexample is that Phexample sometimes reads like lolcats, because real code that asserts that a stack be empty reads like

stack should not be isEmpty.

The complaint feels like an instance of the complaint of how stupid it is that in OSX you need to drag CDs to the waste bin to eject them, or how in Windows the bin stands on the desk. Of course the metaphor is broken there, but you only notice it because the metaphor works great.

And now you compare that to the usual alternative, which is assert. The first thing you know about assert is that nobody can remember which one is the expected and which is the actual value (I'll take this fact and some of the below discussion from Josh Graham's discussion of the assert syntax). We're talking about assertEquals(stack.isEmpty, true). Or the other way around. I certainly can't remember that either.

In Java, improvements have been suggested:
assertEquals(expected(true), actual(stack.isEmpty());

I find that while Phexample does not allow you to express your examples in a way that completely parses as natural language, it certainly reads easy enough and gives a natural and easy distinction between the expected and the actual.

So there's my analogy: no other testing framework is required to express its tests in a way that perfectly parses as human language. Only because the tests already look so close to perfection is it that people single out the mismatch and decry it. I don't think this is an interesting problem worth solving at all.

Yet, the criticism follows Phexample wherever it goes, so I'll try and allow phrases like

stack isEmpty should be true.


The difficulty in making the above code work is that the testing framework is only activated after isEmpty was evaluated already. Which means that it isn't in the stack anymore, so we need to analyze the source code to find out which method caused the failure to display a meaningful error message.

Wednesday, November 4, 2009

Telling other people how to install your code

Previously, I created dirty ScriptLoader tricks to make it easier for others to download my code.

I am pleased to find out that Gofer is even easier than my dirty tricks, well without being a dirty trick: Gofer on Lukas's blog.

Phexample, because examples expand on one another

Imagine you want to test the Stack class, and your first test just
creates a test and checks if it's empty.

Then in your second test, you create the same empty stack, but now
push an element 'apple' and check if the size is one.

Then in your third test, you do all the same, and then you pop the
element and check that it is an apple.

Wouldn't it be great if you could write your test cases such that they
could expand on one another? If you didn't need to copy paste the code
of the previous test and just say that you require the previous one!

Well, that's what PhExample lets you do. And as a bonus, tests are
ignored if the test they expand on does not pass. This gives you less
failed tests, because examples are executed only if the examples they
expand on pass too! That gives you the simplest failing examples in
your set to look at!

And best of all, you can run all examples with ye old TestRunner!

The example discussed above would read in code as follows:


EGExample subclass: #ForExampleStack


shouldBeEmpty
"Create the empty stack"
| stack |
stack := Stack new.
stack should be isEmpty.
stack size should = 0.
^ stack


shouldPushElement
"Push one element"
| stack |
stack := self given: #shouldBeEmpty.
stack push: 42.
stack should not be isEmpty.
stack size should = 1.
stack top should = 42.
^ stack


shouldPopElement
"And pop it again"
| stack |
stack := self given: #shouldPushElement.
stack pop should = 42.
stack should be isEmpty.
stack size should = 0.


Find the code at http://www.squeaksource.com/phexample.html

You can install it as follows using Gofer:


Gofer new
squeaksource: 'phexample';
addPackage: 'Phexample';
load


The repository is write-global, so feel free to drop your ideas!

PhExample is based on JExample by Lea Haensenberger, Adrian Kuhn, and
Markus Gaelli. See JExample

Wednesday, September 30, 2009

Introducing ThothCompiler

ThothCompiler combines the strengths of the two most popular smalltalk compilers. It takes the extensibility and wealth of the AST implementation from the NewCompiler and combines it with the most important asset of the old Compiler that per default ships with Squeak: it actually works.

Installation


The installation is fantastically easy. Go to Thoth on Squeaksource and get ASTBridgeLoader.

In a workspace, then execute:

ASTBridgeLoader load


Add the method compilerClass to the CLASS side of your class to set the compiler for the class. Thoth installs itself as the default compiler of the image. If you're unhappy with that, for example because you don't like the lack of error messages when you accept uncompilable code, you can choose for each class which compiler is to be used. Behavior>compilerClasss sets the default.

How do I hack into it?


Subclass ThothCompiler and override #compile:in:classified:notifying:ifFail: and #evaluate:in:to:notifying:ifFail:logged: Transform the AST before generate is called. Perhaps I should provide some infrastructure here …

How it works


It is a really simple thing: It uses the parser of the NewCompiler, then transforms the AST into the old model, and then uses the backend of the old compiler to compile.

Wednesday, September 16, 2009

Blocks in Smalltalk can have their own temporary variables

Did you know that the following appears to be Smalltalk code? Runs fine in the current Pharo.

[:a | |b| 2] value: 3

Sunday, September 6, 2009

Explaining Mathematica with the Solution for Problem 59 from Project Euler

The solution to problem 59 is perhaps helpful to see what Mathematica is good at and how it basically works. The task is here. Let me summarize it. The task contains a link to a text file full of numbers. These numbers together are an encrypted text, the password being three lowercase letters. The decryption works like this: You convert the password into its ASCII values, and then XOR the first byte of the encrypted text with the ASCII value of the first letter in the password. You iterate this for all three letters, and then start over with the first letter of the password for the fourth number in the encrypted text. And so forth, until you are done. The task is to find the password.

Ok, so, the first thing Mathematica rocks at is importing. Here it probably isn't a big deal, but I find it noteworthy anyhow … getting the encrypted file is this:

text 
= Import["http://projecteuler.net/project/cipher1.txt", "csv"]
// First ;


Here, Import leaves a pair of braces around the real result, which the First function disposes of.

Now the idea of my checking is this. I'll take every third character from the encrypted file and store it in a list. Then I will try for each possible first letter of the password which letter would yield the most printable characters in the decrypted text. I will use this very narrow definition of a printable character:

 printable[a_] := 65 <= a <= 90 
~Or~ 97 <= a <= 122


One thing we cannot see here in Blogger is that Mathematica allows Mathematical typesetting, thus presenting this definition with a beautiful vee for an or and the inequalities look like inequalities.

The next task is to give each letter of the password a score. Let us assemble the procedure from the ground up. The goal is to build a function which takes every third element of the encrypted text, applies BitXor with the key, and then counts how many characters are displayable.

Getting every third element from the encrypted text, starting with the 1st letter, is expressed as

text[[1 ;; ;; 3]]]

We can refer to the result of the previous computation as %. Let us try 103 as the first letter of the password, that would be the letter "g." Then we decrypt as follows:

BitXor[103,%].

And finally, we count the printable letters in the resulting list. Mathematica provides a Count function. Count is a nice function to demonstrate something rather unique to Mathematica: patterns. Let us first see our code, which counts printable letters:

Count[%, _?printable].

Here, _?printable is the pattern. The underscore stands for "anything", and the question mark specifies that the anything must evaluate to true when passed into the following function, printable. This is a very simple pattern, you can do cooler things, like the pattern {1,2,_}, which matches lists of three elements whose first two elements must be 1 and 2.

Now altogether the score function looks like this, where i ranges between 1 and 3:
score[i_, letter_] := 
Count[BitXor[letter, text[[i ;; ;; 3]]] , _?printable]


The task states that valid candidates for the letters of the password must be lowercase. These have ASCII codes between 79 and 122. The function SelectMaximizer is an invention of mine. It chooses the element of a list which maximizes a function that can be supplied. The supplied function will be score, of course, so we can find the first letter as

SelectMaximizer[Range[97, 122], 
score[1, #] &]


The confusing thing here is the closure, score[1, #] &. In Mathematica, everything that ends in an ampersand is a closure, or a local function. The hash symbol is the argument. We need it because score accepts two arguments (the position of the letter in the password, and the actual letter). Where we'd like to use only the second as an argument.

The function call will dutifully find the character "g" as the first letter of the password. To find all 3, we use the Table function. The Table function an expression with an undefined variable, i, and then tell it to use that variable as an iterator, assuming values from 1 to 3. The following code thus finds all three letters of the password:

password = 
Table[ SelectMaximizer[Range[97, 122],
score[i, #] &], {i, 3}];


To decrypt the text with our password, we partition the encrypted text into runs of three, which we can then BitXor. Mathematica ships with a Partition function.

Partition[text, 3] returns a list partitions of length 3 of text text. Map function applies a closure to each element of a list. For example, Map[Range[4],#*#&] computes the first four square numbers.

Our first shot at a decoding function is thus:

Map[BitXor[password, #] & , Partition[text, 3]

We almost get it right. Unfortunately, the encrypted text's length is not divisible by three, and thus partition just 'swallows' the last letter. We can tell Partition to attach the remaining letter to the list. We use a little bit of trickery, because simply attaching the last letter would attach a list of length 1 to the list of partitions, but our partitions must have length 3 for the BitXor to work. Well, we can tell Partition to add the left-overs, and then pad to the length of 3 with the password itself, which will then become 0 after the Xor, so we're really appending null bytes to the string.

Together, the decoding function is now

decode[p_] := Map[BitXor[p, #] & ,
Partition[text, 3, 3, 1, p]]


where p stands for password.

For what it's worth, the password is "god" and the result is chapter one of the Gospel of John. The task still requires us compute the sum of the ASCII codes of the decoded text. Our decode function really returns a nested list, to compute the sum of all entries, we could either use Flatten to reduce it to depth 1, or we could tell the Total function to operate on the second level of a nested list.

Total[decode[password], 2]


which returns 107359.

The complete code is

text = 
Import["http://projecteuler.net/project/cipher1.txt",
"csv"] // First ;

printable[a_] := 65 <= a <= 90 ~Or~ 97 <= a <= 122

score[i_, letter_] :=
Count[BitXor[letter, text[[i ;; ;; 3]]] , _?printable]

decode[p_] := Map[BitXor[p, #] & ,
Partition[text, 3, 3, 1, p]]

password =
Table[ SelectMaximizer[Range[97, 122],
score[i, #] &], {i, 3}];

Total[decode[password], 2]


The complete code is can be downloaded as a complete code for problem 59 Mathematica notebook or as a complete code for problem 59 pdf file.

Friday, August 14, 2009

A Small Mathematica Vs C Benchmark

Sometimes I get frowned upon because I do expensive computations in Mathematica. Some expensive computations will be very fast in Mathematica, of course. Try computing (10^4)//Factorial.

But what about the cheap computations that you could as well do in C? The Euler Project gave me the change to try it out with an easy problem. I was solving problem 9 from project Euler, which in short searches for natural numbers a<b<c, such that a^2+b^2=c^2, and a+b+c=1000.

The C-program that finds these numbers is


int main(void) {
for(int a = 1; a < 10000; a++) {
for(int b = a + 1; b <= 10000; b++) {
int c = 1000 - a - b;
if(a * a + b * b == c * c) {
printf("%d\n",a*b*c);
return 0;
}
}
}
}


Note that I de-optimized a little bit by bounding a and b in [1,10,000], which is ten times greater than it should be. The result is still correct, but now it's slow enough so that we can measure something.

The run-time in C is 0.016 seconds on my machine

Bringing this to Mathematica is more than straightforward:


Module[{a, b, c},
For[a = 1, a <= 10000, a++,
For[b = a + 1, b <= 10000, b++,
c = 1000 - a - b;
If[a a + b b == c c, Return[a b c]]
]
];
0
][]



Now the run-time is at 8.5 seconds. But! Mathematica can compile! To something faster, that is. It's as easy as wrapping a Compile around the computation. We get:


Compile[{},
Module[{a, b, c},
For[a = 1, a <= 10000, a++,
For[b = a + 1, b <= 10000, b++,
c = 1000 - a - b;
If[a a + b b == c c, Return[a b c]]
]
];
0
]
]



And now we're down to 0.38 seconds. Just as a reminder:

Table 1. Run times of the Euler project 9 algorithm in seconds. M stands for Mathematica
C uncompiled M compiled M
0.16 8.53 0.38



Which brings me to the following conclusion: For basic arithmetic computation, Mathematica will slow you down by a factor 40, if you use it only casually, or by a factor of 2 if you have some experience with it (Not everything is nicely compilable, so this is a bit optimistic).

Mathematica taketh away, but it also giveth: The library functions are fast, and there's plenty of them. You can treat Mathematica just like any scripting language: If the operations in your inner loop are costly, Mathematica is good for you! Otherwise, you pay a price for a wide range of library-functions. And it is worth delving into that library.