Niko Schwarz's science and programming

TextMate 2.0 alpha is out

2011-12-13T20:17:00.002+01:00

Things that did NOT make it into the alpha:

Split views: Yes, I actually had this on the alpha milestone, I’m not overly excited about this feature myself, but I know it’s a very common request, so eventually it should find its way into the application.
Full screen mode: This is mainly because we are hesitant to go Lion-only so we are holding back with “lionizing” TextMate till we feel confident we can fully drop backwards compatibility.
Performance: Overall performance is fine, but there are still edge-cases that we haven’t looked into, for example the long lines issue which also exists in 1.x or opening files that exhaust TextMate’s memory space.
Bundle editor: While a proof-of-concept bundle editor is included, it is provisional, has some flaws, and not how we envision the final bundle editor to be.
Settings: Not everything in the Preferences window has an effect at the moment and several menu settings are not sticky, some even revert when switching tabs. Those can be set via .tm_properties — more about this in an upcoming post.

Update: I, for one, can't actually start it. It crashes upon load.

Lack of excitement for Pharo 1.4 seems symptomatic

2011-12-11T17:58:00.001+01:00

Here's Lukas Renggli, the working horse that pulls Smalltalk behind him, announcing his plans for Pharo 1.4, or his lack thereof:

Just to repeat myself: With Pharo 1.4 having uncountable changes in core parts of the system and with the system including more and more forked and increasingly incompatible versions of packages (AST, RB, FS, Shout, Regex, …) I am unwilling to go through the same pain as with Pharo 1.3 again. In the current state, I don't see any of the code I am involved with (including Seaside, Magritte, Pier, OB, PetitParser, …) to move forward. I suspect that moving to another development platform soon causes less pain than to move to adopt the next Pharo :-(

In a word, nobody wants to upgrade to new versions of Pharo. But that was the whole point of Pharo: progress, moving ahead. There isn't a Smalltalk with traction right now. You can watch the Smalltalk community fail to answer which part of the Smalltalk universe is worth looking at at Stackoverflow.
Compare this with the spirit in the Objective-C/Cocoa world:

It’s OK to support only the newest version of iOS.

Now, you might say that this isn't a fair comparison, because writing iOS apps is the key to a new and exciting and fast-growing market, the mobile platform. But I think that's the same argument that Eric Schmidt is getting wrong about Android: “Whether you like Android or not, you will support that platform”. As Gruber points out, he's having cause and effect backwards: developers love programming for iOS, and only therefore is iOS as exciting as it is.
Smalltalk simply isn't as inviting to developers as it used to be.

I'm thru with the GPL

2011-12-05T10:15:00.001+01:00

Free software is great. Seeing your own project being used and maintained by somebody else is great, too. But being expunged from the version history is a really bitter thing to experience. I'm moving my projects to Apache License, which requires attribution.

Point-free programming in Smalltalk

2011-08-30T17:45:00.005+02:00

Smalltalk does not, ordinarily, support point-free programming. Point-free programming is programming without "points", or variables. In a loop, rather than assigning every item to loop over to a variable, you only write which function should be called on them.

Pharo Smalltalk has limited support for that. There, you can write:


#(1 2 3 4) select: #even     -> #(2 4)

The above is implemented using an easy equivalence. Every block that sends only one message is equivalent to the selector of that message. Thus, the above is equivalent to:


#(1 2 3 4) select: [:e | e even]    -> #(2 4)

Which is just standard Smalltalk. The equivalence can be made work by implementing #value: etc. on Symbol.

Well, of course this is rather limited, we still haven't got rid of blocks. We still need blocks for everything that does not only send one message. You can push a little farther by allowing composition of symbols:


{'100' . '20' . '3'} sort: #<= * #first.     ->  #('100' '20' '3')

The idea is to read the asterisk as "after", like the function compositor in Mathematics.

So you read this as "sort the area by less than, after first was applied". Thus, the strings are sorted by their first letter. Awesome, eh?

This is not standard Pharo, but it can be implemented in a couple of minutes. The code is on Snipt, but it's more easily written than read.

This can still not replace blocks. For example, the following block still cannot be expressed point-free:

[:a | Transcript show: a]

.

But nothing is easier than inventing a syntax for that (can you implement it in Pharo, so it'll work?):

 Transcript delayedSend: #show:

Finally, for blocks where the left-hand-side is the argument:

 [:stream | stream nextPutAll: 'hello']

I propose that could be written as follows (can you implement it in Pharo, so it'll work?):

Delayed nextPutAll: 'hello'

Open questions

Would it be more enjoyable to write entirely point-free in Smalltalk? What do you think?

How to obtain the current time in a number of standard APIs:

2011-01-24T13:56:00.007+01:00

I googled for the programming language plus "current time", and obtained the following methods. The programming languages are the most popular languages on github.

Ruby:

now = Time.new

Java:

Date now = Calendar.getInstance().getTime();

Python:

now = datetime.datetime.now()

C#:

DateTime now = DateTime.Now;

PHP:

now = time()

Objective C:

now = [NSDate date]

JavaScript:

var now = new Date();

C++ (well, this is really C):

time_t now = time(NULL);

These are the standard methods in the standard APIs. Nonetheless, they introduce “Hidden global state” into your application (see Slide 16 on “Clean Code Talks - Global State and Singletons” in the Google testing blog). Hidden global state makes your code hard to test, yet it is ubiquitous. Somebody should do something about that! (Yes, I’m working on it :)

Full text search in Pharo?

2010-10-27T16:34:00.003+02:00

Pharo does have something like full text search across all methods, and it works like this: Rightclick the text you'd like to search for, and choose "extended search" from the options. Then, choose "method source with it." A progress bar will appear, and you should get your answer within a minute. That has to do with the source not being loaded into Pharo at all times, it is read on demand out of the sources file.

However, the compiled sources are in memory from the start. While you can't meaningfully search compiled sources, you can search the string literals within. Again, rightclick the text you'd like to search for, and choose "extended search" from the options. But now, choose "method strings with it." That will bring up a list of all methods that contain the selected text as a string literal—instantly.

One of my most-hated methods in Pharo

2010-10-12T15:44:00.004+02:00


    TestCase>fail
         ^self assert: false

What's wrong? Well, I often write self fail at the end of a unit test to remind myself that the test isn't yet done. Now, if I run the test, I care whether an assertion failed or whether the test solely failed. Of course I have no right to expect should detailed feedback from a test runner. But at least I shouldn't be misinformed. But that's exactly what happens. The TestRunner reports that an assertion failed if you call fail, even if none did.

Why's the Smalltalk debugger not focused on more in Smalltalk IDEs?

2010-10-12T11:31:00.003+02:00

I can't help but wonder why Smalltalk's lovable trait of letting you develop in the debugger isn't leveraged far more by the IDEs. Suppose you just noticed that a method call doesn't have enough arguments. That is, you wrote this:

self doSomething.

But you should have written:

self doSomethingWith: anArgument

Well now, that's a fantastic situation for refactoring, isn't it? You could automatically rename the method in the receiver, and you'd know quite precisely who that receiver is. Little of that is leveraged. But of course, if Smalltalk is to play any serious role in the future, we should play by our strengths.

Well, here's a strength: the most powerful debugger in any modern IDE.

A more robust doesNotUnderstand:?

2010-10-08T16:52:00.005+02:00

Smalltalk makes it really convenient to write code in the debugger. So, how I often work is as follows:

Write a rough skeleton that calls "self fail" at some point.
Write a unit test that calls the skeleton with real data.
Finish the skeleton in the debugger, where I have test data right at hand to see if things work as intended.

Well, that's alright as far as it goes, but you can't work like that with doesNotUnderstand: methods.

doesNotUnderstand: aMessage
 self fail.

If you now call a non-existing method on an object of a thus modified class, you've built an endless recursion that is so close to the VM that you can't get into a debugger anymore.

Smalltalk's doesNotUnderstand: method is a dangerous superpower. And perhaps that's why just about nobody overwrites it in a standard Pharo image.

Open problems

Do you know a more robust implementation?

Non-existing calls inside the handler for such should be caught, I think.
It should work nicely with code completion. (Code completion seems to be an integral part of just about every current approach to help programmer's productivity, so it better work nice.)

expandMacrosWith:

2010-10-01T16:04:00.009+02:00

In Pharo, there's a very useful method String>expandMacrosWith:, virtually without documentation. It's much like printf in C. I guess it's easily understood from a few examples, so here go:


"<1s> is replaced by the first argument, a string.
 <n> is a newline, <t> is a tab. 
 % escapes brackets."
'<1s><n><t>%<inject><n><t>^<1s>' expandMacrosWith: 'player'
➔
'player
 <inject>
 ^player'


"p calls printString":
'<1p>' expandMacrosWith: #(1 2 3)
➔
'#(1 2 3)'


"? prints one string or another, depending on a Boolean"
'Dear <1?Mr:Mrs> <2s>, your contract has expired.' 
  expandMacrosWith: false   with: 'Smith'
➔
'Dear Mrs Smith, your contract has expired.'

Pharo dojo in Bern, October 23–24

2010-09-29T09:41:00.005+02:00

There will be a Pharo dojo in Bern. The announcement reads:

We will organize a joint Pharo sprint / Moose dojo during October 23-24, in Bern (at the Software Composition Group, University of Bern). There will be free food and drinks!
Some action points are mentioned on the dedicated page (of course, other ideas and interests are welcome as well).

For planning purposes, please let me, Tudor Girba, know if you will attend.

Dependency injection aids copy and paste coding

2010-09-27T16:58:00.008+02:00

In the presence of good documentation or sample code, I love to copy and paste directly out of the documentation or even unit tests, straight into my code.

For example, I was wondering how to add an instance variable to a class in Pharo Smalltalk. Well, I just search my code base for occurrences of addInstanceVariable, and I find this test:

testAddInstanceVariable
 | refactoring |
    refactoring := AddInstanceVariableRefactoring 
        variable: 'asdf'
        class: RBTransformationRuleTest.
     self executeRefactoring: refactoring.
     self assert: ((refactoring model classNamed: 
         #RBTransformationRuleTest) 
              directlyDefinesInstanceVariable: 'asdf')

Well, that's great, the first few lines are what I need. Some more browsing shows that line 6 is deeply irrelevant for me: the test uses a highly artificial way of executing the test, different from the canonical one.

Is any of that surprising? Not really. There's some papers establishing that this is how a lot of people work: forking work off sample code, but only if it's complex enough to match the use case (cite?).

I think it has an interesting implication for writing unit tests: They should be as close as possible to the intended end user usage. In the example above, my hunch is that the unit test wasn't written in the way that the end user would use the library, because the author feared side effects.

I think here we see one of the great strengths of dependency injection: by moving the part of the unit test that avoids side effects to another level of abstraction, the unit test itself can look exactly as if it had side effects, and thus can serve better as a basis for new work.

Circular problem solving in computer science

2010-09-27T12:11:00.003+02:00

Here's the rules: Find a genuine problem in a framework called X. Then write your solution in framework Y that depends on X. Then, rewrite X using Y such that the problem goes away.

Because rewriting of X depends via Y on some older version of X, I give bonus points for using a framework that allows calling older versions of X from within X.

Three degrees of inversion of control.

2010-09-22T17:00:00.003+02:00

Inversion of control is a paradigm by Martin Fowler that advocates for separating things that don't belong together during object creation.

What exactly should be split from what? Different frameworks answer the question slightly differently. They do at least 1 of these three:

Eliminate direct constructor calls. All dependency injection frameworks do that.
Strictly separate business logic from object creation. Google Guice, among others, allows that, but doesn't enforce that.
Eliminate all static references. Newspeak makes all lookups dynamic, thereby eliminating global state.

All three are compatible with one another. Can they be combined in practice? I'm working on it …

In unit tests, why are failures yellow but errors red?

2010-07-22T10:34:00.003+02:00

It's confusing, isn't it? Errors are displayed red, which looks scary, while failures are displayed yellow, which looks inviting. But, debugging a test failure, you don't get a trace of a clue what caused the failure. While debugging an error, you have the stack trace right there and know exactly where to look.

When SUnit lists yellow and red failed unit tests, I invariably start with the red ones. They're usually easier to fix.

Book tip: Computing the mind

2010-06-22T23:12:00.002+02:00

I'd like to discuss why, without having finished the book yet, I hand over heels love the book: Computing the mind, subtitled "How the mind really works", by Shimon Edelmann. I have to admit that I didn't have the highest hopes when I opened the book. I went so far as to skip the first couple of pages, to get to the fun parts a bit quicker. As I write these blog post, page 1 lies open on my desk, and it's the most beautiful start a non-fiction book could hope to offer:

If you are reading this in the first couple of decades of the third millennium of the common era, chances are that (1) the human brain is the most complex toy you'll ever come near, let alone get to play with, and (2) you're trapped inside one for the foreseeable future.

And then, on page 78, we get this teaser for what's to come later:

Claim: the brain is a programmable digital computer. If this were true, the brain would be like hardware to the mind's software. This claim does not stand up to scrutiny, mainly because in the brain it is not possible to separate the "program" from the architecture that executes it. It is very important to remember, however, that a digital computer can simulate the operation of a brain (because a digital computer can simulate any physical system down to an arbitrary precision), and hence the mind. The ramifications of the realization that the mind is what the brain does are far-reaching and profound; I shall describe and discuss them in later chapters.

He had me long before page 78, though. He quotes Karl Popper throughout the book. This book is amazing in clarity, it's funny, and it's mind-opening.

Friendly advice from Eric S Raymond

2010-05-23T14:08:00.003+02:00

My favorite quote from "The cathedral and the bazaar," p. 207:

If you're attracted to hacking because you don't have a life, that's okay too---at least you won't have trouble concentrating. Maybe you'll get a life later on.

I love the book, but I think the first two essays would have been enough to read, that's "The cathedral and the bazaar" and "homesteading the noosphere." The others are far from neutral and not full of insight like the first two. The first two are an insightful analysis of the psychology and culture of hackerdom.

The burden of the reviewer

2010-05-04T14:37:00.001+02:00

The paper I'm reviewing treats an interesting topic without much love. I'm getting sick of reading other people's papers. If this goes on, I'll just write my own papers.

By the way, I'm still taking really long for reviewing. An hour of work and I'm on page … 2.

Git tree objects, how are they stored?

2010-03-25T22:19:00.006+01:00

A git tree is git’s internal representation for directories. Trees have entries which are either trees themselves, or blobs.

The general format is: First 4 bytes declaring the object type. In our case, those four bytes are “tree”, ASCII-encoded. Then comes a space, and then the entries, separated by nothing.
The exact format is the following. All capital letters are “non-terminals” that I’ll explain shortly.

    tree (ZNA FNS)*

where

N is the NUL character

Z is the size of the object in bytes

A is the unix access code, ASCII encoded, for example> 100644 for a vanilla file.

F is the filename, (I’m not sure about the encoding. It’s definitely ASCII-compatible), NUL-terminated.

S is the 20 byte SHA hash of the entry pointed to, 20 bytes long.

Here’s an example. Say we have a directory with two files, called test and test2. The SHA of the directory is f0e12ff4a9a6ba281d57c7467df585b1249f0fa5. You can see the SHA-hashes of the entries in the output of

    $ git cat-file -p f0e12ff4a9a6ba281d57c7467df585b1249f0fa5
    100644 blob 9033296159b99df844df0d5740fc8ea1d2572a84    test
    100644 blob a7f8d9e5dcf3a68fdd2bfb727cde12029875260b    test2

(To reproduce this, the file test should contain the ASCII-encoded characters “hallo” (not \n-terminated), the file test2 should contain the ASCII-encoded characters “bla\n” (yes, \n-terminated)).

That is converted into the following object (uncompressed):

Finally, the file is deflated and stored on disk. For deflation, zlib is used. For my experiments I used the ruby zlip package. The following tiny script reads the tree from disk and inflates it, so you can look at it:


    require 'zlib'
    require 'fileutils'
    File.open("objects/f0/e12ff4a9a6ba281d57c7467df585b1249f0fa5") { |f|
        puts Zlib::Inflate.inflate(f.read)
    }

Why psychology and computer science should go together

2010-03-22T15:33:00.001+01:00

You can't cast Object[] to String[] in Java

2010-03-20T13:51:00.007+01:00

Consider this snippet:
String[] str = (String[]) new Object[] { "Bla!"};.
That gives a ClassCastException. The short version of an explanation: an array instantiated by new Object[] is of a different class than one by new String[]. For the full glory of that fact, please see the Java Language Specification. As a teaser, I'll quote this snippet and its output from it:

class Test {
 public static void main(String[] args) {
  int[] ia = new int[3];
  System.out.println(ia.getClass());
  System.out.println(ia.getClass().getSuperclass());
 }
}

which prints:
class [I
class java.lang.Object.

Thick tech books

2010-03-18T23:21:00.001+01:00

James Hague:

"In order for a book to sell," said my publisher, "it's got to be thick. 600 pages thick."

Well, that about explains why tech books suck so badly.

How to test post-conditions

2010-03-18T21:34:00.007+01:00

A pretty good student, Joshua Muheim, just asked me what the simplest way was to check postconditions. And my answer was the following:

Joshua, this is a research question! To the best of my knowledge, currently and in practice, developers use steamroller tactics to check postconditions. The standard way to verify that the LHC works as intended, in a method exterminate on a class EarthPopulation:

exterminate
    | oldSize | 
    oldSize := self size.
    LHC switchOn.
    self size should < oldSize.

That is, you copy whatever you intend to change and then compare new with old value. It isn’t difficult to imagine that this can lead to trouble if whatever you want to change isn’t cloneable. And of course it isn't possible to switch off the overhead of these assertions.* And that’s where Histoory (by Pluquet et al) comes to the rescue. Not only does it work independent of the cloneability, it also provides a much cleaner syntax:

exterminate
    [LHC swithOn]
        postCond: [:old | old size should < self size]

Alas, it isn’t in the latest flavor of any programming language, so I truly hope that Histoory will make it into Pharo :).

* I added this sentence after Joshua's correct observation of this problem.

Book on algorithm design

2010-03-16T09:07:00.005+01:00

Recently, Sedgewick's algorithms book slipped through my hands again. It's a great book. I'd just like to point to a newer algorithms book that is thinner, and funner to read, and if you just want to browse for an hour or two, this is surely worth your while. That's Algorithm design, by Jon Kleinberg, and Éva Tardos.

Software merging, commit messages

2010-03-15T14:02:00.003+01:00

What is the intent of a change? How do you merge conflicting changes? The current state of the art is dire. There's a survey, A state-of-the-art survey on software merging , but it's summary may be that there's no silver bullet to automated merging without understanding the intent of a commit.

How do you understand the intent of a commit? The best technique I am aware of today is … having the author provide a clear description of his intent. There's a paper about that, quite worth your while: Semantic patches considered helpful.

I told you that the situation is dire, didn't I?

Distance measure between software artifacts

2010-02-28T21:36:00.004+01:00

There's currently some interest in software visualization, which tries to speed up your understanding of source code by leveraging your brain's great ability to discern patterns in images. Below is shot of Codemap, visualizing something.

To draw a map such as the above, you need an underlying metric that tells you how far apart the code artifacts should be. (You probably won't be able to draw a map where the distances are exactly what you want them to be, but you may get close.) Now, what is used as a metric between software artifacts (islands, in the above image)? Codemap uses the vocabulary of the artifact, that is: artifacts that use many same words will end up closer together in the overall drawing.

A botfly moves along a herd of cattle. It finds every of the beasts hospitable, but it can't live without a herd. Thus, Richard Dawkins calls the herd the botfly's "Archipelago" (in The greatest show on earth, p. 253). This inspires me to a metric which is supposed to measure similarity of software artifacts. The idea is to assume that if two artifacts are hospitable to the same code snippet, then they should provide similar environments. Thus, you could measure how in the evolution of a software program, snippets wandered from one place to another, and assume that whenever a code snippet is moved from one artifact to another, these two artifacts increase in our metric's similarity.

It seems difficult to bring forth evidence that software maps aid understanding in the first place, thus I wouldn't know how to show that a metric like the one outlined above is any better than a comparison of metrics. Yet, there's some momentum in the field, so I'll save up the idea for a point in time when more light will be shed on the benefits of software visualization.

On Erwann Wernli's idea of having the VM take care of dependency injection

2010-01-09T19:34:00.005+01:00

My dear friend Erwann asks in his blog whether dependency injection should not be the task of the virtual machine. He considers code like this:

List l = new ArrayList();

He then notices (correctly, to my mind), that this statement contains more detail than the developer meant to express. "Whether the list is a new one or not, I don’t care, all I need is an empty one," Erwann writes. Fair enough.

There's real problems connected with the observation. The Java specification rules that new has to allocate a new object. Hence, the method

static void slack() {

 new Object();
 return;
}

cannot be optimized into a nop. Now, I believe you've already thought of a code snippet that cannot be optimized due to this rule. And of course it gets worse if the language forces us to use new all the time. it is the only way to get an object, even though we would have been fine with an old one, should it fulfill our criteria.

Here's the solution he proposes:

List l = obtain list ( size=0 );

To undo the awkwardness, he proposes hiding this monstrum in syntactic sugar, although it isn't clear to me what the sugar should look like in this instance.

Besides avoiding the new operator, his proposal leaves the concrete type of variable l open. He now proposes that depending on the "context," the VM can do voodoo to determine exactly which class may be most appropriate at the point in time. As an example, he cites changing the default FileInputStream to a zipped version of itself, as disk space runs low.

I find his thinking interesting, because it asks the good old question of who's responsible for what in programming. Having the machine figure stuff out for you is great if the figuring out is easy or secondary. Everyone loves Ruby's ActiveRecord, which creates the database table belonging to an entity class of yours all by itself. But choosing the right kind of a list? That's harder. There's a disadvantage to having the machine figure stuff out for you: your program gains in being opaque. The text-file that stores your program code would diverge further from the code to be executed at run time, because important decisions about the code are delayed to the run-time of the program. Since us humans can only see text, this is a real drawback.

I'm obviously sceptical to ubiquitious dependency injection, and I'm not the only one. Here's what Russ Olsen has to say about the matter (In his book Design Patterns in Ruby):

The best way to go wrong with any of the object creation techniques that we have examined … is to use them when you don't need them. Not every object needs to be produced in a factory. In fact, most of the time you will want to create most of your objects with a simple MyClass.new. … Remember, chances are You Ain't Gonna Need It.

Well, we have to balance our skepticism against what voodoo may bring us. Suffice it to say that I don't really believe that I want the VM to have any say in whether I zip my files or not. If I write a file to the disk that then the bash will assume to be a plain text email message, it better not be zipped on 5 % of my clients' computers. Another advantage of the voodoo is that we may also get to re-using objects that are laying around anyhow, waiting to be garbage collected. There's charm to that idea, although I wonder if the picture is as rose as Erwann paints it. After all, you'd need to find an object to be disposed first. Object Pooling sucks up memory and CPU cycles for their management.

Let me summarize that I'm skeptical about the solution that is proposed. Nonetheless, Erwann raises legitimate and important questions, such as: Don't our languages force the developer to supply and commit to more detail than the developer wishes? A mismatch between the mental model of the developer and the programming language is a flaw that deserves our tending to.

There's one little afterthought. Erwann claims that the mismatch he identified can be tackled only by making both language and VM aware of the problem. I believe the strength and beauty of Smalltalk is to come a far way in allopwing new ways of doing things, without having to dig to the VM and the language (See the blog post by Benjamin Pollack: Your Language Features Are My Libraries). Well, I think that Newspeak's late binding of class names could implement all of Erwann's ideas in a library. Here's the idea: If you type Array.new, then you may expect Array to point to a specific and global class in a namespace that is the same everywhere. What if all lookups had to go through a "lobby," as it were, and each module could not demand the module, it would have to be passed. Then, we could have the lobby figure out what List.new is supposed to mean, and have it be context-dependent. (Newspeak is a dream of a programming language. It breaks my heart to see how it isn't used widely. See the Wikipedia article for an overview.)

Summary

Programming languages do not currently reflect the mental model of the developer in instructions as easy as

List l = new ArrayList();

They specify more details than the programmer wishes to, and as a result, optimizations and opportunities for automation are lost. I suspect that Newspeak would be a great platform for experiments, and I'd like to thank Erwann Wernli for sharing his concerns. His blog is great by the way. Please go and read it!

Here is Erwann Wernli's article: Erwann Wernli: Should DI and GC be unified?.

ThothCompiler revamped

2009-12-17T18:57:00.005+01:00

ThothCompiler has been revamped very near completely. Get it by executing:

Gofer it
    squeaksource: 'ASTBridge';
    addPackage: 'ThothCompiler';
    load

Then go to the Preference Browser (System->Preferences in the world menu) and choose ThothCompiler for the option "whichCompiler".

ThothCompiler allows you to use a modern AST in Pharo, rather than the 20 year old default one. The code is parsed with Lukas' really fast RBParser and then transformed to the 20 year old nodes, from there compiled. What's the advantage?

Well, you can transform the modern AST nodes before compilation! It has a great visitor API, that lets you do all kinds of transformations. You can use ThothCompiler as is just to feel fancy, or better: subclass it and overwrite the transform: method, in which you can transform the AST to your liking.

ThothCompiler is now used in Helvetica (albeit under the name of RBCompiler, because things can never have enough names!).

Changes since the last blog post:

Now uses the RBParser, which is much faster than SmaCC
No more dependency on the NewCompiler or AST package
Adds a preference to the system (in the system menu), where you can choose the compiler, even when you can't compile anymore :)

What to keep in mind when playing with compilers in Pharo

2009-12-15T12:12:00.002+01:00

If you play with the compiler in Pharo, here's an urgent hint: don't lock yourself out of your image! If you break the compiler, you want to have something to fall back to. So here's what you do.

The compiler for the system is whatever Behavior compilerClass returns, so change that method to read out a preference:

Behavior>compilerClass
 "Answer a compiler class appropriate for source methods of this class."
 |default|
 default := Compiler.
 ^(Smalltalk classNamed:  (Preferences valueOfPreference: #whichCompiler ifAbsent: [^ default]) ) ifNil: [default]

That's step 1. Second step is to create the preference, so you can read it out at all:

Preferences addTextPreference: #whichCompiler category: #compiler default: 'Compiler' balloonHelp: 'Compiler to be used throughout the system' .

That's it, you're set. Happy hacking!

Code beauty, J vs. Mathematica

2009-11-30T00:16:00.004+01:00

I drew the equivalent kernel of a smoothing spline in J.

Code:

t =: i: 10 j. 1000
y=:(%2) * (^-|t% (%2)) * sin(( (|t)* (%:2) % 2 )+ (pi%4))
plot t;y

This is the Mathematica version:

The J programming language

2009-11-29T20:37:00.008+01:00

Today I went through the first few tutorials of the J programming language. J is a treat. The human language has many devices that keep things short and explicit. While in context-oriented programming people are still wondering how to properly identify and model the "context" of a statement, natural language already goes and changes the behaviour of nouns by specifying them more precisely with adjectives. Compare that with modern object-oriented programming languages, which only have nouns (classes) and verbs (methods) at best.

The J language tries to harness the power of the natural languages by providing nouns, verbs, adverbs, conjunctions, and comparatives. I'd love to see an OO version with adjectives.

The tutorials are splendid and the IDE helpful, which I won't elaborate on because James Hague wrote such a nice piece about the fact. Oh, and he raves about the built-in plotting. Here's some turtle geometry, similar to an example that they provide:

The source code was:

show repeat 20;'point (n =: n+1) fd 1 rt 15'[n=: _2
   to 'goreturn a b';'goto b goto a'
   to 'goreturnall a'; 'goreturn for a,.i.20'
   show goreturnall for i.20

The article is over. Just one last thing: There's a funny glitch in the tutorials. The tutorials assume that you have some familiarity with Hilbert matrices and alternating binomial coefficients. But you are not expected to know how to handle a mouse:

The color window can be moved away from text that it may obscure by clicking on the top bar, holding the button down, and dragging it to a new position before releasing the mouse button –– Chapter 2, lesson 2 in the J tutorial

And it's done, Phexample has no more lolcats

2009-11-21T22:24:00.005+01:00

Lolcats have left the building, Phexample reads easier now.

Stack new isEmpty should not be true.

To get a meaningful error message out of this test, you're faced with a small dilemma. The Phexample framework lies in the should method, thus, for all that should sees, it is called on true, so the test might as well have been

true should not be true.

Now, to squeeze a reasonable error message out of this, such as "isEmpty should return false, but got true," it helps finding the unexecuted code snippet in the caller. Thus, the execution depends on who called you. Maybe not perfectly object-oriented, but context-oriented! There we have a new buzz-word: COP, context-oriented programming.

I learned a few things on the way: Bytecodes in Smalltalk have different sizes, thus you can't just walk backwards from where you are. The containing method must be read forwards, and once you hit the position of the current execution, you know you're a bit too far. Also, you find the current stack frame using the pseudo-variable thisContext, though there are some performance issues with it.

By the way, you can't really ice and copy the stack (someone forbade it in the source, for whatever reason), you're analyzing it as it is being used for the current computation. A bit scary, but it works quite nice. Pharo ftw! Try this in JavaScript!

Get the latest version of Phexample at SqueakSource (there's a Gofer-Script on that page that will do the downloading for you!.

Phexample's lolcats syntax

2009-11-19T21:04:00.008+01:00

In this post I'll clear my throat on what there is to come. The number one criticism of Phexample is that Phexample sometimes reads like lolcats, because real code that asserts that a stack be empty reads like

stack should not be isEmpty.

The complaint feels like an instance of the complaint of how stupid it is that in OSX you need to drag CDs to the waste bin to eject them, or how in Windows the bin stands on the desk. Of course the metaphor is broken there, but you only notice it because the metaphor works great.

And now you compare that to the usual alternative, which is assert. The first thing you know about assert is that nobody can remember which one is the expected and which is the actual value (I'll take this fact and some of the below discussion from Josh Graham's discussion of the assert syntax). We're talking about assertEquals(stack.isEmpty, true). Or the other way around. I certainly can't remember that either.

In Java, improvements have been suggested:
assertEquals(expected(true), actual(stack.isEmpty());

I find that while Phexample does not allow you to express your examples in a way that completely parses as natural language, it certainly reads easy enough and gives a natural and easy distinction between the expected and the actual.

So there's my analogy: no other testing framework is required to express its tests in a way that perfectly parses as human language. Only because the tests already look so close to perfection is it that people single out the mismatch and decry it. I don't think this is an interesting problem worth solving at all.

Yet, the criticism follows Phexample wherever it goes, so I'll try and allow phrases like

stack isEmpty should be true.

The difficulty in making the above code work is that the testing framework is only activated after isEmpty was evaluated already. Which means that it isn't in the stack anymore, so we need to analyze the source code to find out which method caused the failure to display a meaningful error message.

Telling other people how to install your code

2009-11-04T12:14:00.005+01:00

Previously, I created dirty ScriptLoader tricks to make it easier for others to download my code.

I am pleased to find out that Gofer is even easier than my dirty tricks, well without being a dirty trick: Gofer on Lukas's blog.

Phexample, because examples expand on one another

2009-11-04T12:05:00.012+01:00

Imagine you want to test the Stack class, and your first test just
creates a test and checks if it's empty.

Then in your second test, you create the same empty stack, but now
push an element 'apple' and check if the size is one.

Then in your third test, you do all the same, and then you pop the
element and check that it is an apple.

Wouldn't it be great if you could write your test cases such that they
could expand on one another? If you didn't need to copy paste the code
of the previous test and just say that you require the previous one!

Well, that's what PhExample lets you do. And as a bonus, tests are
ignored if the test they expand on does not pass. This gives you less
failed tests, because examples are executed only if the examples they
expand on pass too! That gives you the simplest failing examples in
your set to look at!

And best of all, you can run all examples with ye old TestRunner!

The example discussed above would read in code as follows:


EGExample subclass: #ForExampleStack
       …


shouldBeEmpty
       "Create the empty stack"
       | stack |
       stack := Stack new.
       stack should be isEmpty.
       stack size should = 0.
       ^ stack


shouldPushElement
       "Push one element"
       | stack |
       stack := self given: #shouldBeEmpty.
       stack push: 42.
       stack should not be isEmpty.
       stack size should = 1.
       stack top should = 42.
       ^ stack


shouldPopElement
       "And pop it again"
       | stack |
       stack := self given: #shouldPushElement.
       stack pop should = 42.
       stack should be isEmpty.
       stack size should = 0.

Find the code at http://www.squeaksource.com/phexample.html

You can install it as follows using Gofer:


 Gofer new
  squeaksource: 'phexample';
  addPackage: 'Phexample';
  load

The repository is write-global, so feel free to drop your ideas!

PhExample is based on JExample by Lea Haensenberger, Adrian Kuhn, and
Markus Gaelli. See JExample

Introducing ThothCompiler

2009-09-30T17:00:00.007+02:00

ThothCompiler combines the strengths of the two most popular smalltalk compilers. It takes the extensibility and wealth of the AST implementation from the NewCompiler and combines it with the most important asset of the old Compiler that per default ships with Squeak: it actually works.

Installation

The installation is fantastically easy. Go to Thoth on Squeaksource and get ASTBridgeLoader.

In a workspace, then execute:

ASTBridgeLoader load

Add the method compilerClass to the CLASS side of your class to set the compiler for the class. Thoth installs itself as the default compiler of the image. If you're unhappy with that, for example because you don't like the lack of error messages when you accept uncompilable code, you can choose for each class which compiler is to be used. Behavior>compilerClasss sets the default.

How do I hack into it?

Subclass ThothCompiler and override #compile:in:classified:notifying:ifFail: and #evaluate:in:to:notifying:ifFail:logged: Transform the AST before generate is called. Perhaps I should provide some infrastructure here …

How it works

It is a really simple thing: It uses the parser of the NewCompiler, then transforms the AST into the old model, and then uses the backend of the old compiler to compile.

Blocks in Smalltalk can have their own temporary variables

2009-09-16T16:58:00.003+02:00

Did you know that the following appears to be Smalltalk code? Runs fine in the current Pharo.

[:a | |b| 2] value: 3

Explaining Mathematica with the Solution for Problem 59 from Project Euler

2009-09-06T14:13:00.012+02:00

The solution to problem 59 is perhaps helpful to see what Mathematica is good at and how it basically works. The task is here. Let me summarize it. The task contains a link to a text file full of numbers. These numbers together are an encrypted text, the password being three lowercase letters. The decryption works like this: You convert the password into its ASCII values, and then XOR the first byte of the encrypted text with the ASCII value of the first letter in the password. You iterate this for all three letters, and then start over with the first letter of the password for the fourth number in the encrypted text. And so forth, until you are done. The task is to find the password.

Ok, so, the first thing Mathematica rocks at is importing. Here it probably isn't a big deal, but I find it noteworthy anyhow … getting the encrypted file is this:

text 
  = Import["http://projecteuler.net/project/cipher1.txt", "csv"]  
  //  First ;

Here, Import leaves a pair of braces around the real result, which the First function disposes of.

Now the idea of my checking is this. I'll take every third character from the encrypted file and store it in a list. Then I will try for each possible first letter of the password which letter would yield the most printable characters in the decrypted text. I will use this very narrow definition of a printable character:

 printable[a_] := 65 <= a <= 90 
  ~Or~ 97 <= a <= 122

One thing we cannot see here in Blogger is that Mathematica allows Mathematical typesetting, thus presenting this definition with a beautiful vee for an or and the inequalities look like inequalities.

The next task is to give each letter of the password a score. Let us assemble the procedure from the ground up. The goal is to build a function which takes every third element of the encrypted text, applies BitXor with the key, and then counts how many characters are displayable.

Getting every third element from the encrypted text, starting with the 1^st letter, is expressed as

text[[1 ;; ;; 3]]]

We can refer to the result of the previous computation as %. Let us try 103 as the first letter of the password, that would be the letter "g." Then we decrypt as follows:

BitXor[103,%].

And finally, we count the printable letters in the resulting list. Mathematica provides a Count function. Count is a nice function to demonstrate something rather unique to Mathematica: patterns. Let us first see our code, which counts printable letters:

Count[%, _?printable].

Here, _?printable is the pattern. The underscore stands for "anything", and the question mark specifies that the anything must evaluate to true when passed into the following function, printable. This is a very simple pattern, you can do cooler things, like the pattern {1,2,_}, which matches lists of three elements whose first two elements must be 1 and 2.

Now altogether the score function looks like this, where i ranges between 1 and 3:

score[i_, letter_] := 
 Count[BitXor[letter, text[[i ;; ;; 3]]] , _?printable]

The task states that valid candidates for the letters of the password must be lowercase. These have ASCII codes between 79 and 122. The function SelectMaximizer is an invention of mine. It chooses the element of a list which maximizes a function that can be supplied. The supplied function will be score, of course, so we can find the first letter as

SelectMaximizer[Range[97, 122], 
  score[1, #] &]

The confusing thing here is the closure, score[1, #] &. In Mathematica, everything that ends in an ampersand is a closure, or a local function. The hash symbol is the argument. We need it because score accepts two arguments (the position of the letter in the password, and the actual letter). Where we'd like to use only the second as an argument.

The function call will dutifully find the character "g" as the first letter of the password. To find all 3, we use the Table function. The Table function an expression with an undefined variable, i, and then tell it to use that variable as an iterator, assuming values from 1 to 3. The following code thus finds all three letters of the password:

password = 
  Table[ SelectMaximizer[Range[97, 122], 
  score[i, #] &], {i, 3}];

To decrypt the text with our password, we partition the encrypted text into runs of three, which we can then BitXor. Mathematica ships with a Partition function.

Partition[text, 3] returns a list partitions of length 3 of text text. Map function applies a closure to each element of a list. For example, Map[Range[4],#*#&] computes the first four square numbers.

Our first shot at a decoding function is thus:

Map[BitXor[password, #] & , Partition[text, 3]

We almost get it right. Unfortunately, the encrypted text's length is not divisible by three, and thus partition just 'swallows' the last letter. We can tell Partition to attach the remaining letter to the list. We use a little bit of trickery, because simply attaching the last letter would attach a list of length 1 to the list of partitions, but our partitions must have length 3 for the BitXor to work. Well, we can tell Partition to add the left-overs, and then pad to the length of 3 with the password itself, which will then become 0 after the Xor, so we're really appending null bytes to the string.

Together, the decoding function is now

decode[p_] := Map[BitXor[p, #] & ,
   Partition[text, 3, 3, 1, p]]

where p stands for password.

For what it's worth, the password is "god" and the result is chapter one of the Gospel of John. The task still requires us compute the sum of the ASCII codes of the decoded text. Our decode function really returns a nested list, to compute the sum of all entries, we could either use Flatten to reduce it to depth 1, or we could tell the Total function to operate on the second level of a nested list.

Total[decode[password], 2]

which returns 107359.

The complete code is

text = 
Import["http://projecteuler.net/project/cipher1.txt",
   "csv"] //  First ;

printable[a_] := 65 <= a <= 90 ~Or~  97 <= a <= 122

score[i_, letter_] := 
 Count[BitXor[letter, text[[i ;; ;; 3]]] , _?printable]

decode[p_] := Map[BitXor[p, #] & , 
  Partition[text, 3, 3, 1, p]]

password = 
  Table[ SelectMaximizer[Range[97, 122], 
  score[i, #] &], {i, 3}];

Total[decode[password], 2]

The complete code is can be downloaded as a complete code for problem 59 Mathematica notebook or as a complete code for problem 59 pdf file.

A Small Mathematica Vs C Benchmark

2009-08-14T21:30:00.012+02:00

Sometimes I get frowned upon because I do expensive computations in Mathematica. Some expensive computations will be very fast in Mathematica, of course. Try computing (10^4)//Factorial.

But what about the cheap computations that you could as well do in C? The Euler Project gave me the change to try it out with an easy problem. I was solving problem 9 from project Euler, which in short searches for natural numbers a<b<c, such that a^2+b^2=c^2, and a+b+c=1000.

The C-program that finds these numbers is


int main(void) {
 for(int a = 1; a < 10000; a++) {
  for(int b = a + 1; b <= 10000; b++) {
   int c = 1000 - a - b;
   if(a * a + b * b == c * c) {
    printf("%d\n",a*b*c);
    return 0;
   }
  }
 }
}

Note that I de-optimized a little bit by bounding a and b in [1,10,000], which is ten times greater than it should be. The result is still correct, but now it's slow enough so that we can measure something.

The run-time in C is 0.016 seconds on my machine

Bringing this to Mathematica is more than straightforward:


Module[{a, b, c},
  For[a = 1, a <= 10000, a++,
   For[b = a + 1, b <= 10000, b++,
    c = 1000 - a - b;
    If[a a + b b == c c, Return[a b c]]
    ]
   ];
  0
  ][]

Now the run-time is at 8.5 seconds. But! Mathematica can compile! To something faster, that is. It's as easy as wrapping a Compile around the computation. We get:


Compile[{},
 Module[{a, b, c},
  For[a = 1, a <= 10000, a++,
   For[b = a + 1, b <= 10000, b++,
    c = 1000 - a - b;
    If[a a + b b == c c, Return[a b c]]
    ]
   ];
  0
  ]
 ]

And now we're down to 0.38 seconds. Just as a reminder:

Table 1. Run times of the Euler project 9 algorithm in seconds. M stands for Mathematica
C	uncompiled M	compiled M
0.16	8.53	0.38

Which brings me to the following conclusion: For basic arithmetic computation, Mathematica will slow you down by a factor 40, if you use it only casually, or by a factor of 2 if you have some experience with it (Not everything is nicely compilable, so this is a bit optimistic).

Mathematica taketh away, but it also giveth: The library functions are fast, and there's plenty of them. You can treat Mathematica just like any scripting language: If the operations in your inner loop are costly, Mathematica is good for you! Otherwise, you pay a price for a wide range of library-functions. And it is worth delving into that library.

Pharo – perhaps Squeak 2.0?

2009-07-06T10:32:00.005+02:00

Saturday, two days ago, I had the pleasure of being present at the Berne Pharo sprint. It was not my first contact with Pharo, but close.

If I understand it correctly, then Pharo emerged because Squeak was torn into too many directions. Squeak was the one platform for child education, multimedia websites, and webserver development. Some code in the Squeak image was never released under a free licence, and the Squeak-licence itself is not an OSI-certified open-source licence. Squeak has more than 2000 open issues in its bugtracker, Mantis.

Pharo is a fork from Squeak that aims at removing cruft and turnung Squeak smalltalk into a more traditional programming environment, with a small core, stable and fast-evolving.

I am slightly suspicious about the reducing of the Image size. I don't know anything about it, but I still recall how in an interview, Dan Ingalls spoke about the advantages of an image that is NOT split into packages, where everything calls everything. I recall for myself that when I wrote a web application that was to send emails by itself, I was delighted to see that an SMTP client is included in a standard image already. It was fantastically easy to send an email message. On the other hand, I discovered at least one bug in the SMTP client code.

Still, I will use Pharo, and not Squeak, henceforward. The user interface is revised and does not look childish anymore. The number of open issues is small, and despite the alpha status, Pharo appears to be a lot more stable than Squeak. Pharo is already used as the reference implementation of Seaside, and thus, it is only a matter of time until it will draw a significant share of the Smalltalk world into using it for web development. Depending on how well it will manage to faciliate the installation of external packages, I would like to believe that for non-educational purposes, Pharo will take over all of the Squeak world, and serve as Squeak 2.0.

How fast is Comanche?

2009-05-26T10:40:00.001+02:00

The other day I came across this guy who wants to serve webpages in less than a millisecond. Without parallelization. So, his web page is really simple, it looks like this: <h1>Hello TWUG!</h1>.

His point is that things like Twitter are difficult to scale across several machines. The slowness of dynamic pages, he claims, is tried to be made up by advanced caching strategies. He goes on to check how fast certain languages are at serving dynamic content. He gets these numbers:

Clearly, his machine is faster than mine, because serving the same file statically, my Apache (single-threaded using httpd -X) can do only 1850 requests per second, that is 0.5 ms per request. (I use this benchmark: httperf --hog --num-conn 10000 --uri http://localhost:9090/?name=Niko). Here are the numbers I get:

Anyway, I find interesting to mention that Comanche is not far from Apache. Comanche can serve the same page dynamically (the name is a GET parameter) in 1.1 ms, for which Apache takes statically 0.5 ms. As for Apache-php vs. Comanche, I'd call it a tie.

The Squeak code for the Http server would be this:

ma := ModuleAssembly core.
ma addPlug: [:request | HttpResponse fromString: 
   (String streamContents: [:s| 
   s nextPutAll:'<h1>Hello! '; 
   nextPutAll:( request  fields at: 'name'); 
   nextPutAll: '</h1>']).].
(HttpService startOn: 9090 named: 'Example') module: ma rootModule

By this way, as the above snippet suggests, a standard Seaside installation for Squeak is quite able to make RESTful applications. Just avoid the Seaside framework and plug right into the Comanche web server.

For completeness, the PHP-code:

<h1>Hello <?=$_REQUEST['name']?></h1>

Update

By the way, this seems to be about 10 % faster:

s:=(HttpService on: 8080 named: 'Example Http Service').
s onRequestDo: [ :request |
 HttpResponse fromString: 
  (String streamContents: [:s| s nextPutAll:'<h1>Hello! '; 
  nextPutAll:( request  fields at: 'name'); 
  nextPutAll: '</h1>']) ].
s start

Adding new properties to a class

2009-05-21T18:30:00.001+02:00

When you load a new toy into Squeak and toy with it in a workspace, that is usually fun. So much more fun than creating a new class. I always suspect that I use the Squeak tools all wrong, but honestly, to create a new class with a couple of properties, it takes me fivethousand clicks to set all the instance variables, set the accessors and get them all properly initialized.

So, I still don't know if there might not be a much easier way to do it, but I'm fine with what I've dreamt up today. If I want to add a new instance variable to a class, including all accessors, I call this from a workspace:

ManyManyRelation addPropertyName: #oneLot.

And if I want to initialize the property to something, I use this:

ManyManyRelation addPropertyName: #otherLot lazyInitializeWith: [Set new].

The following method goes to the instance side of Class:

addPropertyName: aSymbol
  self addInstVarName: aSymbol.
  self compileAccessorsFor: aSymbol

And then:

compileAccessorsFor: aSymbol

"Compile instance-variable accessor methods
  for the given variable name "
   self compileSilently: aSymbol asString , '
       ^ ' , aSymbol classified: 'access'.
   self compileSilently: aSymbol asString , ': anObject' 
        , aSymbol , ' := anObject' classified: 'access'

You can now call something like ManyManyRelation addPropertyName: #oneLot, and it adds an instance variable to ManyManyRelation, including both accessors.

Let's push it one step further. Hooking into initialize really doesn't feel right––if objects are about modelling the real world, then what exactly corresponds to object initialization? While still not being optimal, the following creates lazy initizers in the accessors.

The following two methods go as instance methods into Class:

addPropertyName: aSymbol lazyInitializeWith: aBlock
   self addInstVarName: aSymbol.
   self compileAccessorsFor: aSymbol lazyInitializeWith: aBlock

compileAccessorsFor: aSymbol lazyInitializeWith: aBlock 
    | initializeCommand decompileString |
    decompileString := aBlock decompileString.
    initializeCommand := decompileString copyFrom: 
                2 to: decompileString size - 1.
    self compileSilently: aSymbol asString , '
        ^ ' , aSymbol , ' ifNil: [ ' , aSymbol , 
          ' := ' , initializeCommand , ']'  classified: 'access'.
    self compileSilently: aSymbol asString , ': anObject
    ' , aSymbol , ' := anObject' classified: 'access'

You use it by calling ManyManyRelation addPropertyName: #otherLot lazyInitializeWith: [Set new].

By the way, I would refrain from using (CreateAccessorsForVariableRefactoring variable: #hasMany class: OneManyRelation classVariable: false) execute, which is what happens when you create accessors using OmniBrowsers, the reason being it behaves strangely when the object already responds to a message, most typically like name.

Use relational databases and lock yourself out from your own data

2009-05-17T20:05:00.000+02:00

Searching a certain pattern in a graph of your data is the most exciting task. I learned that by writing a Master's thesis on algorithms that solve NP-hard problems. I know that years of research have gone into people trying to come up with smart algorithms that try to answer really difficult questions on graphs. I also know that researchers carefully take care that each of these graph problems really stands for a computation problem of great significance to real life computing. Graph algorithms help you find travelling routes, understand the preferences of your customers, and help you find the optimal arrangement of your office.

I am talking about graphs defined by your data. If your application has interesting objects, then maybe they form an interesting graph and perhaps this way of thinking brings you a fantastically fast algorithm that serves your business logic.

If you have an interesting query on your data, it may not even be POSSIBLE to formulate the query in SQL. Such a query might be: I have a band, give me the cluster of similar bands!

A relational database promises you to solve all your data searching problems if you give it all your data. That's a REALLY big promise. In fact it's esoteric. Why would a search path optimizer be able to do what otherwise researchers write papers on? If you have an interesting SQL query, then I bet you that your database is rather slow in answering it. Perhaps it hides it by computing the answer to the query long before the query actually shows up, but the cost is still quite high.

For interesting queries on your data, I suggest you do not trust the esoteric and thoroughly overrated RDBMS query optimizers. Trust your brain and come up with a smart graph algorithm. If you use a relational database, you have locked yourself out from your own data. That is because getting a single object that equals one data row out in a RDBMS takes typically about 20 ms. If you navigate through the graph of your objects and each step costs you 20 ms, you know just exactly how many steps through your graph are still ok if you plan to deliver the rendered website in less than a second.

Object databases, instead, show a fantastic speed when you are navigating through the object graph. Give object databases a shot and find your web application soaring all of a sudden. And let's stop pretending that RDMBSs are easy. I've spent more time than I'd like to admit pondering about the access paths that a query optimizer chose and just how exactly to nudge it to to something smarter. Which index to create? How much memory for what? RDBMSs don't think for you, they force to first ponder a smart access strategy and then ponder how to nudge the RDBMS into taking it.

Relational databases are slothful, bulky, and overall a great waste of time. Yours and the computer's. I avoid them wherever I can and I recommend you to do the same.

SandstoneGOODS

2009-05-14T15:54:00.002+02:00

SandstoneGOODS brings SandstoneDB (you might have to click that link twice) a backend to the GOODS object database. Using SandstoneGOODS, you can have the easiest thinkable persistence and transaction safety for your objects coupled with reasonable performance and scalability in Squeak Smalltalk.

SandstoneDB is the coolest OODBMS I am aware of, but until now there were only two ways of using it: either from Squeak using only one Squeak instance, or from a full-blown Gemstone. SandstoneGOODS allows to use SandstoneDB and one session per client in squeak seaside, which, some argue, is a very performant way to host Seaside web pages.

So, how does it work?

get a working copy of GOODS from http://www.garret.ru/goods.html and install it.
Put the following file into the directory where you want the database to reside:
```
1
0: localhost:6000
```
and call this file goods.cfg.
Enter a shell and navigate to the directory where you put the goods.cfg file. Execute goodsrv goods
Start up Squeak and install SandstoneGOODS (and all its requirements, namely GOODS, BTree, SandstoneDb).
shut down the image and restart it
You're ready to go! Use SandstoneDb as described by Ramon Leon

Performance

I can execute 1000 small commits in 30 seconds:

[ 1000 timesRepeat:[ SDChildMock new save] ] timeToRun.

I can execute one commit of 1000 small objects in 3 seconds:

[ SDActiveRecord commit:[ 1000 timesRepeat:[ SDChildMock new save ] ] ] timeToRun.

Read speed is excellent: reading 5000 objects from the db can be done in 1 second:

[SDChildMock findAll ] timeToRun--and once they're in cache, they are read in 0.1 seconds.

Altogether, I think SandstoneGOODS totally rocks and you can write your whole app using SandstoneGOODS faster than it took you before to write the ORM mapper definitions. Please tell me any bugs you find at niko.schwarz@googlemail.com. The repository allows public writing. You can find the sourcode on squeaksource.