Software merging, commit messages

What is the intent of a change? How do you merge conflicting changes? The current state of the art is dire. There's a survey, A state-of-the-art survey on software merging , but it's summary may be that there's no silver bullet to automated merging without understanding the intent of a commit.

How do you understand the intent of a commit? The best technique I am aware of today is … having the author provide a clear description of his intent. There's a paper about that, quite worth your while: Semantic patches considered helpful.

I told you that the situation is dire, didn't I?

Distance measure between software artifacts

There's currently some interest in software visualization, which tries to speed up your understanding of source code by leveraging your brain's great ability to discern patterns in images. Below is shot of Codemap, visualizing something.

To draw a map such as the above, you need an underlying metric that tells you how far apart the code artifacts should be. (You probably won't be able to draw a map where the distances are exactly what you want them to be, but you may get close.) Now, what is used as a metric between software artifacts (islands, in the above image)? Codemap uses the vocabulary of the artifact, that is: artifacts that use many same words will end up closer together in the overall drawing.

A botfly moves along a herd of cattle. It finds every of the beasts hospitable, but it can't live without a herd. Thus, Richard Dawkins calls the herd the botfly's "Archipelago" (in The greatest show on earth, p. 253). This inspires me to a metric which is supposed to measure similarity of software artifacts. The idea is to assume that if two artifacts are hospitable to the same code snippet, then they should provide similar environments. Thus, you could measure how in the evolution of a software program, snippets wandered from one place to another, and assume that whenever a code snippet is moved from one artifact to another, these two artifacts increase in our metric's similarity.

It seems difficult to bring forth evidence that software maps aid understanding in the first place, thus I wouldn't know how to show that a metric like the one outlined above is any better than a comparison of metrics. Yet, there's some momentum in the field, so I'll save up the idea for a point in time when more light will be shed on the benefits of software visualization.

On Erwann Wernli's idea of having the VM take care of dependency injection

My dear friend Erwann asks in his blog whether dependency injection should not be the task of the virtual machine. He considers code like this:

List l = new ArrayList();

He then notices (correctly, to my mind), that this statement contains more detail than the developer meant to express. "Whether the list is a new one or not, I don’t care, all I need is an empty one," Erwann writes. Fair enough.

There's real problems connected with the observation. The Java specification rules that new has to allocate a new object. Hence, the method

static void slack() {

new Object();

return;

}

cannot be optimized into a nop. Now, I believe you've already thought of a code snippet that cannot be optimized due to this rule. And of course it gets worse if the language forces us to use new all the time. it is the only way to get an object, even though we would have been fine with an old one, should it fulfill our criteria.

Here's the solution he proposes:

List l = obtain list ( size=0 );

To undo the awkwardness, he proposes hiding this monstrum in syntactic sugar, although it isn't clear to me what the sugar should look like in this instance.

Besides avoiding the new operator, his proposal leaves the concrete type of variable l open. He now proposes that depending on the "context," the VM can do voodoo to determine exactly which class may be most appropriate at the point in time. As an example, he cites changing the default FileInputStream to a zipped version of itself, as disk space runs low.

I find his thinking interesting, because it asks the good old question of who's responsible for what in programming. Having the machine figure stuff out for you is great if the figuring out is easy or secondary. Everyone loves Ruby's ActiveRecord, which creates the database table belonging to an entity class of yours all by itself. But choosing the right kind of a list? That's harder. There's a disadvantage to having the machine figure stuff out for you: your program gains in being opaque. The text-file that stores your program code would diverge further from the code to be executed at run time, because important decisions about the code are delayed to the run-time of the program. Since us humans can only see text, this is a real drawback.

I'm obviously sceptical to ubiquitious dependency injection, and I'm not the only one. Here's what Russ Olsen has to say about the matter (In his book Design Patterns in Ruby):

The best way to go wrong with any of the object creation techniques that we have examined … is to use them when you don't need them. Not every object needs to be produced in a factory. In fact, most of the time you will want to create most of your objects with a simple MyClass.new. … Remember, chances are You Ain't Gonna Need It.

Well, we have to balance our skepticism against what voodoo may bring us. Suffice it to say that I don't really believe that I want the VM to have any say in whether I zip my files or not. If I write a file to the disk that then the bash will assume to be a plain text email message, it better not be zipped on 5 % of my clients' computers. Another advantage of the voodoo is that we may also get to re-using objects that are laying around anyhow, waiting to be garbage collected. There's charm to that idea, although I wonder if the picture is as rose as Erwann paints it. After all, you'd need to find an object to be disposed first. Object Pooling sucks up memory and CPU cycles for their management.

Let me summarize that I'm skeptical about the solution that is proposed. Nonetheless, Erwann raises legitimate and important questions, such as: Don't our languages force the developer to supply and commit to more detail than the developer wishes? A mismatch between the mental model of the developer and the programming language is a flaw that deserves our tending to.

There's one little afterthought. Erwann claims that the mismatch he identified can be tackled only by making both language and VM aware of the problem. I believe the strength and beauty of Smalltalk is to come a far way in allopwing new ways of doing things, without having to dig to the VM and the language (See the blog post by Benjamin Pollack: Your Language Features Are My Libraries). Well, I think that Newspeak's late binding of class names could implement all of Erwann's ideas in a library. Here's the idea: If you type Array.new, then you may expect Array to point to a specific and global class in a namespace that is the same everywhere. What if all lookups had to go through a "lobby," as it were, and each module could not demand the module, it would have to be passed. Then, we could have the lobby figure out what List.new is supposed to mean, and have it be context-dependent. (Newspeak is a dream of a programming language. It breaks my heart to see how it isn't used widely. See the Wikipedia article for an overview.)

Summary

Programming languages do not currently reflect the mental model of the developer in instructions as easy as

List l = new ArrayList();

They specify more details than the programmer wishes to, and as a result, optimizations and opportunities for automation are lost. I suspect that Newspeak would be a great platform for experiments, and I'd like to thank Erwann Wernli for sharing his concerns. His blog is great by the way. Please go and read it!

Here is Erwann Wernli's article: Erwann Wernli: Should DI and GC be unified?.

ThothCompiler revamped

ThothCompiler has been revamped very near completely. Get it by executing:

Gofer it
    squeaksource: 'ASTBridge';
    addPackage: 'ThothCompiler';
    load

Then go to the Preference Browser (System->Preferences in the world menu) and choose ThothCompiler for the option "whichCompiler".

ThothCompiler allows you to use a modern AST in Pharo, rather than the 20 year old default one. The code is parsed with Lukas' really fast RBParser and then transformed to the 20 year old nodes, from there compiled. What's the advantage?

Well, you can transform the modern AST nodes before compilation! It has a great visitor API, that lets you do all kinds of transformations. You can use ThothCompiler as is just to feel fancy, or better: subclass it and overwrite the transform: method, in which you can transform the AST to your liking.

ThothCompiler is now used in Helvetica (albeit under the name of RBCompiler, because things can never have enough names!).

Changes since the last blog post:

• Now uses the RBParser, which is much faster than SmaCC
  
• No more dependency on the NewCompiler or AST package
  
• Adds a preference to the system (in the system menu), where you can choose the compiler, even when you can't compile anymore :)
  

What to keep in mind when playing with compilers in Pharo

If you play with the compiler in Pharo, here's an urgent hint: don't lock yourself out of your image! If you break the compiler, you want to have something to fall back to. So here's what you do.

The compiler for the system is whatever Behavior compilerClass returns, so change that method to read out a preference:

Behavior>compilerClass
 "Answer a compiler class appropriate for source methods of this class."
 |default|
 default := Compiler.
 ^(Smalltalk classNamed:  (Preferences valueOfPreference: #whichCompiler ifAbsent: [^ default]) ) ifNil: [default] 

That's step 1. Second step is to create the preference, so you can read it out at all:

Preferences addTextPreference: #whichCompiler category: #compiler default: 'Compiler' balloonHelp: 'Compiler to be used throughout the system' .

That's it, you're set. Happy hacking!

Code beauty, J vs. Mathematica

I drew the equivalent kernel of a smoothing spline in J.



Code:

t =: i: 10 j. 1000
y=:(%2) * (^-|t% (%2)) * sin(( (|t)* (%:2) % 2 )+ (pi%4))
plot t;y

This is the Mathematica version:

The J programming language

Today I went through the first few tutorials of the J programming language. J is a treat. The human language has many devices that keep things short and explicit. While in context-oriented programming people are still wondering how to properly identify and model the "context" of a statement, natural language already goes and changes the behaviour of nouns by specifying them more precisely with adjectives. Compare that with modern object-oriented programming languages, which only have nouns (classes) and verbs (methods) at best.

The J language tries to harness the power of the natural languages by providing nouns, verbs, adverbs, conjunctions, and comparatives. I'd love to see an OO version with adjectives.

The tutorials are splendid and the IDE helpful, which I won't elaborate on because James Hague wrote such a nice piece about the fact. Oh, and he raves about the built-in plotting. Here's some turtle geometry, similar to an example that they provide:



The source code was:

show repeat 20;'point (n =: n+1) fd 1 rt 15'[n=: _2
   to 'goreturn a b';'goto b goto a'
   to 'goreturnall a'; 'goreturn for a,.i.20'
   show goreturnall for i.20

---

The article is over. Just one last thing: There's a funny glitch in the tutorials. The tutorials assume that you have some familiarity with Hilbert matrices and alternating binomial coefficients. But you are not expected to know how to handle a mouse:

The color window can be moved away from text that it may obscure by clicking on the top bar, holding the button down, and dragging it to a new position before releasing the mouse button –– Chapter 2, lesson 2 in the J tutorial

.