Request for Comments: Simpler Self

Below is a brief history of Smalltalk and Self from the viewpoint of growing complexity and simplification. This is followed by 5 suggestions on how to continue this trend of making things simpler: a cleaner story about method/activations/blocks, making non local returns less special, eliminating syntax for literal objects, unifying assignment with argument passing and removing parent pointers from base-level programming.

Brief History of Smalltalk/Self

Most programming languages grow more and more complex with time. That is to be expected since the most common reason to change a language is to have someone propose a new feature and show an example that looks really awkward using only the current features.

Smalltalk has (mostly) been an exception to this rule. The original Smalltalk-72 was an implementation of a design created to win a bet: could the most powerful language in the world (even better than Lisp!) be defined in just one page? Smalltalk-74 made the message stream and other things into objects, reducing the number of fixed concepts in the language. Smalltalk-76 made classes into objects, continuing the previous trend. It made the language less flexible by defining a fixed syntax instead of allowing each class to define its own. Performance was greatly improved by the added complexity of a Smalltalk to bytecode compiler. The most important thing to note is that though there were some examples in -74 which became more awkward in -76, most practical code became simpler.

In Smalltalk-78 a lot of simplifications were made to run on commercial microprocessors and this included moving the bulk of the graphics code from the implementation into Smalltalk: the flexible BITBLT primitive was used in many different and creative ways. Smalltalk-80 did bring a lot of complications, including its very flexible meta-level. Most Smalltalks today are complications of the original Smalltalk-80.

The Self-87 definition (originally Smalltalk-86) was a great simplification. Classes were eliminated by having objects hold their own code and format information and by creating new objects by cloning prototypes. Arguments and temporary variables were now simply slots in the current context object and were accessed by redefining the lookup algorithm for messages sent to self. Making contexts inherit from one another was enough to simulate lexical scoping. Making method contexts inherit from their receiver was enough to access the receiver slots (instance variables) with the exact same lookup. Having the receiver inherit from some shared "traits" object was enough to make its slots (class variables) available. And having most objects inherit from the "lobby" object was enough to make its slots (global variables) available nearly everywhere. And the complex meta-level was entirely eliminated.

Self 1.0 (Self-89) was the first implementation and included several complications: private slots, parent priorities and automatic "tie breaker rules" for multiple inheritance. Self 1.2 (Self-90) had a far more complex compiler and Self 2.0 (Self-91) was more sophisticated with a better interactive performance but neither made changes at the language level. In Self 3.0 (Self-93) the language was simplified by removing parent priorities and privacy. Annotations were added to cope with the loss of the ability to organize large objects with lots of mixins (no longer practical without parent priorities) and to help the transporter fileOut/fileIn system. This was needed since Self now had a graphical development environment and source files were no longer hand crafted. The implementation evolved by replacing one complex compiler with two simpler ones.

Self 4.0 brought us simpler graphical programming with Morphic and outliners. The implementation was changed to save memory. Neither the language nor the compilers were changed. Self 4.1 became easier to port to machines other than Sparc and gained features to help run languages like Java on the virtual machine. This included new bytecodes which also make an interpreter more practical than with the previous (simpler) bytecodes.

So how could a future version of Self be even simpler than Self 4.1.5?

1. simpler method/block story

Self currently needs the following objects to implement the execution environment:

methods: the have slots for the arguments and temporary values and the following "pseudo slots":
- sourceString: the texts from which the method was compiled
- line: the line number where the source was originally read from
- file: the file from which the source was read
- code: the actual bytecodes to be executed
- literals: a vector with the value for bytecodes needing literals. Also has a pointer to the object where this code can be found in order to make resends work
method activations: In theory, these are created by simply cloning a method. It doesn't even exist as an object in practice, but it can have a mirror and this is enough for what it needs to do (the mirror knows the process and position of the activation and can access the machine level stack using primitives). These are called "contexts" in Smalltalk.It has information not found in the method object:
- expression stack: where the bytecodes get their arguments and save their results
- instruction pointer: which bytecodes to execute next (offset into the code vector
- sender: who called us - needed when returning
- receiver (:self*): this makes the activation act as if it were lexically enclosed in the receiver
blocks: these are used as arguments in the "push literal" bytecodes, but instead of being pushed on the stack directly a block activation is created and that is what is pushed. It has a parent slot pointing to "traits block" and a 'value' (or 'value:', or...) method with the code defined in the block source
block activations: in addition to the slots in the block, it also includes a pseudo slot which saves a pointer to the execution in which the "push literal" that created it was executed in
block method activations: like a regular method activation, but instead of information about the receiver it has a "lexicalParent" pseudo slot

In addition, Self 1.0 also had an "inner method" object which was like a block that is always executed immediately - its method activation was like that of a block method activation. This made the language more uniform by allowing an expression in parenthesis to be treated as if it were a method. Later Selfs complicated the syntax by defining parenthesis as nested expressions to be handled by the parser, but the result was a simpler runtime structure with a "flatter" code.

A lot of people are confused about the three different objects needed to implement blocks. A single method might be invoked multiple times simultanously, so the "push literal" must make a separate copy each time so it can save the value of the current activation (which will be different for each invocation of this method). And the 'value' message might be sent multiple times to a single block activation, so it is nice that a clone is made each time (as is usual for methods in any case). Note that block activations and block method activations have different parents, so you can pass blocky messages to the first one but not to the second.

It is interesting to note how similar an activation is to a continuation. Both represent the state of a computation, but an activation is only a part of the stack and depends on other activations as well as the state of objects. A continuation is supposed to represent the whole stack. The idea of continuations was perfected in the Lisp dialect Scheme. Many control structures could be implemented using them. You can invoke a continuation and pass it a single argument and if that continuation represented the state of execution just before you were called then the effect would be exactly that of returning (continuing) a value from a call. This made calls, jumps and returns more uniform.

My proposal is to have a single type of object similar to the current activations but that can be invoked like a continuation. The semantics of a message send would then be

 send: message To: method = (
  | na |
  na: method clone.
  na sender: thisContext.
  na self: message receiver.
  na loadArgumentsFrom: message arguments.
  na run
  "execution never reaches here" )

Here I explicitly set the 'sender' and then did the invocation with 'run' and no arguments.

When a method finishes executing, it can return by doing

  return = ( sender run: expressionStack pop )

since what we stored in 'sender' was simply another continuation.

We could have inner methods and we could make the push literal bytecode invoke them like this

 sendTo: method = (
  | na |
  na: method clone.
  na sender: thisContext.
  na nextLevel: thisContext.
  na runOrSelf
  "execution never reaches here except for blocks" )

But while regular methods have their "self" slot marked as parent, inner methods call their parent "nextLevel". Blocks are exactly like inner methods but have an additional parent pointing to "traits block". The 'runOrSelf' method is implemented as (run) for inner methods and (self) for blocks. And here is what a variation of 'value' would look like in traits block:

  value: arg1 With: arg2 = (
    | na |
    na: clone.
    na sender: thisContext sender.
    na loadArgumentsFrom: arg1 & arg2.
    na run
    "execution never reaches here" )

This gives us the right semantics except that the objects replacing the block, block activation and block method activation are all clones of each other and so both understand all the blocky messages and can access the slots defined in itself and any outer scope. This code would cause an error in Self 4 but would work in this proposal:

    [ | :a. :b | a + b ] a

It would return "nil" in this case. In practice, the vocabulary of blocks is separate enough (except for 'value') from other objects that the pontential for conflicts isn't too bad.

I wrote that there should be a single kind of object but showed three. The differences are small enough (one slot name and one parent value) not to matter.

2. simpler non local return

This supposes that the single continuation-like object of the previous proposal is used. Since we can omit the receiver of a message when it is sent to self, this expression is perfectly valid:

   + 43

We can think of the carat character (or uparrow) as simply another binary message selector, and in that case there is no reason not to define in 'traits method':

   ^ arg = (
      sender run: arg
      "execution never reaches here")

and in 'traits block':

   ^ arg = (
      sender ^ arg
      "execution never reaches here")

An alternative would be to define 'topSender' as the same as 'sender' in outer methods but have inner methods and block inherit this value. Then we could return to topSender in a single step.

In any case, this makes finding syntax errors (specially the popular missing "." at the end of the previous line) harder. One solution would be to issue a warning whenever the first token in a line is a message to an expression in the previous line. That nearly always is a case of a missing ".".

Brian Rice noticed that in this scheme we have to add parenthesis around returned expressions with keywords. One solution would be to forget the binary selector (the caret character is ugly compared to the up arrow originally used in Smalltalk anyway) and use a keyword selector like 'return:' instead. This would make the language look more familiar to people coming from C/Java, though I don't know if that is a good thing.

3. simpler syntax - no literals

Consider the following method:

  myColor: frac = ( | r. b <- color named: 'blue' |
       r: color named: 'red'.
       frac interpolateFrom: r To: b
       )

The the code used to initialize 'r' looks very much like the one for 'b', there are important difference. It is executed every time the 'myColor:' method is invoked instead of when the method is edited (and automatically "compiled" to bytecodes). It is executed in the context of the 'myColor:' method instead of the 'lobby' object, so it wouldn't work if this object doesn't happen to inherit from 'globals' at least indirectly.

In Self 4's graphical programming environment, we have an interesting alternative in the 'b' case for when the textual representation of the initial value is too complex: just type 'b <- ()' and then drag the arrow to point to where we want instead of the newly created empty object.

My proposal is to give 'r' the same advantages as 'b' by allowing any object to be dropped in the middle of source code. The parser ("compiler" in Smalltalk terms) would only have to generate a "push literal" bytecode and copy the embedded pointer from the source to the literal array. In fact, we could eliminate all syntax related to literals and the language would still work. Intead of having the parser understand the syntax for strings or numbers we could embedded the actual string and number objects in the source. We wouldn't have to have syntax for blocks or even slots as long as the programming environment gives us a graphical way to build them. That is, we don't have to be able to write

  x _AddSlots: ( | zzz = 9 | )

if this can do the job

  (reflect: x) addConstantSlotNamed: 'zzz' Value: 9

When Self was a textual language where we used "vi" as a programming tool, the more elegant first version was important. But now that we program by dragging boxes and arrows around the screen, what goes under the hood doesn't matter as much.

Of course, with existing tools it would be far more awkward to have to find or build a simple object and then drag it to where we are typing than what we have now. But this is just a matter of making the editor used for creating sources smart enough. If while typing an expression you hit ' for example, the editor could automatically insert a new empty editable string in the code and then jump the cursor into this sub-editor so the next characters you type would be added to the string. Pressing ' again could pop you back to the top level editor. Same thing for numbers and blocks. Smalltalk has a syntax for literal arrays. Rebol (http://www.rebol.com) has dates, times, money, URLS, email, TCP addresses, tags, logic, lists, hashes, tuples, XY pairs and others. Wouldn't it be nicer if not only these but all kinds of objects could be "literals"?

One interesting side effect of this proposal is that many objects would no longer require access to 'globals'. They would include references to exactly the objects they need in order to work. The graphical environment has access to 'globals' so we can get the objects we need at edit time. Like the controlled pointer copying of capability systems, this seems like a good thing.

4. simpler assignment story

All the different types of variables in Smalltalk are replaced by constant slots and data slots

  ( | pi = 3.14. var <- 12 | )

where data slots are actually considered a short cut for a pair of "access" slots

  ( | pi = 3.14. var = 12. var: = <- | )

The third slot, 'var:', is considered to be a special kind of method slot that contains an "assignment primitive" instead of an actual method. Some people don't find this satisfying and would like for assignment slots to be real methods, but this would merely push the introduction of "magic" down an extra level. Even so, this extra level would allow the association of new side effects with assignment which is currectly handled like this

  ...
  rawColor <- color named: 'red'.
  color = (rawColor).
  color: c = (rawColor: c. update).
  ...

This object's users are supposed to ignore the actual 'rawColor' data slot and program as if it had a 'color' data slot with a side effect on assignment.

But there is another part of the language where values get assigned to slots. With a method defined as

  goof: height And: width = ( .... )

which can also be written as

  goof:And: ( | :height. :width | .... )

all we have to do is use an expression like

   obj goof: 45 And: 29

to assign the first value (45) to the 'height' slot and the second value (29) to the 'width' slot.

I propose that we consider an alternative to the positional binding of arguments. What if both these expressions achieved the same result as the previous one?

   obj goofHeight: 45 Width: 29

   obj goofWidth: 29 Height: 45

We might even go a step further and allow the method to be defined with default values

  goof = ( | :height <- 45. :width <- 45 | .... )

such that this expression would yield the same result as all the previous ones (note that I supposed we no longer needed the colon in the slot name with this kind of syntax)

  obj goofWidth: 29

Having all methods with the same name limited to having also the same argument names might require a change in style, but otherwise shouldn't be too much of a problem.

One very interesting slot in a method activation object is 'self', which indicates the message receiver. If we imagine that the value of this slot, though actually just a regular object, looks a lot like a method, we might interpret

   selfHeight: 30

as changing the value of the 'height' slot in the 'self' object, which is really an assignment to a data slot. This allows direct manipulation of an objects state by its methods, which isn't something I want. But you can't use this to have access from "outside" the object since then 'self' won't have the right value.

We could now implement the assignment side effects

   color: = ( | :color | selfColor: color. update )

Note that the fact that the argument (which I suppose must have the same name as the whole method when there are no capital letters) shadows the data slot that we are trying to change isn't a problem at all.

Since the pattern

   xxx: = ( | :xxx | selfXxx: xxx )

would be so common, this might be indicated with a flag in the corresponding data slot instead of explicitly.

5. removing inheritance from the base-level

Self reuses the idea of constant and data slots to indicate objects for automatic delegation (inheritance). All you have to do is add a "*" to the end of the slot name. Since data slots can be changed by normal message sends during runtime, we also have "dynamic inheritance".

The ability to fetch the parents from the base-level is rarely used. The slot name is used by the "directed resend" bytecode, which is like "super" in Smalltalk extended to deal with multiple inheritance.

What if parents were indicated not with special slots, but as a set or list available through an object's mirror? The virtual machine and graphical programming environment would hardly be affected at all, and they are the most interested parties in this case. Dynamic inheritance would become more awkward

   (reflect: self) setparentName: 'linkMixin' Value: x

instead of simply

   linkMixin: x

but that would actually be a very good thing. It makes the programmer more aware of the deep implications of what he is doing. Bringing meta-level programming into the base-level with innocent looking expressions almost always leads to grief sooner or later.

In the above example I supposed that parents were associated with names even if limited to the meta-level. This would allow resends to look exactly as they do now. There are other alternatives and it might be a good idea to replace directed resends with a general explicit delegation mechanism. Instead of

  c: treeParent.copy: mySize

we could have something like

  c: #treeParent# copy: mySize

which would also allow things like

  #(userPreference pen)# forward: d

Of course, a far nicer notation can be developed - the message selector might be altered instead of the receiver expression, for example. The point is that most people who talk about prototypes and delegation think Self already allows this kind of thing (and it does with methods like 'sendTo:DelegatingTo:With:' for strings, but this is a bit indirect) and this could be made true while making parents simpler at the same time.