procedure ContrivedExample;
var
   factorial: TFunc;
begin
   factorial :=
      function(input: Integer): integer
      begin
         if (input = 0) or (input = 1) then
            result := input
         else result := input * (factorial(input - 1));
      end;

   writeln(factorial(5));
end;

Or how about this one, which doesn’t use recursion, and is based on something I noticed when using the list comparison algorithm to solve a problem at work?

procedure SomewhatLessContrivedExample(list1, list2: TList);
var
   newItem, editItem, deleteItem: TProc;
begin
   newItem :=
      procedure(value: TMyObject)
      begin
         fMainList.Add(value);
      end;

   editItem :=
      procedure(value: TMyObject)
      begin
         if value.CheckSomeFlag = true then
            newItem(value)
         else fMainList[value.id].Update(value);
      end;

   deleteItem :=
      procedure(value: TMyObject)
      begin
         fMainList.Delete(value);
      end;

   listCompare(list1, list2, editItem, newItem, deleteItem);
end;

If you were paying close attention in my first two posts about how anonymous methods are implemented, you might realize that these anonymous method references are interface references to a TInterfacedObject descendant, and that by having the closure capture one, the object now holds a reference to itself.  Which means that when you actually run the code, you could end up with FastMM4 reporting something like this:

An unexpected memory leak has occurred. The unexpected small block leaks are:
21 – 28 bytes: ContrivedExample$ActRec x 1

Oops!  That’s not good, especially if you call ContrivedExample in a main loop somewhere and end up leaking hundreds of thousands of them.  This is too bad, since there are all sorts of interesting things you could do by using anonymous methods recursively.  And since anonymous methods don’t have a name, they can’t invoke themselves recursively without capturing the reference variable.

There’s probably no way for the compiler to prevent this in the general case, but it could be solved easily enough for examples that fit the pattern described in my examples:  When an anonymous method reference variable is only assigned once in an enclosing procedure, and that one point is where the method is defined within the procedure (i.e. it’s not passed in as a parameter or already assigned at global or object scope), then the compiler can be certain that the Self pointer for the underlying object is the same as the one it’s currently executing on, and transform it behind the scenes from an interface reference to a normal method call.

Can anyone else think of possible ways to fix this?

It’s been explained several times by several different people, including me in my last article about anonymous methods, that anonymous methods are implemented through interfaces.  And all it takes is a trivial little console application to writeln the sizeof a reference to variable and come up with 4, the size of a pointer.  Interestingly enough, though, the compiler won’t allow you to typecast a reference to to an IInterface, a Pointer, an Integer or anything else.  It’s an anonymous method reference and it won’t be anything else no matter what you do.

…unless, of course, you have another variable occupying the same four bytes of memory.  Something like this:

procedure TClassViewer.FormShow(Sender: TObject);
var
   proc: TProc;
   func: TFunc<string>;
   intf: IInterface absolute proc;
begin
   ReportMemoryLeaksOnShutdown := true;
   proc := procedure()
   begin
      assert(sender is TObject);
   end;
   func := function(): string
   begin
      result := sender.ClassName;
   end;
   loadClass(intf as TObject);
end;

I set this up to load itself into my class RTTI inspector tool, and used Delphi 2010′s trick of casting an interface to an object, and it worked! And here’s what the object looks like:

The name of the functor object’s class is the full name of the enclosing method, plus $ActRec tacked onto it. Its parent class is TInterfacedObject.

It implements two interfaces, one for each anonymous method. (There’s a third, IInterface, at the TInterfacedObject level, of course.)

Interestingly enough, even though the compiler goes to great pains to make sure this is only treated as an anonymous method and never as an actual object, Delphi 2010 still generates extended RTTI for the variables that the closure captures.

There’s not too much to see in the other tabs, but if you want to explore further with some more complex functor objects, feel free to download the RTTI inspector and take a look around.

Unicode: Hmm… looks interesting.
Generics: YES! FINALLY!
Anonymous methods: …huh?

I think that’s pretty much how everyone reacted to anonymous methods at first.  The syntax is kinda ugly, and you could get lost trying to read through a procedure and finding another procedure declared in the middle of it like that.  And all this so you could have… drumroll please… a procedure with no name! Ta-da! Umm… yeah.  OK, whatever.

Things got a little more interesting, and more clear as to what they were useful for, when people started talking about anonymous methods as closures.  Storing local variables inside an anonymous method that retains state and can be passed around.  That’s actually kinda cool!  But how does it work?  Well, let’s find out.

program Project1;

{$APPTYPE CONSOLE}

uses
   SysUtils;

type
   TAnonProc = reference to procedure;

procedure doStuff;
var
   counter: integer;
   i: integer;
   countProc: TAnonProc;
begin
   counter := 0;
   countProc := procedure
                begin
                   inc(counter);
                end;
   for i := 1 to 10 do
      countProc;
   writeln(counter);
end;

begin
   try
      doStuff;
   except
   on E: Exception do
      Writeln(E.ClassName, ': ', E.Message);
   end;
   readln;
end.

When I run this, the result that gets output is 10, showing that the closure has been modifying the actual local variable for the doStuff procedure and not its own private copy.  But how does that work?  Local variables are stored on the stack, but this closure can be assigned to a reference that can be passed out of the scope of the procedure.  Well, let’s look at the assembly code for DoStuff to get a sense of what’s really going on.  It may not be pretty, but it’s honest.  Nothing can hide from the assembly view.

Project1.dpr.16: begin
push ebp
mov ebp,esp
push $00
push $00
push ebx
push esi
xor eax,eax
push ebp
push $0040e8f1
push dword ptr fs:[eax]
mov fs:[eax],esp
mov dl,$01
mov eax,[$0040e7a8]
call TObject.Create
mov esi,eax
lea eax,[ebp-$08]
mov edx,esi
test edx,edx
jz $0040e891
sub edx,-$08
call @IntfCopy

Project1.dpr.17: counter := 0;
xor eax,eax
mov [esi+$0c],eax

Project1.dpr.18: countProc := procedure
lea eax,[ebp-$04]
mov edx,esi
test edx,edx
jz $0040e8a7
sub edx,-$10
call @IntfCopy

Project1.dpr.22: for i := 1 to 10 do
mov ebx,$0000000a

Project1.dpr.23: countProc;
mov eax,[ebp-$04]
mov edx,[eax]
call dword ptr [edx+$0c]

Project1.dpr.22: for i := 1 to 10 do
dec ebx
jnz $0040e8b1

Project1.dpr.24: writeln(counter);
mov eax,[$00411ccc]
mov edx,[esi+$0c]
call @Write0Long
call @WriteLn
call @_IOTest

Project1.dpr.25: end;
xor eax,eax
pop edx
pop ecx
pop ecx
mov fs:[eax],edx
push $0040e8f8
lea eax,[ebp-$08]
call @IntfClear
lea eax,[ebp-$04]
call @IntfClear
ret
jmp @HandleFinally
jmp $0040e8e0
pop esi
pop ebx
pop ecx
pop ecx
pop ebp
ret

Wow, that’s a lot of code!  The really interesting stuff here is what’s going on in the “begin” line.  After a bit of standard setup, it creates a try block, then does this:

mov eax,[$0040e7a8]
call TObject.Create
mov esi,eax
lea eax,[ebp-$08]
mov edx,esi
test edx,edx
jz $0040e891
sub edx,-$08
call @IntfCopy

It places a class pointer in EAX, then calls TObject.Create to set up a new instance of that class, and stores the resulting object pointer in the ESI register.  Then it copies ESI to EDX (second parameter in the register calling convention), loads the stack location for our anonymous procedure reference into EAX (first parameter), and calls System._IntfCopy, which is declared as

procedure _IntfCopy(var Dest: IInterface; 
            const Source: IInterface);

So now we have an interface reference to an object containing an anonymous method, used to implement the closure.  When we go to assign to counter locally, we get this:

Project1.dpr.17: counter := 0;
xor eax,eax
mov [esi+$0c],eax

Aha!  So it’s not stored on the stack after all; it’s treated more or less as a var parameter referring to a field inside the closure object.  It’s referenced as [esi + $0C], so our “local” variable is stored 12 bytes in, which is right where we’d expect to find the first data field on an object that implements two interfaces: IInterface and the “anonymous method interface” for this closure.  This explains why an anonymous method can’t capture by-reference parameters or the Result variable, which canonically resides either in EAX or on the stack, depending on your calling convention.  The compiler needs to be able to reference it inside the closure object’s heap space, and it can’t do that if the variable’s already living somewhere else.

This raises an interesting question:  What happens if you create two anonymous methods that both refer to the same local variable?  Where is the variable stored then?  I won’t waste space posting another big disassembly, but the answer is, “in the closure object for that procedure.”  Only one gets created, with multiple (anonymous) methods and multiple interface references.  (Incidentally, this is also how CLOS creates objects:  by wrapping multiple closures around the same set of variables.)

There’s more interesting stuff that can be found by poking around in closure objects, but this post is getting long enough, so that’s all for now.  I’ll go into more details about anonymous methods and closure objects in a few days.