Run-time stack information?

Just in case you haven’t listened to it yet, Jim McKeeth over at Delphi.org posted a new podcast last week. He did an interview with Allen Bauer that apparently ran for about two hours, so he split it up into two parts.  The second part isn’t up yet, but there’s a lot of interesting stuff in the first one.  But one of the most interesting things actually came in the comments.

In the podcast, Allen mentioned certain features that were “holding the language back.”  Now, if you were to ask me what needs to be taken out of Delphi, I’d say with.  And I wouldn’t actually take it out; I’d change the syntax so it works like it does in VB, and I’d disallow nested with blocks.  Between those two changes, that should eliminate the ambiguities that make with troublesome and turn it into a very useful feature.

But Allen’s thinking on a bit of a lower level.  He mentioned absolute.  When I asked why absolute was problematic in the comments, he responded:

“absolute” can cause significant type-safety holes by allowing one to “overlay” a variable of one type on top of a variable of another type. In the future this will cause significant confusion for things like further enhanced meta-data and run-time stack information leading up to a richer/safer run-time experience. This would certainly interfere with the potential future inclusion of garbage collector, for instance.

This raises some serious philosophical questions about the future of Delphi.  From what I’ve seen, most of us tend to keep our diets rather low on kool-aid and don’t actually want garbage collection in Delphi.

But above that, mentioned ever-so-casually, is a term I don’t think I’ve ever seen before: run-time stack information. In fact, a Google search for “run-time stack information” (in quotes; without quotes we get nothing useful) turns up this podcast as the #1 result.  So what is run-time stack information?  It’s hard to say exactly, since I’m not Allen Bauer or a member of the Delphi team, but I can make a few educated guesses.

I’ll begin by assuming that the obvious similarity to “run-time type information” doesn’t end with the name.  So then Allen’s likely thinking about information about stack data that’s available at run-time, that would otherwise only be available in the source code and then get thrown away by the compiler.

So what goes on the stack?  Four things:  Parameters, local variables, implicit local variables, and stack frames.

Pretty much all the relevant data on parameters is already available through extended RTTI, at least for methods that have RTTI available.  So RTSI on parameters could probably be implemented simply by referencing the RTTI for the current function.  (Of course that would limit RTSI coverage to methods that actually have RTTI available.)  Not much of interest here.

At first glance, information on local variables would seem to be even less useful.  At its core, RTTI is about discovering things at runtime that can’t be statically verified at compile-time.  A routine already has all the information it needs about its own locals at compile-time.  Things get more interesting after looking at stack frames, though.

Implicit local variables can also go on the stack.  They’re for intermediate values. For example, if I write “x := x + y * z;” the compiler has to calculate y * z and store the result somewhere before it adds it to x.  Depending on various factors, that could go in a register or it could go on the stack.  Either way, I don’t think there’s much usefulness in creating data tables about implicit variables.

Stack frames are what makes the concept really interesting, though.  Here’s where we get information that’s not available at compile time.  Imagine if there was some way for a routine to determine what called it, and if that process could be continued for an arbitrary number of iterations.  We’d have an official API for examining the call stack from within Delphi.  There are various call stack examiners already available, especially as part of exception-logging packages, but they tend to depend on hacks and implementation details.  An official call-stack tracing API would be very useful, especially as Delphi starts to branch out into other platforms and architectures.

(Having said that, I’ve noticed that MadExcept frequently gets the call stack right in places where Delphi’s debugger gets lost.  Maybe Embarcadero should talk to Mathias Rauen about the possibility of integrating some of his code into the debugger?)

Call stack RTSI is also what make local variable and parameter RTSI interesting.  A function knows everything it needs to know about its own locals, but what if it could examine the locals and parameters of functions further up the call stack?  Admittedly, there’s probably not much practical use for this for most cases, but it would make it theoretically possible to build a debugger into your program.

Of course, a debugger runs on debug data, which is traditionally not included in a program, and for a good reason.  To make that information available at runtime, it needs to be included in the binary, where anyone with the right tools could get at it.  This is the sort of stuff that makes .NET assemblies trivially decompilable, unless you use an obfuscator that eliminates the benefits of having that level of detailed metadata around in the first place.  Anyone want to see a Delphi Reflector?

So, what is “run-time stack information”?  How is the team planning to implement it?  Is it merely a concept, or do they have any work in progress on implementing it so far?  It’s hard to say.  These are just my best guesses about what it might be and what might be done with it.  But it sure sounds interesting…

8 Comments

  1. A.Bouchez says:

    With the enhanced RTTI, generated exe size just grow up… Delphi XE executables are twice bigger than Delphi 7, with the same functionalities.

    Adding some RTTI at the stack-level… why not… but why? Just to make your executable four times bigger than Delphi 7?

    You’ve classes, parameters and code architecture to make your code well designed.
    Starting to look at the caller’s stack does make sense for MadExcept… but I don’t see the point for applications.

    About “absolute” or “with” statement (even nested!!!!), I have to confess that I still find it very useful in some cases.
    Since I use Alt-F2 and take a look at mostly every line generated by the compiler when I write some code, I know exactly what is generated. And sometimes, “with” or “absolute” are just the good combination for generating the fastest code. OK – that’s a border-line usage, I must admit.

    You can always put a local variable on stack instead of a with statement, of course. It will work well for classes, but for a record/object you’ll have to use a pointer.

    Another sin against strong typing are untyped var or const parameters. Since such tricks are used in the RTL, I don’t see deprecating a var or const typeless parameter without a lot of code breaking.

    Embarcadero tries its best to maintain as much backward compatibility as possible.
    They know that they have to take care about the existing code base.
    Some product manager would just make the decision to go to DotNet just because of some backward compatibility issue: “Oh it doesn’t compile with the new IDE? OK, let’s switch to DotNet!”. This is not a good idea, but we all hear that all days, in the real world. All Delphi programmers are not Delphi enthusiasts… Some are paid to maintain some application, and are very sensible to the Micro$oft marketing sirens song…

    • A.Bouchez says:

      I’ve just to add that when Allen Bauer speak about the future of Delphi and a garbage collector… I start crying…

      Do those people don’t learn from experiment?
      What about the Delphi for DotNet death?

      I need non-managed code, with no garbage collector, small and efficient applications with no external dependencies.

      Take a look at DXScene: true 3D framework, with no external dependency, fully hardware accelerated, cross-platform, fast and nice looking. Impossible to create in Java nor DotNet without huge libraries.

      • André says:

        What’s wrong with a Garbage Collection? It can be useful in several situations. Also when developing large and complex apps it is sometimes very frustrating and/or time consuming to find and solve all memory leaks.
        Btw: I’m thinking about to implement some kind of simple Object GC in ScaleMM2. Initial to find (object) leaks runtime (so not only when closing the app, would help me finding leaks in my services), but depending on the results… :-).
        And a GC thread would make “inter thread” memory faster in ScaleMM (delay free to owner thread, no lock needed every free).
        GC looked very ugly to me some years ago, but as a consultant I see many day time uses for it: for business you want apps that just works, and which are easy to develop and maintain.

        Anyway, Delphi must go on, and these new features makes it easier to develop large apps. I hope they make some stuff optional, so you can still make low-level and efficient apps when needed (and use higher level features when developing large apps).

  2. Michael Thuma says:

    I think relying to much on RTTI finally means to rely on compiler features that impact the architecture and not the other way around … This is similar to the insurance for the car – it does not hurt that other get their money late as long as one’s own insurance is cheap. I think avoiding such situations makes sense.

    Garbage collector. Imagine in a few years we have lots of developers that never managed memory programmatically … and rely …

    • A.Bouchez says:

      What do those young people learn in school? Some C just enough to discourage them, then forget about it and go to the Java/DotNet garbage in “serious” development business? How will maintain the Linux kernel?

      Perhaps there will be still some place for dinosaurs like us. 🙂

  3. Nick Hodges says:

    With regard to a Garbage Collector, I’d venture to guess that any Garbage Collector that gets included in Delphi will be optional and not required.

  4. I really like reading a post that can make men and women think.

    Also, thanks for allowing for me to comment!

Leave a Reply to A.Bouchez