TURBU Tech

Over the past couple weeks, I’ve been working on refining and testing my RTTI generation and the scripting system I’ve been building on top of it, which I’ve decided to call RTTI Script.  I think I’m finally starting to get something ready for public consumption.  I set up a Google Code repository this morning for RTTI Script, but so far all it contains is the RTTI generation code.  The actual compiler and script executor still need some work.

I’m able to build and run some simple scripts, but I don’t have all the language features implemented yet, and I’m waiting on a bugfix for an issue in Collections that’s causing trouble for me.  (A big thanks to Alex for putting up with all my bug reports and feature requests, BTW!)

RTTI generation probably sounds intimidating, but the framework for it all condenses down to about 1200 lines of code spread across 4 units.  There are a couple more minor things I still need to add to it, but it’s not all that complicated when you get down to it.  Here’s a quick overview of the four units and what they do:

vmtStructure: A very simple unit that redefines the VMT of a class and several of the RTTI structures from TypInfo.pas in ways that are easier to work with.  A lot of it is based on work by Hallvard Vassbotn.

vmtBuilder: This unit has four routines for creating parts of the VMT and RTTI tables.  The RTTI structures can be very fiddly; they pack data in ways that is not actually supported by the Object Pascal language, such as inline variable-length arrays.  Reading them (or writing them) requires a lot of pointer-based math, so I wrote up these routines to encapsulate the scary work.  Just pass in the data that the structures are supposed to contain, and they’ll arrange it for you.

The vmtBuilder unit also introduces an interface called IBuffer, which provides a simple way to pack arbitrary data together.  It’s mostly just for internal use by the other RTTI-creation routines.

newClass: Here’s where the real fun begins.  The TNewClass class contains a bunch of methods for defining your new class itself.  You create the TNewClass, call the Add*Field methods (AddIntegerField, AddObjectField, etc.) in order, call AddMethod for each method, and then call the ClassPtr method, which takes all the information you’ve fed it and turns it into a new VMT and accompanying RTTI tables.  You can optionally pass a TPrivateHeap instance to ClassPtr, which will place the new data into the private heap instead of the standard application memory.  (I put that in because it’s very useful for dealing with an arcane but important detail of the RTTI system.)

TNewClass currently does not support creating properties, because they’re just syntactic sugar and I was focused on real functionality.  This will change once my script compiler gets to the point that it requires them.  It also doesn’t support creating classes that implement interfaces.  That would add a lot of complexity and I don’t think it’s worth the effort.  But the whole thing’s open source, so if someone needs it and feels like creating a patch for it, they’d be welcome to.

rttiPackage: Here’s the part where I stop drawing on Hallvard’s work and instead owe a big debt of gratitude to Barry Kelly for explaining some of the trickier details.  If you only want to create a new class, you could stop with the three units above.  But if you want to integrate it into the RTTI system so you can get at its members, (which is basically the only way to use it since they aren’t available at compile time,) you need a way to feed all this new RTTI you’ve created into RTTI.pas.  That’s what the rttiPackage unit does.

It contains two new interfaces, INewUnit and IRttiPackage, and classes that implement them.  INewUnit can contain any number of TNewClass instances, and it organizes them so the RTTI system thinks they were all declared in the same unit.  Then IRttiPackage is the construct that pulls everything together.  It collects INewUnit instances and creates and registers a new TLibModule record, which makes RTTI.pas think you’ve loaded a new runtime package, so it loads the RTTI tables for the (fake) new package, containing the data for your new classes.

IRttiPackage.Install, the method that sets up all the data, can take an optional reference to a procedure that allows you to alter the code address of your new classes’ methods at install time.  This was to support the script engine.  When the script compiler builds the metadata for the methods, the actual code address doesn’t exist yet because the RTTI hasn’t been loaded, so it sets the code address as the offset within the script.  (This will also make it a lot easier to save the script and load it later.)  Then when the RTTI table gets loaded as the script engine is starting up, it can use TRttiMethod.CreateImplementation to build a bridge between native Delphi land and the script engine, and set the implementation stub’s address to the official code address for the new method.  It’s kind of an ugly trick, but it works, and it’s the only way I could find to make it work.

I mentioned in my last post that the compiler will require Delphi XE because it uses this trick. I’ll have to revise that further.  There’s a glitch in the setup code for Method Implementations in XE RTM that can cause bogus FPU exceptions if you use it enough. This was fixed in XE Update 1, so that will be the minimum requirements for the script compiler when it’s ready.  The RTTI generation code itself, though, will work under D2010, if anyone has other uses for it.

More to come soon…