Our development office is in Lynnwood, Washington, (about half an hour from Seattle,) and on-site work is preferred, though some exceptions do (infrequently) get made on a case-by-case basis.  It’s a good location, right across the street from a major mall (plenty of places to go for lunch!), with good pay and working conditions.  The office has a friendly, engineer-centric culture, and it really helps that the manager is a former coder and the boss still is one.  No pointy hair here!

Only those who know what they’re doing need apply.  The developers take an active part in the interview process, and we’ve got really high standards.  We understand that the only way to develop good software is with good developers, and we try hard to make sure that that’s all we get.  But if you’re good and you can demonstrate that you know what you’re doing, you’re likely to get an offer.

If anyone’s interested, send me a resume at mwheeler@wideorbit.com and I’ll see about getting you an interview.

I’ve fixed a few bugs in the parser, and updated the executor to support the Environment object.  I added an Examples folder with two simple Hello World projects.  They both display a message box with a simple greeting.  The message box, of course, is not built in to the scripting system, so it has to be imported from the Delphi project.  These two sample programs demonstrate the two ways to do this: through external unit registration and with an Environment object.

The rsCompiler unit provides a class called TrsTypeImporter that’s used to import new types, variables, constants and routines (hmm… maybe I should change the name to just TrsImporter?) to make the compiler aware of them.  To register a new external unit, call RegisterStandardUnit on the compiler.  It takes two parameters: a unit name and an anonymous reference to a procedure that takes a TrsTypeImporter as an argument.  The first time a unit with that name is used in the script, the compiler will call the method reference, providing an importer to register things with the compiler.  A unit named “system” is always used, so if you want to import things that will always be available to the script, call RegisterStandardUnit and use “system” as the unit name.

Any units registered with the compiler also have to be registered with the script executor by a similar process.  The interface for the executor’s RegisterStandardUnit system is not stable yet, but it currently allows you to register your standalone routines in the callback.  I’ll add support for registering classes and external variables soon.

The second, simpler way to do imports is through the Environment object.  The second sample program shows how this works.  You declare a class and register it with the compiler, and every public method and property on it will be available to the script engine as if they were global routines and variables declared in the “system” import.  Then you register an instance of the class with the script executor before loading the program.  (It’s important to note that the Environment has to be registered with the compiler before compiling, and with the script executor before loading the program.)

It’s possible to use both systems together, of course.  If you want to break up your external routines into groups, you could declare and register multiple external units, but still have a group of globally-available methods in an Environment object.

Hopefully these examples will show people how to get started.  I’ll add some more examples as needed and when I get some more features ready.

Note: This is an alpha version.  There are probably plenty of bugs in it.  There are still features I haven’t gotten around to implementing.  The sight of the code would probably send Barry Kelly fleeing from the room, screaming in horror.  So use at your own risk, and I take no responsibility if this causes your system to crash, your data to come out all wrong, or demons to fly out your nose.  Specifically, the program keyword is not implemented yet, (only units,) var parameters don’t work yet, and a handful of features aren’t implemented in the script executor.

Having said that, here’s a few cool features I’ve come up with:

• Class creation:  Defining a new class inside the script code is not supported yet, but the basic framework is there, and script units implement new classes with their routines as methods, accessible to the script executor through the RTTI system.  I’ll add class creation pretty soon.
• Automatic registration: Ever try to register a class with the script engine in PascalScript?  It’s a painful process involving generating a registration unit that basically redeclares your entire class, and then has to be kept in sync or things break.  (And if you get the dependency order wrong, it fails silently,  leading to weird errors further down the line.)  I figured that was way too complicated, so for RTTIScript, all you have to do is register a class reference, and it uses the RTTI system to figure out the internal members and dependencies for you.
• Execution environment: You’ll often have a bunch of little standalone routines and global variables that you want to import into a script to have available at all times, to provide an interface between the script and your program.  So I introduced the concept of the “environment:” You create a class and register it with the compiler, and it treats all the class’s public methods and properties as globals for the script.  (This also means that you can use property getters and setters to have the script’s interaction with its “global variables” invoke side effects.)
• Script saving: I know Eric Grange thinks that a fast compiler is good enough and a bytecode version of a script is not necessary, but I don’t agree.  By his own tests, even a very fast script compiler can take 2 seconds to compile a 100,000 line script.  I’ve actually got an example of one that big in a project that the author wants to convert over once the TURBU project is ready.  Adding a two-second lag into the loading process and making the player wait while a level loads, which should feel instantaneous, is unacceptable.  I don’t have script saving implemented yet, but it’s definitely on the to-do list.

Anyway, the alpha version is available for any guinea pigs curious developers out there who’d like to take a look at it.  It requires Delphi XE, it’s pretty fast, and when I get the rest of the features set up it’s going to be awesome!  Feel free to check it out, and to submit a bug report if you find some errors. (Which I’m sure plenty of people will.)

They shouldn’t be.  I create the renderer in CreateWnd and I destroy it in DestroyWnd, which should get called as part of the component destruction process.  But as it turns out, DestroyWnd is not actually called anywhere!

By tracing through the VCL code, it looks like DestroyWnd is supposed to get called from a call stack that looks something like this:

TMyControl.DestroyWnd //virtual
TWinControl.DestroyHandle
TWinControl.RemoveControl
TControl.SetParent(nil)
TControl.Destroy

But earlier in the form destruction process, TWinControl.Remove gets called.  It does not call RemoveControl on the control being removed, and at the end of the routine, it says:

AControl.FParent := nil;

By directly setting the field to nil instead of calling SetParent(nil), it makes it so that when SetParent(nil) is called, the entire body of code gets short-circuited, so that RemoveControl is never called.  In fact, it’s not just my control.  I put a breakpoint on the first line of TWinControl.DestroyHandle, ran the program, and closed it, which generates and then frees almost 30 TWinControl descendants.  The breakpoint was never hit.

I’m seeing this in Delphi XE.  I haven’t tested it in any other version, but from what I can see, it appears that any VCL control created in XE will never call DestroyHandle during component destruction, leading to all window handles (and other resources, such as my rendering contexts) being leaked.

Can anyone confirm this, or say if it affects other versions?  I didn’t see this in QC, so I wrote up a bug report.  Please vote for it, especially if you work on a program with lots of complex forms and have users who would be, shall we say, unamused to suddenly hit an “out of resources” error for no good reason…

There is a better way, by the way.  Python has it; it’s called “slicing.”  You know how in Delphi, you can use array[subscript] notation to get a single item out of an array?  Well in Python you can use an extended version of that to get a subrange (a slice) out of an array.  For example:

returns “string”.  (In Python, strings start at index 0.)  Also, you don’t need a value on both sides of the colon.  If you don’t provide one, it translates to the beginning/end of the array, as appropriate.  So “mystring[:6]” returns “Python” and “mystring[14:]” returns “slicing”.

Now, using a colon here isn’t very Pascal-ish.  Colons are for variable declarations.  But we do have a range operator that’ s used with arrays: the double dot.  Wouldn’t it be nice to be able to do stuff like this in Delphi?

The compiler could translate this syntax to calls to Copy easily enough.  Also, they could improve on the concept a little.  If you feed Python a negative number in a slicing expression, it will error out.  But if Delphi could automatically translate myArray[..-5] into myArray[..high(myArray) - 5], grabbing everything except the last  X number of elements would be that much easier.

Any thoughts?

Michael Swindell opened it up.  He was talking about the recent advances in UX (user experience) technology throughout the computer industry, and he explained how FireMonkey is designed to provide a modern interface and user experience.  He had a very interesting story about a dentist visit a few years back.  He went in one day and found that the dentist’s office was modernizing a lot.  All the medical records were being computerized, moving out of the file cabinets.  He was really impressed when they showed him the new system, especially as the software came from a Delphi shop.  When he went in to talk with the dentist, all his patient records were up there on a big screen, and he was really proud of what they’d done with Delphi.  Then the dentist tried to show him a new feature and, as Michael put it, “all the pride turned to shame.”

He had to grab a mouse and try to use it with one hand, clicking around through the interface, all while trying to hold a miniature camera inside Michael’s mouth with the other, and it was a big, clumsy mess.  It would have been a lot easier for him if he’d had a touch-based system on a tablet, or example.

He mentioned that a lot of new interface and display technology is starting to show up on the market.  Samsung, for example, has just started to mass-produce transparent LCD screens.  (He mentioned the movies Avatar and Minority Report as examples of transparent LCD screens in use, which kinda bugged me.  I wish people would stop talking about “Minority Report interfaces” as if they were something realistic, because they’re anything but.  Sure, they look cool in movies, but there’s a good reason why all successful human interface devices have always required little wrist movement, minimal elbow movement and zero shoulder movement to operate.  Trying to run a real-life “Minority Report interface” would wear your arms out very quickly.)

He also mentioned GPUs.  He said that all computers from desktops to smartphones have had GPUs as standard features for the last decade, and they’ve got more processing power than your CPU, but they generally sit idle.  He said that finding ways to use the power of the GPU is a major focus for Embarcadero.  All the FireMonkey visuals are drawn with the GPU, and they’re looking into ways to do general-purpose computing on the GPU with Delphi, for a later version.  CUDA was mentioned a few times as an example of the sort of things that can be done with a GPU, which I find interesting.  CUDA and OpenCL, CUDA’s non-NVIDIA competitor, both use script languages that look like someone took C and beat it with an ugly stick.  If they could come up with a Pascal-based GPGPU script language, that would be awesome!

After Michael, John Thomas spoke for a while, talking about FireMonkey.  He called FireMonkey “a full application stack” that makes it very easy to simply recompile and run on a new platform.  But unlike other cross-platform solutions, FireMonkey a full native code framework; no VMs to slow down execution and limit what you can do with the system.  He also mentioned that all FireMonkey controls are drawn with vector graphics, leveraging the GPU, so everything scales properly. (Even at high DPI! Finally!)  He also mentioned that FireMonkey “is not a game engine,” despite all the heavy graphical focus, but it’s a general-purpose application development framework.  I found that interesting because of what happened next.

Next came a bunch of FireMonkey demos by Anders Ohlsson.  He showed various instances of graphics and 3D objects rotating and moving around on-screen.  One very noticeable issue with the demos was that when he made the window larger, everything got slower.  He mentioned that this was because the animation was linked to the Application.OnIdle event.  I found this a bit funny because “FireMonkey is not a game engine,” but if it was, they would have a game timer available, which you can hook into Application.OnIdle to decouple animation from the system “heartbeat” and provide a constant rate of speed even with varying levels of processing power.  (I talked to him afterwards and explained the basic principle, and he came back about half an hour later to show me everything running nice and smooth and at a consistent rate on his demos, so that’s cool.  Maybe someone will make a game timer component for FireMonkey.)

He also showed off an impressive collection of GPU-based shaders that can be applied to FireMonkey objects in real time.  There are shaders that can add color effects, distortions, or transitions from one image to another, and they can be “stacked” to apply multiple shaders to the same object or scene.

I went to look at the shader language code that implements the shaders, though, and I was very disappointed to see that they were not implemented with readability in mind.  There are 3 versions of each shader, one for DirectX, one for OpenGL, and one for OpenGL ES (for mobile devices).  The DirectX ones are implemented as byte arrays of compiled shader code, and the GL ones are in ARB (low-level “shader assembly code”).  The GLES shaders are in GLSL (a higher-level “GL Shader Language”,) but written in a highly unreadable style.  I asked in Q&A if they had deliberately obfuscated the shaders, which makes them hard to read and study and would certainly make it harder for third parties to create new shader effects.  Michael Swindell assured me that that was certainly not the objective, and said he’d look into it and try to ensure that the shaders were easier to work with in the future.

They talked about a few other things related to mobile development, but it was really the FireMonkey stuff that stood out and caught everyone’s attention.  As Nick Hodges said yesterday when a bunch of us were talking, FireMonkey is conceptually the correct solution for Delphi cross-platform development, and a lot better than the QT-based idea they were working on last year.  Time will tell how well they’ll do with maturing it and promoting it, but for the moment I’m cautiously optimistic for the future of Delphi and the FireMonkey framework.

I expanded on his idea and sent a suggested implementation to Barry Kelly.  Unfortunately, it didn’t make it into XE, but it’s in XE2 now.  It’s not exactly the code I suggested–a few things have changed in the way TValue works internally since D2010, and there’s at least one bugfix that I can see–but the basic idea is there.  When storing values into a TValue and retrieving them again, there’s special code to set and retrieve the most common types directly, without having to go into the generic RTTI-based (much slower) conversion routines.

I reran the timing test under XE2, and the results were much better this time:

Variants: 2140
TValues: 2895

Still about 30% slower than Variants, but at least it’s not 2000% slower anymore.  Thanks, Barry, and thanks to Robert too for the original suggestion!

I got my SA notification and went to install Delphi XE2 today.  I’m a SA holder for Delphi Professional, which is what I got a serial number for, which I entered when the installer prompted me for it.  But that’s not what it installed.  See, if you’ve had a previous installation of a different SKU level, (a demo, for example, which always seems to be Architect level,) it will disregard the serial number you enter (even though it makes sure that you enter a valid serial!) and install the feature set for what the leftover license file specified.

I’ve seen a lot of bugs in a lot of software in my day, but there have been very few that mystify me as much as this one.  I don’t understand how it is possible to screw this up.  Taking input directly from the user and acting on it is the most basic principle of software design there is!  It’s the first thing any of us learn, usually even before “simpler” programs such as Hello World.  You prompt the user for input, and you act on it.  But somehow, the Delphi installer prompts the user for input, and then… disregards it and acts on something completely different if it’s available?  What is this I don’t even

How?!?  This is not an easy mistake to make; you really have to go out of your way to screw something this simple up!  (To add insult to injury, the License Manager tool conveniently provided by Embarcadero, presumably for managing your licenses, can find the invalid licenses, and there’s a Delete menu item, but it’s always grayed out.  I’ve never figured out anything that will actually enable the Delete option.)

It gets worse.  The installer doesn’t really understand the implications of choosing not to install things.  For example, I chose not to install the iOS development stuff because I have no use for it.  It still spent several minutes downloading the package, which is quite large and whatever server it’s downloading from doesn’t have the world’s greatest bandwidth.

And it will still display and require you to agree to EULAs for bundled third-party components you tell it not to install.  I can’t help but wonder about the legal ramifications to that…

Dear Embarcadero,

Your installer sucks.  Please fix it.  It’s the first impression a new customer gets of the quality of your engineering, even before using the new product, and it makes Delphi look bad.

Sincerely,

Everyone who has to put up with it

64-bit is scheduled to be released this August.

That’s not particularly unexpected, since that’s when the new releases tend to come out.  But it does confirm one thing I thought was kind of strange when I got the DelphiLive! 2011 Call For Papers email a few weeks ago.

The conference is scheduled for mid-September, which means that for the first time, it’s going to be held after the year’s new Delphi release instead of before it.  That’s a pretty sharp contrast with last year, where the keynote address was basically the official RAD Studio XE release announcement…

And yet, on one of my larger test projects, that wasn’t enough.  One of the things that has to be loaded upfront was map tile data, so that the maps can draw.  Unfortunately, this project has over 200 different tilesets, and it was taking quite a while to load that much data.  I’ve got a RTTI-based deserializer that can turn dataset records into objects, but it was taking a completely unreasonable 3.3 seconds to read the tile data.

Profiling said that most of the delay–close to 60%–was coming from FastMM’s FastFreeMem calling something in ntdll.dll.  It didn’t say what, and I didn’t figure I needed to poke around inside the memory manager.  I’d be better off making sure there weren’t so many calls into FastFreeMem, right?

So I poked around in the deserializer code and found several places in inner loops where strings were being created and disposed of quite unnecessarily.  I fixed the code so that that wouldn’t happen, optimizing out all the unnecessary FreeMem calls.  That should have fixed things up, I figured.  My reward was a measly 0.4 seconds, down from 3.3 to 2.9, with the bulk of the time still taking place in ntdll.

So I poked around in the FastFreeMem code a little, and was surprised to run across this:

So when it tries to lock the memory block to free some memory, unless a special “NeverSleepOnThreadContention” compiler flag is set, it’ll call the Winapi Sleep function, giving up the entire timeslice (several milliseconds) because it’s blocked by an operation that will take a few dozen lines of ASM to complete.

I looked for this option in FastMM4Options.inc, and found the following explanation:

{Enable this option to never put a thread to sleep if a thread contention occurs. This option will improve performance if the ratio of the number of active threads to the number of CPU cores is low (typically < 2). With this option set a thread will enter a “busy waiting” loop instead of relinquishing its timeslice when a thread contention occurs.}

So sleeping the thread instead of spinlocking can be helpful when there are a high number of threads running.  But there’s no code to detect this.  It’s never call Sleep or always call Sleep, with the decision hardcoded in at compile time.  I wonder if it would be possible to always spinlock for a certain number of cycles first and see if that helps, before calling Sleep?

Anyway, it turns out that that was my problem.  I was doing some other data-intensive loading in a background thread, and the memory allocations were clashing with each other.  When I set the NeverSleepOnThreadContention flag and rebuilt, the load time for tile data dropped to a far more acceptable 1.1 seconds.