Attributes were introduced to Delphi as part of the extended RTTI package in D2010.  They’re essentially little tags you can put on a class (or various other places) that provide metadata about it, which can be retrieved via the RTTI system.  But [Weak] is something more: it’s an attribute that has a specific meaning to the compiler itself.  It’s not just a metadata attribute, but a semantic attribute.  And that concept of semantic attributes, if employed in a slightly different way, could be used to make Delphi code a lot prettier without having to slow everything down, break tons of existing code, and tick off the vast majority of Delphi developers who do not want garbage collection in the language.

Consider the following code:

procedure myProc;
var
   myObj: TMyObject;
begin
   myObj := TMyObject.Create;
   try
      myObj.DoWhatever;
   finally
      myObj.free;
   end;
end;

You’re probably familiar with this pattern.  You’ve probably written code like this a million times.  It’s The Right Way to write it: you clean up after yourself once you’re done.

It’s also kind of bulky.  The body of the procedure is six lines long. One for setup, one for doing what we actually care about, and four for cleanup.  The ARC version would be a lot shorter:

procedure myProc;
var
   myObj: TMyObject;
begin
   myObj := TMyObject.Create;
   myObj.DoWhatever;
end;

But while that may look like less code, there’s actually a lot more going on.  That try block is still there; it’s just been added implicitly by the compiler.  And instead of calling Free directly, there’s a ref count release call (which slows down everything because it has to use an atomic decrement) before the destructor.  And the stupid part is, none of it is necessary.  Even the most trivial static analysis could show that this reference to myObj is never going to be passed to anything else, so there’s no need to keep a count of it, at all.  A smart compiler should be able to automatically insert a direct call to Free without having to keep a reference count here.

The problem is, the only static analysis that can really do any good is “the most trivial static analysis,” which can only be useful in the most trivial of cases.  For example, if the TMyObject constructor passed Self to some routine somewhere, it may or may not have held a reference to it in an object, and then all bets are off.  Have you heard of the Halting Problem, how it’s impossible to come up with a general-case analysis routine that will determine if a given program will halt or not?  Well, it gets worse.  There’s this thing called Rice’s theorem which states that it is impossible to come up with any general-purpose analysis routine that can answer any useful, non-trivial question about computer programs.  Which means that there’s no way to write a static analyzer that can detect which variables can be freed safely in a situation like this.

Rice’s theorem says that it’s impossible to create a smart compiler that can take care of something like that, for the same basic reasons that it’s impossible to solve the Halting Problem.  Simply put, you can’t automate “smart,” which is why attempts to do so invariably end up with stupid solutions that require dumbing things way down.  (See: every implementation of garbage collection ever devised, including ARC.)

This is where semantic attributes come in.  With a bit of compiler support, it would be simple to accomplish the same thing, like this:

procedure myProc;
var
   [Local] myObj: TMyObject;
begin
   myObj := TMyObject.Create;
   myObj.DoWhatever;
end;

The semantics here should be intuitively obvious.  A [Local] attribute would declare to the compiler that this variable is scoped to the current declaration.  Instead of trying to automate “smart”, which is impossible, we keep the smart decisions where they belong–with the intelligent person behind the keyboard–and instead get the compiler to automate something computers are very good at automating: mindless drudgery.  The [Local] attribute would transform the procedure above into this:

procedure myProc;
var
   myObj: TMyObject;
begin
   myObj := nil;
   try
      myObj := TMyObject.Create;
      myObj.DoWhatever;
   finally
      myObj.free;
   end;
end;

(If static analysis could prove that myObj can never be accessed before it gets assigned (as is the case here) it could even optimize out the initialization.)

There would be two other use cases for the [Local] attribute.  Putting it inside a field list for a class or record would mean that when the class or record gets cleaned up, the object in question should be destroyed.  (And I’ve got more to say about object and record cleanup a bit later on.  There’s something that could be done to improve those greatly)  And for functions, it would be a little bit different.

You’ve used factory functions before, right?  You call a function that produces a new object, instead of using an existing one that you pass in.  Well, [Local] would be useful for them, for safety:

function MyFactory(value, value2: integer): [Local] TMyObject;
begin
   result := FactoryRegistryClass[integer].Create;
   result.Foo(value2);
end;

The problem with a function like this is, what happens if the call to Foo can raise an exception?  You don’t want a try/finally block here, because if everything goes right you’re supposed to return the value, but you do have to wrap it in a “try/except free; raise; end;” block to get it right.  Marking the result as [Local] would have the compiler take care of that for you.

Tagging something that’s not an object or a typed pointer with [Local] should result in a compiler warning.

This scheme would result in much cleaner code, it would keep the responsibility for making the smart decisions where it belongs–with the programmer, not the language designer–it doesn’t introduce any overhead beyond that which exists for correct code, and it doesn’t break any already-existing code.  Unlike ARC, where there’s really no upside, with this system there’s no downside.

…except for one thing.  Moving object fields to an object’s or record’s automatic destruction list would mean moving them to FinalizeRecord, and for the life of me, I cannot understand why FinalizeRecord exists at all.

For those of you who’ve never ended up in there while debugging with Use Debug DCUs turned on, or who just plain don’t know what it is, FinalizeRecord (and its partner in crime, FinalizeArray) is a routine that uses RTTI to automatically clean up managed managed-type variables (strings, interfaces, variants) that are owned by something else.  When you destroy an object that contains a string, you don’t have to clean up the string in the destructor, because FinalizeRecord takes care of it for you.  The compiler creates a table that says where all of the managed references are and what types they are, and FinalizeRecord and FinalizeArray go over this table, and for each item in it they look up the correct cleanup routine and call it with a pointer to the right location in memory.  It’s been around since the beginning of Delphi, and it’s been a big mistake since the beginning.  It’s absolutely the wrong way to clean up this sort of situation.

I can sort of imagine how it came about.  They’d just invented the Delphi RTTI system to take care of form streaming, which was quite the impressive accomplishment, and remains one today.  (Remember that Java came out at the same time as Delphi, and still doesn’t have a decent form designer!)  And someone on another part of the team said, “well, look at this.  We’ve got managed types, that the compiler cleans up when they go out of scope without the programmer needing to write any code for it, and that works fine inside of procedures… but how are we going to handle it for objects that contain managed types as fields?”

And then someone, tragically, said “well, look at this cool RTTI thing we invented!  We can write up one cleanup routine that inspects the object’s structure and cleans up managed types that way.”  And so that’s what they did.

The only problem with that theory is that it requires the original Delphi team to have been smart enough to create Delphi, and yet stupid enough to make a mistake like this.  It requires that Anders Hejlsberg, who created Delphi, who famously stated “static when possible, dynamic when necessary,” to have gotten his own principle completely wrong in this case.  (But, since FinalizeRecord has been in Delphi since the beginning, any explanation of why it happened must necessarily involve this in some way.  It really does baffle me.)

The thing is, RTTI is Run-Time type information.  It’s for looking things up that are not and cannot be known at compile time, such as (de)serializing a form file.  But this is not such a situation.  Remember that the compiler has to build the finalization table in the first place.  This means that it already knows exactly what has to be finalized, and seeing as how Delphi is not a dynamically-typed language, that knowledge it has isn’t going to become invalid at any point after compilation.  So why doesn’t it simply do what compilers do best, and generate code to do the finalization?

Even though they’re written in hand-optimized assembly, FinalizeRecord and FinalizeArray are painfully slow.  They’ll frequently turn up in profiling reports if you’re working with code that churns through a lot of objects or records.  And adding objects to them will just make things slower.  (Which the ARC system has done, BTW.  It just keeps inflicting layer upon layer of extra slowness upon us!)  What I’d like to see is those routines deleted from System.pas, and the VMT slot that points to the finalization table replaced with a pointer to an automatically-generated cleanup routine.  No more unnecessary RTTI to take care of a task that could easily be covered under “static when possible”.

And before anyone objects, saying that debugging would suffer under all of the new auto-generated code… have you ever tried to debug into FinalizeArray?  It’s a huge mess of ASM.  What you would see in the debugger if you tried to trace into this would be a much smaller and more straightforward mess of ASM, so if anything, debugging would be improved!

If Embarcadero would do these two things–throw out ARC in favor of a [Local] attribute (don’t worry about compatibility if it’s only for one version, guys; Delphi developers are very forgiving and we’d let you take a mulligan on an obvious mistake; it wouldn’t be the first time we’ve done it) and throw out FinalizeArray and FinalizeRecord in favor of faster, compiler-generated static code, we’d see some huge improvements in both readability and performance for the Delphi language.  That would give us a NextGen I could actually rant about positively!

I have to disagree with one of the points he made, before I get into the main focus of my post.  He said that “Optimizing for English language is not a primary goal, as Delphi is used a lot around the world.”  That’s a mistake in modern coding, because even if your application’s user interface isn’t in English, it’s very likely that it will rely on one or more of the following features:

• HTML/HTTP, either for display purposes or for communicating with websites and web services
• XML
• SQL
• Some scripting language

Each of these features make heavy use of English keywords, and numbers and symbols in the ASCII range.  Optimizing for them with a native UTF-8 string type will help most programs no matter what language the users speak or read.

But that’s not really what I wanted to talk about.  There are more serious problems in the NextGen compiler.  Does anyone remember Visual Fred?  When Microsoft came out with Visual Basic.NET, it was incompatible with classic VB code in many ways, which made it essentially impossible to port existing VB projects forward.  There are good arguments out there to support the claim that VB.NET is a better, more powerful language than classic VB, but that doesn’t change the fact that it’s a very different, incompatible language, and that calling it by the same name as classic VB is deceptive.  So people decided they should call it by a different name, and the term “Visual Fred” was coined.

I’m worried that Delphi is heading in the same direction.  Let’s have a look at a couple of the other things that were added in the Nextgen compiler.  (These are things that actually were added, not just things that are being discussed as possibilities.)  They’re huge messes, and for some reason, people don’t seem to be talking about them.

STRING INDEXING CONFUSION: Nextgen strings are 0-based, rather than 1-based.  Now, that would be bad enough if that was all.  (Note: I’m not claiming that 1-based strings are inherently better.  There are good arguments for both styles.  But changing the way it works when you have literally billions of existing lines of code that use the old system is just insane.)  But it’s worse than that.  “To make the transition easier,” string indexing is controlled by a compiler directive.  Now strings might be zero-based or they might be one-based, depending on what the compiler directive is set to where the routine that uses the string is compiled.  Now we don’t have a language with 0-based strings, and we don’t have a language with 1-based strings; we have both in the same codebase!

There’s an old joke in programming circles: “There are only two difficult problems in computer science: cache invalidation, naming things, and off-by-one errors.”  Expect to see off-by-one errors skyrocket under the new string indexing regime.

AUTOMATED REFERENCE COUNTING: Under the new compiler, TObject is now a managed type, like String or IInterface.  Very much like IInterface, in fact: Every TObject now has a reference count (it’s been moved off of TInterfacedObject) and special add/release methods, and the compiler has a special-case hack in it where calling TObject.Free sets the object reference to nil instead.

Remember the additional memory pressure created a few releases back, when TMonitor was introduced and every TObject got an extra 4 bytes of overhead whether it needed it or not?  Well, here we go again.  Even more bizarre is the fact that it was added to the iOS compiler.  Adding that overhead to the 64-bit compiler might not be such a bad thing.  But when your target platform is a mobile device with very limited memory?  You’ve gotta be kidding me!

But it gets worse.  Again following the IInterface model, every reference to a TObject, as a variable, a parameter, a member of an object or record, or an array element will incur compiler-managed AddRef/Release calls.  Again, on devices with limited hardware resources.  And since mobile devices are going multicore now, that means that we get all the additional overhead of synchronization on an atomic value change FOR EVERY SINGLE ONE.

…well, almost.  You can get around that for parameters by marking them const, like with strings and interfaces.  This makes sense with strings, but a lot less sense with objects.  It’s not uncommon to pass a string somewhere and then change the local copy in some way, but you just don’t do that with objects.  At least I don’t.  When was the last time that you wrote a routine that takes an object as input, then assigns something new to the parameter variable, without the parameter being specifically marked as var?  So what this means is that in order to avoid crippling overhead on your function calls, you have to go back and spam up a huge percentage of them with const declarations.

But it gets worse still.  There’s a second problem with reference counting: cycles.  If A holds a reference to B, and B holds a reference to A, neither one of them will fall to 0 and nothing will get cleaned up.  To deal with this problem, you can mark variables and object members with a [Weak] attribute, which marks it as a weak reference.  That’s great, right?  No cycles, no reference counting overhead… right? Not quite.  Look closely at the bottom of the documentation, and you’ll see:

Note: When an instance has its memory released, all active [weak] references are set to nil. Just as a strong (normal) reference, a [weak] variable can only be nil or reference a valid instance. This is the main reason why a weak reference should be assigned to a strong reference before being tested for nil and dereferenced. Assigning to a strong reference does not allow the instance to be released prematurely.

So all of those weak references have to get tracked, and then cleaned up when you’re done with them.  That’s actually far more expensive than the atomic updates, and is likely to be a massive performance killer if you use weak references.  And where do you think that all gets stored?  If you said “on the object itself, with even more overhead per instance, like the new reference count,” you’d be… wrong, actually.  It’s not *quite* as bad as that.  But all that bookkeeping has to go somewhere, so instead, there’s a new global multimap structure declared deep in the bowels of system.pas that manages weak references.  Look up TInstHashMap for the gory details.  It’s not pretty, and because it’s a global, access has to be protected by a TMonitor.  Yay for additional atomic synchronization and memory management overhead!

And did you notice how it says that with both a strong or a weak reference, an object variable can only be nil or reference a valid object?  OK, everyone who’s ever called something like myStringList.AddObject(myString, TObject(myIntegerValue)); please raise your hand.  Yeah, lots of hands going up out there.  It’s a pretty common idiom in Delphi code.  Well congratulations, your code no longer works!  You see, this isn’t really about reference counting.  If it was, no one would care about cleaning up those weak references.  No, what it’s really about is memory safety.  Delphi programmers have soundly rejected the concept of putting garbage collection in the language for years, consistently, every time the question comes up.  But now someone at Embarcadero is trying to sneak it in through the back door.

And this isn’t just me ranting.  Look at that same documentation page.  Just below the section on weak references, we see that there’s a new [Unsafe] attribute for special-case references that really need to not be reference-counted for some reason.

It should be only used outside the System unit in very rare situations. It is considered dangerous and its use is not recommended as no code associated with reference counting is generated.

In other words, “we realize that “unsafe” code is absolutely necessary to actually create a working programming language, as evidenced by the fact that we have to employ it in the System unit in order to make basic functionality work, but it’s only for us to use; you really should stick to high-level code only.”

What a bunch of patronizing crap!  If we wanted guaranteed memory safety and enforced garbage collection, we would be using Prism Oxygene.  Not being burdened with all that baggage is one of Delphi’s strongest selling points, and someone at Embarcadero doesn’t get that, and thinks it should all just be thrown out the window.  Well, they’re wrong.  We already have a .NET Delphi implementation, and one for Java, in Oxygene.  There’s no good reason not to keep native Delphi native and close to the metal.

And it keeps getting worse!  You see, there’s no [Weak] generic constraint, which means it’s impossible to create a TObjectList<[Weak]T>.  And you know where generic lists are popping up?  Everywhere!  Since you only want to assign object references to strongly typed values so that ARC can work, that means the old TList is out.  So let’s look at one specific scenario: GUI design.

In previous versions of Delphi, TComponent had a member declared as FComponents: TList.  It held the component’s child components.  Now that’s a generic list, containing strong references to the child components.  What that means (among other things) is that you can no longer delete a UI element by calling Free on it.  (Free is “magic” now, remember?  All it does is set your reference to nil.  But with the object’s owner holding a strong reference to it, that won’t bring the count down to 0.)

Whatever shall we do?  Never fear! It’s even more incompatible changes to the rescue!  There’s a new method on TObject, called DisposeOf, that forces the destructor to execute immediately in exactly the same way that Free used to do.  So instead of using what we already have that we know works, someone decided to change the semantics of Free, and then introduce something else that does exactly what Free is supposed to do, except that you still have to wait for the garbage collector to actually clean up the object.  And they called it Dispose.  Hmm… where have I heard that before?

The entire ARC model is one big mess from beginning to end.  It will make Delphi code slow–it will essentially make it impossible to write fast Delphi code, in fact–and it will break tons of existing, working code.  I literally see zero benefit to any of this.  Managed code advocates’ FUD notwithstanding, manual memory management is not a difficult issue.  Heck, in Delphi it’s a solved problem with FastMM and FullDebugMode.  Even if someone were to make the (somewhat dubious) claim that this “modernizing” of the language will attract new developers, the saying “a bird in the hand beats two in the bush” exists for a reason.

There’s one very interesting thing that was introduced among all of this ARC crap, that could be used to produce real improvements in memory management and help people write cleaner code going forward, without breaking any existing code, had the compiler team followed a slightly different path.  But this article is too long already, so I’ll explain how to do this sort of stuff right tomorrow.

Over the last few days I’ve taken a big step towards the real goal: developing a general-purpose data querying system.  I factored out all of the SQL-specific stuff into its own unit and created an interface called IQueryBuilder that responds to the various query operators to build an expression tree that can be evaluated by DWS.  You create a recognizer and register it with dwsLinq, that checks the value of your FROM expression and returns an IQueryBuilder if it can create an expression for it.  And once I had that working, I created a second unit with a new implementation of IQueryBuilder, for querying JSON data.

I checked it in recently, and it supports all of the query operators except JOIN (which raises a compiler error, since I couldn’t figure out how to do in any meaningful way without language support for anonymous methods) and DISTINCT (which is a no-op that raises a compiler warning).  It takes a JSONVariant value (supplied by the JSON Connector language extension) as input and produces a new one as output.  The INTO operator expects a JSONVariant as input and can output any type.  (There’s no special to distinguish array types from single elements, like with the SQL version, because this is a lot more complicated to pull off without an enforced schema.  I might change that later on.)

The JSON query system is still very rough, and very much a work in progress.  It’s a lot harder to get things right than SQL, because SQL relations have a schema and JSON does not.  (It’s essentially the same as the difference between ensuring type safety in a static language or a dynamic one.  Getting it right in the dynamic language is a lot more work!)

The query system expects that the input value will consist of an array of objects (much like a SQL result set) and should work more or less as expected under those conditions.  I’ve tried to put in cases for the various query operators to handle them, but it still needs work, and I’ll keep working on it.

But what this means is that anyone could write an IQueryBuilder implementation and their own query expression objects, to be able to run LINQ expressions against arbitrary data.  This is starting to become a really interesting project…

This is not quite the same thing as INTO in C#’s version of LINQ.  INTO there basically acts as a naming operator, taking a query expression and giving it a name so that you can use it as a sub-expression in further query work.

I did something different with this.  INTO in dwsLinq takes a function pointer as its argument.  The function has to have a single parameter of type Dataset, and return some value, and the INTO clause changes the output of the entire query expression.

If the INTO function returns an array type, it will be called once, and whatever it returns becomes the query result.  Otherwise, it gets called once for each result in the dataset, and the individual results get collected into an array of whatever type the INTO function returns, which becomes the query result.  The idea is that this can be used to quickly load data into script objects or records.  You provide the basic query, and a function that transforms a dataset row into an object, and let dwsLinq take care of the plumbing.

Having to write a function pointer expression into the query is a bit awkward, but it’s a limitation of the language at the moment.  Once DWS is able to deal with anonymous methods, I’ll expand the capabilities of INTO.

Here’s the test that I wrote, that demonstrates the syntax for both forms of INTO.  Both styles shown here give the same result:

uses System.Data;

function processor(ds: Dataset): integer;
begin
   result := ds.FieldByName('VALUE_1').AsInteger;
end;

function processor2(ds: Dataset): array of integer;
begin
   var value := ds.FieldByName('VALUE_1');
   while ds.step do
      result.add(value.AsInteger);
end;

var db := DataBase.Create('SQLite', ['.\Linq\linqtest.s3db']);

var values := from DB.TBL1
              order by ID
              select VALUE_1
              into @processor;
for var value in values do
   printLn(value);

PrintLn('');

var values2 := from DB.TBL1
               order by ID
               select VALUE_1
               into @processor2;
for var value2 in values2 do
   printLn(value2);

Yes, the @ is necessary for the DWS parser to recognize it as a function pointer.

Unfortunately, there’s nothing like that available for Delphi.  There could be, if the team would shift their focus to augmenting the language with actual useful features borrowed from managed languages, such as a proper extension method implementation, rather that burdening it with useless, counterproductive crap like pseudo-garbage collection and new String models.  But for the time being, we’re out of luck.

It could be done in DWS, but LINQ is a really complex system that makes heavy use of lambdas (aka anonymous methods,) which Eric Grange hasn’t gotten around to supporting yet outside of the JavaScript cross-compilation.  Apparently it’ll take a complete re-architecting of the execution model to make anonymous methods available.

But a while ago, an idea struck me.  In .NET, LINQ uses anonymous methods to support expression class evaluation at runtime.  But DWS is already an interpreted scripting system that operates on an expression tree!  And since DWS has a language extension mechanism in place, I decided to poke around and see what was possible.

I came up with a simple proof of concept that could read very simplistic queries and generate SQL from them, then execute it using DWS’s database module and return a dataset.  It required modifying both the compiler and the language extension system.  So I sent it to Eric for feedback, just as an idea.  I was expecting some constructive criticism, either explaining why it was a bad idea, or some things that would make it work better.  I certainly never expected him to actually take it and put it in the DWS repository.

Guess what he did with my code?

So now I’ve been added as a contributor to the DWS project, so I can maintain this.  I’ve been updating it since then, and now it will cover all of the basic syntax features (FROM, JOIN, WHERE, GROUP BY, ORDER BY, and SELECT).  It’s still just a SQL generator, and not a full-blown LINQ implementation, which will take time, but if you’re working with DWS and database code, it already has a few advantages over the normal SQL model.

You can integrate parameters directly into your query, rather than having to put them someplace else.  For example, consider this:

myDataset := myDatabase.Query(#'
 select VALUE1, dbFunction(VALUE2, :param1)
 from TABLE_1
 where ID = :param2', [arg, userID]);

Having to move the params apart from the values you’re giving them makes it harder to read, especially on large queries, you can have DRY problems with it.  What happens if you rearrange the query, you change the order of the params?  What if you add or delete a param?  You need to find the right place in your param array to insert/delete the new value.

This can be expressed in LINQ syntax more simply, without the problems noted above:

myDataset :=
 from myDatabase.TABLE_1
 where ID = userID
 select VALUE1, dbFunction(VALUE2, arg);

The dwsLinq parser can call back into the DWS parser to read Pascal expressions, and when building your query, it will automatically replace anything that came from a Pascal expression with a parameter, so you get the right SQL query.

This is still very much a work in progress, but if anyone would like to try it out, it’s in the repository now.  Any bug reports would be welcome.  Just keep in mind that this isn’t expected to be stable, or to work well, yet.  It’s still mostly something I’m playing around with.  But so far, it’s kinda cool.

Seems to me that it should be possible to detect this with a bit of instrumenting, though.  Here’s the basic idea.

When FastMM starts, it gets a brand new heap with nothing allocated in it yet, and it sets up its bookkeeping.

When FastMM shuts down, it checks to see if anything is still allocated in that heap.  If so, it reports it.

If you want to detect “live leaks,” just apply that principle recursively: Run some routine called “LiveLeakCheckBegin” which allocates a new heap and some new bookkeeping for it.  From this point on, all new allocations should go into this new heap.

At some later point, run “LiveLeakCheckEnd”.  When this is run, FastMM would go through and generate a leak report based on everything still allocated *in the new heap*.

Obviously, this would only be for debugging purposes, and should not be used in a release build.  Maybe it could be added to the FullDebugMode DLL as a new feature?

Pierre, if you’re reading this, how difficult would it be to implement something like this in FastMM?

Anyone else, do you have any ideas on how to implement it without having to modify FastMM?

John Wiley & Sons is a publishing company, involved in one of the most despicable fields within the publishing industry: the publication of academic textbooks.  Anyone who’s ever been to college and had to pay the exorbitant rates they charge every semester, just because they can, will need no explanation as to why these guys are scum.

The interesting thing is, while they charge whatever they can, “whatever they can” means different things in different contexts.  Some parts of the world are wealthier than others, and in some countries there’s just less blood available to be squeezed from that particular stone, and so they have to set the prices lower so as not to be unaffordable.

Anyone with a basic knowledge of economics, when presented with these facts, should recognize that an opportunity for arbitrage exists here: buying goods in a low-priced market and reselling them at a profit, where the prices are high.  And that’s exactly what this Thai guy called Supap Kirtsaeng did.

He would purchase textbooks in poor nations at a low cost, then resell them to American students at a decent markup which still managed to significantly undercut Wiley’s prices.  It’s a win-win situation. Students get a more affordable education, Kirtsaeng gets some money in his pocket, and the parasites lose out on some opportunities for ill-gotten gain.  Everyone’s happy.

…except, of course, the parasites.  Wiley said “hey, we’re losing money on this!” and sued.

Kirtsaeng’s defense was that, under the First Sale Doctrine in US copyright law, once he’s legally purchased something, he has the right to do whatever he wants to with it, including resell it, and the publisher has no right to interfere.  (Publishing interests hate the First Sale Doctrine.  In digital media, they’ve managed to do a good job circumventing it by getting the DMCA passed, so that they can just stick DRM on something and then even if you have rights to something you purchased, you can’t legally exercise them.  But Kirtsaeng was dealing with books.)

Wiley responded with the somewhat ridiculous claim that, because the books were printed and purchased overseas, they were not covered by the First Sale Doctrine, which is a provision of US law and therefore does not apply to foreign purchases.  (And yet somehow they wanted US copyright law to apply to these purchases in every sense in which it benefited them! Otherwise, they would have had no case in the first place.)

Well, to make a long story short, different courts ruled different ways, and it kept getting appealed, until the Supreme Court agreed to hear the case.  The justices asked some pretty pointed questions about how the exclusion of First Sale rights from copyrighted items created overseas might apply to other areas of life.  Would it be illegal to sell your car, for example, if it runs copyrighted computer code that was written outside of the USA?

Well, they finally reached a verdict earlier today, in which they upheld Kirtsaeng’s First Sale rights and officially ruled that the First Sale Doctrine is not bound by geography.  But the part that really caught my attention was the way their ruling explicitly went back and touched on the basic underlying issue: arbitrage.

Wiley and the dissent claim that a nongeographical interpretation will make it difficult, perhaps impossible, for publishers (and other copyright holders) to divide foreign and domestic markets. We concede that is so. A publisher may find it more difficult to charge different prices for the same book in different geographic markets. But we do not see how these facts help Wiley, for we can find no basic principle of copyright law that suggests that publishers are especially entitled to such rights.

[....] the Constitution’s language nowhere suggests that its limited exclusive right should include a right to divide markets or a concomitant right to charge different purchasers different prices for the same book, say to increase or to maximize gain. Neither, to our knowledge, did any Founder make any such suggestion.We have found no precedent suggesting a legal preference for interpretations of copyright statutes that would provide for market divisions.

To the contrary, Congress enacted a copyright law that (through the “first sale” doctrine) limits copyright holders’ ability to divide domestic markets. And that limitation is consistent with antitrust laws that ordinarily forbid market divisions.

In other words, the Supreme Court just officially proclaimed, as part of a larger ruling, that publishers have no inherent right to charge different prices for the same product in different markets.

If anyone wonders why I’ve spent so many words writing up a brief recap of a copyright decision on what’s supposed to be a Delphi blog, look at that last paragraph.  Things are about to get very interesting for Delphi customers overseas…

I built on his foundation and sent a suggested patch to the Delphi team.  They made their own tweaks, and newer versions of Delphi have had a much faster TValue because of that.  But one of the most interesting things I heard while I was working on that improvement came from Barry Kelly. He said that he wasn’t sure how much those speedups would actually help, because the bulk of your CPU time in RTTI work was going to be spent in invocation (using the RTTI system to call methods) and not in moving data in and out of TValue variables.

And he was right.  If you wanna really break your brain sometime, trace into a call to TRttiMethod.Invoke and take a look at what all is taking place under the hood.  That’s a huge amount of work going on, and the interesting thing is how much of it will be exactly the same every time you invoke the same method, assuming you have a valid parameter list.

To do it quickly, you have to recompute as little of that setup code as possible every time.  The fastest way would be to write your own invocation routines, something like this:

function InvokeFuncA(var params: TArray<TValue>; const self: TValue): TValue;

begin

   result := (self.AsObject as TMyClass).FuncA(params[0].AsInteger);

end;

Of course, that’s a static invocation routine which only works for one method, which is the exact opposite of what TRttiMethod.Invoke does: provides you with a generic invocation routine that works for any method.  But it’s fast.

So… what if we could find a way to get TRttiMethod to create something like that for us, at runtime?  Instead of doing a bunch of work every time you call it to figure out what goes into memory where in order to convert your list of TValues into a native parameter list for a method, maybe it could work out how to perform that mapping, and express it in machine code.

It’s not as strange as it sounds. TMethodImplementation already does a certain amount of machine code generation at runtime to make some of the fancier RTTI tricks possible.  And with RTTI becoming used in more places in Delphi (it’s all over the place in Live Bindings, for example,) the last thing anyone wants is for it to be slow.

So, I’d like to challenge the community to step up to the plate again. Is there anyone out there who knows enough about low-level code to build what would essentially be a JIT compiler that takes a TRttiMethod as input and outputs a shim for fast invocation? I’ll be poking at things from one angle, but I’d like to invite anyone else who’s interested to help out. Let’s see how fast we can get RTTI invocation.

About a year ago, I was browsing around and I found an ad for a Web-based game that looked like a lot of fun.  Turns out it really wasn’t.  It was kind of simplistic and insipid, in fact, but it had a bunch of players who were paying real money for in-game bonuses.

I’d been working on a bunch of HTTP-based services for work, and I looked at this game and thought, “I could do this, and I could do it better.”  So I got in touch with a few friends of mine who have some design skills, and we set up a team.  Things are not quite ready to announce anything officially yet, but we’re making a lot of progress and are getting pretty close to launching a public beta.

In the middle of all this, the ADUG call for papers came out, so I submitted an idea for presenting about how to build an interactive website on a pure Pascal stack.  This is based on what I’ve learned from my work on the game, using a Delphi web server, with DWS for the server-side scripting and building advanced front-end effects in Smart.

For those of you who are in the area and can make it, I hope to see you there!

Slide to unlock is not Apple’s patent

Smartphone Wars: Micron’s Slide-to-Unlock Patent

Apparently the patent office essentially just ruled that the infamous “slide to unlock” patent that Apple has been beating Samsung and other companies that make Android devices over the head with is actually invalid, because another company, Micron PC, invented it several years before Apple did.  And the name associated with Micron’s patent is Jim McKeeth.

Of course, when something like this happens, you wonder if it’s the same guy or not.  Sometimes people have the same name, you know.  But I did some checking, and it appears that he did used to work at Micron.

So, does Apple owe Jim McKeeth money for this bogus “innovation” that they’ve been extorting money from all sorts of smartphone companies over?  It would be deliciously ironic if they did.

Jim, if you’re reading this, would you mind leaving a comment on the subject?