Adding non-data fields to a client dataset

A lot of the UI design for the TURBU editor is based on data-aware controls bound to client datasets.  I was trying to build a new form this morning that required me to filter one of the datasets.  Problem is, that would break other things that expected it not to be filtered.  Well, that’s not such a big problem, because TClientDataset has an awesome method called CloneCursor that lets you set up a second client dataset that shares the first one’s data store, but with independent view settings.  So I used a cloned dataset, and immediately got an exception when I tried to run.  The control I was using couldn’t find the field.

After a bit of digging, I found out that when CloneCursor builds the field structure for the cloned dataset, it copies the FieldDefs from the original.  And FieldDefs only define data fields.  The field I was trying to display was a Calculated field, so I ended up without it.  Well, OK, that’s not such a big problem.  Just add a new calculated field, right?

[code lang="delphi"]
begin
   calc := TWideStringField.Create(nil);
   calc.FieldName := 'DisplayName';
   calc.Size := 255;
   calc.FieldKind := fkCalculated;
   calc.Dataset := DataSet;
[/code]

That ought to work, but it raises an exception.  “Cannot perform this operation on an open dataset.”   OK, I can understand that, almost.  It makes sense for data fields, because the underlying data store has to be set up in a certain way.  But for lookup and calculated fields that aren’t bound to the data store, it doesn’t make any sense.  Unfortunately, TField.SetDataset doesn’t care what kind of field you’re adding.  It checks to see if the dataset is inactive, and if not, boom!  It blows up in your face.

So I can’t add a calculated field to the dataset after CloneCursor has run, because CloneCursor calls Open and after that it’s too late.  So maybe I can add it before CloneCursor has run?  That almost works.  You end up with that calculated field in your cloned dataset… and nothing else.  Why?  Because about 2/3 of the way through TClientDataset.InternalOpen, it says “if DefaultFields then CreateFields;”  And DefaultFields is set by the method that calls InternalOpen, which starts like this:

[code lang="delphi"]
procedure TDataSet.DoInternalOpen;
begin
  FDefaultFields := FieldCount = 0;
  InternalOpen;
[/code]

Oops!  Again, not making any distinction between data fields and auxiliary fields.  FDefaultFields is a private field, and DefaultFields is a read-only property, so that makes it very difficult to change.

I could have created an entirely new TClientDataset on my DFM that duplicates the entire field structure of the dataset I’m trying to clone, including the calculated field, and that would have solved the immediate problem.  But then I’d have to make another copy of another dataset every time I find myself in a situation like this.  I’d really prefer to make it recognize that there are some predefined fields that it can integrate into the field structure.

Fortunately, InternalOpen is virtual.  All I need to do is create a subclass of TClientDataset that overrides InternalOpen and resets FDefaultFields to the correct value before calling inherited.  Except… how do you do that?  It’s a private field of TDataset, accessible only through a read-only property.

Prior to Delphi 2010, I wouldn’t have been able to do this.  But now, there’s a way to fix this sort of problem with extended RTTI. The solution looks like this. (This solution works specifically for TClientDataset, but you can apply it to any TDataset descendant easily enough.)

[code lang="delphi"]
unit extensible_cds;

interface
uses
   DBClient;

type
   TExtensibleClientDataset = class(TClientDataset)
   protected
      procedure InternalOpen; override;
   end;

implementation
uses
   db, RTTI;

{ TExtensibleClientDataset }

procedure TExtensibleClientDataset.InternalOpen;
var
   context: TRttiContext;
   field: TField;
   value: boolean;
begin
   value := true;
   for field in self.Fields do
      value := value and (field.FieldKind <> fkData);
   context.GetType(self.ClassInfo).GetField('FDefaultFields').SetValue(self, TValue.From(value));
   inherited InternalOpen;
end;

end.[/code]

If this sounds overly complicated, that’s probably because it is. TField.SetDataset should only care about the dataset being open if you’re trying to add a data field. That would have solved everything. I submitted a report to QC about this a few months ago.  Hopefully it’ll be one of the things they fix for Delphi 2011.

4 Comments

  1. Jolyon Smith says:

    Prior to Delphi 2010 you *could* have accessed fDefaultFields by creating a memory-layout facsimile of the class that introduces the member thereby creating a “memory overlay compatible” class, then casting “self” as that facsimile class with those members now exposed in scope, enabling you to access all private member data thru “the back door”.

    Yes, it’s a dirty hack but it doesn’t require that you bloat your executable with vast amounts of RTTI data to support an operation that requires a fraction of a fraction of a vanishingly small percentage of that total RTTI data. And it would be faster (perhaps not important in this case, but can be in others).

    More importantly, it’s a technique that works in *all* versions of Delphi.

  2. Fritz says:

    Great work! Thanks for that!

    Just one thing: What did you want to write where “<>” is displayed in the code for TExtensibleClientDataset?

    Thx

    Fritz

  3. Mason Wheeler says:

    Fritz: Fixed it. Thanks.

    Joylon: Eww! That would probably work, but it would be a big mass of IFDEFs to get it to actually work with “all versions of Delphi”, and you’d need to be very careful to get fill-in values for all the hidden interface pointers in the right locations, etc. And if you do anything wrong, you end up corrupting data. I agree, I’d like for more of the RTTI to be optional, but the alternative is just plain scary!

  4. David Moorhouse says:

    Jolyon

    An ugly hack – and how do you manage the case when the field layout changes in a later version of Delphi and the app mysteriously corrupts data – god help the people who have to maintain your code.