TValue is very slow!

Delphi 2010’s help describes TValue, used by the RTTI unit to store values of arbitrary types, as “a lightweight version of the Variant type.” I saw that and it made me wonder, how lightweight is it? How fast is using TValue?

Thankfully, among D2010’s lesser-known new features is the Diagnostics unit, which gives us TStopwatch, a simple record for timing operations. That makes it very easy to write a simple speed test.

I was expecting TValue to be about equal to Variant, or maybe a little bit faster. So I wrote up the following app:

program Project1;

{$APPTYPE CONSOLE}

uses
  SysUtils, rtti, diagnostics;

const HUNDRED_MILLION = 100000000;

procedure tryTValue;
var
   i: integer;
   j: TValue;
   value: integer;
begin
   for I := 1 to HUNDRED_MILLION do
   begin
      j := i;
      value := j.AsInteger;
   end;
end;

procedure tryVariants;
var
   i: integer;
   j: variant;
   value: integer;
begin
   for I := 1 to HUNDRED_MILLION do
   begin
      j := i;
      value := j;
   end;
end;

var
   stopwatch: TStopWatch;
begin
  try
    stopwatch := TStopWatch.StartNew;
    tryVariants;
    stopwatch.Stop;
    writeln('Variants: ', stopwatch.ElapsedMilliseconds);

    stopwatch := TStopWatch.StartNew;
    tryTValue;
    stopwatch.Stop;
    writeln('TValues: ', stopwatch.ElapsedMilliseconds);
  except
    on E: Exception do
      Writeln(E.ClassName, ': ', E.Message);
  end;
  readln;
end.

Of course, this is by no means an exhaustive test of TValue’s capabilities, but the results are instructive. When I ran this on my dev machine, (a high-end Alienware laptop with CPU cycles to burn,) the variants test returned almost immediately, while TValue took so long I ended up pausing it just to make sure it wasn’t hung.

I stopped it and let the test run again, and eventually got the following results (in milliseconds):
Variants: 1528
TValues: 30889

Punch that into Calculator and, for this specific operation at least, TValue is 20.21531413612565 times slower than Variant. Ya call that lightweight?!?

17 Comments

  1. delphigeist says:

    Nice post, didn’t knew that Variant is more efficient than TValue(actually I still do not need to work with d2010).
    Keep up the good work!

  2. Three reasons:

    – Your testing is flawed. The compiler has intrinsic knowledge of the Variant type, thus it can apply its (fairly basic) dataflow analysis and omit unused code. This means that no code is being generated for the line “value := j;” because you don’t ever use “value”.
    – TValue is a record, records are initialized and finalized using RTTI, and using RTTI is slow. Andreas tried to avoid that by JITing System.Initialize (http://andy.jgknet.de/blog/?p=376), but he didn’t pursue the idea because it turned out to be insignificant in real-world applications (http://andy.jgknet.de/blog/?p=384).
    – Also, the Rtti unit is written with generality in mind, whereas Variants seems to prefer speed over generality. For example, Variants basically consists of 2n conversion routines (*toVar, varTo*) for n supported types, whereas in TValue basically every value input/output goes through TValue.Make()/TValue.ExtractRawData(). (And a few functions in Rtti could really use the “inline” directive.)

  3. Xepol says:

    A few points: Note that in TryVariants, Value := J; is elimnated by the optimizer (it is not eliminated in the TryTValue). Adding a Writeln(Value) after the loop in both cases prevents this (and keeps the timing equal).

    This only double the TryVariants time, making it only about 22 times slower.

    Also if you then document out the assignment to Value and change the writeln to J or J.asinteger, the time for TryVariants drops to only 10x what TryVariants takes – obviously the read is the length process here.

    When you look at how the TValue works, clearly its dependance on RTTI and typeinfo at runtime makes it significantly slower than the compiler time magic that happens for Variants. In this case, it runs the type test a million times at run time instead of once at compile time. Much of this *MIGHT* be offset by better code design, but I do not think it can ever truely be overcome, long loops will always underscore the difference here.

    Perhaps by lightweight, they were refering to code overhead? Although, the code looks extra ugly, and I doubt it is any smaller than the variants code. This code *MIGHT* be more deterministic in type conversion (compared to Variants), assuming it has any benefit at all. If you submit a QC and suggest discarding TValue in favour of Variant as the resolution, let us know – it would be worth a vote.

    Nice demo of the TStopwatch object – it displays it nicely.

  4. Mason Wheeler says:

    Thanks for catching that, Moritz and Xepol. I turned Optimizations off, so all the lines would be executed, and adjusted the numbers in the post. It’s now about 20x slower on my system, close to what Xepol got.

  5. Robert Love says:

    Very Interesting test thanks for taking the time to do it. I also did what Xepol suggested to make tests equal.

    Looking at the resulting ASM the routines are nearly the same.
    So basically what you are comparing here is: varToInt() to TValue.Implicit and then VarToInteger to TValue.AsInteger

    .AsInteger makes several calls but several for type safety, if you modify .AsInteger to:

    result := FData.FAsSLong;

    The modify class operator TValue.Implicit(Value: Integer): TValue to:

    result.FData.FTypeInfo := System.TypeInfo(Integer);
    result.FData.FAsSLong := Value;

    My observations is that TValue then is faster than Variant.
    Variants: 1315
    TValues: 462

    I suspect if time were taken TValue could be optimized to have better speed and not lose the type safety.

    Unlike Xepol, I would not support discarding TValue, it can store and deal with far more than Variants can. It just needs some time spent making it faster.

  6. Mason Wheeler says:

    Robert: Good idea, but your .AsInteger function throws type safety out the window. I like the Implicit, though. I wonder if there’s a way to optimize AsInteger while maintaining type safety…

  7. Robert Love says:

    Here is a type safe version of the two methods that results in similar performance, with TValue being slightly faster. I am sure it could be optimized more if conversions need to occur.

    class operator TValue.Implicit(Value: Integer): TValue;
    begin
    result.FData.FTypeInfo := System.TypeInfo(Integer);
    result.FData.FAsSLong := Value;
    result.FData.FHeapData := IInterface(Nop_Instance);
    end;

    function TValue.AsInteger: Integer;
    begin
    if (FData.FTypeInfo = System.TypeInfo(Integer)) and Assigned(FData.FHeapData) then
    begin
    result := FData.FAsSLong;
    end
    else
    Result := AsType;
    end;

  8. Ken Knopfli says:

    I learn a lot from your blog.

    You post your posit, then dozens of pundits chime in to show where you are going wrong.

  9. Mason Wheeler says:

    Robert: I tried those and got:

    Variants: 1551
    TValues: 1977

    Much better!

    Ken: *grin* I guess I’m still learning.

  10. Barry Kelly says:

    TValue isn’t optimized for speed, and I wouldn’t describe it as “a lightweight Variant”. It’s explicitly designed to not be like Variant – Variants can change type depending on context, while TValue doesn’t. TValue is designed to model the Delphi type system dynamically so that values can be transported with high fidelity to and from dynamic invocation. Variant was designed to model Excel spreadsheet cells, and extended to be the basic type of a scripting engine – naturally it has had far more focus on performance.

    But yes, things can be improved with TValue performance wise, but be aware that any gains in practice will be limited owing to the fact that the overhead of reflection-based invocation will still add up. The dynamic invocation logic can itself be optimized by precomputing more things and reducing the number of copies, but again it’s still not going to be competitive with native invocations.

  11. Barry Kelly says:

    And FWIW, your timings are likely to be processor sensitive. I ran it on my high-end desktop:

    Variants: 473
    TValues: 21853

    By this test, over 40x slower 🙂

  12. François says:

    @Mason and commentators,
    Very interesting reading. Thanks!

  13. Jolyon Smith says:

    I’m curious … if TValue is more specialised and Variant is more generalised, then it seems to me that Variant’s could have been used everywhere that a TValue is used, so why re-invent an inferior wheel ?

    As a type intended for the basis of a scripting engine, it seems to me that Variant would also be a natural choice for any other form of dynamic invocation, reflection based or not (most, if not all, “scripting engine’s” involve some form of dynamic invocation, no?).

    I suspect collective noses were turned up at Variants because they weren’t “Generic” enough (note deliberate capitalisation and air-quotes).

  14. Chris says:

    Joylon – what’s with the snark? I take it you haven’t actually tried to put something as basic as (say) an enum value or an object reference into a variant? Obviously, you could hack this, but the code required would presumably mean replicating a lot of TValue if the actual type (and not just the ordinal value) is to be roundtrippable.

  15. Jolyon Smith says:

    Then extend Variant or use Variant internally, rather than come up with a.n.other entirely new “like a variant but not Variant” implementation. At the very least take the bit’s of Variant that are performant (yuck word) and add the twiddly bits that are needed, rather than creating something entirely new from scratch. But as I say, I suspect that this approach wasn’t “sexy” enough.

  16. Chris says:

    Joylon – ‘rather than creating something entirely new from scratch’

    That’s a rather ironic thing to say given the performance issue in question arises from TValue ultimately piggy-backing on something old, namely the long-standing internal routine that gets invoked whenever one record with managed fields is assigned to another. Moreover, there remains the basic point that, contra what the help says, TValue *isn’t* a ‘lightweight version of Variant’ at all, but something different with a different purpose.

    Now of course, maybe the Variant type *should* be extended to better reflect its name rather than its OLE Automation-supporting origins – for myself, I’ve always found it odd that it supports native Delphi strings yet not native Delphi objects, and as a result, barely gives you much more than the strict OLE Automation subset that is OleVariant, D6+’s custom variant support notwithstanding. Nonetheless, that’s a distinct issue to the one at hand.

  17. […] few years back, when I posted an analysis of how TValue is very slow, it prompted a lot of response from the community.  Various people ran their own benchmarks, and […]