TValue is very slow!
Delphi 2010’s help describes TValue, used by the RTTI unit to store values of arbitrary types, as “a lightweight version of the Variant type.” I saw that and it made me wonder, how lightweight is it? How fast is using TValue?
Thankfully, among D2010’s lesser-known new features is the Diagnostics unit, which gives us TStopwatch, a simple record for timing operations. That makes it very easy to write a simple speed test.
I was expecting TValue to be about equal to Variant, or maybe a little bit faster. So I wrote up the following app:
[code lang="Delphi"]program Project1;
{$APPTYPE CONSOLE}
uses
SysUtils, rtti, diagnostics;
const HUNDRED_MILLION = 100000000;
procedure tryTValue;
var
i: integer;
j: TValue;
value: integer;
begin
for I := 1 to HUNDRED_MILLION do
begin
j := i;
value := j.AsInteger;
end;
end;
procedure tryVariants;
var
i: integer;
j: variant;
value: integer;
begin
for I := 1 to HUNDRED_MILLION do
begin
j := i;
value := j;
end;
end;
var
stopwatch: TStopWatch;
begin
try
stopwatch := TStopWatch.StartNew;
tryVariants;
stopwatch.Stop;
writeln('Variants: ', stopwatch.ElapsedMilliseconds);
stopwatch := TStopWatch.StartNew;
tryTValue;
stopwatch.Stop;
writeln('TValues: ', stopwatch.ElapsedMilliseconds);
except
on E: Exception do
Writeln(E.ClassName, ': ', E.Message);
end;
readln;
end.[/code]
Of course, this is by no means an exhaustive test of TValue’s capabilities, but the results are instructive. When I ran this on my dev machine, (a high-end Alienware laptop with CPU cycles to burn,) the variants test returned almost immediately, while TValue took so long I ended up pausing it just to make sure it wasn’t hung.
I stopped it and let the test run again, and eventually got the following results (in milliseconds):
Variants: 1528
TValues: 30889
Punch that into Calculator and, for this specific operation at least, TValue is 20.21531413612565 times slower than Variant. Ya call that lightweight?!?
Nice post, didn’t knew that Variant is more efficient than TValue(actually I still do not need to work with d2010).
Keep up the good work!
Three reasons:
– Your testing is flawed. The compiler has intrinsic knowledge of the Variant type, thus it can apply its (fairly basic) dataflow analysis and omit unused code. This means that no code is being generated for the line “value := j;” because you don’t ever use “value”.
– TValue is a record, records are initialized and finalized using RTTI, and using RTTI is slow. Andreas tried to avoid that by JITing System.Initialize (http://andy.jgknet.de/blog/?p=376), but he didn’t pursue the idea because it turned out to be insignificant in real-world applications (http://andy.jgknet.de/blog/?p=384).
– Also, the Rtti unit is written with generality in mind, whereas Variants seems to prefer speed over generality. For example, Variants basically consists of 2n conversion routines (*toVar, varTo*) for n supported types, whereas in TValue basically every value input/output goes through TValue.Make()/TValue.ExtractRawData(). (And a few functions in Rtti could really use the “inline” directive.)
A few points: Note that in TryVariants, Value := J; is elimnated by the optimizer (it is not eliminated in the TryTValue). Adding a Writeln(Value) after the loop in both cases prevents this (and keeps the timing equal).
This only double the TryVariants time, making it only about 22 times slower.
Also if you then document out the assignment to Value and change the writeln to J or J.asinteger, the time for TryVariants drops to only 10x what TryVariants takes – obviously the read is the length process here.
When you look at how the TValue works, clearly its dependance on RTTI and typeinfo at runtime makes it significantly slower than the compiler time magic that happens for Variants. In this case, it runs the type test a million times at run time instead of once at compile time. Much of this *MIGHT* be offset by better code design, but I do not think it can ever truely be overcome, long loops will always underscore the difference here.
Perhaps by lightweight, they were refering to code overhead? Although, the code looks extra ugly, and I doubt it is any smaller than the variants code. This code *MIGHT* be more deterministic in type conversion (compared to Variants), assuming it has any benefit at all. If you submit a QC and suggest discarding TValue in favour of Variant as the resolution, let us know – it would be worth a vote.
Nice demo of the TStopwatch object – it displays it nicely.
Thanks for catching that, Moritz and Xepol. I turned Optimizations off, so all the lines would be executed, and adjusted the numbers in the post. It’s now about 20x slower on my system, close to what Xepol got.
Very Interesting test thanks for taking the time to do it. I also did what Xepol suggested to make tests equal.
Looking at the resulting ASM the routines are nearly the same.
So basically what you are comparing here is: varToInt() to TValue.Implicit and then VarToInteger to TValue.AsInteger
.AsInteger makes several calls but several for type safety, if you modify .AsInteger to:
result := FData.FAsSLong;
The modify class operator TValue.Implicit(Value: Integer): TValue to:
result.FData.FTypeInfo := System.TypeInfo(Integer);
result.FData.FAsSLong := Value;
My observations is that TValue then is faster than Variant.
Variants: 1315
TValues: 462
I suspect if time were taken TValue could be optimized to have better speed and not lose the type safety.
Unlike Xepol, I would not support discarding TValue, it can store and deal with far more than Variants can. It just needs some time spent making it faster.
Robert: Good idea, but your .AsInteger function throws type safety out the window. I like the Implicit, though. I wonder if there’s a way to optimize AsInteger while maintaining type safety…
Here is a type safe version of the two methods that results in similar performance, with TValue being slightly faster. I am sure it could be optimized more if conversions need to occur.
class operator TValue.Implicit(Value: Integer): TValue;
begin
result.FData.FTypeInfo := System.TypeInfo(Integer);
result.FData.FAsSLong := Value;
result.FData.FHeapData := IInterface(Nop_Instance);
end;
function TValue.AsInteger: Integer;
begin
if (FData.FTypeInfo = System.TypeInfo(Integer)) and Assigned(FData.FHeapData) then
begin
result := FData.FAsSLong;
end
else
Result := AsType;
end;
I learn a lot from your blog.
You post your posit, then dozens of pundits chime in to show where you are going wrong.
Robert: I tried those and got:
Variants: 1551
TValues: 1977
Much better!
Ken: *grin* I guess I’m still learning.
TValue isn’t optimized for speed, and I wouldn’t describe it as “a lightweight Variant”. It’s explicitly designed to not be like Variant – Variants can change type depending on context, while TValue doesn’t. TValue is designed to model the Delphi type system dynamically so that values can be transported with high fidelity to and from dynamic invocation. Variant was designed to model Excel spreadsheet cells, and extended to be the basic type of a scripting engine – naturally it has had far more focus on performance.
But yes, things can be improved with TValue performance wise, but be aware that any gains in practice will be limited owing to the fact that the overhead of reflection-based invocation will still add up. The dynamic invocation logic can itself be optimized by precomputing more things and reducing the number of copies, but again it’s still not going to be competitive with native invocations.
And FWIW, your timings are likely to be processor sensitive. I ran it on my high-end desktop:
Variants: 473
TValues: 21853
By this test, over 40x slower 🙂
@Mason and commentators,
Very interesting reading. Thanks!
I’m curious … if TValue is more specialised and Variant is more generalised, then it seems to me that Variant’s could have been used everywhere that a TValue is used, so why re-invent an inferior wheel ?
As a type intended for the basis of a scripting engine, it seems to me that Variant would also be a natural choice for any other form of dynamic invocation, reflection based or not (most, if not all, “scripting engine’s” involve some form of dynamic invocation, no?).
I suspect collective noses were turned up at Variants because they weren’t “Generic” enough (note deliberate capitalisation and air-quotes).
Joylon – what’s with the snark? I take it you haven’t actually tried to put something as basic as (say) an enum value or an object reference into a variant? Obviously, you could hack this, but the code required would presumably mean replicating a lot of TValue if the actual type (and not just the ordinal value) is to be roundtrippable.
Then extend Variant or use Variant internally, rather than come up with a.n.other entirely new “like a variant but not Variant” implementation. At the very least take the bit’s of Variant that are performant (yuck word) and add the twiddly bits that are needed, rather than creating something entirely new from scratch. But as I say, I suspect that this approach wasn’t “sexy” enough.
Joylon – ‘rather than creating something entirely new from scratch’
That’s a rather ironic thing to say given the performance issue in question arises from TValue ultimately piggy-backing on something old, namely the long-standing internal routine that gets invoked whenever one record with managed fields is assigned to another. Moreover, there remains the basic point that, contra what the help says, TValue *isn’t* a ‘lightweight version of Variant’ at all, but something different with a different purpose.
Now of course, maybe the Variant type *should* be extended to better reflect its name rather than its OLE Automation-supporting origins – for myself, I’ve always found it odd that it supports native Delphi strings yet not native Delphi objects, and as a result, barely gives you much more than the strict OLE Automation subset that is OleVariant, D6+’s custom variant support notwithstanding. Nonetheless, that’s a distinct issue to the one at hand.
[…] few years back, when I posted an analysis of how TValue is very slow, it prompted a lot of response from the community. Various people ran their own benchmarks, and […]