record and object issue in Delphi 2010

ab · 2011-01-29 08:44:23

In a previous post, I explained why I still like to use, in some special cases, records or object where using a class doesn't perfectly fit my needs.
I just discovered a non reproducing bug in Delphi 2010, about objects: sometimes, reference-counted objects were not initialized!

Records and objects

Records can be evil (slow hidden copy, record hidden slow cleanup code, a fillchar would make any string in record become a memory leak...), but they are sometimes very convenient to access a binary structure (e.g. some "smallish value"), via a pointer.

If using pointers, you must know what you are doing. It's definitively more preferable to stick with classes, and TPersistent if you want to use the magical "VCL component ownership model".
I only try to use records if there is no reference counted fields in them.

But sometimes, I still use records to use some algorithms on the stack, or as part of a class internal data storage: no need to a try..finally block for the stack use, or no need to a Create/Free (or TPersistent.Create) for the class use.

Here comes the problem

There are some problems with using record or object with recent versions of Delphi, in comparison to the "regular" usage from Turbo Pascal 5.5 up to Delphi 2007.

Some "enhancements" (mostly marketing, IMHO) were introduced to the record model since Delphi 2009: the object keyword was told as "deprecated", you could add methods to a record, and some properties

But existing code won't compile any more. Just try to compile a packed object structure in Delphi 2009/2010/XE... and you'll find out that it's not possible anymore. You'll have to either define a packed record either use the {$A-} conditional define.
The problem is that a packed record can't have methods with older Delphi version. So if, like me, you are writing libraries and want them to compile to most Delphi IDE version (not only the last one), you can't use that. So I've used the {$A-} conditional define in my source code.

Those past days, I just found out another issue.

Normally, if you define a record on the stack, containing some reference-counted variables (like a string), it will be initialized by some compiler magic code, at the begin level of the method/function (to

type TObj = object Int: integer; Str: string; end;
procedure Test;
  var O: TObj
begin // here, an _InitializeRecord(@O,TypeInfo(TObj)) call is made
  O.Str := 'test';
  (...)
end;  // here, a _FinalizeRecord(@O,TypeInfo(TObj)) call is made

Those _InitializeRecord and _FinalizeRecord will "prepare" then "release" the O.Str variable.

With Delphi 2010, I found out that sometimes, this _InitializeRecord() was not always made.
If the record has only some no public fields, the hidden calls are sometimes not generated by the compiler.
Just build the source again, and there will be...

The only solution I found out was using the record keyword instead of object.

So here is how my code looks like:

/// used to store and retrieve Words in a sorted array
// - is defined either as an object either as a record, due to a bug
// in Delphi 2010 compiler (at least): this structure is not initialized
// if defined as a record on the stack, but will be as an object
TSortedWordArray = {$ifdef UNICODE}record{$else}object{$endif}
public
  Values: TWordDynArray;
  Count: integer;
  /// add a value into the sorted array
  // - return the index of the new inserted value into the Values[] array
  // - return -(foundindex+1) if this value is already in the Values[] array
  function Add(aValue: Word): PtrInt;
  /// return the index if the supplied value in the Values[] array
  // - return -1 if not found
  function IndexOf(aValue: Word): PtrInt; {$ifdef HASINLINE}inline;{$endif}
end;

The {$ifdef UNICODE}record{$else}object{$endif} is awful... but the code generation error didn't occur since..
The resulting modifications in the source code are not huge, but a bit disappointing.

What's next?

Perhaps you'll find, and you'll be perfectly right, that I should get rid on the record/object types here, and use plain Delphi classes.
And, in order to get rid of the Free call, use an interface.
But, defining an interface just to add some basic garbage collection without any object-oriented need... it's not a perfect solution either, even if can be handy in some cases...

I'm perhaps "border-line", playing with the compiler fire...

But it's just a pity that EMB doesn't have, in their automated test suite, some tests dedicated to the record/object part of the language, in order to avoid such regressions.

esmondb · 2011-01-31 14:41:46

Thanks for explaining objects (which even the delphi 3 docs hardly mention). You say above that with records there's "no need to a try..finally block for the stack use". Could you explain this a bit more?

ab · 2011-01-31 17:02:50

Here is how you'd code it using class:

type
  TMyObject = class
     Int: integer;
     Text: string;
  end;

procedure Test;
var M: TMyObject;
begin
  M := TMyObject.Create;
  try
    M.Text := '12039'
  finally
    M.Free;
  end;
end;

and the same code using a record/object:

type
  TMyObject = object
     Int: integer;
     Text: string;
  end;

procedure Test;
var M: TMyObject;
begin
  M.Text := '12039'
end;

For the record/object, the structure is initialized by the compiler.

There is some hidden _InitializeRecord/_FinalizeRecord calls at the begin/end level, together with an invisible try...finally block, as stated in my blog post.
Like this:

procedure Test;
var M: TMyObject;
begin
  _InitializeRecord(@M,TypeInfo(TMyObject)); // will set pointer(M) := nil;
  try
    M.Text := '12039'
  finally
    _FinalizeRecord(@M,TypeInfo(TMyObject)); // will execute M := '';
  end;
end;

One difference is that M.Int will always be 0 in case of a class, and that M.Int could have any value (it's not initialized) in case of a record/object.
_InitializeRecord will only set Text to '', and leave Int as it is, i.e. in whatever value is on the stack.

esmondb · 2011-01-31 21:28:06

Didn't think of protecting memory, Thanks.

Apart from the change of nomenclature in unicode delphi which I can see is a compatibility pain is there much loss of functionality? If you use record in place of object do you only loose 'inheritance' which at a low level doesn't seem that important?

Leander007 · 2011-05-04 12:58:27

The code is not compilable under Delphi 2009 from version 2011-04-06 19:25 [a286d5c455] • added TMemoryMap ...
[DCC Fatal Error] SynCommons.pas(2874): F2084 Internal Error: DT5830

I changed the code to be compilable, but I don't know if this is the way you want it :

{$ifdef ISDELPHI2009}
TMemoryMap = record
{$else}
TMemoryMap = object
protected
{$endif}

ab · 2011-05-04 13:09:28

The Delphi 2009 compiler is broken with the object keyword.

I've modified the trunk code as you suggested.
I only have Delphi 2010 installed on my computer.

With Delphi 2010, there is no problem with this syntax.
Did you install the latest fixes for Delphi 2009?

Thanks for the feedback.

ab · 2011-05-04 13:21:58

esmondb wrote:

Apart from the change of nomenclature in unicode delphi which I can see is a compatibility pain is there much loss of functionality? If you use record in place of object do you only loose 'inheritance' which at a low level doesn't seem that important?

So called "inheritance" is a good aspect of the object model, and a true missing feature according to the record feature.
For instance, some C oriented libraries uses memory structures with object orientation. Just for example, see the internal structures used by the SQLite engine: some of them are meant to have some "inheritance" when used from user code. You can't use a TClass because you need to match the SQLite C struct.
The only way of implementing this "inheritance" is by putting parent fields within children records, as nested records... not very elegant..

Another potential issue is that you also miss the "private" and "protected" fields, which are sometimes very handy, if you want to use properties, and hide variables.

Leander007 · 2011-05-04 14:09:08

ab wrote:

The Delphi 2009 compiler is broken with the object keyword.
...
Did you install the latest fixes for Delphi 2009?

Yes, I have the latest fixes for Delphi 2009 (to the update 4).
I do updates from your repository at least once or twice on month, so I'll report if there would be more errors of this kind.

ab · 2011-05-04 15:25:16

Leander007 wrote:

I do updates from your repository at least once or twice on month, so I'll report if there would be more errors of this kind.

Thanks!

Leander007 · 2011-05-11 02:01:04

Hi, I didn't know where to post this small error (should I open ticket as I did for 6593f0fbd1 in similar case or ?). I post it here, because is somehow related with provided pas units with Delphi 2009 (unicode), but it has nothing with object issue .

In SQLite3i18n.pas at line 2872 and 2905 typecast to PAnsiChar should be done to be compilable, because default for MakeIntResource is not any more PAnsiChar in Windows.pas.

...
EnumResourceNamesA(HInstance, PAnsiChar(RT_STRING), @CB_EnumStringProc, PtrInt(@CB_EnumStringValues));
...

I'm assuming that you are doing ExtractAllResources with your Delphi 7 .

Anyway, many thanks for your excellent framework.
I'm enjoying learning it (through trying to make useful app) bit by bit (now I'm on i18n area as you see), sometimes is hard to understand some practices, but on the end I always reach the goal with help of your superb documentation in code and generated one.

ab · 2011-05-11 05:17:48

Leander007 wrote:

I'm assuming that you are doing ExtractAllResources with your Delphi 7 .

You are right: I did only made full testing of i18n under non Unicode version of Delphi yet.

I've corrected both lines.
Thanks for your feed back and interest!

You could have opened a new thread in the SQLite3 framework forum for this particular issue.

Leander007 · 2011-06-02 08:27:04

Hi, again some problems at least in Delphi 2009.

I compiled Main Demo, but I could not use it properly, because only "Audit trial" tab was created and two Access Violation has been reported.

I looked with debugger where it stops and I saw that it has some problems with TSQLRibbon creation.
The problem is with TFileRibbonTabParameters object (keyword, not instance ) which is derived from TSQLRibbonTabParameters.
When I combine fields from TSQLRibbonTabParameters with TFileRibbonTabParameters (this was enough, no need to change to record type, but inheritance is gone this way) all works fine.

Could you look through what could be an issue and how could be solved in general way.
In this way as you did, I'm assuming that this could be done only with object keyword, to have declarative way of UI definition and inheritance of common fields.

Some thoughts:
I think that such areas which are not performance critical is better to design with classes (avoiding problems as this in every corner ), even if this means different way of UI creation.
Maybe is better that UI related "elements" (as is TSQLRibbonTabParameters) should be in some other unit (e.g. SQLite3UICommons) and not to be referenced directly with SQLite3Commons unit as is in constructor TSQLModel.Create(TabParameters...),if not for other reason, just for clean separation sake.

ab · 2011-06-02 10:16:46

Delphi 2009 was indeed buggy... buggy... barely usable.
Delphi XE is seen by some as the "stable" version of what Delphi 2009 should have been... apart from the new RTTI.

The current code compiles fine with Delphi 2010, in all cases.
Did you install all Delphi 2009 updates?

Classes may not be possible here, because I wanted to initialize the UI content from a const.
With classes, we should use some lines of code, one per property.

The only solution around should be to use a record instead, and a nested record for "sub-classing".
But it won't be as easy as with the current "object" implementation.
Two diverse records, perhaps...

The fact that this UI element was part of SQlite3Commons was that it may also be used for an AJAX interface.
It's not linked to the GUI VCL, and is still linked to the "View" aspect of the MVC architecture.
The "Controler" aspect is already included in SQlite3Commons (with TSQLFilter and TSQLValidate classes).

I'll see if I may use another implementation.
Thanks for your feedback.

Leander007 · 2011-06-02 10:58:13

ab wrote:

Delphi 2009 was indeed buggy... buggy... barely usable.
The current code compiles fine with Delphi 2010, in all cases.

I said that compiles in Delphi 2009 too, but execution is not ok. Maybe you (I'm missing some low level knowledge and understanding of your implementation) can find what could went wrong and you can find some workaround (even with low level tricks ). I think too, that new implementation can cost to much refactoring and is not worth it if this is only Delphi 2009 issue.

ab wrote:

Did you install all Delphi 2009 updates?

Yes.

About TestSQL3 unit tests:
I don't know if failing tests are Delphi 2009 related (I didn't look them with debugger yet), but if they are, maybe is best to omit support for Delphi 2009 compiler (sooner or later I'll switch to newer version , but for now I'm stuck with D2009 ), because is buggy in areas you use most (this object, record issue) .

List of failing tests:

1.4. Cryptographic routines:
! - Base64: 2 / 11,994 FAILED
Total failed: 2 / 12,618 - Cryptographic routines FAILED

1.6. Synopse PDF:
! - TPdfDocument: 2 / 4 FAILED
! - TPdfDocumentGDI: 1 / 3 FAILED
Total failed: 3 / 7 - Synopse PDF FAILED

2.1. Basic classes:
! - TSQLRecord: 4 / 42 FAILED
Total failed: 4 / 245 - Basic classes FAILED

2.2. File based:
! - Direct access: 2 / 10,132 FAILED
Total failed: 2 / 618,271 - File based FAILED

2.3. File based WAL:
! - Direct access: 2 / 10,132 FAILED
Total failed: 2 / 618,271 - File based WAL FAILED

2.4. Memory based:
! - Direct access: 1 / 10,131 FAILED
! - TSQLRestClientDB: 2 / 615,382 FAILED
Total failed: 3 / 644,543 - Memory based FAILED

2.5. Client server access:
! - TSQLite3HttpClient: 1 / 3 FAILED
Total failed: 1 / 15,048 - Client server access FAILED

Synopse framework used: 1.13
SQlite3 engine used: 3.7.6.3
Generated with: Delphi 2009 compiler

Time elapsed for all tests: 52.55s
Tests performed at 2.6.2011 12:47:56

Total assertions failed for all test suits: 17 / 5,384,675

Last edited by Leander007 (2011-06-02 11:16:09)

ab · 2011-06-02 19:24:20

As stated in the documentation, it compiles AND all tests run just fine under Delphi 7, 2007 and 2010...
Delphi 2009... is another buggy story...

Could you enable logging during testing, and report all failed tests?
In order to have the faulty source lines and stack trace in the log, you should enable the .map file creation at compile time.
Thanks!

Leander007 · 2011-06-06 20:31:48

TestSQL3 compiled with Delphi 2009

I run the tests with debugger using asserts that I could see where it stops.
I did it all to the "file based" tests, because a lot of failures exists for same root problem (Hash32...).
New build was done for each check (after commenting out failing line), because using only compile resulted in very strange behaviour.

Here are results for these tests (in comments are breakable results) from code snapshot 2011-05-26 09:59 c83b1d9aa5:

Unit SynSelfTest:

409:

procedure TTestCryptographicRoutines.Base64;
...
Value64: RawByteString = 'SGVsbG8gL2Mn6XRhaXQg5+Ar';
...
Check(Base64Encode(Value)=Value64);//'SGVsbG8gL2Mn6XRhaXQg53Ir'
Check(BinToBase64(Value)=Value64);//'SGVsbG8gL2Mn6XRhaXQg53Ir'

692:
It seems that this check should always fail, because we never set Helvetica font and should be changed to Canvas.SetFont(Name[embed],10,[]);
Nevertheless it fails eather way.

procedure TTestSynopsePDF._TPdfDocument 
Name: array[boolean] of PDFString =
    ('Arial','Helvetica');
...
Canvas.SetFont('arial',10,[]); 
Check(Canvas.Page.Font.Name=Name[embed]);

620:

Check(Hash32(MS.Memory,MS.Position)=Hash[embed]); //$1780F68C=$A1201D84

774:

H := Hash32(MS.Memory,MS.Position); //H = $93E059A5
Check( (H=$93D859A5) or (H=$9FB359A7) );

Unit SQLite3Commons:

16559:

 
T.fAnsi := 'abcde'#233'ef'#224#233; //'abcdeéefŕé' or $61 $62 $63 $64 $65 $E9 $65 $66 $E0 $E9 
T.fTest := WinAnsiToUTF8('abcde'#233'ef'#224#233);//'abcdeéefré' or $61 $62 $63 $64 $65 $C3 $A9 $65 $66 $72 $C3 $A9 
T.fUnicode := Utf8DecodeToRawUnicode(T.fTest);//'a'#0'b'#0'c'#0'd'#0'e'#0#$E9#0'e'#0'f'#0'r'#0#$E9#0#0 or $61 $00 $62 $00 $63 $00 $64 $00 $65 $00 $E9 $00 $65 $00 $66 $00 $72 $00 $E9 $00 $00 
//RawUnicodeToWinAnsi(T.fUnicode) -> 'abcdeéefré' or $61 $62 $63 $64 $65 $E9 $65 $66 $72 $E9 
Check(RawUnicodeToWinAnsi(T.fUnicode)=T.fAnsi);

16569:

Check(Hash32(s)=$C18C26D);//$D3EFC46D

16572:

Check(Hash32(s)=$6DE61E87);//$D18E1389

16590:

Check(Hash32(s)=$DDAF2211);//$9543E289

Unit SQLite3:

5623:

Check(Hash32(JS)=$221D7DBD,'Expected ExecuteJSON result not retrieved'); //$FE400815

5633:

Check(Hash32(Demo.ExecuteJSON(Req))=$221D7DBD,'ExecuteJSON crypted'); //$FE400815

6376:

Check(Hash32(S)=$6060CC67); //$6C2710BF

6380:

Check(Hash32(S)=$A1DF1271); //$670609C8

Last edited by Leander007 (2011-06-06 20:33:51)

ab · 2011-06-07 05:50:06

Thanks for the input.

I don't find any matching line number with the current version of the framework for SynSelfTests...
Perhaps some files are not the latest version of the framework.

What is your system code page?
There are problems about the character encoding in the source code, for e.g. in

T.fAnsi := 'abcde'#233'ef'#224#233;

It should be 'abcdeéefàé' but you have 'abcdeéefŕé'

Here is the problem: not in the framework, but in the system settings.
Not in Delphi 2009, but in character encoding.
It will affect only those low-level tests, which rely on those hard-coded characters.

I tried to fix it by using #233 instead of é.
See http://synopse.info/forum/viewtopic.php?pid=122#p122
But it didn't work as expected...

Could you try:
- Change the charset of every file to Win-1252;
- Or try to put #0... instead of #... for instance T.fAnsi := 'abcde'#0233'ef'#0224#0233; (not sure for this)
- Or use chr() for instance T.fAnsi := 'abcde'+chr(233)+'ef'+chr(224)+chr(233);
... and the tests should pass.

I've changed all constant ansichar declaration from #.... into chr(), and forced the AnsiString to be a WinAnsiString.
See http://synopse.info/fossil/info/fa3685333d

Leander007 · 2011-06-07 12:24:55

SLO code page is 1250 and the problems related to code page are gone now with your corrections.

Because the problems (at least mainly) are not Delphi 2009 compiler related, I moved this issues to the new topic "Synopse SQlite3 Unit Tests".

ab · 2011-06-07 13:10:05

To be continued on http://synopse.info/forum/viewtopic.php?id=335

stanleyxu2005 · 2012-04-26 16:51:19

Sorry for bubble up the old post.

IMO, it is very dangerous to initialize a record with FillChar. If you want to design a new record type, I recommend to make it immutable.

  // Example of an immutable record
  TIdentifier = record
  private
    FValue: ISuperObject;
  public
    property Value: ISuperObject read FValue;
  public
    class function NullIdentifier: TIdentifier; static;
    class function StringIdentifier(const Value: string): TIdentifier; static;
    class function NumberIdentifier(Value: Integer): TIdentifier; static;
    class function FromValue(const Value: ISuperObject): TIdentifier; static;
    function ToString: string;
  end;

ab · 2012-04-26 19:58:19

No problem of turning back to this subject.

The issue was a compiler issue.
Using FillChar here is just a workaround.

"record" are working as expected.
Only "object" needs sometimes to be initialized, with some versions of the Delpih compilers.
So your proposal has nothing to do with the issue: it won't fix anything, even for an object.

edwinsn · 2019-03-19 07:42:50

While defining TDynArray as record/object saved you from writing try...final block, one the other hand, if it's implemented as class, one can easily inherit from it and add new functions.

mORMot Open Source

#1 2011-01-29 08:44:23

record and object issue in Delphi 2010

#2 2011-01-31 14:41:46

Re: record and object issue in Delphi 2010

#3 2011-01-31 17:02:50

Re: record and object issue in Delphi 2010

#4 2011-01-31 21:28:06

Re: record and object issue in Delphi 2010

#5 2011-05-04 12:58:27

Re: record and object issue in Delphi 2010

#6 2011-05-04 13:09:28

Re: record and object issue in Delphi 2010

#7 2011-05-04 13:21:58

Re: record and object issue in Delphi 2010

#8 2011-05-04 14:09:08

Re: record and object issue in Delphi 2010

#9 2011-05-04 15:25:16

Re: record and object issue in Delphi 2010

#10 2011-05-11 02:01:04

Re: record and object issue in Delphi 2010

#11 2011-05-11 05:17:48

Re: record and object issue in Delphi 2010

#12 2011-06-02 08:27:04

Re: record and object issue in Delphi 2010

#13 2011-06-02 10:16:46

Re: record and object issue in Delphi 2010

#14 2011-06-02 10:58:13

Re: record and object issue in Delphi 2010

#15 2011-06-02 19:24:20

Re: record and object issue in Delphi 2010

#16 2011-06-06 20:31:48

Re: record and object issue in Delphi 2010

#17 2011-06-07 05:50:06

Re: record and object issue in Delphi 2010

#18 2011-06-07 12:24:55

Re: record and object issue in Delphi 2010

#19 2011-06-07 13:10:05

Re: record and object issue in Delphi 2010

#20 2012-04-26 16:51:19

Re: record and object issue in Delphi 2010

#21 2012-04-26 19:58:19

Re: record and object issue in Delphi 2010

#22 2019-03-19 07:42:50

Re: record and object issue in Delphi 2010

Board footer