Problem with TInterfacedCollection for Lazarus 1.8.2

danny · 2018-03-14 12:55:45

Hello,
I am writing from Poland, where I use polish characters on a daily basis, for example: ó, ł, ą, ę, etc.

I create simple app with DTO objects and I have problem with special polish characters. Problem not exists on Delphi, but only for Lazarus (I have checked both).

I suppose the problem is related to TCollectionItem and TInterfacedCollection.

The same problem occurs in demo "20 - DTO interface based service".

for example in Delphi, result shows as:

{
	"Airport": 
	[{
			"Location": "Rzeszów",            <--------------------------------------------------------------------------------------
			"Terminal": ["terminalA","terminalB","terminalC"],
			"Gate": ["gate1","gate2","gate3","gate4","gate5"],
			"BHS": "Siemens",
			"DCS": "Altiea"
		}
	],
	"Airline": 
	[{
			"CX": ["B777","B737","A380","A320"],
			"QR": ["A319","A380","B787"],
			"ET": "380",
			"SQ": "A320"
		}
	],
	"GroundHandler": ["Swissport","SATS","Wings","TollData"]
}

but for lazarus:

{
	"Airport": 
	[{
			"Location": "Rzesz??w",            <--------------------------------------------------------------------------------------
			"Terminal": ["terminalA","terminalB","terminalC"],
			"Gate": ["gate1","gate2","gate3","gate4","gate5"],
			"BHS": "Siemens",
			"DCS": "Altiea"
		}
	],
	"Airline": 
	[{
			"CX": ["B777","B737","A380","A320"],
			"QR": ["A319","A380","B787"],
			"ET": "380",
			"SQ": "A320"
		}
	],
	"GroundHandler": ["Swissport","SATS","Wings","TollData"]
}

I have tried different combinations with UTF8ToString etc - nothing helped.

I am asking for some tips.....
Regards

ab · 2018-03-14 15:02:30

On Windows?

danny · 2018-03-14 15:50:00

Yes, windows 7 64b

ab · 2018-03-14 16:07:50

Did you try with https://synopse.info/fossil/info/67711926bf2d907a

danny · 2018-03-14 16:47:43

Unfortunately not work...

Now, cutting string after meet polish special character:

[{
		"Name": "Rzesz

The same at retrieving name from object (not as json)

mpv · 2018-03-14 18:32:42

Commit from prev. post is only for linux (@ab thanks, I check it - work as expected).
@danny - on fpc trunk (fpc 3.1.1, lazarus 1.9.0) I work with ukainian (ї, і) chars without problems. Please, check on fpc 3.1.1 - there are many RTTI fixes there and I think problem is because of fpc version

danny · 2018-03-15 12:01:26

Ehh this is some insane....problm still exists

I have checked on fpc 3.1.1, lazarus 1.9.0 and the same story....

My example codes:
INTERFACES

TDTOCity = class(TCollectionItem)  // deklaracja obiektu DTO dla Miasta
  private
    FName: RawUTF8;
  published
    property Name: RawUTF8 read FName write FName;
  end;

  TDTOCities = class(TInterfacedCollection)  // deklaracja kolekcji obiektow TDTOCity
  private
    function GetCollItem(aIndex: Integer): TDTOCity;
  protected
    class function GetClass: TCollectionItemClass; override;
  public
    function Add: TDTOCity;
    property Item[aIndex: Integer]: TDTOCity read GetCollItem; default;
  end;

  ICitiesService = interface(IInvokable)   // deklaracja interfejsu
    ['{49ACE92E-F9E8-4AE9-BA04-AC9A3198F034}']
    procedure GetCities(const Name: RawUTF8; out Cities: TDTOCities);
  end;

const
  ROOT_NAME = 'Demo';
  PORT_NAME = '888';

implementation

{ TDTOCities }

function TDTOCities.Add: TDTOCity;
begin
  result := TDTOCity(inherited Add);
end;

class function TDTOCities.GetClass: TCollectionItemClass;
begin
  result := TDTOCity;
end;

function TDTOCities.GetCollItem(aIndex: Integer): TDTOCity;
begin
  result := TDTOCity(GetItem(aIndex));
end;

IMPLEMENTATION:

 { TCitiesService }

  TCitiesService = class(TInterfacedObject, ICitiesService) // implementacja interfejsu
  private
    FConnection: TSQLDBZEOSConnectionProperties;
    FSQLDBRows: ISQLDBRows;
  public
    constructor Create;
    destructor Destroy; override;

    procedure GetCities(const Name: RawUTF8; out Cities: TDTOCities);
  end;

implementation

{ TCitiesService }

constructor TCitiesService.Create;
begin
  inherited;
end;

destructor TCitiesService.Destroy;
begin
  FConnection.Free;
  inherited Destroy;
end;

procedure TCitiesService.GetCities(const Name: RawUTF8; out Cities: TDTOCities);
begin
  //proste polaczenie do bazy
  FConnection := TSQLDBZEOSConnectionProperties.Create(TSQLDBZEOSConnectionProperties.URI(dMSSQL,'SERVER'),'DATABSE','sa', '');
  FSQLDBRows := FConnection.Execute('SELECT NazwyMiast FROM Miasta WHERE NazwyMiast LIKE ? ', [Name]);

    with Cities.Add do
    begin
      Name := 'Rzeszów;
    end;
        with Cities.Add do
    begin
      Name := 'Miasto2';
    end;
            with Cities.Add do
    begin
      Name := 'Miasto3';
    end;

  //while FSQLDBRows.Step do
  //begin
  //  with Cities.Add do
  //  begin
  //    Name := FSQLDBRows.ColumnUTF8('NazwyMiast');
  //  end;
  //end;
end;

FPC HELP INTERFACES

{$ifndef HASINTERFACERTTI} // circumvent old FPC bug of missing RTTI

{ TInterfaceFactoryDefinition }

type
  /// define and manage missing interface RTTI for the following interfaces:
  // - ICalculator
  TInterfaceFactoryDefinition = class(TInterfaceFactoryGenerated)
  protected
    procedure AddMethodsFromTypeInfo(aInterface: PTypeInfo); override;
  end;

procedure TInterfaceFactoryDefinition.AddMethodsFromTypeInfo(aInterface: PTypeInfo);
begin
  if aInterface=TypeInfo(ICitiesService) then   //dodajemy metode pomocnicza, bo Lazarus ma problem z interfejsami
  begin
    AddMethod('GetCities',[
      ord(smdconst),'Name',TypeInfo(RawUTF8),
      ord(smdOut),'Cities',TypeInfo(TDTOCities)
      ]);
    exit;
  end;
end;

initialization
  TInterfaceFactoryDefinition.RegisterInterface(TypeInfo(ICitiesService)); //rejestrujemy interfejs

{$endif HASINTERFACERTTI}

Maybe someone can send me some example demo with DTO (with interfaced colection)?

hnb · 2018-03-15 14:09:38

Did you try to use in modules :

{$MODE DELPHI}
{$CODEPAGE UTF8}

?

danny · 2018-03-15 15:17:42

Thx, hnb, that was it!

Its ok, but unfortunately there is one more such situation when I get ???? instead chars...
when I assign values by FSQLDBRows.ColumnUTF8, Am I doing this well?

while FSQLDBRows.Step do
  begin
    with Cities.Add do
    begin
      Name := FSQLDBRows.ColumnUTF8('NazwyMiast');  // <------------------------------------------------ return RawUTF8  - Name field is RawUTf8 too......
    end;
  end;

when I assign by const then is all ok..:

    with Cities.Add do
    begin
      Name := 'Rzeszów';
    end;

EgonHugeist · 2018-03-15 15:39:04

The problem is this:

  FConnection := TSQLDBZEOSConnectionProperties.Create(TSQLDBZEOSConnectionProperties.URI(dMSSQL,'SERVER'),'DATABSE','sa', '');
  FSQLDBRows := FConnection.Execute('SELECT NazwyMiast FROM Miasta WHERE NazwyMiast LIKE ? ', [Name]);

in combination with the LCL issue SystemCodePage := ut8;

Comment in ZDbcDBlib:

{note: this is a hack from a user-request of synopse project!
Purpose is to notify Zeos all Character columns are
UTF8-encoded. e.g. N(VAR)CHAR. Initial idea is made for MSSQL where we've NO
valid tdsType to determine (Var)Char(Ansi-Encoding) or N(Var)Char(UTF8) encoding
So this is stopping all encoding detections and increases the performance in
a high rate. If Varchar fields are fetched you should use a cast to N-Fields!
Else all results are invalid!!!!! Just to invoke later questions, reports!}
.....

The user attaches ansi encoded fields not nvarchar-fields.
It would help to use the sybdb.dll (your_zeos_folder\lib\freetds\32 bits + iconv) + FreeTDS protocol + Connection.Properties.Values['ClientCodepage'] := 'UTF8';

Propose to use the ODBC driver of SynDB or the Zeos ODBC driver like in \SQLite3\Samples\15 - External DB performance project described.

danny · 2018-03-15 17:56:37

ok, i replace Connection on

 
FConnection := TOleDBMSSQL2012ConnectionProperties.Create('Server','DB','sa', '');

And i think is ok now, thx for all.

mpv · 2018-03-15 18:30:24

Synoledb have some problems in case compiled by FPC for Win x64. I will fix it soon ( work in progress inside fix/oledb64 github brunch)

mORMot Open Source

#1 2018-03-14 12:55:45

Problem with TInterfacedCollection for Lazarus 1.8.2

#2 2018-03-14 15:02:30

Re: Problem with TInterfacedCollection for Lazarus 1.8.2

#3 2018-03-14 15:50:00

Re: Problem with TInterfacedCollection for Lazarus 1.8.2

#4 2018-03-14 16:07:50

Re: Problem with TInterfacedCollection for Lazarus 1.8.2

#5 2018-03-14 16:47:43

Re: Problem with TInterfacedCollection for Lazarus 1.8.2

#6 2018-03-14 18:32:42

Re: Problem with TInterfacedCollection for Lazarus 1.8.2

#7 2018-03-15 12:01:26

Re: Problem with TInterfacedCollection for Lazarus 1.8.2

#8 2018-03-15 14:09:38

Re: Problem with TInterfacedCollection for Lazarus 1.8.2

#9 2018-03-15 15:17:42

Re: Problem with TInterfacedCollection for Lazarus 1.8.2

#10 2018-03-15 15:39:04

Re: Problem with TInterfacedCollection for Lazarus 1.8.2

#11 2018-03-15 17:56:37

Re: Problem with TInterfacedCollection for Lazarus 1.8.2

#12 2018-03-15 18:30:24

Re: Problem with TInterfacedCollection for Lazarus 1.8.2

Board footer