#51 mORMot 1 » Json Array de-serialization problem » 2020-03-05 09:22:11

ertank
Replies: 6

Hello,

I am using Delphi 10.3.3, Synopse is updated from github. I double check that I am using latest version of SynCommons.pas

I have a simple json string (json array) that I fail to find why I cannot de-serialize. I simply fail to find a reason.

I do appreciate any help.

Record definition:

  TStVATDetail = packed record
    u32VAT: Integer;
    u32Amount: Integer;
    u16VATPercentage: UInt16;
  end;
  TStVATDetails = Array of TStVATDetail;

Json array that I cannot de-serialize:

[{"u32VAT":15,"u32Amount":100,"u16VATPercentage":1800},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":0},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":800},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":100},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":2400},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":0},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":0},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":0}]

My code for de-serialization:

const
  Json = '[{"u32VAT":15,"u32Amount":100,"u16VATPercentage":1800},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":0},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":800},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":100},' +
         '{"u32VAT":0,"u32Amount":0,"u16VATPercentage":2400},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":0},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":0},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":0}]';

procedure TForm3.Button1Click(Sender: TObject);
var
  Taxes: TStVATDetails;
begin
  if not SynCommons.DynArrayLoadJSON(Taxes, RawUTF8(Json), TypeInfo(TStVATDetails)) then
  begin
    ShowMessage('Cannot deserialize Json!');
  end
  else
  begin
    ShowMessage('Json deserialization OK');
  end;
end;

#52 mORMot 1 » Json de-serialization order » 2020-03-05 08:11:16

ertank
Replies: 1

Hello,

I have a problem de-serializing a json string. After I get false from de-serialization function, I see some variables have value in my record.

In order to find my mistake, I would like to understand how mORMot doing de-serializing order.

For example I have below json string:

{"name":"test","surname":"test"}

And my record definition is:

type
  TPerson = packed record
    surname: string;
    name: string;
  end;

When de-serializing, is it name or surname that is filled in record variable first?

Thanks & regards,
Ertan

#53 Re: mORMot 1 » TTextWriter.RegisterCustomJSONSerializerFromText - UInt64 problem » 2020-02-29 19:17:20

I do really would like to hear if somebody can re-produce issue I am facing.

#54 Re: mORMot 1 » TTextWriter.RegisterCustomJSONSerializerFromText - UInt64 problem » 2020-02-26 07:24:42

I could complete a small project and I observe same problem in that stand-alone project.

Project can be downloaded from here: https://fil.email/qYgVeiEo
P.S.: Download valid for 1 month only.

FYI, I am using Synopse from GitHub. However, I am using SVN client to get sources. I see my copy is from 26th January, 2020. There is note "Merge pull request #268 from LongDirtyAnimAlf/master fix compilation of Alpine Linux."

#55 Re: mORMot 1 » TTextWriter.RegisterCustomJSONSerializerFromText - UInt64 problem » 2020-02-25 14:26:29

I am going to work on a small project to re-produce (if I can re-produce) and either post project or a feedback here. That is in a few days.

Thanks.

#56 Re: mORMot 1 » TTextWriter.RegisterCustomJSONSerializerFromText - UInt64 problem » 2020-02-25 12:54:06

In provided test json stLoyaltyService is null. However, when I put its text definition in the middle, it is not empty but filled in with A letters I suppose. There are other problems as well. For example, UserData variable is not filled in even there is a value in the json string for it.
Please see: https://imgur.com/a/lsjTnsM

#57 Re: mORMot 1 » TTextWriter.RegisterCustomJSONSerializerFromText - UInt64 problem » 2020-02-25 10:34:15

ab wrote:

Note that stLoyaltyService[...] defines a nested ARRAY [..] of records.

I realize that I did not share all necessary info. Sorry about that.

I indeed have this record as an array of records in my definition:

  TStLoyaltyServiceInfo = packed record
    name: Array[0..23] of Byte; // original was this. I changed to "Array of byte" after having access violation errors 
    CustomerId: string;
    ServiceId: UInt16;
    u16AppId: UInt16;
    CustomerIdType: UInt16;
    TotalDiscountAmount: UInt32;
  end;
  TStLoyaltyServiceInfos = Array of TStLoyaltyServiceInfo;
...
  TStTicket = packed record
...
    stLoyaltyService: TStLoyaltyServiceInfos;  // [0..MAX_LOYALITY_TRANS_NUMBER]
  end;

So, I believe my definition of "array of records" matches in my text definition. It is just changing position other than last most in the definition leads to a memory leak in my code. Maybe it is my code to blame I do not know for sure. Just trying to understand.

I also tried to use TByteDynArray and if definition is not latest, I still get access violation.

My complete text definition is here: https://pastebin.com/u27beRi9 (Where I put it at the end)
Text definition that gives access violation is here: https://pastebin.com/kACw9Y1z (Where it is defined in the middle)
And if you check both above that is what I mean by "in the middle" or "at the end" or "bottom most (last)"

Formatted json string is here: https://pastebin.com/9NxWvx4v
De-Serialized (single line in text file) json string is here: https://pastebin.com/wQsqjtVQ
Above two are same except one is formatted by online website.

Complete (hopefully I did not forget anything) record definitions are here: https://pastebin.com/uyb2uaam

#58 Re: mORMot 1 » TTextWriter.RegisterCustomJSONSerializerFromText - UInt64 problem » 2020-02-25 08:48:39

That QWord works for UInt64. Thank you.

I have one thing I would like to understand though.

I have below sub-record. If I move that record definition up in my text definition, that gets me memory corruption (access violation) after de-serialization before exiting procedure and corruption goes away if put it bottom most (last) in my text definition.

  TStLoyaltyServiceInfo = packed record
    name: Array of Byte;
    CustomerId: string;
    ServiceId: UInt16;
    u16AppId: UInt16;
    CustomerIdType: UInt16;
    TotalDiscountAmount: UInt32;
  end;

For above record I have following text definition:

stLoyaltyService[name Array of Byte CustomerId string ServiceId Word u16AppId Word CustomerIdType Word TotalDiscountAmount Cardinal]

For the record, stLoyaltyService is NULL in my de-serialized json string. So, basically it should not have any items at all. But, I see some items are created after de-serialization in Local variable view *if* it is in the middle but not at the end in my text definition.

It is possible I am missing something here?

Thanks & regards,
Ertan

#59 mORMot 1 » TTextWriter.RegisterCustomJSONSerializerFromText - UInt64 problem » 2020-02-25 06:15:47

ertank
Replies: 18

Hello,

I am using Delphi 10.3.3. SynCommit is 1.18.5394

I have a rather complex json and corresponding record definitions. I was happily using it as it is. This is used to communicate with a physical POS device. Recently, other part has changed something in *some* of their devices. They actually removed some of their not used json pairs. This new version of theirs is going to spread all devices in the end. Unfortunately, this process is slow. So, I cannot just convert my records to match new json string or old version running devices will fail.

I decided to use TTextWriter.RegisterCustomJSONSerializerFromText() and put in just the necessary values in both json strings and be on my way. Unfortunately, I got below exception;

Unregistered ptCustom for TJSONRecordTextDefinition.AddItem(ItemPrice: UINT64)

Until I came to this one, I did already converted some Int16, Int32 to their corresponding variable types in my constant string. It is just Int64 seems not have any alias at all.

1- Is it possible to include it in so that above exception does not raise?
2- Is there any workaround that I can use?

Thanks & regards,
Ertan

#60 Re: htm2pdf » htm2pdf free tool release » 2020-01-26 19:27:44

Hello,

I am searching for a way to print html files I have in certain order and without any user dialog intervention.

I thought easiest might be to convert them in to pdf and search ways of printing pdf without any user intervention. I am interested in free solutions.

I have found this htm2pdf tool. Tried it and I have below problems:
1- picture embedded in html is not visible.
2- text seems not UTF-8.
3- This maybe UTF-8 again, but even empty table cells are filled with some characters

Html source can be found in here: https://controlc.com/233e0de2
Generated PDF view can be found in here: https://files.fm/u/7ea5fdzn

I wonder if there is any way to fix these problems.

Thanks & regards,
Ertan

#61 mORMot 1 » Faster StrToDate() » 2019-11-21 21:48:43

ertank
Replies: 1

Hello,

I am using Delphi 10.3.3.

I probably used wrong selection of words using google. SynCommons.pas seems not having that function.

I am searching for a faster running code than current System.SysUtils.StrToDate().

There is below function in my code base and when I used a profiler, it turned out to be time consuming function in the application (most likely because mos of the calls are raising exception I think but not sure).

function IsDate(const DateString: string): Boolean;
begin
  try
    StrToDate(DateString);
    Result := True;
  except
    Result := False;
  end;
end;

So, I am searching for a faster executing code than above.

#62 Re: mORMot 1 » DynArrayLoadJSON access violation » 2019-11-08 08:59:44

ab wrote:

I have just make it more obvious in the comments/documentation.

And introduced a new overloaded DynArrayLoadJSON() function which will make a copy of the input buffer.
See https://synopse.info/fossil/info/bd63059e2f

I like new overload function more because of its Boolean return makes it easier to understand if result is a success or not.

Thank you.

#63 Re: mORMot 1 » DynArrayLoadJSON access violation » 2019-11-07 09:02:39

It works as suggested.

Interesting. I have legacy code about 3 years old working just fine using Pointer() with earlier versions of mORMot:

DynArrayLoadJSON(Taxes, Pointer(sRaw), TypeInfo(TStTaxRatesString));

I cannot say I am good at pointers. I wonder why one works and not the other?

#64 mORMot 1 » DynArrayLoadJSON access violation » 2019-11-06 14:59:12

ertank
Replies: 7

Hello,

Using Delphi 10.3.2, targeting Win32 executable.

I have demo code at https://pastebin.com/wqdBqRmT

When I run it I get access violation in SynCommons.pas:55273

        D^ := P^; // 3 stages pipelined process of unescaped chars

Synopse commit version is: 1.18.5037

I get same error at same line (line no 54907 this time) using latest codes on GitHub, too. That code Synopse commit version is: 1.18.5448

I am not sure if it is me doing something wrong. I know that I can successfully de-serialize identical json string using an alternative record based library. I do want to use mORMot.

So, I am posting here to see if there is anything to check in mORMot.

Any help is appreciated.

Thanks & regards,
Ertan

#65 Re: mORMot 1 » Access violation with FPC 3.3.1 » 2019-10-28 22:14:03

Are these changes waiting tests from users before going in to the main code base?

Thanks.

#66 Re: mORMot 1 » Empty sub-json in a json » 2019-10-10 10:50:06

I was wrong. It seems I also need to de-serialize such json data. Another problem might be that Sometimes they won't be empty at all. Example response from web service documentation:

{
   "ackCode":"success",
   "responseTime":"10/11/2010 11:51:11",
   "timeElapsed":"1369 ms",
   "sales":{
      "sale":{
         "saleCode":"325479960001",
         "status":"Alıcı ürüne onay verdi",
         "productId":"32547996",
         "productTitle":"Fotograf makinesi için çanta",
         "price":"1",
         "cargoPayment":"Alıcı öder",
         "amount":"1",
         "endDate":"09/11/2010 15:30:08",
         "buyerInfo":{
            "username":"test_user2",
            "name":"Mehmet",
            "surname":"Yılmaz",
            "phone":"212 3456435",
            "address":"Eski Üsküdar Yolu Cad. Çayıryolu Sok. No:1 Partaş Center K:16",
            "district":{
 
            },
            "city":"İstanbul",
            "zipCode":{
 
            }
         },
         "thumbImageLink":"http://mcdn01.gittigidiyor.net/3254/tn9/32547996_tn9_0.jpg",
         "lastActionDate":"09/11/2010 15:38:55"
      }
   }
}

Above, district, zipCode are empty, but it is not always empty.

I am searching for an example where at least one of them is not empty, but so far failed to find one.

#67 mORMot 1 » Empty sub-json in a json » 2019-10-10 10:38:32

ertank
Replies: 1

Hello,

I just realize now that I need to serialize something like an empty json:

      "pageTemplate":"1",
      "description":"Kodak Z915 Fotograf Makinesi 10Mp 50X zoom",
      "startdate":{
  
      },
      "catalogId":{
  
      },
      "catalogDetail":{
  
      },
      "format":"S",
      "startPrice":{
  
      },
      "buyNowPrice":"240",
      "netEarning":{
  
      },
      "listingDays":"30",

Above, startdate, catalogId, catalogDetail, startPrice, netEarning are all what I call empty json. For the record, above example is part of a bigger json string.

My understanding from the documentation of the web service is that. I will never have the need to fill inside of startdate, catalogId etc.

I am not sure if mORMot can do that and how to serialize something like above with mORMot.

Any help is appeciated.

Thanks & regards,
Ertan

#68 Re: mORMot 1 » @ sign in json string » 2019-10-10 09:29:06

Sorry for the mis understanding. After explanations I used following code in an empty project using Delphi:

...

  TTest = packed record
    abc: string;
  end;

const
  __TTest = '"@abc" string';

var
  Form1: TForm1;

implementation

{$R *.dfm}

uses
  SynCommons;

procedure TForm1.Button1Click(Sender: TObject);
var
  x: TTest;
  Json: RawUTF8;
begin
  x.abc := 'here I am';
  Json := SynCommons.RecordSaveJSON(x, TypeInfo(TTest));
  ShowMessage(string(Json));
end;

initialization
  TTextWriter.RegisterCustomJSONSerializerFromText(TypeInfo(TTest), __TTest);

end.

I see displayed message:

{"@abc":"here I am"}

Thanks for the help.

#69 Re: mORMot 1 » @ sign in json string » 2019-10-10 04:12:21

I just need to generate (serialize) json string like in my example for this web service. Responds to requests from the web service seem to be fine (no @ sign in them) as to their documentation.

I maybe wrong. My understanding this commit is for deserialization. It is good to have anyway. Thank you for that.

I’m searching for a solution without manual inserts in json string after serialization. Well, as much as possible of course.

#70 mORMot 1 » @ sign in json string » 2019-10-09 19:26:48

ertank
Replies: 5

Hello,

I have following web service format that I need to prepare for using a rest web service. This is for an online sales portal api. I do not have any chance of having it changed to something else.

      "specs":{
         "spec":[
            {
               "@required":"true",
               "@value":"Kullanılmış",
               "@name":"Durumu"
            },
            {
               "@required":"true",
               "@value":"10.0 - 11.9 MP",
               "@name":"Çözünürlük (Megapiksel)"
            },
            {
               "@required":"true",
               "@value":"Kodak",
               "@name":"Marka"
            }
         ]
      }

Above is just a part of a bigger json string. I also have some individual usage like:

"photos":{
         "photo":{
            "@photoId":"0",
            "url":"http://images.gittigidiyor.com/2941/KODAK-Z915-10MP-10X-ZOOM-HD-SIFIR-URUN__29416457_0.jpg"
         }
      }

These are json strings that I need to generate using records. I could not figure a solution myself.

Any help is appreciated.

Thanks & regards,
Ertan

#72 mORMot 1 » Exception on Raspberry Pi 4 - SynCommons.pas » 2019-09-20 14:18:45

ertank
Replies: 3

Hello,

I am using Lazarus 2.0.5, revision 61904M, fpc 3.2.0-beta revision 43036 on Raspberry Pi 4.

Today I installed everything on a fresh new system using fpcupdeluxe. Then I started testing my projects compilation for any missing components I may have forgotten. After compiling one of my projects, I tried to run it and I get an exception. Call stack is as following:

#0 GETENUMNAME(0xaee828, 6) at ../fpcupdeluxe/ccr/mORMot/SynCommons.pas:21101
#1 TOTEXT(OSARCH) at ../fpcupdeluxe/ccr/mORMot/SynCommons.pas:26054
#2 SETLINUXDISTRIB(0xf65238 'Raspbian GNU/Linux 10 (bu'...) at ../fpcupdeluxe/ccr/mORMot/SynCommons.pas:26242
#3 RETRIEVESYSTEMINFO at ../fpcupdeluxe/ccr/mORMot/SynCommons.pas:26320
#4 SYNCOMMONS_$$_init$ at :62578
#5 fpc_initializeunits at :0
#6 main at counter.lpr:20

It is GetEnumName() function in SynCommons.pas. This is commit version 1.18.5362 I am using.

Wondering if there is anything that I can do.

Thanks & regards,
Ertan

#73 Re: mORMot 1 » Json de-serialization problem due to position change » 2019-08-10 10:10:21

Hello,

I found that there is an additional field at the very bottom of that several KB long json string. So, it is not a problem of fields changing their position in json string but an addition of a field.

Now I need to use that TTextWriter.RegisterCustomJSONSerializerFromText() not to have such a problem again.

Thanks & regards,
Ertan

#74 mORMot 1 » Json de-serialization problem due to position change » 2019-08-10 07:59:09

ertank
Replies: 1

Hello,

I am using Delphi 10.3.2, targeting Win32 executable.

I had that legacy code of mine working fine until some days ago. I am told that it is giving some json error. My comparison showed me that server side has a minor change in their sent out json. BeyondCompare screen looks like following:
https://imgur.com/u28x5uN

Left side in above picture is version that is working fine. Right side is version that is failing. Basically below fields are moved up before an array in json string:

   "PaymentCheck":null,
   "PaymentAcikHesap":null,
   "HataDuzeltTutari":0.000000,
   "FaturaliSatisTutari":0.000000,
   "CariHesapAdedi":null,

My record definitions are as following:

  TZTax = packed record
    TaxKDV: string;
    TaxAmount: Variant;
    TaxRate: Variant;
    TaxIndex: Variant;
  end;

  TZReportItem = packed record
    Idx: Int64;
    Terminal: string;
    EkuNo: Integer;
    ZNo: Integer;
    ZDate: TDateTime;
    ZTime: string;
    KumuleTutar: Double;
    KumuleVergiKDV: Double;
    GunlukToplamTutar: Double;
    GunlukToplamKDV: Double;
    YuzdeArttirimAdedi: Variant;
    YuzdeIndirimAdedi: Variant;
    IptalAdedi: Variant;
    HataDuzeltAdedi: Variant;
    MaliFisAdedi: Variant;
    MaliOlmayanFisAdedi: Variant;
    MusteriFisAdedi: Variant;
    IptalFisAdedi: Variant;
    SatisIptalFisAdedi: Variant;
    FaturaliSatisAdedi: Variant;
    YuzdeArttirimTutari: Variant;
    YuzdeIndirimTutari: Variant;
    NakitTutari: Variant;
    KrediTutari: Variant;
    IptalTutari: Variant;
    FoodSaleCnt: Variant;
    FoodRcptTotalAmount: Variant;
    OtoparkEntryCnt: Variant;
    InvoiceOtherPayment: Variant;
    OtherPaymentTotalAmount: Variant;
    AvansIslemSayisi: Variant;
    Avans: Variant;
    KasaOdeme: Variant;
    FiscalReportReceiptCnt: Variant;
    TasnifDisiReceiptCnt: Variant;
    InvoiceCash: Variant;
    MobilPayment: Variant;
    TaxRefund: Variant;
    TaxlessAmount: Variant;
    TaxlessCount: Variant;
    CreditInvoice: Variant;
    PaymentGift: Variant;
    PaymentTreat: Variant;
    PaymentFree: Variant;
    PaymentDeposit: Variant;
    PaymentPoint: Variant;
    PaymentExpense: Variant;
    InvoiceTotal: Variant;
    EInvoiceTotal: Variant;
    EArchiveInvoiceTotal: Variant;
    BankaTransferTutari: Variant;
    PaymentCheck: Variant;
    PaymentAcikHesap: Variant;
    HataDuzeltTutari: Variant;
    FaturaliSatisTutari: Variant;
    CariHesapAdedi: Variant;
    ZTaxList: TArray<TZTax>;
    TaxRate0Amount: Variant;
    TaxRate1Amount: Variant;
    TaxRate8Amount: Variant;
    TaxRate18Amount: Variant;
    TaxRateOtherAmount: Variant;
    Currency: string;
    OKCVersion: Variant;
  end;

  TResponse = packed record
    IsSummary: Boolean;
    ZReportItems: TArray<TZReportItem>;
    UniqueId: Variant;
    ErrCode: string;
    ErrDesc: string;
  end;

I am not using TTextWriter.RegisterCustomJSONSerializerFromText() as I do use all fields in that json string.

I know that I can do same change on my record definitions and everything will be back to normal. However, I do really would like to have some flexible code because such changes on server side is going to happen more and more.

My question is: Is there anything that can be done to ignore position of fields in json string in my specific case?

Thanks & regards,
Ertan

#75 Re: mORMot 1 » Access violation at json de-serialization » 2019-08-07 15:38:16

My bad. You are correct. It was working fine and that person seems to have changed the code and I simply did not see that.

Thanks for the reply.

#76 mORMot 1 » Access violation at json de-serialization » 2019-08-07 14:30:51

ertank
Replies: 2

Hello,

This problem is not on my computer but at someone else's who is using mORMot because I suggested.

Delphi 10.3.1, SynopseCommit 1.18.5243, targeting Win32 executable.

Error message says:

First chance exception at $7536FD62. Exception class EAccessViolation with message 'Access violation at address 0040CBC4 in module 'Teklif.exe'. Read of address F9CC0011'. Process Teklif.exe (1220)

Json de-serialization code where exception raises is:

  try
    if not SynCommons.RecordLoadJSON(Response, RawUTF8(Response), TypeInfo(TGetDueDateResponse)) then Exit(False);  // <--- HERE
  except
    Exit(False);
  end;

Response value at the time of exception:

{  
   "license":1,
   "message":true,
   "duedate":"2019-06-12T05:38:48.000000Z",
   "systemdate":"2019-08-07T14:13:23.787244Z",
   "result":true,
   "error":""
}

TGetDueDateResponse definition:

  TGetDueDateResponse = packed record
    license: Integer;
    &message: Boolean;
    duedate: TDateTime;
    systemdate: TDateTime;
    result: Boolean;
    error: string;
  end;

Lastly, here is the call stack in Delphi when exception occurs: https://i.imgyukle.com/2019/08/07/oP2Rqp.png

I simply could not understand what problem may be.

Any help is appreciated.

Thanks & regards,
Ertan

#77 Re: mORMot 1 » SIGSEGV - SynCommons, fpc 3.2 » 2019-05-23 09:55:30

That seemingly working at first glance. But, I need to do some more test as I still get SIGSEGV further in the application. It just doesn't stop in any code line.
Will report in here once I figure what else is going wrong.

#78 mORMot 1 » SIGSEGV - SynCommons, fpc 3.2 » 2019-05-22 23:20:13

ertank
Replies: 7

Hello,

I am using Lazarus 2.0.3, (fixes 2.0) revision 61198, fpc 3.2.0-beta-r42099 (fixes 3.2), mORMot commit 1.18.5222 OS is Raspbian. Everything is updated and upgraded as of today.

I have a SIGSEGV in SynCommons.pas and remark says

    // FPC compatibility: if you have a GPF here at startup, your 3.1 trunk
    // revision seems older than June 2016
    // -> enable HASDIRECTTYPEINFO conditional below $ifdef VER3_1 in Synopse.inc
    // or in your project's options

One thing I would like to understand is my fpc version is 3.2 and newer than June 2016. So, why do I still get a segfault?

Also, I never enabled a conditional in my project's options. I very much appreciate menus to follow here.

Thanks & regards,
Ertan

#79 mORMot 1 » Finding de-serialization fail position » 2019-02-23 19:26:24

ertank
Replies: 1

Hello,

I have completed one big json structure record definitions. It consists of a lot of array records and sub records in it.

As expected, I cannot de-serialize it. RecordLoadJSON() returns False. Odd thing is that "as far as I can tell", all elements are assigned to their values in my record.

Is there a way to learn position of a json string that de-serialization fails at?

Thanks & regards,
Ertan

#80 Re: mORMot 1 » How to parse such json? » 2019-02-23 17:54:52

ab wrote:

You could also write 'v6AssignMode{ 6plane,rfc4193,zt:boolean}' for the definition.

That's good to know. It is better for me to shorten as much as possible since main json structure is a big one.

Thanks.

#81 Re: mORMot 1 » How to parse such json? » 2019-02-22 23:09:48

I tried below using Delphi 10.3.1 and all is good. Both serialization and de-serialization works.

type
  TTest = packed record
    plane: Boolean;
    rfc4193: Boolean;
    zt: Boolean;
  end;

  TMyType = packed record
    v6AssignMode: TTest;
  end;

const
  __TMyType = 'v6AssignMode{ 6plane boolean rfc4193 boolean zt boolean}';

procedure TForm1.Button1Click(Sender: TObject);
var
  Target: string;
  Json: string;
  TestType1: TMyType;
  TestType2: TMyType;
begin
  TestType1.v6AssignMode.plane := True;
  TestType1.v6AssignMode.rfc4193 := False;
  TestType1.v6AssignMode.zt := True;
  Json := RecordSaveJSON(TestType1, TypeInfo(TMyType));
  ShowMessage('Generated: ' + string(Json)); // Here I read JSON with "6plane" in it.

  Target := '{"v6AssignMode":{"6plane":false,"rfc4193":false,"zt":false}}';
  if RecordLoadJSON(TestType2, RawUTF8(Target), TypeInfo(TMyType)) then ShowMessage('OK') else ShowMessage('NOK');  // Here a json string with "6plane" in it correctly de-serialized
end;

Thank you.

#82 Re: mORMot 1 » How to parse such json? » 2019-02-22 21:16:24

Now, I have another challenge with json handling. I just had following json string. That is actually part of a bigger structure.

"v6AssignMode": {
  "6plane": false,
  "rfc4193": false,
  "zt": false
 }

This is last part of that bigger structure and I just stop when I saw that "6plane". My knowledge it is not possible to use a variable name starting with a number. I have to serialize and de-serialize that structure.

I wonder if there is a better solution other than using a StringReplace() and hope not to break anything else in that json?

Thanks & regards,
Ertan

#83 Re: mORMot 1 » How to parse such json? » 2019-02-22 21:11:05

Below code did parse above json nicely.

var
  ParsedValues: TStringDynArray;
begin
  if DynArrayLoadJSON(ParsedValues, Pointer(Json), TypeInfo(TStringDynArray)) <> nil then ShowMessage('Parse OK');
end;

#84 mORMot 1 » How to parse such json? » 2019-02-22 18:07:05

ertank
Replies: 7

Hello,

Is there a way in mORMot to parse a json like below?

["9af5fdb57d837c02","9af5fdb57dc8e575","9af5fdb57d4fdb61"]

When I check in online tools it resolves as a valid json.

Thanks & regards,
Ertan

#85 Re: mORMot 1 » Help with json de-serialization » 2019-02-22 16:52:54

Thanks. That did solve my problem.

Is there a version of fpc which has record RTTI?

#86 mORMot 1 » Help with json de-serialization » 2019-02-22 13:40:16

ertank
Replies: 3

Hello,

I am using Lazarus 2.0.1, fpc 3.2.0, mORMot commit 1.18.5014. Platform is Raspberry Pi.

I have a pretty simple json, but I cannot handle it for a reason I don't know. I will try to keep my code examples minimum.

// json I am dealing is:
{
	"controller": true,
	"apiVersion": 4,
	"clock": 1550842219493,
	"databaseReady": true
}

type
  TController = packed record
    controller: Boolean;
    apiVersion: Integer;
    clock: UInt64;
    databaseReady: Boolean;
  end;

function IsControllerUp(): Boolean;
var
  Controller: TController;
  Json: string;
  J: RawUTF8;
begin
  Result := CallURL('controller', Json);  // do rest request and retrieve result string

  if Result then
  begin
    J := RawUTF8(Json);
    if RecordLoadJSON(Controller, J, TypeInfo(TController)) then
    begin
      Exit(Controller.controller);
    end
    else
    begin
      Exit(False);
    end;
  end;
end;

IsControllerUp() function above returns false and while debugging I see that RecordLoadJSON() return false and record is not modified at all.

I did try several things like using a RawUTF8 variable ("J" in above code for that matter) instead of typecasting, double checked my record definition. I could not see a problem.

Any help is appreciated.

Thanks & regards,
Ertan

#87 Re: mORMot 1 » Delphi 10.3.1 mORMot Compile fails » 2019-02-17 08:12:26

ab wrote:

Chaa is right: the current trunk seems correct.

I guess Ertan version is outdated: there should be several SynCommons.pas in his IDE path, referring to a old one, not the latest.

Absolutely, For a reason I seem to copy a SynCommons.pas in projects directory. My bad. Sorry.

#88 mORMot 1 » Delphi 10.3.1 mORMot Compile fails » 2019-02-14 21:39:37

ertank
Replies: 3

Hello,

I am on version 1.18.5037 of library.

When I try to compile on Delphi 10.3.1 targetting Win32 I get following error:

[dcc32 Error] SynCommons.pas(54244): E2029 Expression expected but 'END' found

I read following lines in conditionals starting at SynCommons.pas:54236

    {$elseif defined(VER320)}'Delphi 10.2 Tokyo'
    {$elseif defined(VER320)}'Delphi 10.3 Carnival'

10.3 compiler version should read VER330 according to below link:
http://docwiki.embarcadero.com/RADStudi … r_Versions

Once I modify that version number I can compile SynCommons.pas just fine.

Can anyone confirm, please?

Thanks & regards,
Ertan

#89 Re: mORMot 1 » TJSONRecordTextDefinition.Create error text definition is not accurate » 2019-01-11 18:06:42

It turned out that I did not read whole section and missed information on how to define "array of record" part.

Defining text representation as following seems to work OK:

__TSearchCardResult = 'ret{code Integer desc string subReturn{code Integer desc string}} cardStatus Integer message string instructions[InstructionId Int64 Amount string InsertionDate string Description string]';

Sorry for the noise.

#90 mORMot 1 » TJSONRecordTextDefinition.Create error text definition is not accurate » 2019-01-11 17:45:37

ertank
Replies: 1

Hello,

I started to define text representations for all json record types in my project because service started to add new fields that are for service internal use (not needed for my application), but cause json deserialize to fail.

I failed to define a test representation for my following TSearchCardResult record below:

  TSubReturn = packed record
    code: Integer;
    desc: string;
  end;

  TReturn = packed record
    code: Integer;
    desc: string;
    subReturn: TSubReturn;
  end;

  TInstruction = packed record
    InstructionId: Int64;
    Amount: string;
    InsertionDate: string;
    Description: string;
  end;
  TInstructions = TArray<TInstruction>;

  TSearchCardResult = packed record
    ret: TReturn;
    cardStatus: Integer;
    message: string;
    instructions: TInstructions;
  end;

I tried to use following test representation:

const
  __TSubReturn = 'code Integer desc string';
  __TReturn = 'code Integer desc string subReturn{code Integer desc string}';
  __TInstruction = 'InstructionId Int64 Amount string InsertionDate string Description string';
  __TSearchCardResult = 'ret TReturn cardStatus Integer message string instructions TInstruction';
  // I also tried below that raise same exception for me
  //__TSearchCardResult = 'ret{code Integer desc string subReturn{code Integer desc string}} cardStatus Integer message string instructions {InstructionId Int64 Amount string InsertionDate string Description string}';

And tried to register it as following in OnCreate() event of the form they are used:

  TTextWriter.RegisterCustomJSONSerializerFromText(TypeInfo(TSubReturn),         __TSubReturn).Options        := [soReadIgnoreUnknownFields];
  TTextWriter.RegisterCustomJSONSerializerFromText(TypeInfo(TReturn),            __TReturn).Options           := [soReadIgnoreUnknownFields];
  TTextWriter.RegisterCustomJSONSerializerFromText(TypeInfo(TInstruction),       __TInstruction).Options      := [soReadIgnoreUnknownFields];
  TTextWriter.RegisterCustomJSONSerializerFromText(TypeInfo(TSearchCardResult),  __TSearchCardResult).Options := [soReadIgnoreUnknownFields];

When I run the application I get subject exception displayed. Says RTTI size is 28 bytes but text definition is 44 bytes.

I do not know where I made a mistake in the text representation.

Any help is appreciated.

Thanks & regards,
Ertan

#91 Re: mORMot 1 » How to generate same json as C# does » 2018-12-28 20:18:02

igors233 wrote:

Take a look at soReadIgnoreUnknownFields in 10.1.3.2.4 at online documentation.
Also good alternative for reading external json may be using TDocVariantData instead of records.

I have learned from igors233 that there is a way to make de-serialization more flexible.

I have below very simple and small record definition as an example

  TCommand = packed record
    cmd: string;
    name: string;
  end;

Recently, application that uses this record has slightly different json as a response because REST server changed due to company take over. New owner added some additional fields for their own use. These additional fields will not be used by my application in any way. All my json parsing fails now. For above definition new json response is as below.

{"cmd":"905A00000342220100","data":null,"name":"iso cmd"}

This is one small sample record off the application and I have a lot of different records with different additional fields in them. I wanted to set a single option for global JSON de-serialization for that specific project to ignore unknown fields. I could not see such an option myself. "soReadIgnoreUnknownFields" is mentioned only in 10.1.3.2.4 of online documentation. I tried to set something without using RegisterCustomJSONSerializerFromText because I did not want to additionally define text representation of all my record definitions.

initialization

TTextWriter.RegisterCustomJSONSerializerSetOptions(TypeInfo(TCommand), [soReadIgnoreUnknownFields]);

end.

I could not make that work for myself. With above code, parsing still fails for me.

My questions are:
1- Is there any option that I can set globally to ignore unknown fields during de-serialization? (I do very much appreciate such an option myself)
2- If above RegisterCustomJSONSerializerSetOptions is the shortest way to go without additional text representation definition for each record, I appreciate a small code example, please.

Thanks & regards,
Ertan

#92 Re: mORMot 1 » Decrypt OpenSSL library crypt data » 2018-12-21 10:14:40

ab wrote:

You have a padding issue I guess.
See how padding works - in a nutshell, AES works on 16 bytes blocks, so you need to do something with your last bytes.

In your Aes.Decrypt() code, you uncypher 1000 bytes... this is not the way to do it!

Use a Aes method using padding algorithm, e.g. PKCS7.

Exactly, changing one line and I can read same text on Windows.

Utf8String := Aes.DecryptPKCS7Buffer(Pointer(MyBytes), Length(MyBytes), False);

Thank you.

#93 mORMot 1 » Decrypt OpenSSL library crypt data » 2018-12-21 09:30:11

ertank
Replies: 2

Hello,

I have to decrypt data which is crypt using OpenSSL library AES-CBC on a Linux system. Decryption will be on Windows. Plain text encrypt on Linux is: "The quick brown fox jumps over the lazy dog"

Below is my code for decryption:

function DecryptItAESCBC(const s: string; AKey, IV: Array of Byte; out Value: string): Boolean;
var
  Aes: TAESCBC;
  LocalIV: THash128;
  Utf8String: RawByteString;
  MyBytes: TBytes;
  I: Integer;
begin
  if s = EmptyStr then Exit(False);
  for I := Low(LocalIV) to High(LocalIV) do LocalIV[I] := IV[I];
  Aes := TAESCBC.Create(AKey, 256);
  try
    Aes.IV := LocalIV;
    SetLength(Utf8String, 1024);
    try
      MyBytes := TNetEncoding.Base64.DecodeStringToBytes(s);  // I confirm bytes are identical to crypt data on other end
      Aes.Decrypt(Pointer(MyBytes), Pointer(Utf8String), 1024);
    except
      Value  := EmptyStr;
      Exit(False);
    end;
    Value := UTF8ToString(Utf8String);
  finally
    Aes.Free();
  end;
  Result := True;
end;

For test purpose simple Key and IV used. My code to call above function is:

procedure TForm1.Button1Click(Sender: TObject);
var
  Key: TBytes;
  IV: TBytes;
  I: Integer;
  Plain: string;
begin
  SetLength(Key, 32);
  SetLength(IV, 16);

  for I := Low(Key) to High(Key) do Key[I] := I;  // same Key used to crypt on other end
  for I := Low(IV) to High(IV) do IV[I] := I;  // same IV used to crypt on other end
  if not DecryptItAESCBC(TFile.ReadAllText('Z:\b64crypt'), Key, IV, Plain) then
  begin
    Memo1.Lines.Add('Cannot decrypt!');
    Exit();
  end;
  Memo1.Lines.Add('Plain text: ' + AnsiQuotedStr(Plain, '"'));
end;

What I read in memo is: "The quick brown fox jumps over the lazy dog#5#5#5#5#5'n'#$14#$E'9'" (without double quotes)

I could not find my mistake. I tested decryption using OpenSSL on Linux and it works just fine. On Windows, I read above 9 more characters and most of them are invisible.

Any help is appreciated.

#94 Re: mORMot 1 » Delphi-Android to access mORMot server » 2018-12-02 09:06:40

I checked examples. They handle json file as a table.

I suppose I will "pull" from srver json, save it in a file and process?

I could not see how I can make a "pull" from such a server of mine. Is there any such examples for FMX platform?

#95 mORMot 1 » Delphi-Android to access mORMot server » 2018-12-01 12:43:16

ertank
Replies: 2

Hello,

I have a server that I use for interface based access. My windows VCL project connects to the server as below:

  Client := TSQLHttpClientWebsockets.Create(SERVER, PORT_NAME, Model);
  TSQLHttpClientWebsockets(Client).WebSocketsUpgrade('BF67CA3DCE1B473ABC568A25BD41DA7A');  // real value is different

I have looked at examples for Cross Platform. I could not see how I can connect my above server. TSQLHttpClientWebsockets seems not defined in CrossPlatform units. If I include mORMotHttpClient unit that leads to compilation error in SynZip.pas file.

It is possible I missed something. I just do not know what it is.

I appreciate any help.

Thanks.

#96 Re: mORMot 1 » Delphi 10.3 - Linux platform support » 2018-12-01 08:24:49

I didn't know that client side is already supported for Delphi nextgen compiler.
Thanks for the link.

#97 Re: mORMot 1 » Delphi 10.3 - Linux platform support » 2018-11-30 11:37:06

ab wrote:

Embarcadero told that ARC will remain for Android and iOS.

Actually, this is what is written in blog:
"The go-forward plan is to not embrace ARC for the coming macOS 64-bit platform (so all desktop platforms are and will remain on the non-ARC model) and to disable ARC also for mobile platforms in the future (most likely around the next major release after 10.3)."

So, current plan is to remove ARC from all supported platforms. That will probably be introduced in Delphi 10.4 version.

ab wrote:

And for those mobile platforms, you have the client-side code generator available, which is enough for a SOA client.
Wanting to put more than just a dump client in a mobile app is IMHO a wrong design. For instance, no SQL storage in the mobile app - just cache!

I was wondering to use Andriod/iOS for client connection to mORMot server. Sorry, that was not clear.

#98 Re: mORMot 1 » Delphi 10.3 - Linux platform support » 2018-11-30 11:10:43

I am more a newbie to mORMot. I do not feel like I am up for such a task, yet. I once tried to adapt mORMot for compiling under WinCE and could not manage that.

On the other hand; Delphi seemingly will use same traditional memory management for all of the platforms it supports including Android and iOS platforms. If that happen, is there any chance that mORMot will support these platforms using Delphi?

I am probably asking that question a couple years early, it is also possible Delphi change its approach to memory management, but just wondering.

Thanks.

#99 mORMot 1 » Delphi 10.3 - Linux platform support » 2018-11-24 23:22:10

ertank
Replies: 11

Hello,

ARM memory management is not used for Linux platform in Delphi 10.3.

I wonder if that change can get mORMot to support Delphi 10.3 Linux platform.

Thanks & regards,
Ertan

#100 Re: mORMot 1 » TAESCFB.EncryptPKCS7() raise SIGABRT when used in a thread » 2018-08-03 21:07:07

After reading below post, I installed a fresh fpc and Lazarus from trunk sources and problem disappeared.
https://synopse.info/forum/viewtopic.ph … 846#p27846

Sorry for the noise.

Thanks.
Ertan

Board footer

Powered by FluxBB