Posts by ertank

ertank · mORMot 1

After calculations, I get 1.92KB per item average memory usage. It was around 5KB before.
That would suffice to continue with Win32 application.
Since increase is slow, I will figure an alternative solution in the future.
Thanks.

ertank · mORMot 1

ab wrote:

TDocVariant loads everything into memory, before you can iterate into its values.

Yes, I am going to make measurements. Getting ready for it.
I count on dvoInternNames, dvoInternValues parameters. I will see if it will be enough to keep memory very low as names identically repeat and there are values that repeat a lot.

ertank · mORMot 1

ab wrote:

1. I have changed the BOM support in mORMot 2, and now you can use Doc.InitJsonFromFile() with a BOM file.
Even if BOM is not a good idea for UTF-8...
InitJsonInPlace() is indeed low-level.

I do really appreciate that. Thank you. I also think BOM is not a good idea, but you eventually find it out of your control here and there.

ab wrote:

2. I guess you can use dvoInternNames safely, but only dvoInternValues if most JSON values are likely to reappear in the input JSON.

Number values indeed repeat a lot like 1 and 2 only and that is for multiple different values.

ab wrote:

3. If TDocVariant has troubles with the JSON, and if the structure is really simple, I would rather try to use a dynamic array of records for the storage.
It would be more efficient in terms of memory, and also performance. And using a dynamic array and records would be much easier to work with.

That file immediately goes into database for each item read. I will not keep its data in memory for any other purpose. So far TDocVariant seems to work just fine.

ertank · mORMot 1

My mistake again, I assumed InitJSONInPlace returns Boolean, but it does not.

  if Doc.InitJSONInPlace(Pointer(AnyTextFileToRawUtf8('users.json')), [dvoInternNames, dvoInternValues]) = nil then
  begin
    ShowMessage('Json file could not be loaded');
    Exit();
  end;

Now, I will go back to parsing and measure memory usage on my test data.

Thanks.

ertank · mORMot 1

igors233 wrote:

Try TDocVariantData.InitJSONInPlace(AnyTextFileToRawUtf8(FilePath));

I could not make compiler happy. InitJSONInPlace require PUTF8Char and AnyTextFileToRawUtf8 returns RawUTF8. However, compiler says

[dcc32 Error] Unit1.pas(44): E2010 Incompatible types: 'PUtf8Char' and 'UTF8String'

I tried several other conversion and explicitly using pointers. But, all of them lead to a compiler error.

ertank · mORMot 1

Hello,

Problem was users.json file has BOM and so cannot be read by TDocVariantData.InitJsonFromFile() If I read it using TFile.ReadAllText('users.json', TEncoding.UTF8) it is just fine. However, reading file content in a text variable and then using it to Init TDocVariantData feels like additional memory usage especially on a considerably large file.

This file is received as ZIP format from a web service and I cannot change the way it is saved. Is there any option for TDocVariantData.InitJsonFromFile() that can possible make it work with files including BOM?

Thanks & Regards,
Ertan

ertank · mORMot 1

I've never used TDocVariantData and I cannot find my way thru. Below is my current test code, but even if it is an json of array, code thinks otherwise and exits at Doc.IsArray check.

uses
  mormot.core.variants, mormot.core.data;

procedure TForm1.Button1Click(Sender: TObject);
var
  Doc: TDocVariantData;
  I, TempInteger: Integer;
  TempString: string;
begin
  Doc.InitJsonFromFile('users.json', [dvoInternNames, dvoInternValues]);
  if not Doc.IsArray then Exit();
  repeat
    TempString := Doc.GetValueOrEmpty('Identifier');
    TempInteger := string('0' + Doc.GetValueOrEmpty('AppType')).ToInteger();
  until (not Doc.Items.MoveNext);
end;

Can someone put me in right direction, please?

Thanks & Regards,
Ertan

ertank · mORMot 1

Hello,

I have a well defined structured relatively big (195MiB) json file to parse. Using Delphi System.JSON unit it will likely crash on Win32 exe because average single structure needs around 5KB in memory. The file has more than 750.000 records and slowly increasing.

I could find about 5 threads in forum about "SAX". I could not find anything in documents. I believe (correct me if I am wrong) mORMot and very much likely mORMot2 has SAX json parsing.

I do appreciate an example on how to use SAX parsing, please.

BTW, I prefer mORMot2.

Thanks & Regards,
Ertan

ertank · mORMot 1

A good reference for future.

Thank you.

ertank · mORMot 1

This was what I have been looking for.
Both name and unit change and I failed to find it in mORMot2 sources.

Thank you.

ertank · mORMot 1

Hi,

I have an old project that I need to make some changes and additons. I am using mORMot 1 in that project for json handling and now I want to switch using mORMot2.
But, I could not find any example project or some information (even on this forum) to adapt my code. It is more than possible I overlooked at it.

There is ThirdPartyDemos\martin-doyle\03-MethodBasedServices example project which has very basic json handling and that is all I could find.

I do not know if having just "mormot.core.json" is enough in uses list.
TTextWriter.RegisterCustomJSONSerializerFromText() is something that I use in old project and I could not see how to use/convert it to mORMot2.
I am also not quite sure how to serialize/de-serialize arrays and regular json.

Any help is appreciated.

Thanks & Regards,
Ertan

ertank · mORMot 1

Hello,

I was more asking about (2) "IVAtBeginning" implementation. This is the part I could not figure.

ertank · mORMot 1

Hello,

I am using Delphi 10.4.2, targeting Android32.

There is a previously developed REST web service with MARS Curiosity project which is using mORMot encryption routines. That web server running on Windows and is just fine. Now, I have to develop a small mobile application that is consuming some methods on that web service. Unfortunately, I failed to compile below function on Android32. I get error for SynLZ unit for its asm instructions. It uses SynCommons and SynCrypt units on Win32 platform.

function EncryptItAES(const s: string; const AKey: TBytes): string;
var
  Key: TSHA256Digest;
  Aes: TAESCFB;
  _s: RawByteString;
begin
  Result := EmptyStr;
  Key := SHA256Digest(Pointer(AKey), Length(AKey));
  Aes := TAESCFB.Create(Key, 256);
  try
    _s := StringToUTF8(s);
    _s := BinToBase64(Aes.EncryptPKCS7(_s, True));
    Result := UTF8ToString(_s);
  finally
    Aes.Free();
  end;
end;

1) I wonder if I use mORMot2 encryption routines under Android32 and write similar function to above?
2) I can use alternative AES encryption library for Android32. I just don't know how above code each time produce different encrypt data for same input. I appreciate instructions on how to develop same logic using AES library that compiles under Android32? I am confused by checking sources to figure this out.

Thanks & Regards,
Ertan

ertank · mORMot 1

Nevermind, just figured, right after I post that there is a space character in raw JSON string for

"isSelected "

Cost me a lot of time until I could actually see that.

Sorry for the noise.

ertank · mORMot 1

Hello,

I am using Delphi 10.3, mORMot SynopseCommit.inc value '1.18.6244'

I could not figure why I get FAILED! message for following simple JSON string

{
  "optionUuid": "57202ea4-cc62-4eff-bbea-fa5be41449ee",
  "optionName": "Tuzsuz",
  "optionPrice": 5.2,
  "isSelected ": true
}

Using a code like below:

  TOrderProductNote = packed record
    optionUuid: string;
    optionName: string;
    optionPrice: Double;
    isSelected: Boolean;
  end;

procedure TForm2.Button1Click(Sender: TObject);
var
  Response: TOrderProductNote;
  TempString: string;
begin
  TempString := '{"optionUuid": "57202ea4-cc62-4eff-bbea-fa5be41449ee","optionName": "Tuzsuz","optionPrice": 5.2,"isSelected ": true}';
  if not SynCommons.RecordLoadJSON(Response, RawUTF8(TempString), TypeInfo(TOrderProductNote)) then
    ShowMessage('FAILED!')
  else
    ShowMessage('OK');
end;

My debugging, I see de-serialization fails on "isSelected", but I could not understand why.

Any help is appreciated.

Thanks & Regards,
Ertan

ertank · mORMot 1

Hello,

I tried to search in the forum without any success.

I receive a JSON string as following:

"lastPriceChangeDate": 1616569735683,
"lastStockChangeDate": 1618009761506,
"lastUpdateDate": 1618009761506,

This is a Unix Timestamp in milliseconds format.

Can I directly de-serialize that kind of values into to a TDateTime variable using mORMot?

Thanks & Regards,
Ertan

ertank · mORMot 1

Hello,

I am trying to make an integration with an online food sale chain. They have kind of weird design. I am getting a JSON (part of it) as below:

                "optionCategories": [
                    {
                        "optionCategory": "5dd28622e0450549407168f4",
                        "name": {
                            "tr": "1. Tantuni Tercihi",
                            "en": "1. Tantuni Tercihi"
                        },
                        "options": [
                            {
                                "option": "5dd28622e045052fb27168f6",
                                "product": "5dd28614c424f0658e8cedb8",
                                "name": {
                                    "tr": "Yarım Ekmek Et Tantuni",
                                    "en": "Yarım Ekmek Et Tantuni"
                                },
                                "price": 0,
                                "optionCategories": [
                                    {
                                        "optionCategory": "5dd28614c424f010da8cedb9",
                                        "name": {
                                            "tr": "Porsiyon Tercihi",
                                            "en": "Porsiyon Tercihi"
                                        },
                                        "options": [
                                            {
                                                "option": "5dd28614c424f05bd58cedbc",
                                                "name": {
                                                    "tr": "1 Porsiyon",
                                                    "en": "1 Porsiyon"
                                                },
                                                "price": 0
                                            }
                                        ]
                                    }
                                ]
                            }
                        ]

I could not figure how I should be preparing my record structure in order to be able to de-serialize something like that. I am actually stuck where "optionCategories" is becoming recursive. And there is another part where it is not going recursive. So, it is not always it is recursive, but sometimes. Here is an example for not recursive part (this also comes in identical array):

                    {
                        "optionCategory": "5dd28622e04505a7b37168ee",
                        "name": {
                            "tr": "İçecek Tercihi",
                            "en": "İçecek Tercihi"
                        },
                        "options": [
                            {
                                "option": "5dd28622e045052a3f7168ef",
                                "product": "5dd2861f35e9bba43d19b039",
                                "name": {
                                    "tr": "Eker Ayran (300 ml)",
                                    "en": "Eker Ayran (300 ml)"
                                },
                                "price": 0
                            }
                        ]
                    }

My current record structure (part of it) is not working and is as following:

  TName = packed record
    tr: string;
    en: string;
  end;

  TOption = packed record
    option: string;
    product: string;
    name: TName;
    price: Double;
  end;
  TOptions = TArray<TOption>;

  TOptionCategory = packed record
    optionCategory: string;
    name: TName;
    options: TOptions;
  end;
  TOptionCategories = TArray<TOptionCategory>;

I really do not want to parse whole JSON manually.

I appreciate any help.

Thanks & Regards,
Ertan

ertank · mORMot 1

That worked just fine.
Pointer(TestBytes) generates identical hash.

Thank you.

ertank · mORMot 1

Hello,

I am using Delphi 10.3.3, Synopse commit version 1.18.6045, generating Win32 executable.
I had some legacy code that I am modifying. I used to pass parameter as array of byte and now I am changing it into TBytes. However, I am getting different result for same byte values. My test code is as following:

uses
  SynCrypto;

const
  TestArray: Array [0..5] of Byte = (1, 2, 3, 4, 5, 6);
  TestBytes: TBytes = [1, 2, 3, 4, 5, 6];


procedure TForm1.Button1Click(Sender: TObject);
var
  Sha2561: TSHA256Digest;
  Sha2562: TSHA256Digest;
begin
  Sha2561 := SHA256Digest(Pointer(@TestArray), SizeOf(TestArray));
  Sha2562 := SHA256Digest(Pointer(@TestBytes), Length(TestBytes)); // For TBytes I must use Length, SizeOf always returns 4 
  if not CompareMem(@Sha2561, @Sha2562, 32) then
    ShowMessage('different')
  else
    ShowMessage('identical');
end;

For above test, I get different message displayed.

I am not sure if I am doing something wrong. I could not remember why I did write a code like

Pointer(@TestArray)

That code is possibly a copy-paste from this forum, years ago. In any way, using it as

@TestArray

also displays different message for me.

Any help is appreciated.

Thanks & Regards,
Ertan

ertank · mORMot 1

Missed that point in samples.
Thanks for clarifying.

ertank · mORMot 1

Hello,

I am using Delphi 7 and latest available version of Synopse from GitHub. Just removed all SynCommons.pas files from my system and have just a single copy.

Registering a custom json text writer fails for me. Details are as following:

type
  TTicker = packed record
    symbol: RawUTF8;
    highPrice: Double;
    lowPrice: Double;
  end;
  TTickers = Array of TTicker;

const
  __TTickers = '[symbol RawUTF8 highPrice Double lowPrice Double]';


initialization
  TTextWriter.RegisterCustomJSONSerializerFromText(TypeInfo(TTickers), __TTickers);
end;

I am getting error saying:

TTicker text definition is not accurate, or the type has not been defined as packed record: RTTI size is 20 bytes but text definition covers 0 bytes

I cannot see where I am doing it wrong.

Any help is appreciated.

Thanks & regards,
Ertan

ertank · mORMot 1

Thanks for all suggestions.

Actually, I did take very basic precautions like try..except block in my code. I did not put it all here in my earlier post.

...
try
  V := _Json(RetJsonString);
except
  // do what needs to be done
end;
...

After reading other replies, I also changed my code to retrieve data like following:

var
  Amount: Integer;
  LStPayment: TVariantDynArray;
begin
  LStPayment := _Safe(V.stPayment).Values;
  for Idx := 0 to High(LStPayment) do
  begin
    if LStPayment[Idx].typeOfPayment = 4 then
    begin
      Amount := LStPayment[Idx].StBankPayment.Amount;
    end;
  end;
end;

Which IMHO is easier to compared to my initial version. However, I am not quite sure this code above is free from exceptions if there is no StBankPayment exists in LStPayment (even though it should exists because payment type 4 means credit card payment) I am asking just in case. Because even rarely it does happen when company does an update to device software and breaks produced json in a way.

I appreciate if someone can confirm/deny.

Regards,
Ertan

ertank · mORMot 1

Seems I need to use array access as following:

...
V := _Json(RetJsonString);
...
var
  Amount: Integer;
  AType: Integer;
begin
  if V.StPayment._(0).typeOfPayment = 4 then
  begin
    Amount := V.StPayment._(0).StBankPayment.Amount;
  end;
end;

Problem solved.

Thanks.

ertank · mORMot 1

That did indeed work.

ertank · mORMot 1

I gave up and follow igors233 suggestion. As I do not know how to use TDocVariantData. I did read documents but could not comprehend them fast enough. So, I went down using Variant.

However, I could failed to read information I am looking for from sub objects in incoming json. For regular record filled with data I could write something like:

var
  Amount: Integer;
  AType: Integer;
begin
  if Rec.StPayment[0].typeOfPayment = 4 then
  begin
    Amount := Rec.StPayment[0].StBankPayment.Amount;
  end;

Now, I load json into a Variant named V. Try to write a code as below:

...
V := _Json(RetJsonString);
...
var
  Amount: Integer;
  AType: Integer;
begin
  if V.StPayment[0].typeOfPayment = 4 then
  begin
    Amount := V.StPayment[0].StBankPayment.Amount;
  end;

That compiles fine, but fails at the if statement at run-time.

I appreciate some examples as to how I can extract json data directly from a variant.

Thanks & regards,
Ertan

ertank · mORMot 1

I do not know details of RTTI. However, below code works for the first one and not for the second one. I do get "json2 OK" displayed on the screen. That is without using any of suggested 3 options above.

const
  Json = '[{"u32VAT":15,"u32Amount":100,"u16VATPercentage":1800},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":0},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":800},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":100},' +
         '{"u32VAT":0,"u32Amount":0,"u16VATPercentage":2400},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":0},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":0},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":0}]';
  Json2 = '{"u32VAT":15,"u32Amount":100,"u16VATPercentage":1800}';

procedure TForm3.Button1Click(Sender: TObject);
var
  Taxes: TStVATDetails;
  Tax: TStVATDetail;
begin
  if SynCommons.RecordLoadJSON(Tax, RawUTF8(Json2), TypeInfo(TStVATDetail)) then
    ShowMessage('json2 OK')
  else
    ShowMessage('json2 failed');

  if not SynCommons.DynArrayLoadJSON(Taxes, RawUTF8(Json), TypeInfo(TStVATDetails)) then
  begin
    ShowMessage('Cannot deserialize Json!');
  end
  else
  begin
    ShowMessage('Json deserialization OK');
  end;
end;

ertank · mORMot 1

There is an overloaded version of DynArrayLoadJSON in SynCommons.pas. My code supposed to use it

function DynArrayLoadJSON(var Value; const JSON: RawUTF8; TypeInfo: pointer): boolean;
var tmp: TSynTempBuffer;
begin
  tmp.Init(JSON); // make private copy before in-place decoding
  try
    result := DynArrayLoadJSON(Value,tmp.buf,TypeInfo)<>nil;
  finally
    tmp.Done;
  end;
end;

ertank · mORMot 1

Hello,

I am using Delphi 10.3.3, Synopse is updated from github. I double check that I am using latest version of SynCommons.pas

I have a simple json string (json array) that I fail to find why I cannot de-serialize. I simply fail to find a reason.

I do appreciate any help.

Record definition:

  TStVATDetail = packed record
    u32VAT: Integer;
    u32Amount: Integer;
    u16VATPercentage: UInt16;
  end;
  TStVATDetails = Array of TStVATDetail;

Json array that I cannot de-serialize:

[{"u32VAT":15,"u32Amount":100,"u16VATPercentage":1800},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":0},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":800},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":100},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":2400},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":0},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":0},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":0}]

My code for de-serialization:

const
  Json = '[{"u32VAT":15,"u32Amount":100,"u16VATPercentage":1800},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":0},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":800},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":100},' +
         '{"u32VAT":0,"u32Amount":0,"u16VATPercentage":2400},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":0},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":0},{"u32VAT":0,"u32Amount":0,"u16VATPercentage":0}]';

procedure TForm3.Button1Click(Sender: TObject);
var
  Taxes: TStVATDetails;
begin
  if not SynCommons.DynArrayLoadJSON(Taxes, RawUTF8(Json), TypeInfo(TStVATDetails)) then
  begin
    ShowMessage('Cannot deserialize Json!');
  end
  else
  begin
    ShowMessage('Json deserialization OK');
  end;
end;

ertank · mORMot 1

Hello,

I have a problem de-serializing a json string. After I get false from de-serialization function, I see some variables have value in my record.

In order to find my mistake, I would like to understand how mORMot doing de-serializing order.

For example I have below json string:

{"name":"test","surname":"test"}

And my record definition is:

type
  TPerson = packed record
    surname: string;
    name: string;
  end;

When de-serializing, is it name or surname that is filled in record variable first?

Thanks & regards,
Ertan

ertank · mORMot 1

I do really would like to hear if somebody can re-produce issue I am facing.

ertank · mORMot 1

I could complete a small project and I observe same problem in that stand-alone project.

Project can be downloaded from here: https://fil.email/qYgVeiEo
P.S.: Download valid for 1 month only.

FYI, I am using Synopse from GitHub. However, I am using SVN client to get sources. I see my copy is from 26th January, 2020. There is note "Merge pull request #268 from LongDirtyAnimAlf/master fix compilation of Alpine Linux."

ertank · mORMot 1

I am going to work on a small project to re-produce (if I can re-produce) and either post project or a feedback here. That is in a few days.

Thanks.

ertank · mORMot 1

In provided test json stLoyaltyService is null. However, when I put its text definition in the middle, it is not empty but filled in with A letters I suppose. There are other problems as well. For example, UserData variable is not filled in even there is a value in the json string for it.
Please see: https://imgur.com/a/lsjTnsM

ertank · mORMot 1

ab wrote:

Note that stLoyaltyService[...] defines a nested ARRAY [..] of records.

I realize that I did not share all necessary info. Sorry about that.

I indeed have this record as an array of records in my definition:

  TStLoyaltyServiceInfo = packed record
    name: Array[0..23] of Byte; // original was this. I changed to "Array of byte" after having access violation errors 
    CustomerId: string;
    ServiceId: UInt16;
    u16AppId: UInt16;
    CustomerIdType: UInt16;
    TotalDiscountAmount: UInt32;
  end;
  TStLoyaltyServiceInfos = Array of TStLoyaltyServiceInfo;
...
  TStTicket = packed record
...
    stLoyaltyService: TStLoyaltyServiceInfos;  // [0..MAX_LOYALITY_TRANS_NUMBER]
  end;

So, I believe my definition of "array of records" matches in my text definition. It is just changing position other than last most in the definition leads to a memory leak in my code. Maybe it is my code to blame I do not know for sure. Just trying to understand.

I also tried to use TByteDynArray and if definition is not latest, I still get access violation.

My complete text definition is here: https://pastebin.com/u27beRi9 (Where I put it at the end)
Text definition that gives access violation is here: https://pastebin.com/kACw9Y1z (Where it is defined in the middle)
And if you check both above that is what I mean by "in the middle" or "at the end" or "bottom most (last)"

Formatted json string is here: https://pastebin.com/9NxWvx4v
De-Serialized (single line in text file) json string is here: https://pastebin.com/wQsqjtVQ
Above two are same except one is formatted by online website.

Complete (hopefully I did not forget anything) record definitions are here: https://pastebin.com/uyb2uaam

ertank · mORMot 1

That QWord works for UInt64. Thank you.

I have one thing I would like to understand though.

I have below sub-record. If I move that record definition up in my text definition, that gets me memory corruption (access violation) after de-serialization before exiting procedure and corruption goes away if put it bottom most (last) in my text definition.

  TStLoyaltyServiceInfo = packed record
    name: Array of Byte;
    CustomerId: string;
    ServiceId: UInt16;
    u16AppId: UInt16;
    CustomerIdType: UInt16;
    TotalDiscountAmount: UInt32;
  end;

For above record I have following text definition:

stLoyaltyService[name Array of Byte CustomerId string ServiceId Word u16AppId Word CustomerIdType Word TotalDiscountAmount Cardinal]

For the record, stLoyaltyService is NULL in my de-serialized json string. So, basically it should not have any items at all. But, I see some items are created after de-serialization in Local variable view *if* it is in the middle but not at the end in my text definition.

It is possible I am missing something here?

Thanks & regards,
Ertan

ertank · mORMot 1

Hello,

I am using Delphi 10.3.3. SynCommit is 1.18.5394

I have a rather complex json and corresponding record definitions. I was happily using it as it is. This is used to communicate with a physical POS device. Recently, other part has changed something in *some* of their devices. They actually removed some of their not used json pairs. This new version of theirs is going to spread all devices in the end. Unfortunately, this process is slow. So, I cannot just convert my records to match new json string or old version running devices will fail.

I decided to use TTextWriter.RegisterCustomJSONSerializerFromText() and put in just the necessary values in both json strings and be on my way. Unfortunately, I got below exception;

Unregistered ptCustom for TJSONRecordTextDefinition.AddItem(ItemPrice: UINT64)

Until I came to this one, I did already converted some Int16, Int32 to their corresponding variable types in my constant string. It is just Int64 seems not have any alias at all.

1- Is it possible to include it in so that above exception does not raise?
2- Is there any workaround that I can use?

Thanks & regards,
Ertan

ertank · htm2pdf

Hello,

I am searching for a way to print html files I have in certain order and without any user dialog intervention.

I thought easiest might be to convert them in to pdf and search ways of printing pdf without any user intervention. I am interested in free solutions.

I have found this htm2pdf tool. Tried it and I have below problems:
1- picture embedded in html is not visible.
2- text seems not UTF-8.
3- This maybe UTF-8 again, but even empty table cells are filled with some characters

Html source can be found in here: https://controlc.com/233e0de2
Generated PDF view can be found in here: https://files.fm/u/7ea5fdzn

I wonder if there is any way to fix these problems.

Thanks & regards,
Ertan

ertank · mORMot 1

Hello,

I am using Delphi 10.3.3.

I probably used wrong selection of words using google. SynCommons.pas seems not having that function.

I am searching for a faster running code than current System.SysUtils.StrToDate().

There is below function in my code base and when I used a profiler, it turned out to be time consuming function in the application (most likely because mos of the calls are raising exception I think but not sure).

function IsDate(const DateString: string): Boolean;
begin
  try
    StrToDate(DateString);
    Result := True;
  except
    Result := False;
  end;
end;

So, I am searching for a faster executing code than above.

ertank · mORMot 1

ab wrote:

I have just make it more obvious in the comments/documentation.
And introduced a new overloaded DynArrayLoadJSON() function which will make a copy of the input buffer.
See https://synopse.info/fossil/info/bd63059e2f

I like new overload function more because of its Boolean return makes it easier to understand if result is a success or not.

Thank you.

ertank · mORMot 1

It works as suggested.

Interesting. I have legacy code about 3 years old working just fine using Pointer() with earlier versions of mORMot:

DynArrayLoadJSON(Taxes, Pointer(sRaw), TypeInfo(TStTaxRatesString));

I cannot say I am good at pointers. I wonder why one works and not the other?

ertank · mORMot 1

Hello,

Using Delphi 10.3.2, targeting Win32 executable.

I have demo code at https://pastebin.com/wqdBqRmT

When I run it I get access violation in SynCommons.pas:55273

        D^ := P^; // 3 stages pipelined process of unescaped chars

Synopse commit version is: 1.18.5037

I get same error at same line (line no 54907 this time) using latest codes on GitHub, too. That code Synopse commit version is: 1.18.5448

I am not sure if it is me doing something wrong. I know that I can successfully de-serialize identical json string using an alternative record based library. I do want to use mORMot.

So, I am posting here to see if there is anything to check in mORMot.

Any help is appreciated.

Thanks & regards,
Ertan

ertank · mORMot 1

Are these changes waiting tests from users before going in to the main code base?

Thanks.

ertank · mORMot 1

I was wrong. It seems I also need to de-serialize such json data. Another problem might be that Sometimes they won't be empty at all. Example response from web service documentation:

{
   "ackCode":"success",
   "responseTime":"10/11/2010 11:51:11",
   "timeElapsed":"1369 ms",
   "sales":{
      "sale":{
         "saleCode":"325479960001",
         "status":"Alıcı ürüne onay verdi",
         "productId":"32547996",
         "productTitle":"Fotograf makinesi için çanta",
         "price":"1",
         "cargoPayment":"Alıcı öder",
         "amount":"1",
         "endDate":"09/11/2010 15:30:08",
         "buyerInfo":{
            "username":"test_user2",
            "name":"Mehmet",
            "surname":"Yılmaz",
            "phone":"212 3456435",
            "address":"Eski Üsküdar Yolu Cad. Çayıryolu Sok. No:1 Partaş Center K:16",
            "district":{
 
            },
            "city":"İstanbul",
            "zipCode":{
 
            }
         },
         "thumbImageLink":"http://mcdn01.gittigidiyor.net/3254/tn9/32547996_tn9_0.jpg",
         "lastActionDate":"09/11/2010 15:38:55"
      }
   }
}

Above, district, zipCode are empty, but it is not always empty.

I am searching for an example where at least one of them is not empty, but so far failed to find one.

ertank · mORMot 1

Hello,

I just realize now that I need to serialize something like an empty json:

      "pageTemplate":"1",
      "description":"Kodak Z915 Fotograf Makinesi 10Mp 50X zoom",
      "startdate":{
  
      },
      "catalogId":{
  
      },
      "catalogDetail":{
  
      },
      "format":"S",
      "startPrice":{
  
      },
      "buyNowPrice":"240",
      "netEarning":{
  
      },
      "listingDays":"30",

Above, startdate, catalogId, catalogDetail, startPrice, netEarning are all what I call empty json. For the record, above example is part of a bigger json string.

My understanding from the documentation of the web service is that. I will never have the need to fill inside of startdate, catalogId etc.

I am not sure if mORMot can do that and how to serialize something like above with mORMot.

Any help is appeciated.

Thanks & regards,
Ertan

ertank · mORMot 1

Sorry for the mis understanding. After explanations I used following code in an empty project using Delphi:

...

  TTest = packed record
    abc: string;
  end;

const
  __TTest = '"@abc" string';

var
  Form1: TForm1;

implementation

{$R *.dfm}

uses
  SynCommons;

procedure TForm1.Button1Click(Sender: TObject);
var
  x: TTest;
  Json: RawUTF8;
begin
  x.abc := 'here I am';
  Json := SynCommons.RecordSaveJSON(x, TypeInfo(TTest));
  ShowMessage(string(Json));
end;

initialization
  TTextWriter.RegisterCustomJSONSerializerFromText(TypeInfo(TTest), __TTest);

end.

I see displayed message:

{"@abc":"here I am"}

Thanks for the help.

ertank · mORMot 1

I just need to generate (serialize) json string like in my example for this web service. Responds to requests from the web service seem to be fine (no @ sign in them) as to their documentation.

I maybe wrong. My understanding this commit is for deserialization. It is good to have anyway. Thank you for that.

I’m searching for a solution without manual inserts in json string after serialization. Well, as much as possible of course.

ertank · mORMot 1

Hello,

I have following web service format that I need to prepare for using a rest web service. This is for an online sales portal api. I do not have any chance of having it changed to something else.

      "specs":{
         "spec":[
            {
               "@required":"true",
               "@value":"Kullanılmış",
               "@name":"Durumu"
            },
            {
               "@required":"true",
               "@value":"10.0 - 11.9 MP",
               "@name":"Çözünürlük (Megapiksel)"
            },
            {
               "@required":"true",
               "@value":"Kodak",
               "@name":"Marka"
            }
         ]
      }

Above is just a part of a bigger json string. I also have some individual usage like:

"photos":{
         "photo":{
            "@photoId":"0",
            "url":"http://images.gittigidiyor.com/2941/KODAK-Z915-10MP-10X-ZOOM-HD-SIFIR-URUN__29416457_0.jpg"
         }
      }

These are json strings that I need to generate using records. I could not figure a solution myself.

Any help is appreciated.

Thanks & regards,
Ertan

ertank · mORMot 1

I suspect there is kind of a link with below fixed problem
https://synopse.info/forum/viewtopic.php?id=5015

ertank · mORMot 1

Hello,

I am using Lazarus 2.0.5, revision 61904M, fpc 3.2.0-beta revision 43036 on Raspberry Pi 4.

Today I installed everything on a fresh new system using fpcupdeluxe. Then I started testing my projects compilation for any missing components I may have forgotten. After compiling one of my projects, I tried to run it and I get an exception. Call stack is as following:

#0 GETENUMNAME(0xaee828, 6) at ../fpcupdeluxe/ccr/mORMot/SynCommons.pas:21101
#1 TOTEXT(OSARCH) at ../fpcupdeluxe/ccr/mORMot/SynCommons.pas:26054
#2 SETLINUXDISTRIB(0xf65238 'Raspbian GNU/Linux 10 (bu'...) at ../fpcupdeluxe/ccr/mORMot/SynCommons.pas:26242
#3 RETRIEVESYSTEMINFO at ../fpcupdeluxe/ccr/mORMot/SynCommons.pas:26320
#4 SYNCOMMONS_$$_init$ at :62578
#5 fpc_initializeunits at :0
#6 main at counter.lpr:20

It is GetEnumName() function in SynCommons.pas. This is commit version 1.18.5362 I am using.

Wondering if there is anything that I can do.

Thanks & regards,
Ertan

ertank · mORMot 1

Hello,

I found that there is an additional field at the very bottom of that several KB long json string. So, it is not a problem of fields changing their position in json string but an addition of a field.

Now I need to use that TTextWriter.RegisterCustomJSONSerializerFromText() not to have such a problem again.

Thanks & regards,
Ertan

mORMot Open Source

#1 Re: mORMot 1 » SAX parsing json » 2022-11-28 17:35:26

#2 Re: mORMot 1 » SAX parsing json » 2022-11-28 13:28:45

#3 Re: mORMot 1 » SAX parsing json » 2022-11-28 12:09:23

#4 Re: mORMot 1 » SAX parsing json » 2022-11-28 07:37:53

#5 Re: mORMot 1 » SAX parsing json » 2022-11-28 07:28:16

#6 Re: mORMot 1 » SAX parsing json » 2022-11-28 05:35:28

#7 Re: mORMot 1 » SAX parsing json » 2022-11-27 18:35:14

#8 mORMot 1 » SAX parsing json » 2022-11-27 06:33:32

#9 Re: mORMot 1 » mORMot2 - json operations » 2022-08-09 21:52:40

#10 Re: mORMot 1 » mORMot2 - json operations » 2022-08-09 08:25:17

#11 mORMot 1 » mORMot2 - json operations » 2022-08-06 14:03:53

#12 Re: mORMot 1 » AES on Android » 2021-10-15 08:02:35

#13 mORMot 1 » AES on Android » 2021-10-14 20:43:57

#14 Re: mORMot 1 » JSON De-serialization help needed » 2021-05-03 21:42:27

#15 mORMot 1 » JSON De-serialization help needed » 2021-05-03 21:37:56

#16 mORMot 1 » Json with datetime as unix timestamp » 2021-04-10 15:48:06

#17 mORMot 1 » Help needed with JSON » 2020-07-18 05:00:20

#18 Re: mORMot 1 » Problem with SynCrypto.SHA256Digest() » 2020-07-15 22:14:02

#19 mORMot 1 » Problem with SynCrypto.SHA256Digest() » 2020-07-15 21:11:24

#20 Re: mORMot 1 » Help needed with custom json text definition - Delphi 7 » 2020-06-23 09:51:01

#21 mORMot 1 » Help needed with custom json text definition - Delphi 7 » 2020-06-23 03:59:57

#22 Re: mORMot 1 » TTextWriter.RegisterCustomJSONSerializerFromText - UInt64 problem » 2020-03-05 19:45:55

#23 Re: mORMot 1 » TTextWriter.RegisterCustomJSONSerializerFromText - UInt64 problem » 2020-03-05 15:03:59

#24 Re: mORMot 1 » Json Array de-serialization problem » 2020-03-05 14:45:36

#25 Re: mORMot 1 » TTextWriter.RegisterCustomJSONSerializerFromText - UInt64 problem » 2020-03-05 13:53:13

#26 Re: mORMot 1 » Json Array de-serialization problem » 2020-03-05 11:33:53

#27 Re: mORMot 1 » Json Array de-serialization problem » 2020-03-05 10:08:26

#28 mORMot 1 » Json Array de-serialization problem » 2020-03-05 09:22:11

#29 mORMot 1 » Json de-serialization order » 2020-03-05 08:11:16

#30 Re: mORMot 1 » TTextWriter.RegisterCustomJSONSerializerFromText - UInt64 problem » 2020-02-29 19:17:20

#31 Re: mORMot 1 » TTextWriter.RegisterCustomJSONSerializerFromText - UInt64 problem » 2020-02-26 07:24:42

#32 Re: mORMot 1 » TTextWriter.RegisterCustomJSONSerializerFromText - UInt64 problem » 2020-02-25 14:26:29

#33 Re: mORMot 1 » TTextWriter.RegisterCustomJSONSerializerFromText - UInt64 problem » 2020-02-25 12:54:06

#34 Re: mORMot 1 » TTextWriter.RegisterCustomJSONSerializerFromText - UInt64 problem » 2020-02-25 10:34:15

#35 Re: mORMot 1 » TTextWriter.RegisterCustomJSONSerializerFromText - UInt64 problem » 2020-02-25 08:48:39

#36 mORMot 1 » TTextWriter.RegisterCustomJSONSerializerFromText - UInt64 problem » 2020-02-25 06:15:47

#37 Re: htm2pdf » htm2pdf free tool release » 2020-01-26 19:27:44

#38 mORMot 1 » Faster StrToDate() » 2019-11-21 21:48:43

#39 Re: mORMot 1 » DynArrayLoadJSON access violation » 2019-11-08 08:59:44

#40 Re: mORMot 1 » DynArrayLoadJSON access violation » 2019-11-07 09:02:39

#41 mORMot 1 » DynArrayLoadJSON access violation » 2019-11-06 14:59:12

#42 Re: mORMot 1 » Access violation with FPC 3.3.1 » 2019-10-28 22:14:03

#43 Re: mORMot 1 » Empty sub-json in a json » 2019-10-10 10:50:06

#44 mORMot 1 » Empty sub-json in a json » 2019-10-10 10:38:32

#45 Re: mORMot 1 » @ sign in json string » 2019-10-10 09:29:06

#46 Re: mORMot 1 » @ sign in json string » 2019-10-10 04:12:21

#47 mORMot 1 » @ sign in json string » 2019-10-09 19:26:48

#48 Re: mORMot 1 » Exception on Raspberry Pi 4 - SynCommons.pas » 2019-09-22 14:29:49

#49 mORMot 1 » Exception on Raspberry Pi 4 - SynCommons.pas » 2019-09-20 14:18:45

#50 Re: mORMot 1 » Json de-serialization problem due to position change » 2019-08-10 10:10:21

Board footer