JSONToObject and TObjectDictionary

dc · 2014-05-19 12:30:57

Hi, I would like to know what is the best (most efficient) way to parse json containing a dictionary structure like this one:

{
  "24203": {
    "prop1": 0,
    "prop2": "test 24203"
  },
  "24202": {
    "prop1": 1,
    "prop2": "test 24202"
  },
  "24201": {
    "prop1": 2,
    "prop2": "test 24201"
  }
}

What I'd like to achieve is deserializing it directly into this structure:

type
   TMyClass = class
      prop1: integer;
      prop2: string;
   end;

   TMyDictionary = TObjectDictionary<string, TMyClass>;

var
   myDict: TMyDictionary;

   // and then use sth. like this
   JSONToObject(myDict, jsonStr, valid, TMyDictionary);

I know I have to use TJSONSerializer.RegisterCustomSerializer, but i'd like to know what is the best way to work with PUTF8Char type? I've seen some functions in SynCommons like GetJSON* but don't have a clear view of the parsing process yet. I would like to have a generic way to load TObjectDictionary<TKey,TValue> from json where TValue will be a simple class. Could you help me to get started?

dc

ab · 2014-05-19 13:27:28

Your JSON input is pretty weird.
Using an evolving integer value, changed into string, as key, is not common!

Are you sure you would not be able to use an array instead?
Something like that:

[{"name":"24203",
  "value": {
    "prop1": 0,
    "prop2": "test 24203"
  }},
  {"name": "24202",
   "value": {
    "prop1": 1,
    "prop2": "test 24202"
  }}, ...]

In all cases, with custom serialization, every layout is possible.

To fetch a property name, you can just call GetJSONPropName().
Then JsonToObject() for the value.

dc · 2014-05-20 10:06:22

ab wrote:

Your JSON input is pretty weird.
Using an evolving integer value, changed into string, as key, is not common!
Are you sure you would not be able to use an array instead?
In all cases, with custom serialization, every layout is possible.
To fetch a property name, you can just call GetJSONPropName().
Then JsonToObject() for the value.

It's not possible to change the json structure. Integer key as string in this case was used because of json limitations (or features) - you can not have a numerical property. It's just one of many examples, other dictionaries have real string keys. Anyway I've managed to write a generic parser and it works fast! Thank you very much for your help.

dc

ab · 2014-05-20 10:31:05

You could take a look at http://synopse.info/fossil/finfo?name=C … rmJSON.pas

I just introduced it, and it is fast, and cross-platform.

Of course, it is slower than our optimized versions in SynCommons/mORMot.pas, but it is already much faster than DBXJSON or SuperObject.

   JSON benchmarking
  -------------------


1. Small content

 1.1. Synopse record:
  - Read: 50,000 assertions passed  147.66ms  338,611/s
  - Access: 100,000 assertions passed  790us  63,291,139/s
  - Write: 50,000 assertions passed  98.08ms  509,777/s
  Total failed: 0 / 200,000  - Synopse record PASSED  247.09ms

 1.2. Synopse variant:
  - Read: 50,000 assertions passed  248.91ms  200,873/s
  - Access direct: 100,000 assertions passed  58.90ms  848,824/s
  - Access late binding: 100,000 assertions passed  194.35ms  257,257/s
  - Write: 50,000 assertions passed  119.86ms  417,139/s
  Total failed: 0 / 300,000  - Synopse variant PASSED  625.91ms

 1.3. Synopse cross platform variant:
  - Read: 50,000 assertions passed  681.91ms  73,323/s
  - Access direct: 100,000 assertions passed  34.19ms  1,462,073/s
  - Access late binding: 100,000 assertions passed  659.29ms  75,838/s
  - Write: 50,000 assertions passed  571.20ms  87,534/s
  Total failed: 0 / 300,000  - Synopse cross platform variant PASSED  1.94s

 1.4. Super object record:
  - Read: 50,000 assertions passed  4.10s  12,191/s
  - Access: 100,000 assertions passed  810us  61,728,395/s
  - Write: 50,000 assertions passed  3.55s  14,063/s
  Total failed: 0 / 200,000  - Super object record PASSED  7.66s

 1.5. Super object properties:
  - Read: 50,000 assertions passed  4.38s  11,398/s
  - Access: 100,000 assertions passed  3.88s  12,871/s
  - Write: 50,000 assertions passed  387.34ms  129,084/s
  Total failed: 0 / 200,000  - Super object properties PASSED  8.66s

 1.6. dws JSON:
  - Read: 50,000 assertions passed  279.64ms  178,798/s
  - Access: 100,000 assertions passed  76.09ms  657,038/s
  - Write: 50,000 assertions passed  208.42ms  239,899/s
  Total failed: 0 / 200,000  - dws JSON PASSED  565.98ms

 1.7. DBXJSON:
  - Read: 50,000 assertions passed  4.51s  11,070/s
  - Access: 100,000 assertions passed  50.60ms  988,122/s
  - Write: 50,000 assertions passed  661.31ms  75,607/s
  Total failed: 0 / 200,000  - DBXJSON PASSED  5.23s


2. Big content

 2.1. Depth content:
  - Download files if necessary: no assertion  412us
  - Synopse read variant: 1 assertion passed  95.11ms  525,707/s  337 KB
  - Synopse read to BSON: 2 assertions passed  2.63ms  18,989,745/s  155 KB
  - Synopse cross platform: 1 assertion passed  3.99ms  12,509,382/s  300 KB
  - Super object read: 2 assertions passed  8.78ms  5,694,112/s  529 KB
  - dws JSON read: 1 assertion passed  6.62ms  7,549,448/s  439 KB
  - DBXJSON read: 1 assertion passed  86.10ms  580,672/s  679 KB
  Total failed: 0 / 8  - Depth content PASSED  211.04ms

 2.2. Table content:
  - Download files if necessary: no assertion  427us  19,269,320/s
  - Synopse parse: 1 assertion passed  2.73ms  3,013,553/s  1.1 MB
  - Synopse ORM loop: 41,135 assertions passed  6.18ms  1,330,153/s  1.1 MB
  - Synopse ORM list: 41,135 assertions passed  6.47ms  1,270,775/s  952 KB
  - Synopse table direct: 41,135 assertions passed  18.45ms  445,690/s  1.1 MB
  - Synopse table variant: 41,135 assertions passed  21.63ms  380,281/s  1.1 MB
  - Synopse doc variant: 41,137 assertions passed  43.27ms  190,092/s  4.6 MB
  - Synopse late binding: 41,137 assertions passed  36.37ms  226,153/s  4.6 MB
  - Synopse crossplatform: 41,135 assertions passed  20.56ms  400,048/s  1.9 MB
  - Synopse to BSON: 2 assertions passed  9.34ms  880,269/s  1.1 MB
  - Super object properties: 41,136 assertions passed  2.20s  3,739/s  6.3 MB
  - Super object record: 41,136 assertions passed  145.01ms  56,732/s  6.3 MB
  - dws JSON: 41,136 assertions passed  32.05ms  256,628/s  4.7 MB
  - DBXJSON: 41,136 assertions passed  240.84ms  34,159/s  9.9 MB
  Total failed: 0 / 452,496  - Table content PASSED  2.80s

 2.3. Huge content:
  - Download files if necessary: no assertion  542us
  - Synopse read record: 4 assertions passed  1.44s  142,730/s  113.5 MB
  - Synopse read variant: 2 assertions passed  2.66s  77,511/s  512.9 MB
  - Synopse cross platform: 2 assertions passed  5.65s  36,523/s  424.8 MB
  - Synopse read to BSON: 3 assertions passed  2.10s  98,078/s  168.1 MB
  - Super object read: 2 assertions passed  9.26s  22,305/s  1.1 GB
  - dws JSON read: 2 assertions passed  3.38s  61,056/s  672.7 MB
  - DBXJSON read: no assertion  604us  82,781,456/s
     DBXJSON will raise EOutOfMemory for 185 MB JSON in Win32 -> skip
  Total failed: 0 / 15  - Huge content PASSED  25.18s


Generated with: Delphi XE6 compiler

Time elapsed for all tests: 53.14s
Tests performed at 20/05/2014 12:14:50

Total assertions failed for all test suits:  0 / 2,052,519
! All tests passed successfully.

mORMot Open Source

#1 2014-05-19 12:30:57

JSONToObject and TObjectDictionary

#2 2014-05-19 13:27:28

Re: JSONToObject and TObjectDictionary

#3 2014-05-20 10:06:22

Re: JSONToObject and TObjectDictionary

#4 2014-05-20 10:31:05

Re: JSONToObject and TObjectDictionary

Board footer