RecordLoadJSON(): how to know what has changed in JSON?

mdbs99 · 2020-10-16 11:14:22

I'm consuming a third API using RecordLoadJSON() a lot with proper TTextWriter.RegisterCustomJSONSerializerFromText() to register and fill records/objects "automagically".

My issue is that they, eventually, change the API without notify us before, and it breaks the system. What I do is search API responses—using the log—to find out which has changes, which new fields they introduced, etc.

Is there a way for mORMot tells me which is different between the JSON response and my record(s) structure?

best regards.

ab · 2020-10-16 17:24:42

There is an option to let the parser ignore unknown fields IIRC, so it will work even if there some new fields in the returned JSON.

mdbs99 · 2020-10-16 17:43:28

I may set this option to make it safer, however I've already got change even in field types on the API.

I think this API is new and still under construction...

IIRC, TTextWriter.RegisterCustomJSONSerializerFromText() shows a nice error, if record/object do not match with its "data string configuration". It would be nice, if RecordLoadJSON() could do the same.

ab · 2020-10-16 17:48:33

If your API is not backward compatible, then you need to use a versioned API.
But even a new API can be backward compatible. You can add fields.
I don't think that something magic may happen in RecordLoadJSON.

Perhaps a record is not the best structure for you, something more versatile like a TDocVariant or a class with custom serialization (not published properties, but manual reader/writer methods) may be used.
Then, once it is stabilized, use records.

mdbs99 · 2020-10-16 22:05:14

If your API is not backward compatible, then you need to use a versioned API.

It's not my API, otherwise it wouldn't have this issue

I don't think that something magic may happen in RecordLoadJSON

"Roger that."

Perhaps a record is not the best structure for you, something more versatile like a TDocVariant or a class with custom serialization (not published properties, but manual reader/writer methods) may be used.
Then, once it is stabilized, use records.

I've coded the client that consume the API and it's written using records/objects all over the place—I can't change this now.

Thanks for try to help. If the API continues changing, I will continue investigating what has changed.

macfly · 2020-10-19 16:19:05

If you use a DocVariant you can do the validation.
There are methods that allow you to check if a property exists, extract the properties and check if there is a new one, etc.

You do not need to stop using the Records you already have. After validating you load in to Record.

You could also do this validation once a hour/day so as not to overload the system.

mdbs99 · 2020-10-19 22:42:32

macfly,
I'm thinking to do some like that. The issue is how do it in an automatically way. I don't want to test every field manually, using Exists(), at first.

However, if it isn't possible, I will code this in regression tests.

Thanks.

Last edited by mdbs99 (2020-10-19 22:43:05)

mORMot Open Source

#1 2020-10-16 11:14:22

RecordLoadJSON(): how to know what has changed in JSON?

#2 2020-10-16 17:24:42

Re: RecordLoadJSON(): how to know what has changed in JSON?

#3 2020-10-16 17:43:28

Re: RecordLoadJSON(): how to know what has changed in JSON?

#4 2020-10-16 17:48:33

Re: RecordLoadJSON(): how to know what has changed in JSON?

#5 2020-10-16 22:05:14

Re: RecordLoadJSON(): how to know what has changed in JSON?

#6 2020-10-19 16:19:05

Re: RecordLoadJSON(): how to know what has changed in JSON?

#7 2020-10-19 22:42:32

Re: RecordLoadJSON(): how to know what has changed in JSON?

Board footer