Interface-based services: Receiving a list of objects

Beertjie · 2018-09-14 14:15:24

I have an interface-based service on my mORMot REST server, to which I'm submitting JSON parameters via an AJAX call.

Currently, the method takes a record with a property that is an array of other records. I'd like to replace those records with classes, but I can't get it to work.

Adding the "ClassName" property in my JSON doesn't seem to help.

Is this something that is supported by the framework?

Example of what I'm trying to do
For example, if my method takes a parameter of type TOrderItem, e.g.

TOrderItem = record
  ...
end;

TOrder = record
  Description: string;
  Items: array of TOrderItem;
end;

This allows me to post JSON to it such as:

{
  "Order": {
    "Description": "Test",
    "Items": [{...}, {...}, {...}]
  }
}

That works 100%. But now I need to replace those records with objects, e.g.

TOrderItem = class(TPersistent)
private
  ...
published
  ...
end;

What have I tried?
An array of objects, e.g.

property Items: array of TOrderItem

TOrderItems = array of TOrderItem;

TOrder = record
  ...
  Items: TOrderItems;
end;

A TObjectList, e.g.

property Items: TObjectList;

Subclassing TInterfacedCollection, e.g.

TOrderItem = class(TCollectionItem)
  ...
end;

TOrderItemCollection = class(TInterfacedCollection)
protected
  class function GetClass: TCollectionItemClass; override;
end;

TOrder = record
  ...
  Items: TOrderItemCollection;
end;

What do I mean "it doesn't work"?
Most of the time, my JSON is rejected by the framework (so my service method is never hit).

If I leave the object list properties out in my JSON, the object passed to my service method has its list properties set to nil.

mpv · 2018-09-29 19:45:37

Why you need classes there? If you need methods you can add it to records. In any case using of ANY TPersistent descendant magically turns your multi-thread application into single thread one. Take a look at global lock inside TPersistent.Destroy. I mORMot we use TSynPersistent instead (but I'm not sure this solve your problem)

ab · 2018-09-29 23:37:33

Use TSynPersistent and store lists as dynamic arrays using TJSONSerializer.RegisterObjArrayForJSON() for automatic serialization.

Nice picture by the way.
2421.png?m=1536934742

Beertjie · 2018-09-30 13:07:44

Thank you so much. Automatic serialization works now.

ab wrote:

Use TSynPersistent and store lists as dynamic arrays using TJSONSerializer.RegisterObjArrayForJSON() for automatic serialization.

It turns out TJSONSerializer.RegisterObjArrayForJSON() was the key. I never knew about this method!

Sadly, with two decades' worth of technical debt, my application is already single-threaded. Nevertheless, I am happy to switch to TSynPersistent.

mpv wrote:

Why you need classes there? If you need methods you can add it to records.

I forgot to mention that I'm stuck in Delphi 6 for this project, so no methods on records.

Having gotten this far...
Is there any way to make this work with polymorphic arrays?

Edit:
For example, if my various types of "order items" have vastly different sets of properties:

{
  "Order": {
    "Description": "Test",
    "Items": [
      {
        "Class": "TOrderItemA",
        "Field1": "..."
      },
      {
        "Class": "TOrderItemB",
        "Field2": "...",
        "Field3": "..."
      },
      {
        "Class": "TOrderItemC",
        "Field4": "..."
      }]
  }
}

Then I'd prefer to have this:

TOrderItemA = class(TOrderItem)
published
  property Field1: string ...
end;

TOrderItemB = class(TOrderItem)
published
  property Field2: string ...
  property Field3: string ...
end;

TOrderItemC = class(TOrderItem)
published
  property Field4: string ...
end;

Otherwise I'll end up with a flat class like this:

TOrderItem = class(TSynPersistent)
published
  property Field1: string ...
  property Field2: string ...
  property Field3: string ...
  property Field4: string ...
end;

Which, in my real scenario, gets very unhandy very quickly.

Last edited by Beertjie (2018-09-30 13:33:53)

AntonE · 2018-10-12 09:00:33

TDocVariant might simplify your app?

Beertjie · 2018-10-15 17:14:25

Thank you for the suggestion, AntonE.

In fact, I have switched to TDocVariantData! This does give me much more flexibility, although time will tell whether it's simplifying or just bloating my code base.

I wouldn't know how to apply this to an interface-based service, but my app has since transformed from an HTTP-based service to a queue-based one, so I did away with that part anyway.

ab · 2018-10-16 07:02:15

TDocVariant are working just great with interface-based services: just define "variant" parameters.

edwinsn · 2018-10-16 12:21:56

Beertjie wrote:

...but my app has since transformed from an HTTP-based service to a queue-based one...

@Beertjie, would love to know some details about the queue-based architecture, if you can tell!
I has just recently discovered the 'queue' concept is so powerful and useful in a multi-thread environment, so I'm very curious about it

ab · 2018-10-16 12:27:25

FYI you have a simple but thread-safe queue in SynCommons.pas, named TSynQueue.
It is efficient, and allows optional persistence of the stored items, as JSON or binary, since it is TDynArray-based.

edwinsn · 2018-10-16 15:44:15

@ab, thanks, I discovered TSynQueue, but I wish I noticing it earlier, otherwise I won't have to use an alternative

And the ability to persist the queued items is really really an added value !

Beertjie · 2018-10-17 09:44:49

ab wrote:

TDocVariant are working just great with interface-based services: just define "variant" parameters.

That simple? Thanks, I'm sure that will come in handy in the future.

edwinsn wrote:

@Beertjie, would love to know some details about the queue-based architecture, if you can tell!
I has just recently discovered the 'queue' concept is so powerful and useful in a multi-thread environment, so I'm very curious about it

I'm only starting to play around with queues myself. My legacy application is slow and non-reentrant so I'm having it process requests from a queue rather than calling it directly via an API (currently using RabbitMQ as my message broker).

Last edited by Beertjie (2018-10-17 09:45:23)

edwinsn · 2018-10-17 10:58:38

@Beertjie,
Thanks for the details, I haven't used RabbitMQ before, however, I have the following custom-made, queue-based architecture in mind, utilizing TSynQueue and web-sockets provided by mORMot, how does it compare to your solution?

On the server side:
- expose a mORMot-based API function for the clients (also based on mORMot) to submit requests, and all the submitted requests won't be process immediately but instead, they will be put into a queue (based on TSynQueue), so the clients won't be blocked.
- there will be one (or multiple) worker thread(s) to process the requests in the queue one by one, when done, the corresponding client will be notified using mORMot's web-sockets-based interface.

I'm sure engines like RabbitMQ must have their advantages, for sure, but I'm still want to know how my idea compares, it's upside is that it's written in pure Delphi/Pascal

Beertjie · 2018-10-17 12:08:28

That's pretty much exactly the pattern I am following, except that my API is written in C#, and my queue lives on my RabbitMQ server.

The idea is to scale this component by starting up more Delphi worker instances as needed, which I believe is the same thing you're describing.

edwinsn · 2018-10-17 14:05:42

Great to hear that! mORMot is so powerful

ab · 2018-10-17 14:23:23

We use a similar mechanism to scale some process to hundredths (soon thousands) of processing servers.

Only using polling instead of WebSockets, and regrouping the commands to one call per node for a small period of time (e.g. 100ms).
The fact that we use polling (and a request/process ID) with DoSomethingStart and DoIsSomeThingFinalized calls with several process ID at once allows a real stateless communication: a server could be down or isolated from the network for a while, or even restarted, and we don't suffer from the callback to disappear.
We use WebSockets for one-to-one communication, with a well defined lifetime of the session.

edwinsn · 2018-10-17 14:55:09

Great! It seems that polling has its own advantages - easy to implement and simple connection life-time management.

edwinsn · 2018-10-31 06:09:04

Google recently introduced , it's well explained how it works here:

Cloud Tasks

I believe with mORMot we can definitely implement a similar queue-based architecture!

mORMot Open Source

#1 2018-09-14 14:15:24

Interface-based services: Receiving a list of objects

#2 2018-09-29 19:45:37

Re: Interface-based services: Receiving a list of objects

#3 2018-09-29 23:37:33

Re: Interface-based services: Receiving a list of objects

#4 2018-09-30 13:07:44

Re: Interface-based services: Receiving a list of objects

#5 2018-10-12 09:00:33

Re: Interface-based services: Receiving a list of objects

#6 2018-10-15 17:14:25

Re: Interface-based services: Receiving a list of objects

#7 2018-10-16 07:02:15

Re: Interface-based services: Receiving a list of objects

#8 2018-10-16 12:21:56

Re: Interface-based services: Receiving a list of objects

#9 2018-10-16 12:27:25

Re: Interface-based services: Receiving a list of objects

#10 2018-10-16 15:44:15

Re: Interface-based services: Receiving a list of objects

#11 2018-10-17 09:44:49

Re: Interface-based services: Receiving a list of objects

#12 2018-10-17 10:58:38

Re: Interface-based services: Receiving a list of objects

#13 2018-10-17 12:08:28

Re: Interface-based services: Receiving a list of objects

#14 2018-10-17 14:05:42

Re: Interface-based services: Receiving a list of objects

#15 2018-10-17 14:23:23

Re: Interface-based services: Receiving a list of objects

#16 2018-10-17 14:55:09

Re: Interface-based services: Receiving a list of objects

#17 2018-10-31 06:09:04

Re: Interface-based services: Receiving a list of objects

Board footer