Switching Response Format (XML/JSON)

cheemeng · 2014-08-03 12:45:48

I understand ab's intentions in using JSON (speed, memory, etc), however there are client applications that can only process XML response.

In .Net's WebAPI framework, the response serializer automatically switches depending on the http Accept parameter (application/json, application/xml, text/json, text/xml).

Is there a way that we could do this in mORMot? Perhaps registering a new serializer/deserializer class? Or is returning TServiceCustomAnswer the only viable solution?

Thanks!

ab · 2014-08-03 13:48:53

JSON is implemented as transmission layer from the ground up (with UTF-8 encoding for text).
The the SAD 1.18 pdf about this choice.
For both ORM and SOA, the JSON content is generated directly by lowest-level of code.
So you can not "switch" from XML to JSON by changing a class.

But, as you stated, by using TServiceCustomAnswer you can return any content, including XML formatted documents.

Why on earth would you like to use XML today, if it is only use to transmit the same information as JSON?
Even a JSON parser and emitter would be easier to implement than a XML to JSON / JSON to XML translator.
Which kind of client applications can only process XML?

cheemeng · 2014-08-03 14:55:41

The kinds written in Java by people in the early 2000s.

Arnaud, does this mean there is no way to implement a 'switch' for response type?

Or perhaps I should create a helper function which looks for the Http Accept parameter if it exists, and return the response type accordingly? e.g.

function ReturnContent(Ctx, Model): TServiceCustomAnswer;
begin
  if AcceptParameter = 'xml' then
    Result.Content := XMLify(Model)
  else Result.Content := JSONify(Model);
end;

function TServiceDevice.GetStatus(const IP: string): TServiceCustomAnswer;
begin
  Model := GetDeviceStatus(IP);
  Result := ReturnContent(Ctx, Model);
end;

Or should I create a subclass of TServiceContainerClient to perform some 'magic'?

PS: By the way how do I 'subscribe' to a thread so that I get notified when someone replies? For instance even though I started this thread, I didn't get any e-mail notifications when you posted your reply.

ab · 2014-08-04 08:57:48

AFAIK Java does handle pretty well JSON content, even in old versions of the JVM.

XML to/from JSON conversion can be a bit non obvious.
For instance, the first online converters shown by Google are not able to convert the following JSON consistently:

[1,2,3]

Some converters fails to convert it, others generate <0>1</0><1>2</1><2>3</2>...
We will follow the 2nd pattern.

I found out that some of the converters are not able to convert back the result of their own conversion!

We will try to do better.

Yes, I'm currently adding XMLToJSON and JSONToXML functions to SynCommons.pas.

See newly added JSONToXML() JSONBufferToXML() and TTextWriter.JSONBufferToXML():
http://synopse.info/fossil/info/60541ac2a0ad

Then, I suspect it may be possible to put a global flag for TSQLRestServer, then let TSQLRestServer.URI() inspect the HTTP "Accept" header, and if the client does only accept XML and not JSON, convert JSON into XML directly.
In this case, JSONBufferToXML() will do the conversion in one step.

cheemeng · 2014-08-04 12:26:21

Thanks Arnaud!

Sometimes certain client applications are not within our control, and the vendor that did the original application insisted on using XML and not changing it.

Last edited by cheemeng (2014-08-04 12:57:15)

ab · 2014-08-04 14:13:06

Interface-based services can now return the result value as XML object instead of JSON array or object if TServiceFactoryServer.ResultAsJSONObject is set.
It can be useful e.g. when consuming services from XML only clients...
As an alternative, TServiceFactoryServer.ResultAsXMLObjectIfAcceptOnlyXML option will recognize 'Accept: application/xml' HTTP header (exact match) and return XML instead of JSON in this case

See http://synopse.info/fossil/info/5a89b5e46f

In conjunction with URI-encoded parameters - available with TSQLRestRoutingREST default routing scheme - it could be an useful alternative.
You can specify the input parameters named at the URI level, then let the server return XML content, as expected by your client.

Up to now, this XML alternative response type can NOT be consumed by mORMot.pas - so TServiceFactoryServer.ResultAsXMLObjectIfAcceptOnlyXML option should be used if you want the service to be accessible by both your XML clients and JSON clients (like regular Delphi or Ajax clients).

cheemeng · 2014-08-04 16:24:46

Arnaud could you add text/xml too?

From the RFC (3023), under section 3, XML Media Types:

If an XML document -- that is, the unprocessed, source XML document -- is readable by casual users, text/xml is preferable to application/xml. MIME user agents (and web user agents) that do not have explicit support for text/xml will treat it as text/plain, for example, by displaying the XML MIME entity as plain text. Application/xml is preferable when the XML MIME entity is unreadable by casual users.

Thank you once again!

ab · 2014-08-04 19:49:11

Now 'Accept: text/xml' will be recognized in addition to previous 'Accept: application/xml'.
See http://synopse.info/fossil/info/40b2c25 … 1798d76280

I've also updated the documentation, and will soon post a blog article about this feature.
Thanks for the feedback!

mpv · 2014-08-06 08:24:12

ab, to continue this good innovation I want to share my code able to serialize TSQLTableJSON to schemas-microsoft-com:rowset XML format (format used by MS Recordset).
We use this type of serialization to pass server result directly to MS application. Actually we build very powerfull Excell based client using this feature.
I send server-side code to You (can be easy integrated to mORMot).
Client side code part VBA(Excell, Word, e.t.c.) example (my authorization (GetAuthToken) function is not as in mORMot, so may be someone write pure mORMot example):

' in case of success return "WinHttp.WinHttpRequest.5.1" instanse
' in case of error - Nothing and put error message to ErrorMessage property
' Warning: do not forgot to set result to Nothing after usage
Private Function CustomRequest(aURL As String, aURLParams As String, aHTTPMethod As String, aBody)
  Dim lngTimeout
  Dim strUserAgentString
  Dim intSslErrorIgnoreFlags
  Dim blnEnableRedirects
  Dim blnEnableHttpsToHttpRedirects
  Dim objWinHttp
  Dim strURL As String
  
  Set CustomRequest = Nothing
  fErrorMessage = ""
  lngTimeout = 59000
  strUserAgentString = "http_requester/0.1"
  intSslErrorIgnoreFlags = 13056 ' Ignore Errors
  blnEnableRedirects = True
  blnEnableHttpsToHttpRedirects = True
  Set objWinHttp = CreateObject("WinHttp.WinHttpRequest.5.1")
  objWinHttp.SetTimeouts lngTimeout, lngTimeout, lngTimeout, lngTimeout
  
  strURL = Server & "/" & aURL & "?" & aURLParams & GetAuthToken
  
  On Error Resume Next
  objWinHttp.Open aHTTPMethod, strURL
  If Err.Number <> 0 Then
    fErrorMessage = "Error " & Err.Number & " " & Err.Source & " " & Err.Description
    Exit Function
  End If
  If aHTTPMethod = "POST" Then
    objWinHttp.SetRequestHeader "Content-type", "text/xml; charset=UTF-8"
  End If

  objWinHttp.Option(0) = strUserAgentString
  objWinHttp.Option(4) = intSslErrorIgnoreFlags
  objWinHttp.Option(6) = blnEnableRedirects
  objWinHttp.Option(12) = blnEnableHttpsToHttpRedirects

  objWinHttp.Send (aBody)
  If Err.Number = 0 Then
    If objWinHttp.Status = 200 Then
      Set CustomRequest = objWinHttp
    ElseIf objWinHttp.Status = 401 Then
      fSessionID = Empty ' in case of unauthorized request client must relogon
    Else
      fErrorMessage = "HTTP " & objWinHttp.Status & " " & objWinHttp.StatusText & " " & objWinHttp.ResponseText
    End If
  Else
    fErrorMessage = "HTTP " & objWinHttp.Status & " " & objWinHttp.StatusText & " " & objWinHttp.ResponseText
    If fErrorMessage = "" Then fErrorMessage = Err.Description
  End If
  On Error GoTo 0
  Set objWinHttp = Nothing
End Function

Private Function RecordsetFromStream(ByRef aStream) As recordset
    Set RecordsetFromStream = New ADODB.recordset
    RecordsetFromStream.Open aStream    'Open a recordset from the stream
End Function

Set req = CustomRequest("runList", "", "POST", postData)
If req Is Nothing Then
   Set rs = Nothing
Else
   Set rs = RecordsetFromStream(req.ResponseStream)
End If

ab · 2014-08-06 11:04:36

We have added TSQLTable.GetMSRowSetValues() methods, to return XML content in ADODB.recordset format.
I did not test the output, but sounds matching what you sent to me.
See http://synopse.info/fossil/info/99f74a1c78d

Thanks mpv and Vadim Orel for the input!

cheemeng · 2014-09-05 07:42:59

Hi Arnaud,

I would like to propose that the method JSONBufferToXML (SynCommons.pas, line 34345) accepts an optional ArrayName parameter that can be passed to AddJSONToXML. At the same time, perhaps the ArrayName can default to the Model's name, and may be set by a property of TSQLModel?

Thank you!

ab · 2014-09-05 15:55:14

Could you create a feature request ticket?

cheemeng · 2014-09-05 16:23:51

Done. Did I kill the formatting? http://synopse.info/fossil/tktview/1da1 … 1c04285598

mpv · 2014-10-02 07:15:19

Just for information - we moved to last mORMot and test TSQLTable.GetMSRowSetValues(). Everything works well, however, as always in AB code

esmondb · 2014-10-09 12:21:30

Could there be another switch, set by the HTTP header, to select between the expanded and non-expanded mORMot JSON formats?

I've been experimenting with javascript clients and it's not that difficult to expand the JSON response client side and this would give the benefit of the terser format for transmission.

mpv · 2014-10-10 05:47:14

I'ts not a problem to expand on client side:

function syn2JS(synJSON){
    var fc = synJSON.fieldCount, values = synJSON.values,
        len = values.length, fNum,
        index = fc, item, result = [];
    while (index < len){
        item = {}; fNum = -1;
        while(++fNum < fc){
            item[values[fNum]] = values[index];
            index++;
        }
        result.push(item);     
    }  
    return result;  
}

var synJSON = JSON.parse('{ "fieldCount":2,"values":["col1","col2","val11","val12","val21","val22"] }');
syn2JS(synJSON);

Last edited by mpv (2014-10-10 05:48:07)

esmondb · 2014-10-10 06:22:46

It would just be convenient to be able to select between expanded/unexpanded in the client request.

But both formats are fine. Here's a jquery plugin which detects the format and feeds it into an html table:

(function ( $ ) {

  $.fn.jsonToTable = function( JSONobj ) {
    var i, ii, rowCount, fieldCount, tableRow, fields,
    table = this;
    if (table.find("thead").length == 0) {
      table.append($("<thead></thead>").append("<tr></tr>"));
    }
    if (table.find("thead").find("tr").length == 0) {
      table.find("thead").append("<tr></tr>");
    }
    table.find("thead").find("tr").empty();
    if (table.find("tbody").length == 0) {
      table.append($("<tbody></tbody>"));
    }
    table.find("tbody").empty();
    if(JSONobj.hasOwnProperty("fieldCount")){
      fieldCount = JSONobj.fieldCount;
      for (i = 0; i < fieldCount; i++) {
        table.find("thead").find("tr").append("<th>"+JSONobj.values[i]+"</th>");
      };
      rowCount = JSONobj.rowCount;
      for (i = 0; i < rowCount; i++) {
        tableRow = $("<tr></tr>");//.addClass(rowClass);
        for ( ii = 0; ii < fieldCount; ii++) {
          tableRow.append($("<td>" + JSONobj.values[ii+((i+1)*fieldCount)] + "</td>"));
        };
        table.append(tableRow);
      };
    } else if (JSONobj.values.length > 0) {
      fields = Object.keys(JSONobj.values[0]);
      fieldCount = fields.length;
      rowCount = JSONobj.values.length;
      for (i = 0; i < fieldCount; i++) {
        table.find("thead").find("tr").append("<th>"+fields[i]+"</th>");
      };
      for (i = 0; i < rowCount; i++) {
        tableRow = $("<tr></tr>");//.addClass(rowClass);
        for ( ii = 0; ii < fieldCount; ii++) {
          tableRow.append($("<td>" + JSONobj.values[i][fields[ii]] + "</td>"));
        };
        table.append(tableRow);
      };
    }
    return table;
  };
  
}( jQuery ));

berndvf · 2015-01-28 07:03:11

ab wrote:

Yes, I'm currently adding XMLToJSON and JSONToXML functions to SynCommons.pas.

Hi Arnaud,

Was this ever implemented?

Thanks,
Bernd

ab · 2015-01-28 09:34:03

You have JSONToXML() JSONBufferToXML() and TTextWriter.JSONBufferToXML() methods available.

But the other direction, i.e. XMLToJSON(), can not be implemented easily, since XML does not match JSON, without a conversion scheme (e.g. how to convert attributes? how to store types?).
You can not easily convert SOAP XML into raw JSON, since the values may be nested in other nodes.

AQSANA56 · 2024-07-29 08:37:49

I am a new member here, but I just ask you to please let me know if can I discuss this on an old post. I would be very happy if you could let me know, thanks admin

ab · 2024-07-29 12:58:08

Welcome!

If it is about adding some new XML features, it is more likely to be in mORMot 2, so perhaps worth a new forum thread in the mORMot 2 thematic.
https://synopse.info/forum/viewforum.php?id=24

mORMot Open Source

#1 2014-08-03 12:45:48

Switching Response Format (XML/JSON)

#2 2014-08-03 13:48:53

Re: Switching Response Format (XML/JSON)

#3 2014-08-03 14:55:41

Re: Switching Response Format (XML/JSON)

#4 2014-08-04 08:57:48

Re: Switching Response Format (XML/JSON)

#5 2014-08-04 12:26:21

Re: Switching Response Format (XML/JSON)

#6 2014-08-04 14:13:06

Re: Switching Response Format (XML/JSON)

#7 2014-08-04 16:24:46

Re: Switching Response Format (XML/JSON)

#8 2014-08-04 19:49:11

Re: Switching Response Format (XML/JSON)

#9 2014-08-06 08:24:12

Re: Switching Response Format (XML/JSON)

#10 2014-08-06 11:04:36

Re: Switching Response Format (XML/JSON)

#11 2014-09-05 07:42:59

Re: Switching Response Format (XML/JSON)

#12 2014-09-05 15:55:14

Re: Switching Response Format (XML/JSON)

#13 2014-09-05 16:23:51

Re: Switching Response Format (XML/JSON)

#14 2014-10-02 07:15:19

Re: Switching Response Format (XML/JSON)

#15 2014-10-09 12:21:30

Re: Switching Response Format (XML/JSON)

#16 2014-10-10 05:47:14

Re: Switching Response Format (XML/JSON)

#17 2014-10-10 06:22:46

Re: Switching Response Format (XML/JSON)

#18 2015-01-28 07:03:11

Re: Switching Response Format (XML/JSON)

#19 2015-01-28 09:34:03

Re: Switching Response Format (XML/JSON)

#20 2024-07-29 08:37:49

Re: Switching Response Format (XML/JSON)

#21 2024-07-29 12:58:08

Re: Switching Response Format (XML/JSON)

Board footer