#1 Re: Low level and performance » Delphi doesn't like multi-core CPUs (or the contrary) » 2022-06-24 07:12:39

Bo

It seems a problem in TIdTCPServer component, in its OnExecute event handler, if I use ReadLn to wait for the pack starter,  in some cases (in my cases, hited by multiple packs at the same time and cut the connection in the middle etc.), it will keep firing OnExecute event even there is no connectioin at all (i.e., close the sender applications), change it from ReadLn to WaitFor seems can avoid the infinite loop:

        conn.IOHandler.ReadLn(mllp_header, -2 {IdTimeoutInfinite}, conn.IOHandler.MaxLineLength); ==> conn.IOHandler.WaitFor(mllp_header);

Thanks for being helping me.

#2 Re: Low level and performance » Delphi doesn't like multi-core CPUs (or the contrary) » 2022-06-24 03:25:06

Bo

I now can reproduce the issue on my dev PC, but there is something I don't understand. The new problem is that so far I can only reproduce it if I run the application via Windows Run, after it becomes 100% CPU0, if I attach Delphi debugger to it, CPU0 dropped immediately to normal and stay like that after I continue running it, and after attached to the debugger, I could not reproduce it any more, neither if I run the application start from debugger, so I am now have to add debugging string/log to trace it instead of debugging it within the debugger.

#3 Re: Low level and performance » Delphi doesn't like multi-core CPUs (or the contrary) » 2022-06-23 12:01:16

Bo

The customer's VM was created as 4 vCPUs in Azure, running CPU Get NumberOfCores, NumberOfLogicalProcessers /Format:List returns
    NumberOfCores=2
    NumberOfLogicalProcessors=4

Now I can reproduce on my own test VM which is half of customer's core, i.e., NumberOfCores=1, NumberOfLogicalProcessors=2.

With Process Explorer from Sysinternals tool set, I can see there are two threads using 100% of CPU0, even I suspend one of them the remained one still uses 100% CPU0.

Maybe next step is to see if I can reproduce it on a real PC.

#4 Re: Low level and performance » Delphi doesn't like multi-core CPUs (or the contrary) » 2022-06-21 01:02:27

Bo

Greate to know that recent CPUs have improved on this issue and that sounds correct as my application had been running okay for years in an on-premise VM until it moved to Azure VM and it started to have this 100$ CPU0 and non-responsive issue.
Is Azure VM not friendly to (Delphi) multi-thread application? Do you know if we need to do any optimization for Delphi application to run okay on Azure VM?

#5 Re: Low level and performance » Delphi doesn't like multi-core CPUs (or the contrary) » 2022-06-20 05:46:47

Bo

Hi ab,

I have an application compiled by Dephi 2007, it is a multi-thread application, but when it is busy, it seems only uses one core, i.e., CPU0 will be used 100% but the others are sitll very low usage, and thus the box (a Windows 2019 server VM) becomes no response, do you think this sympton (only one core is used 100%) is also related to the same LOCK issue in Delphi?

#6 Re: Source Code repository » Introducing enhanced logging mechanism » 2018-07-25 23:16:18

Bo
mpv wrote:

You can add some exceptions to be ignored to LogFamily.ExceptionIgnore list

Isn't that globally ignore that type of exception? But I just want to ignore in that function.

#7 Re: Source Code repository » Introducing enhanced logging mechanism » 2018-07-25 06:05:03

Bo

Hi AB,

How do I not to log the exception I explicitly want them to be silent somewhere but do not want this type of exception globally silent:

try
  TryLoadAsFormat1(data);
  // no exception, is format 1
  // do some thing here
except
// do nothing,  silent the exception, but the ISynLog will log this exception which I don't need
end;

Thanks,

#8 Re: mORMot 1 » How do I get every name:value pair from a Json? » 2017-08-30 02:21:48

Bo

I replace

    try
      LVal := TDocVariantData(LDoc.Values[i]); //conversion is failed when the value is an integer
    except
      LIdx := TSQLResourceIndex.Create;
      LIdx.JPath := AParentPath + '.' + LDoc.Names[i];
      LIdx.JValue := LDoc.Values[i];
      LPathValues.Add(LIdx);
      Continue;
    end;

with

      LVal := _Safe(LDoc.Values[i])^;   

and it seems working fine.

FYI, I then updated database (Sqlite) with the list of LIdx (TSQLResourceIndex), it took about 4~5 seconds to insert 35 records, I did not satisfy the performance but then I found the Batch updating in the performance sample, I switched to the batch updating and bang, it only took <200ms! Amazing, great work ab.

#9 Re: mORMot 1 » How do I get every name:value pair from a Json? » 2017-08-28 22:48:18

Bo

Hi ab,

Thanks for your reply.

I filled up JDoc by using _Json function and then cast  it to a TDocVariantData:

var
  v : Variant;
  l  : TList<TSQLResourceIndex>;
begin
  v := _Json('{"abc":123,"b":"c", "c": {"d":"e"}, "e": [1,2,3]}');
  l := GetPathAndValue(TDocVariantData(v));

#10 mORMot 1 » How do I get every name:value pair from a Json? » 2017-08-28 10:59:00

Bo
Replies: 5

Hi,

After I get the variant from _Json, how do I get every name:value pair down to the very most deep ones in the hierarchy of a Json?
For example:

var
  v : Variant;
begin
  v := _Json('{"abc":123,"b":"c", "c": {"d":"e"}, "e": [1,2,3]}');

then I want to list the name(Json path):value pair of the most deep ones (leaves in the hierarchy):

$.abc:123
$.b:c
$.c.d:e
$.e[0]:1
$.e[1]:2
$.e[2]:3

here is my code:

function GetPathAndValue( JDOC: TDocVariantData; AParentPath: string ='$'): TList<TSQLResourceIndex>;
var
  LPathValues: TList<TSQLResourceIndex>;
  LDoc : TDocVariantData;
  i, j: Integer;
  LVal : TDocVariantData;
  LIdx, LNewIdx : TSQLResourceIndex;
  LList : TList<TSQLResourceIndex>;
begin
  LDoc := JDOC;
  LPathValues := TList<TSQLResourceIndex>.Create;
  for i := 0 to LDoc.Count -1 do
  begin
    try
      LVal := TDocVariantData(LDoc.Values[i]); //conversion is failed when the value is an integer
    except
      LIdx := TSQLResourceIndex.Create;
      LIdx.JPath := AParentPath + '.' + LDoc.Names[i];
      LIdx.JValue := LDoc.Values[i];
      LPathValues.Add(LIdx);
      Continue;
    end;
    if LVal.Kind = dvObject then
    begin
      LList := GetPathAndValue(LVal, AParentPath+'.'+LDoc.Names[i]) ;
      for LIdx in LList do
      begin
        LNewIdx := TSQLResourceIndex.Create;
        LNewIdx.JPath := LIdx.JPath;
        LNewIdx.JValue := LIdx.JValue;
        LPathValues.Add(LNewIdx);
      end;
      LList.Free;
    end
    else if (Lval.Kind = dvArray) then
    begin
      for j := 0 to LVal.Count -1 do
      begin
        LList := GetPathAndValue(TDocVariantData(LVal.Values[j]),AParentPath+'.'+LDoc.Names[i]);
        for LIdx in LList do
        begin
          LNewIdx := TSQLResourceIndex.Create;
          LNewIdx.JPath := LIdx.JPath;
          LNewIdx.JValue := LIdx.JValue;
          LPathValues.Add(LNewIdx);
        end;
        LList.Free;
      end;
    end
    else begin
      LIdx := TSQLResourceIndex.Create;
      LIdx.JPath := AParentPath + '.' + LDoc.Names[i];
      LIdx.JValue := LDoc.Values[i];
      LPathValues.Add(LIdx);
    end;
  end;
  Result := LPathValues;
end;

GetPathAndValue(TDocVariantData(v));

Although above code does the job, but I was wondering if there is a better code for checking what is in the value, instead of doing

    try
      LVal := TDocVariantData(LDoc.Values[i]); //conversion is failed when the value is an integer
    except
    ...
    end;

#11 Re: mORMot 1 » Different query result of two equivalent queries » 2017-08-06 09:53:44

Bo
ab wrote:

The SQlite3 engine is expected to run in exclusive mode.
So if two processes (a mORMot server and SynDBExplorer) do access the very same database at the same time, the cache between the two is inconsistent.
This is as expected.

To consult the database in real time, expose the SQlite3 main mORMot instance using e.g. the mORMotDDD remote administration interface.

Don't know the "administration interface" yet, but I have made a little change to the explorer. Instead of generating "SELECT * ..." query when a table on the left list is double clicked, I make the double click to generate the complete list of the column in the SELECT query, like what SQL Studio does.

procedure TDBExplorerFrame.ListTableDblClick(Sender: TObject);
var
  i, j: integer;
  Fields: TSQLDBColumnDefineDynArray;
  LField: TSQLDBProcColumnDefine;
  LSQL : string;
begin
  i := ListTable.ItemIndex;
  if i>=0 then
  begin
    Props.GetFields(S2U(ListTable.Items[i]),Fields);
    LSQL := 'SELECT ' + Fields[0].ColumnName ;
    for j:=1 to Length(Fields) -1 do
    begin
      LSQL := LSQL + ', ' + Fields[j].ColumnName
    end;
    LSQL := LSQL + ' FROM ' + StringToUTF8(ListTable.Items[i]);

    AddSQL(UTF8ToString(LSQL),
      true,ssShift in fListTableShiftState);
  end;
end;

#12 Re: mORMot 1 » Different query result of two equivalent queries » 2017-08-05 12:53:02

Bo

Tested with data change (i.e., inserted records from out of SynDB Explorer), the "select * ..." query did returned new records, so it seems only the columns are cached some where. Where do I disable the caching for this situation?

#13 mORMot 1 » Different query result of two equivalent queries » 2017-08-05 09:37:29

Bo
Replies: 4

I created a class:

 TSQLIntegerDataTypes = class (TSQLRecord)
  private
    fByte : Byte ;   
  published
    property AByte: Byte read fByte write fByte;    
  end;

Ran the application and  a Sqlite db file created, opened SynDB Explorer and connect it to the db, executed  query "select * from IntegerDataTypes", I got column ID and AByte in the result.

While kept SynDB Explorer running, then I upgraded the class to add one more property as following code:

 TSQLIntegerDataTypes = class (TSQLRecord)
  private
    fByte : Byte ;
    fShortInt :  ShortInt;
  published
    property AByte: Byte read fByte write fByte;
    property AShortInt: ShortInt read fShortInt write fShortInt;
  end;

Compiled and ran the application again, the db was now updated, went to SynDB Explorer and executed the same query again, I got the same result without column AShortInt, but if I executed this equivalent query "select ID,AByte, AShortInt from IntegerDataTypes", I got the column AShortInt!

Is this a bug or a feature?

#14 Re: mORMot 1 » HttpGet failed in TestSQL3 » 2017-05-24 06:46:21

Bo

Tried today's Github's copy, still failed.

Checked Delphi's TNetHTTPClient and TNetHTTPRequest, they are using winhttp.dll as well.

I also try to call functions of WinHTTP directly:

procedure TDetailViewForm.Button1Click(Sender: TObject);
var
  hSession : HINTERNET;
  hConnect : HINTERNET;
  hRequest : HINTERNET;
  hResult : Boolean;
  dwSize : DWord;
begin
//  memo1.Text :=  HttpGet(Edit1.Text);
  hSession := WinHttpOpen('Mozilla/5.0 (Windows; mORMot 1.18 TWinHTTP)',WINHTTP_ACCESS_TYPE_NO_PROXY,'','',0);
  if (hSession=nil) then raise Exception.Create('Failed to create session');
  hConnect := WinHttpConnect(hSession,'shop.global-health.com',443,0);
  if (hConnect=nil) then raise Exception.Create('Failed to connect');
  hRequest := WinHttpOpenRequest(hConnect,'GET','/',nil,nil,nil,8388864);
  if (hRequest=nil) then raise Exception.Create('Failed to open request');
  if not WinHttpSendRequest(hRequest,nil,0,0,0,0,0) then
  begin
    raise Exception.Create(IntToStr(GetLastError()))
  end;
  hResult := WinHttpReceiveResponse(hRequest,nil);
  if not hResult then raise Exception.Create('Failed to receive data, error:' + inttostr(GetLastError()));

end;

the value of parameters to the function calls are copied from debuging HttpGet, these code has not failure with "WINHTTP_ACCESS_TYPE_NO_PROXY", but will get error 12029 when calling WinHttpSendRequest if change it to WINHTTP_ACCESS_TYPE_DEFAULT_PROXY.

I then change code in SynCrtSock (line 8266 of 24/5/2017) from

if fProxyName='' then
    OpenType := WINHTTP_ACCESS_TYPE_DEFAULT_PROXY else
    OpenType := WINHTTP_ACCESS_TYPE_NAMED_PROXY;

to

if fProxyName='' then
    OpenType := WINHTTP_ACCESS_TYPE_NO_PROXY else
    OpenType := WINHTTP_ACCESS_TYPE_NAMED_PROXY;

and it works for my machine.

But why? Is it a bug or I should change my system/network settings?

#15 Re: mORMot 1 » HttpGet failed in TestSQL3 » 2017-05-22 23:38:03

Bo

No proxy, but could be firewall as it runs OK outside of firewall, but what settings could be?

I then wrote a simple test application with RAD 10.2:

One form with two buttons, one button call mORMot's HttpGet, another one use TNetHttpRequest which comes with RAD 10.2,

code behind mORMot's button:   

procedure TDetailViewForm.Button1Click(Sender: TObject);
begin
  memo1.Text :=  HttpGet(Edit1.Text);
end;

code behind TNetHTTPRequest button: 

procedure TDetailViewForm.Button2Click(Sender: TObject);
begin
  NetHttpRequest1.URL := Edit1.Text;
  NetHttpRequest1.Execute();
end;

code for response from TNetHTTPRequest:

procedure TDetailViewForm.NetHTTPRequest1RequestCompleted(const Sender: TObject;
  const AResponse: IHTTPResponse);
begin
  Memo1.Text := AResponse.ContentAsString();
end;

Ran on a machine with URL "https://shop.global-health.com", mORMot button did not get response, but TNetHTTPRequest did. If URL changed to "http://shop.global-health.com", both worked.

If we bought an application developed in mORMot and it has this problem, what can our IT guy do to find out what issue could be in the company's firewall/network settings?

#16 mORMot 1 » HttpGet failed in TestSQL3 » 2017-05-22 03:54:14

Bo
Replies: 5

Strangely, all HttpGet failed in SynSelfTest unit when I ran TestSQL3 compiled with Berlin 10.2 Starter and Delphi 2007 Enterprise. The URL's corresponding json file needs to be delete to get the HttpGet line executed.

The error is "Project TestSQL3.exe raised exception class EWinHTTP with message 'winhttp.dll error 12029 (A connection with the server could not be established)'.

The URLs are all fine if open them with browser. Also tested URLs with PostMan and AdvancedREST Client, two of them need to specify header "User-Agent", otherwise all fine.

HttpGet seems working with "http" version of the URLs instead of "https" version.

Any pointer?

#17 Re: SyNode » Beta release of SyNode - ES6 JavaScript + NPM modules » 2017-02-12 12:22:47

Bo
mpv wrote:

Actual issue is inside mainForm.toLog function. Fixed by [70cc3773fb]

This fix does stop the AV, it would be nice if it can actual tell the script author that the variable passed in is undefined.

Also, I have noticed that there is another set of TSMEngineManager and TSMEngine etc in units SynSM, and they use Spider Monkey v24 instead of v45 in SyNode,
what are the main differences? What are the different using scenarios?

#18 Re: SyNode » Beta release of SyNode - ES6 JavaScript + NPM modules » 2017-02-09 11:46:06

Bo

Hi mpv,

Re: SyNode Sample 02, I have found that if the code was referring to a non-existed property of the main form (wrong spelling or capitalized etc.), it raised an generic AV which is not helpful at all, for example, this line of code "mainForm.toLog(mainForm.Caption);" will raise an AV in which does not tell  you the line no, what type of error etc., but "mainForm.ToLog(mainForm.caption);" does. What can we do to improve catching/avoid this type of error?

#19 Re: SyNode » Beta release of SyNode - ES6 JavaScript + NPM modules » 2017-02-09 10:40:49

Bo
mpv wrote:

@Bo In this case end users should program his scripts as a module - see how modules work
User script myCalculation.js:

const fs = require('fs')
...
function onInit(){ ....}
function onDone(){..}
function doCalculation(){...}
module.exports = doCalculation
// another way is to export a several methods
//  module.exports.onInit = onInit; module.exports.onDone = onDone; module.exports.doCalculation = doCalculation;

Your code

FEngine.Evaluate('require("myCalculation")()', 'eval.js', 1, res)

Modules (user code in your case) are wrapped in closure by require function - you can see this in debugger

This will be the equivalent of specifying an entry function in using MSScript.ocx as the script engine wrapper.

#20 Re: SyNode » Beta release of SyNode - ES6 JavaScript + NPM modules » 2017-02-08 22:31:20

Bo
mpv wrote:

There is no way to "clear" the module once it is loaded (like there is no way to "un-uses" unit in Delphi ). And normally you never need such.
Sample 02 execute only selected lines, so to read file twice (for example) in first time select end execute
...
In any case module required only once

The solution you provided is OK in this sample (i.e., interactive), what if the engine is pooled and its script is loaded from script files which are written by the end users (for example, an integration engine runs customized scripts for data processing)? How the end users should program the scripts to avoid "redefine" issue we are talking here?

#21 Re: SyNode » Beta release of SyNode - ES6 JavaScript + NPM modules » 2017-02-08 22:19:56

Bo
markzql wrote:

I also want know how to clear the moudles loaded, I just used it Free every times and Create a New TSMEngine...

How did you do it?

Initially, I used following code and it threw AVs:

// evaluate a text from mSource memo
  FEngine := FSMManager.ThreadSafeEngine(nil);
  try

    if mSource.SelText <> '' then
      FEngine.Evaluate(mSource.SelText, 'mSourceSelected.js', 1, res)
    else
      FEngine.Evaluate(mSource.lines.Text, 'mSource.js', 1, res);

  finally
    FreeAndNil(FEngine);   
  end;

Then I change the way it freed and it seemed OK now:

// evaluate a text from mSource memo
  FEngine := FSMManager.ThreadSafeEngine(nil);
  try

    if mSource.SelText <> '' then
      FEngine.Evaluate(mSource.SelText, 'mSourceSelected.js', 1, res)
    else
      FEngine.Evaluate(mSource.lines.Text, 'mSource.js', 1, res);

  finally
    //FreeAndNil(FEngine);   
    FSMManager.ReleaseCurrentThreadEngine();
  end;

#22 Re: SyNode » Beta release of SyNode - ES6 JavaScript + NPM modules » 2017-02-08 02:29:00

Bo

Hi mpv,

It seems that the script cannot be run more than once in "SyNode\Samples\02 - Bindings ", the error is "redeclaration of const fs.",
I guess the question is how to clear the script before next calling to FEngine.Evaluate?

#23 Re: synopse.info » Blog not reachable » 2017-02-05 13:03:56

Bo

Strange, the blog.synopse.info is not reachable from my desktop computer with all browsers, but it is reachable from my mobile phone with Chrome.

What should I look into to fix the desktop computer?

#24 Re: synopse.info » Blog not reachable » 2017-01-30 05:14:55

Bo

I was experiencing the same problem a few days ago, it was the Chrome and Edge that could not open the blog, IE was fine. I now can confirm that it works fine again on Chrome.

#25 mORMot 1 » A bug in SynDBExplorer » 2017-01-16 23:55:07

Bo
Replies: 1

Hi Ab,

There is a bug on line 275 in SynDBExplorerMain.pas. It should only free the connection if it is created for setting up a new connection, otherwise it throws an AV if an update connection is cancelled.

So the line should be something like

  if res=100 then
    FreeAndNil(C);

#26 Re: mORMot 1 » how to check whether server uses the same model » 2017-01-04 03:00:32

Bo

@EMartin,

I guess what juwo wanted to know was what happen if a client connect to a server on which was running on a different "Data Model" in which had slightly different TSQLRecords (different fields, different field type etc.). For example, the server and the clients were all on the version 1.0, and then the server was upgraded to 2.0, but not all of the clients were upgraded to 2.0, thus some of them would be using a different version of "Data Model", is there a way to avoid this situation or it has been guarded by the mORMot already?

#27 Re: mORMot 1 » Could not run Websocket Sample 31: Simple Echo Server » 2016-12-10 13:31:17

Bo

Another test case:
Running echo server on Windows 10, running browser locally, both Chrome and Edge are failed to connect, but IE can connect and gets echos.

Does Synopse Websocket only support IE?

#28 Re: mORMot 1 » Could not run Websocket Sample 31: Simple Echo Server » 2016-12-10 13:06:50

Bo

Here is another issue:

I ran the echo server on a Windows Server 2003 R2 Standard Edition, and ran the test html in both Chrome and Edge, got different result:
Chrome: could not connect at all.
Edge: the server reported it was connected, but not the browser, and then the connection dropped automatically after a while.

What can I check why the echo server cannot be connected? And why the behavior are different between Chrome and Edge?

#29 Re: mORMot 1 » Could not run Websocket Sample 31: Simple Echo Server » 2016-12-05 05:54:52

Bo
ab wrote:

Usually, this is a port issue.
The port you expect to bind is already used by some service or program.

but I believe this is not the case for me as I created the NodeJS echo server on the same port, if I forgot to stop NodeJS one, the sample echo server could not started until I closed the NodeJS one.

#30 Re: mORMot 1 » Could not run Websocket Sample 31: Simple Echo Server » 2016-12-04 10:39:35

Bo

Followed this link (http://cjihrig.com/blog/websockets-in-n … windows-7/) and created a websocket echo server by using Nodejs, it worked with the test html, so it looks like the Synopse sample echo server has something wrong, anybody else has encountered this before?

#31 Re: mORMot 1 » Could not run Websocket Sample 31: Simple Echo Server » 2016-12-03 13:41:44

Bo

Tried another machine, did work.

So the problem could be the machine's environment, but what could affect a websocket server? As the browser is working with other websocket echo server.

#32 mORMot 1 » Could not run Websocket Sample 31: Simple Echo Server » 2016-12-03 12:16:43

Bo
Replies: 8

Hi,

I am trying to run the sample "Project31SimpleEchoServer", but it seems not working.

Loaded the html, got following in the text area:

WebSocket - status 0
onerror - code:undefined, reason:undefined, wasClean:undefined, status:3
onclose - code:1006, reason:, wasClean:false, status:3


Checked it in the Developer Tools, got following error:

Project31SimpleEchoServer.html:18 WebSocket connection to 'ws://desktop-emq701g/' failed: HTTP Authentication failed; no valid credentials available
init @ Project31SimpleEchoServer.html:18
onload @ Project31SimpleEchoServer.html:72

What do I need to config to be able to run this sample? I did not see other people here reported this.

#33 Re: mORMot 1 » httpd.sys and SSL » 2016-11-15 11:06:49

Bo
erick wrote:

I have been struggling for hours trying to get HTTPD.SYS to work with SSL under Windows 10

...

I'm guessing it doesn't like the GUID?  People seem to just make up a GUID and it works for them.  Any suggestions?

Erick

The GUID does not play an important role at all, I made up one too. Most likely would be certificate's issue. I tried self-sign certificate by following the step's in the document but was not successful, then I reused the set of certificates from a production server and it worked immediately since then.

#34 Re: mORMot 1 » Implement RESTful with specification already designed » 2016-11-04 12:31:57

Bo

Hi ab,

I have read the framework document again regarding "send the parameters as a JSON object body" and response format, I believe that what you have suggested is doable. But I have not found how to return errors as http status code, because my specification requests it to return http 400 error when parameter validation failed. How do I achieve that?

Thanks,

#35 Re: mORMot 1 » Implement RESTful with specification already designed » 2016-11-03 23:17:31

Bo
ab wrote:

You can send the parameters as a JSON object body, even with interface based services.
You may define e.g. IAdmin interface and the corresponding method:

  procedure User(const adminUserId, adminPassword, orgUserid, orgPassword, orgName, orgTelephone, orgCorrespondenceEmail, orgAddress: RawUTF8);

Then you can override the routing class to match your expectations at URI level.
...

Thanks ab, I will try "JSON object body".

#36 mORMot 1 » Implement RESTful with specification already designed » 2016-11-03 00:03:52

Bo
Replies: 3

Is it possible to implement following RESTful API specification with mORMot? (please note this is not a database application, and the RESTful API is already published)


To create a user:

POST root/api/admin/user

with request Body:
{
"adminUserid" : "joe",
"adminPassword" : "password",
"orgUserid" : "GHRestOrg" ,
"orgPassword" : "password123" ,
"orgName" : "Test Rest Organisation" ,
"orgTelephone" : "04-345-1234",
"orgCorrespondenceEmail" : "info@test.com" ,
"orgAddress" : "Plato Towers, No 300/10" ,
}

I guess what I wanted to know is if there is a way to use custom URI and JSON format of request and response.
As far as I am aware, mORMot's interface based/method based services have different URI and JSON schema compare to above specification.

#37 Re: mORMot 1 » SynDB Explorer failed to execute query generated by Query Builder » 2016-10-25 12:14:58

Bo

It is a quick fix in SynDBExplorer, but I think the root cause is in IsSelect function.

MS SQL has no problem with line feeds, the problem is IsSelect failed to check the Select statement which has line feeds immediately after 'select', thus SynDBExplorer executes the select statement as no result SQL and seems to me that it is failed to run the query.

#38 mORMot 1 » SynDB Explorer failed to execute query generated by Query Builder » 2016-10-25 00:45:52

Bo
Replies: 3

I was playing with SynDB Explorer and trying with Query Builder, but strangely, the query generated by Query Builder did not return anything, even a simplest query like:

select
  lastuid.*
from
 lastuid

but the same query ran correctly in SQL Studio with lines of data returned (SynDB Explorer was connecting to a MS SQL database).

After a few tries, I found if I merge the lines to one line then the query runs OK with data returned:

select  lastuid.* from lastuid

With some debugging, I now narrow it down to following check will failed:

Check(isSelect('select'+#13#10+ ' * from toto'));

Looks like it does not treat #13#10 as separator.

#39 Re: Low level and performance » Re: How function results are allocated » 2016-10-22 05:17:55

Bo

Hi Ab,

Now I can reproduce your result. The difference is the "s" must not be a global variable.

#40 Re: Low level and performance » Re: How function results are allocated » 2016-10-20 10:57:02

Bo
ab wrote:

With Delphi 7 and Delphi 10.1, the last s is 'toto', as expected.

result=test
s=toto

This is strange, mine running result did not get "result=test" either!!!

Same result with XE5 and 2007.

open?id=0B8uyyClosvkAclFGR240YV9VTjA

#41 Low level and performance » Re: How function results are allocated » 2016-10-19 23:43:23

Bo
Replies: 3

Hi Ab,

I am reading your blog "How function results are allocated" (http://blog.synopse.info/post/2012/04/1 … -allocated) and running the code you provided in the blog to understand the problem, but what I have found is that the result is different, it was still 'test' instead of 'toto'. I am running code in Berlin 10.1 as following:

function GetString: string;
// is compiled as procedure GetString(var result: string);
begin
  if result='' then
    result := 'test' else
  writeln('result=',result);
end;
function GetRaise: string;
// is compiled as procedure GetRaise(var result: string);
begin
  result := 'toto';
  raise Exception.Create('Problem');
end;

var s: string;
begin
  // here s=''
  s := GetString; // called as GetString(s);
  // here s='test'
  s := GetString; // called as GetString(s);
  // will write 'result=test' on the console
  try
    s := GetRaise; // called as GetRaise(s);
  finally
    // here s='toto'
    writeln('s=',s);
    Readln(s);
  end;
end.

Have something changed in Delphi since you wrote the blog or have I done something wrong?

#42 Re: Low level and performance » Very fast asm-implementation of Base64 Encoding/Decoding » 2016-10-11 23:19:54

Bo

Arnaud is right, Base64 Encoding/Decoding in SynCommons is faster with bigger data.

I was replacing our own version of Base64 Encoding/Decoding functions with mORMot's and I wanted to make sure I was doing the right thing (i.e., I made something better),
so I used the TestSQL3 to test our version (Base64GLH) together with mORMot's (Base64).

I cloned test "procedure TTestCryptographicRoutines._Base64;" to "procedure TTestCryptographicRoutines._Base64GLH;" and made it calling our versions TMySimpleHL7.Base64Encode/Base64Decode:

procedure TTestCryptographicRoutines._Base64GLH;
const
  Value64: RawUTF8 = 'SGVsbG8gL2Mn6XRhaXQg5+Ar';
var tmp: RawByteString;
    b64: RawUTF8;
    Value: WinAnsiString;
    i, L: Integer;
begin
  Value := 'Hello /c''0tait 67+';
  Value[10] := #$E9;
  Value[16] := #$E7;
  Value[17] := #$E0;
  Check(not IsBase64(Value));
  Check(TMySimpleHL7.Base64Encode(Value)=Value64);
  Check(BinToBase64(Value)=Value64);
  Check(IsBase64(Value64));
  tmp := StringFromFile(ExeVersion.ProgramFileName);
  b64 := TMySimpleHL7.Base64Encode(tmp);
  Check(IsBase64(b64));
  Check(TMySimpleHL7.Base64Decode(b64)=tmp);
  Check(BinToBase64(tmp)=b64);
  Check(Base64ToBin(b64)=tmp);
  tmp := '';
  for i := 1 to 1998 do begin
    b64 := TMySimpleHL7.Base64Encode(tmp);
    Check(TMySimpleHL7.Base64Decode(b64)=tmp);
    Check((tmp='') or IsBase64(b64));
    Check(BinToBase64(tmp)=b64);
    Check(Base64ToBin(b64)=tmp);
    if tmp<>'' then begin
      L := length(b64);
      Check(not IsBase64(pointer(b64),L-1));
      b64[Random(L)+1] := '&';
      Check(not IsBase64(pointer(b64),L));
    end;
    tmp := tmp + AnsiChar(Random(255));
  end;
end;

the outcome was not good, Base64GLH ran faster:

 1.5. Cryptographic routines:
  - Adler32: 1 assertion passed  365us
  - MD5: 86 assertions passed  267us
  - SHA1: 10 assertions passed  4.93ms
  - SHA256: 8 assertions passed  6.56ms
  - AES256: 17,428 assertions passed  284.57ms
      cypher 1..2409 bytes with AES-NI: 312us, without: 11.42ms
  - RC4: 1 assertion passed  125us
  - Base64: 11,994 assertions passed  70.41ms
  - Base64GLH: 11,994 assertions passed  55.84ms
  - CompressShaAes: 1,683 assertions passed  3.49ms
  - TAESPNRG: 13,300 assertions passed  51.33ms
  Total failed: 0 / 56,505  - Cryptographic routines PASSED  484.30ms

I thought it might be the number of the test was not big enough, so I increased the loop to 10 times from 1998 to 19980, the outcome was still not good:

 1.5. Cryptographic routines:
  - Adler32: 1 assertion passed  378us
  - MD5: 86 assertions passed  272us
  - SHA1: 10 assertions passed  5.02ms
  - SHA256: 8 assertions passed  6.57ms
  - AES256: 17,428 assertions passed  207.10ms
      cypher 1..2409 bytes with AES-NI: 287us, without: 5.59ms
  - RC4: 1 assertion passed  103us
  - Base64: 119,886 assertions passed  2.82s
  - Base64GLH: 119,886 assertions passed  2.13s
  - CompressShaAes: 1,683 assertions passed  3.38ms
  - TAESPNRG: 13,300 assertions passed  73.87ms
  Total failed: 0 / 272,289  - Cryptographic routines PASSED  5.26s

until I saw this thread, I then gave a go with bigger data: changed tmp's value by adding string in Value instead of on char at a time

procedure TTestCryptographicRoutines._Base64GLH;
const
  Value64: RawUTF8 = 'SGVsbG8gL2Mn6XRhaXQg5+Ar';
var tmp: RawByteString;
    b64: RawUTF8;
    Value: WinAnsiString;
    i, L: Integer;
begin
  Value := 'Hello /c''0tait 67+';
  Value[10] := #$E9;
  Value[16] := #$E7;
  Value[17] := #$E0;
  Check(not IsBase64(Value));
  Check(TMySimpleHL7.Base64Encode(Value)=Value64);
  Check(BinToBase64(Value)=Value64);
  Check(IsBase64(Value64));
  tmp := StringFromFile(ExeVersion.ProgramFileName);
  b64 := TMySimpleHL7.Base64Encode(tmp);
  Check(IsBase64(b64));
  Check(TMySimpleHL7.Base64Decode(b64)=tmp);
  Check(BinToBase64(tmp)=b64);
  Check(Base64ToBin(b64)=tmp);
  tmp := '';
  for i := 1 to 19980 do begin
    b64 := TMySimpleHL7.Base64Encode(tmp);
    Check(TMySimpleHL7.Base64Decode(b64)=tmp);
    Check((tmp='') or IsBase64(b64));
    Check(BinToBase64(tmp)=b64);
    Check(Base64ToBin(b64)=tmp);
    if tmp<>'' then begin
      L := length(b64);
      Check(not IsBase64(pointer(b64),L-1));
      b64[Random(L)+1] := '&';
      Check(not IsBase64(pointer(b64),L));
    end;
    tmp := tmp+value;
  end;
end;

then Base64 ran faster than Base64GLH:

 1.5. Cryptographic routines:
  - Adler32: 1 assertion passed  367us
  - MD5: 86 assertions passed  258us
  - SHA1: 10 assertions passed  4.76ms
  - SHA256: 8 assertions passed  6.49ms
  - AES256: 17,428 assertions passed  206.63ms
      cypher 1..2409 bytes with AES-NI: 278us, without: 5.47ms
  - RC4: 1 assertion passed  89us
  - Base64: 119,886 assertions passed  28.90s
  - Base64GLH: 119,886 assertions passed  39.11s
  - CompressShaAes: 1,683 assertions passed  7.71ms
  - TAESPNRG: 13,300 assertions passed  51.35ms
  Total failed: 0 / 272,289  - Cryptographic routines PASSED  68.29s

#43 Re: mORMot 1 » ORM on legacy database » 2016-07-17 03:07:32

Bo

I have checked the code in CreateMissingTables. It creates the table if the table does not exist, the script for creating a new table includes column ID, but it only goes through published fields of TSQLRecord if the table is already existed in the database, and ID is not a published field in the TSQLRecord, thus it won't add column ID into a existed table.

I am going to add a method TSQLRestServerDB.CreateMissingIDs to add ID column to tables for my legacy database.

#44 mORMot 1 » ORM on legacy database » 2016-07-15 12:18:39

Bo
Replies: 1

I am trying to apply ORM on a legacy database.

First, I tried manually create a TSQLRecord descendant for a table in the database, CreateMissingTable did not report any error but the ID column was not added to the table.
But if I remove one existed column from that table, CreateMissingTable added the column back to the table but still had no ID column.

Then found this post http://synopse.info/forum/viewtopic.php?id=2981, in the comment of following generated code, it does say the ORM will add ID to the table:

type
  /// totalsSent Table
  // - type definition auto-generated by SynDBExplorer 1.18.2765 at 2016-07-15 21:58:23
  // from totalsSent
  // - note that the ORM will add one missing ID field via:
  // $ ALTER TABLE totalsSent ADD ID INTEGER NOT NULL PRIMARY KEY
  TSQLtotalsSent = class(TSQLRecord)
  protected
    fdate: Double;
    fattachments: Int64;
    ferrors: Int64;
  published
    /// match totalsSent.date [FLOAT]
    property date: Double read fdate write fdate;
    /// match totalsSent.attachments [INTEGER]
    property attachments: Int64 read fattachments write fattachments;
    /// match totalsSent.errors [INTEGER]
    property errors: Int64 read ferrors write ferrors;
  end;

...

My CreateMissingTable code:

  LModel := TSQLModel.Create([TSQLproperties,TSQLtotalsSent]);
  LDB := TSQLRestServerDB.Create(LModel,'status.sqlite1');
  LDB.CreateMissingTables(1);
...

So what have I done wrong? How can I get it to create ID by the model so that I don't need to manually create one for each table.

#45 Re: mORMot 1 » Help about table relationship » 2016-07-10 02:00:55

Bo

Hi turrican,

Have you figured out yet?

#46 Re: mORMot 1 » Http Basic Authentication » 2016-07-06 01:05:29

Bo
squirrel wrote:

Does anybody have an example of using TSQLRestServerAuthenticationHttpBasic that works?  I need to use basic authentication and look up the username and password from my own list (not orm).

Same here and would like to know the solution.

#47 Re: mORMot 1 » TestSQL3.exe - Lazarus_fpcup 1.7-Win32 fpc 3.1.1 (2016-06-17) » 2016-07-06 00:54:54

Bo
ab wrote:

Win32 platform?

I have compiled and test with both 32bit and 64bit, both have a lot of AV. Last night I tried New Pascal package and result was the same.

#48 Re: mORMot 1 » TestSQL3.exe - Lazarus_fpcup 1.7-Win32 fpc 3.1.1 (2016-06-17) » 2016-07-04 13:28:16

Bo

I compiled TestSQL3.exe (from source 2016-06-30_140818) with Lazarus 1.6, FPC 3.0.0, ran it with Windows 10, got a lot of AV too, first one was:

!  - Url encoding: 5 / 150 FAILED  2.19ms

Board footer

Powered by FluxBB