#2 Re: mORMot Framework » A newbie question about 'How to use TSynLog' » 2014-10-13 13:58:38

Also, note that AutoFlushTimeOut does not work when you are in Debug mode (DebugHook = 1).

#3 Re: mORMot Framework » A newbie question about 'How to use TSynLog' » 2014-10-08 07:48:03

FYI, it works for me when I tried it (I have not committed the latest patch so I still manually added the trailing slash).

  with TMyLog.Family do
  begin
    Level := [sllInfo, sllError, sllServer];
    DestinationPath := ExtractFilePath(Application.ExeName) + 'logs\';
  end;

  TMyLog.Add.Log(sllInfo, 'Testing');

Question for @win2014: Does the logs folder exist?

#4 Re: mORMot Framework » TSynLogFile.GetEventText returns an Empty String? » 2014-10-06 09:52:46

Just wondering if anyone has had time to test it out? Thank you.

#5 Re: mORMot Framework » TSynLogFile.GetEventText returns an Empty String? » 2014-10-04 14:09:07

Here's a sample console code:

program TestLogger;

{$APPTYPE CONSOLE}

{$R *.res}

uses
  SynCommons,
  System.SysUtils;

type
  TTestLogger = class(TSynLog)
    procedure ComputeFileName; override;
  end;

  TSynLogFileEcho = class(TSynLogFile)
  public
    function LogEcho(Sender: TTextWriter; Level: TSynLogInfo; const Text: RawUTF8): boolean;
  end;

var
  LogFile: TSynLogFileEcho;
  FEventCaption: array[TSynLogInfo] of string;
  E: TSynLogInfo;

{ TTestLogger }

procedure TTestLogger.ComputeFileName;
begin
  inherited;
  fFileName := 'abc.log';
end;

{ TSynLogFileEcho }

function TSynLogFileEcho.LogEcho(Sender: TTextWriter; Level: TSynLogInfo;
  const Text: RawUTF8): boolean;
var
  P: PUTF8Char;
  Line: RawUTF8;
begin
  P := Pointer(Text);
  repeat
    Line := GetNextLine(P, P);
    if Length(Line) < 24 then
      Continue;
    Self.AddInMemoryLine(Line);
  until P = nil;
  Result := True;
end;

procedure ShowLogContent;
var
  index: Integer;
begin
  for index := LogFile.Count - 1 downto 0 do
    WriteLn('Level: ', FEventCaption[LogFile.EventLevel[index]], ' Text: ', LogFile.EventText[index]);
end;

procedure WaitForEnter(const Msg: string);
begin
  WriteLn(Msg);
  Write('Press ENTER to continue...');
  ReadLn;
end;

begin
  try
    with TTestLogger.Family do
    begin
      Level := LOG_VERBOSE;
      FileExistsAction := acAppend;
    end;

    for E := succ(sllNone) to high(E) do
      FEventCaption[E] := GetCaptionFromEnum(TypeInfo(TSynLogInfo),ord(E));

    if FileExists('abc.log') then
      LogFile := TSynLogFileEcho.Create('abc.log')
    else LogFile := TSynLogFileEcho.Create;

    WaitForEnter('Listing initial content');
    ShowLogContent;

    TTestLogger.Add.Log(sllNewRun, 'Starting Up!');
    TTestLogger.Family.EchoRemoteStart(nil, LogFile.LogEcho, False);

    WaitForEnter('After first log');
    ShowLogContent;

    TTestLogger.Add.Log(sllInfo, 'Information, information, information');
    TTestLogger.Add.Log(sllDebug, 'Debug, debug, debug');
    TTestLogger.Add.Log(sllWarning, 'Warning, warning, warning');

    WaitForEnter('After sample logs');
    ShowLogContent;

    WaitForEnter('The End');

    LogFile.Free;
  except
    on E: Exception do
      Writeln(E.ClassName, ': ', E.Message);
  end;
end.

The first time you run it, echo would be correct.

The second time you run it, the log text from the log file will be dumped correctly, but subsequent echoes will show blank text.

The third time you run it, log text from first and second run will be shown, but subsequent echoes will also show blank text.

#6 Re: mORMot Framework » TSynLogFile.GetEventText returns an Empty String? » 2014-10-04 13:34:25

Yup, weirdly LogView works. However, in my sample code, it's exactly the same thing, just without the remote connection.

The message was added via TSynLog.Add.Log(slxxx, 'xxx'); so I assume the format should be correct? When debugging, I see that it returns an empty text because strlen always return 0. However, when I restart the test app (it first works by loading an existing log file, before doing the in-memory append for subsequent logs), the same looping code above works, so I believe the format is correct.

I had this issue much earlier and managed to resolve it by appending a #13 to the Line I got from LogEcho, however that has stopped working since a few patches ago... no idea why yet though.

#7 mORMot Framework » TSynLogFile.GetEventText returns an Empty String? » 2014-10-03 18:41:21

cheemeng
Replies: 4

Hi Arnaud,

If I read back the line that has been added via AddInMemoryLine, I will get an empty string. EventLevel will return the correct event level though? The line sent to AddInMemoryLine is retrieved via a TOnTextWriterEcho event.

Some code snippet:

function TTestLogger.LogEcho(Sender: TTextWriter; Level: TSynLogInfo;
  const Text: RawUTF8): boolean;
var
  P: PUTF8Char;
  Line: RawUTF8;
begin
      repeat
        Line := GetNextLine(P, P);
        if Length(Line) < 24 then
          Continue;
        fActiveLog.AddInMemoryLine(Line); 
      until P = nil;
end;

// and when I do this:

for index := fActiveLog.Count - 1 downto 0 do
  WriteLn('Level: ', EventLevelCaption[fActiveLog.EventLevel[index]], ' Text: ', fActiveLog.EventText[index]);

// I get an empty string for Event Text, but the Event Level is correct

Any ideas? Or did I do something wrongly?

#8 Re: mORMot Framework » Best Way to Store Realtime Stats? » 2014-10-01 18:16:27

Thanks for the reply Arnaud.

I wonder if the built-in SQLite is usable enough to store such data for now, since I'm not having that much traffic? Of course, that means having to do my own presentation layer to display the stats.

#9 mORMot Framework » Best Way to Store Realtime Stats? » 2014-09-30 11:26:01

cheemeng
Replies: 2

I've been meaning to track and store server access stats.

Something like, a per-second snapshot of total calls to each Method/Interface of each TSQLRestServer. Which then can be used to create 5 min, 30 min, hourly and daily totals, for charting purposes.

What would be a good way to collect and persist this data, without affecting performance too much? I have toyed with the idea of having a thread to send data from TSQLRestServerStats to a Redis server (say, every second), but it would be great if this can all be done in a self-contained manner, with no extra dependencies. Would storing it in SQLite create too big a performance hit? Will it bulk up the size of the database too much? (I'm looking at, say, an median of 10-20 calls per minute, up to a max of 200-300 calls per second).

Anyone has experience doing this, in conjuction with the mORMot Framework? Thanks!

#11 Re: mORMot Framework » Does TMemoryMapText.AddInMemoryLine need Locking? » 2014-09-29 07:35:20

OK, I'll remember to lock it if I need to use it from multiple threads. Thanks Arnaud!

#12 mORMot Framework » Does TMemoryMapText.AddInMemoryLine need Locking? » 2014-09-28 19:04:56

cheemeng
Replies: 2

Hi Arnaud,

I do not see any locking in the procedure, so I'm wondering if it is as thread-safe as its disk-based counterpart, TSynLog.LogInternal?

Thanks!

#13 Re: mORMot Framework » LogFile Custom Name » 2014-09-28 18:51:12

Line 42774 also needs amendment to

    if (fFileRotationSize>0) or (fFamily.FileExistsAction <> acOverwrite) then

#14 Re: mORMot Framework » LogFile Custom Name » 2014-09-28 18:21:12

There is a bug that prevents log file appends from working correctly... it ends up overwriting the log file nevertheless if RotationSize is not set (Line 42753), which is why I proposed by code above.

To cater for your updated log file access retry code, the code can be updated to be as follows:

      for retry := 0 to 2 do begin
        for i := 1 to 10 do
        try
          if FileExists(fFileName) and (fFamily.FileExistsAction <> acOverwrite) then
          begin
            if fFamily.FileExistsAction = acAppend then
            Include(fInternalFlags,logHeaderWritten);
          end else if (fFileRotationSize=0) or not FileExists(fFileName) then
            TFileStream.Create(fFileName,fmCreate).Free;   // create a void file
          fWriterStream := TFileStream.Create(fFileName,
            fmOpenReadWrite or fmShareDenyWrite); // open with read sharing
          break;
        except
          on Exception do
            Sleep(100);
        end;
        if fWriterStream<>nil then
          break;
        fFileName := ChangeFileExt(fFileName,'-'+fFamily.fDefaultExtension);
      end;

Lines 42742 to 42748 are unnecessary.

#16 Re: mORMot Framework » Daily Log File » 2014-09-21 08:20:59

Hi Arnaud,

Would it make more sense to move the call to DeleteFile(fFileName) into the IF block before it?

The reason being that if the OnRotate event was handled (True was returned), it most likely means the programmer is saying "OK I have everything under control".

Also, would appreciate it if you could take a look at http://synopse.info/forum/viewtopic.php?id=2022

Thank you!

#17 Re: mORMot Framework » TSQLRecord and TSQLRestServerDB » 2014-09-20 18:27:51

Why not access it this way?

var
  Entries: TEntryDB;

Entries := TEntryDB.CreateAndFillPrepare(FDatabase, yourQueryHere);
while Entries.FillOne do
begin
  Display(Entries.SomeProperty);
end;

#19 Re: mORMot Framework » LogFile Custom Name » 2014-09-20 14:18:53

Hi Arnaud,

Would just like to know what do you think of the proposed changes.

If it should not go to the main code, then let me know and I'll subclass it myself. Thanks!

#20 Re: mORMot Framework » LogFile Custom Name » 2014-09-19 16:05:34

Additional updates (my previous patch was buggy and incomplete)...

procedure TSynLog.CreateLogWriter;
begin
  if fWriterStream=nil then begin
    GenerateFileName;
    if fFamily.NoFile then
      fWriterStream := TFakeWriterStream.Create else
    begin
      if FileExists(fFileName) and (fFamily.fLogExistsAction <> acOverwrite) then
      begin
        if fFamily.fLogExistsAction = acAppend then
          Include(fInternalFlags,logHeaderWritten);
      end else
      if (fFileRotationSize=0) or not FileExists(fFileName) then
        TFileStream.Create(fFileName,fmCreate).Free;   // create a void file
      fWriterStream := TFileStream.Create(fFileName, // open with read sharing
        fmOpenReadWrite or fmShareDenyWrite);
    end;
    if (fFileRotationSize>0) or (fFamily.fLogExistsAction <> acOverwrite) then
      fWriterStream.Seek(0,soFromEnd); // in rotation mode, append at the end
  end;
  if fWriterClass=nil then
    fWriterClass := TTextWriter;
  if fWriter=nil then
    fWriter := fWriterClass.Create(fWriterStream,fFamily.BufferSize);
  fWriter.EndOfLineCRLF := fFamily.EndOfLineCRLF;
  if integer(fFamily.EchoToConsole)<>0 then
    fWriter.EchoAdd(ConsoleEcho);
  if Assigned(fFamily.EchoCustom) then
    fWriter.EchoAdd(fFamily.EchoCustom);
  if Assigned(fFamily.fEchoRemoteClient) then
    fWriter.EchoAdd(fFamily.fEchoRemoteEvent);
end;

#21 Re: mORMot Framework » LogFile Custom Name » 2014-09-19 12:28:22

Hi Arnaud,

What do you think of this? (SynCommons.pas)

 SynCommons.pas | 95 +++++++++++++++++++++++++++++++++++-----------------------
 1 file changed, 57 insertions(+), 38 deletions(-)

diff --git a/SynCommons.pas b/SynCommons.pas
index d673641..3fec809 100644
--- a/SynCommons.pas
+++ b/SynCommons.pas
@@ -10705,6 +10705,7 @@ type
 
   /// how stack trace shall be computed
   TSynLogStackTraceUse = (stManualAndAPI,stOnlyAPI,stOnlyManual);
+  TSynLogExistsAction = (acOverwrite, acAppend, acAppendWithHeader);
 
   {/ regroup several logs under an unique family name
    - you should usualy use one family per application or per architectural
@@ -10762,6 +10763,7 @@ type
     fRotateFileCount: cardinal;
     fRotateFileSize: cardinal;
     fRotateFileAtHour: integer;
+    fLogExistsAction: TSynLogExistsAction;
     function CreateSynLog: TSynLog;
     {$ifdef MSWINDOWS}
     procedure SetAutoFlush(TimeOut: cardinal);
@@ -10945,6 +10947,8 @@ type
     // be appended instead
     // - TSynLogFile class and our LogView tool will handle both patterns
     property EndOfLineCRLF: boolean read fEndOfLineCRLF write fEndOfLineCRLF;
+    // action to take if log file already exists. Defaults to append with header
+    property LogExistsAction: TSynLogExistsAction read fLogExistsAction write fLogExistsAction;
   end;
 
   /// thread-specific internal context used during logging
@@ -11025,12 +11029,12 @@ type
     procedure CreateLogWriter; virtual;
 {$ifndef DELPHI5OROLDER}
     procedure LogInternal(Level: TSynLogInfo; TextFmt: PWinAnsiChar;
-      const TextArgs: array of const; Instance: TObject); overload; 
+      const TextArgs: array of const; Instance: TObject); overload;
 {$endif}
     procedure LogInternal(Level: TSynLogInfo; const Text: RawUTF8;
       Instance: TObject; TextTruncateAtLength: integer); overload;
     procedure LogInternal(Level: TSynLogInfo; aName: PWinAnsiChar;
-     aTypeInfo: pointer; var aValue; Instance: TObject=nil); overload; 
+     aTypeInfo: pointer; var aValue; Instance: TObject=nil); overload;
     // any call to this method MUST call UnLock
     function LogHeaderLock(Level: TSynLogInfo): boolean;
     procedure LogTrailerUnLock(Level: TSynLogInfo); {$ifdef HASINLINE}inline;{$endif}
@@ -11048,6 +11052,7 @@ type
     function Instance: TSynLog;
     function ConsoleEcho(Sender: TTextWriter; Level: TSynLogInfo;
       const Text: RawUTF8): boolean; virtual;
+    procedure GenerateFileName; virtual;
   public
     /// intialize for a TSynLog class instance
     // - WARNING: not to be called directly! Use Enter or Add class function instead
@@ -41663,6 +41668,7 @@ begin
   fExceptionIgnore := TList.Create;
   fLevelStackTrace :=
     [sllError,sllException,sllExceptionOS,sllFail,sllLastError,sllStackTrace];
+  fLogExistsAction := acAppendWithHeader;
 end;
 
 function TSynLogFamily.CreateSynLog: TSynLog;
@@ -42644,48 +42650,61 @@ begin
   end;
 end;
 
-procedure TSynLog.CreateLogWriter;
+procedure TSynLog.GenerateFileName;
 {$ifndef MSWINDOWS}
 var i: integer;
 {$endif}
-var timeNow,hourRotate,timeBeforeRotate: TDateTime;
+var timeNow, hourRotate, timeBeforeRotate: TDateTime;
+begin
+  {$ifdef MSWINDOWS}
+  ExeVersionRetrieve;
+  fFileName := Ansi7ToString(ExeVersion.ProgramName);
+  if fFamily.IncludeComputerNameInFileName then
+    fFileName := fFileName+' ('+Ansi7ToString(ExeVersion.Host)+')';
+  {$else}
+  fFileName := ExtractFileName(ParamStr(0));
+  i := Pos('.',fFileName);
+  if i>0 then
+    SetLength(fFileName,i-1);
+  {$endif}
+  fFileRotationSize := 0;
+  if fFamily.fRotateFileCount>0 then begin
+    if fFamily.fRotateFileSize>0 then
+      fFileRotationSize := fFamily.fRotateFileSize shl 10; // size KB -> B
+    if fFamily.fRotateFileAtHour in [0..23] then begin
+      hourRotate := EncodeTime(fFamily.fRotateFileAtHour,0,0,0);
+      timeNow := Time;
+      if hourRotate<timeNow then
+        hourRotate := hourRotate+1; // trigger will be tomorrow
+      timeBeforeRotate := hourRotate-timeNow;
+      fFileRotationNextHour := GetTickCount64+trunc(timeBeforeRotate*MSecsPerDay);
+    end;
+  end;
+  if (fFileRotationSize=0) and (fFileRotationNextHour=0) then
+    fFileName := fFileName+' '+Ansi7ToString(NowToString(false));
+   {$ifdef MSWINDOWS}
+  if IsLibrary then
+    fFileName := fFileName+' '+ExtractFileName(GetModuleName(HInstance));
+  {$endif}
+  if fFamily.fPerThreadLog=ptOneFilePerThread then
+    fFileName := fFileName+' '+IntToString(GetCurrentThreadId);
+  fFileName := fFamily.fDestinationPath+fFileName+fFamily.fDefaultExtension;
+end;
+
+procedure TSynLog.CreateLogWriter;
 begin
   if fWriterStream=nil then begin
-    {$ifdef MSWINDOWS}
-    ExeVersionRetrieve;
-    fFileName := Ansi7ToString(ExeVersion.ProgramName);
-    if fFamily.IncludeComputerNameInFileName then
-      fFileName := fFileName+' ('+Ansi7ToString(ExeVersion.Host)+')';
-    {$else}
-    fFileName := ExtractFileName(ParamStr(0));
-    i := Pos('.',fFileName);
-    if i>0 then
-      SetLength(fFileName,i-1);
-    {$endif}
-    fFileRotationSize := 0;
-    if fFamily.fRotateFileCount>0 then begin
-      if fFamily.fRotateFileSize>0 then
-        fFileRotationSize := fFamily.fRotateFileSize shl 10; // size KB -> B
-      if fFamily.fRotateFileAtHour in [0..23] then begin
-        hourRotate := EncodeTime(fFamily.fRotateFileAtHour,0,0,0);
-        timeNow := Time;
-        if hourRotate<timeNow then
-          hourRotate := hourRotate+1; // trigger will be tomorrow
-        timeBeforeRotate := hourRotate-timeNow;
-        fFileRotationNextHour := GetTickCount64+trunc(timeBeforeRotate*MSecsPerDay);
-      end;
-    end;
-    if (fFileRotationSize=0) and (fFileRotationNextHour=0) then
-      fFileName := fFileName+' '+Ansi7ToString(NowToString(false));
-     {$ifdef MSWINDOWS}
-    if IsLibrary then
-      fFileName := fFileName+' '+ExtractFileName(GetModuleName(HInstance));
-    {$endif}
-    if fFamily.fPerThreadLog=ptOneFilePerThread then
-      fFileName := fFileName+' '+IntToString(GetCurrentThreadId);
-    fFileName := fFamily.fDestinationPath+fFileName+fFamily.fDefaultExtension;
+    GenerateFileName;
     if fFamily.NoFile then
-      fWriterStream := TFakeWriterStream.Create else begin
+      fWriterStream := TFakeWriterStream.Create else
+    begin
+      if FileExists(fFileName) then
+      begin
+        if fFamily.fLogExistsAction = acOverwrite then
+          DeleteFile(fFileName)
+        else if fFamily.fLogExistsAction = acAppend then
+          Include(fInternalFlags,logHeaderWritten);
+      end;
       if (fFileRotationSize=0) or not FileExists(fFileName) then
         TFileStream.Create(fFileName,fmCreate).Free;   // create a void file
       fWriterStream := TFileStream.Create(fFileName, // open with read sharing

Alternatively, if this is not possible, then I will subclass TSynLog. Thank you!

#22 mORMot Framework » LogFile Custom Name » 2014-09-16 15:32:11

cheemeng
Replies: 7

Hi Arnaud,

Is it possible to have a callback event for SynLog that returns a custom filename? Either that, or move the filename generation routine into another procedure/function that is virtual and may be overridden by a descendant class?

This is useful if I want to only have a daily log, so if the programme gets restarted, logging may resume in the same file. Also, if the file already exists, perhaps logHeaderWritten should be set in fInternalFlags, to prevent writing of header again.

Thank you!

#23 Re: mORMot Framework » Async Insert/Updates » 2014-09-16 11:06:50

If you're using the built-in ORM, caching happens automatically (see Section 5.5 of the SAD pdf file) on the server side.

Not sure what you mean by "postponed" operations.


EDIT: Oops, looks like Arnaud replied first smile

#25 Re: mORMot Framework » Interface Services without Parameters » 2014-09-11 04:15:16

Hi Arnaud,

I'm just calling it from the browser via http://localhost:8080/admin/AdminServer/isRunning

Very simple to reproduce:

  IAdminServer = interface(IInvokable)
    ['{E3097205-9424-427E-B6B4-D61E044CC6C3}']
    function IsRunning: Boolean;
  end;

  TAdminServer = class(TInterfacedObject, IAdminServer)
  public
    function IsRunning: Boolean;
  end;

  function TAdminServer.IsRunning: Boolean;
  begin
    Result := True;
  end;

// ---- snip ----

  AdminModel := TSQLModel.Create([TSQLUser], 'admin');
  AdminDB := TSQLRestServerDB.Create(AdminModel, 'admin.db', False);
  AdminServer := TSQLHttpServer.Create('8080', [AdminDB], '+', useHttpApiRegisteringURI, 32, secNone);
  AdminServer.AccessControlAllowOrigin := '*';
  AdminDB.ServiceRegister(TAdminServer, [TypeInfo(IAdminServer)], sicShared);

Code fails in mORMot.pas line 27791 because Par is nil, and the if statement is trying to perform character matching.

If I amend the line to

    if (Par<>nil) and ((Par^='[') or IdemPChar(Par,'%5B')) then

it still fails at line 27047 of mORMot.pas because as long as the parameter list is nil, it will exit with an error.

#26 mORMot Framework » Interface Services without Parameters » 2014-09-10 16:35:09

cheemeng
Replies: 4

Hi Arnaud,

It currently seems impossible to support functions in interfaces that do not require parameters?

for instance

interface IServerInfo
  function IsRunning: Boolean;
end;

will return an error if the caller does a /server/isRunning

However, if I add any parameters to the URL, it works... e.g. /server/isRunning?random=13348

I believe this is due to checks for parameter list <> nil

Could there be an exception to this check for occurrences where the function is not expecting any parameters?

#27 mORMot Framework » TSQLHttpServer.AddServer does not follow TSQLHttpServerOptions » 2014-09-10 05:34:57

cheemeng
Replies: 1

Hi Arnaud,

When the HttpServer is instantiated with useHttpApiRegisteringURI, all initial roots get registered.

However, subsequent calls to AddServer does not check this value and will pass False to AddUrl's aRegisterURI parameter, thereby causing those roots to not be automatically registered.

Suggestion:

1. Store the value of the initial TSQLHttpServerOptions.

2. mORMotHttpServer.pas line 419

        aServer.Model.Root,fPort,(aHttpServerSecurity=secSSL),fDomainName)<>NO_ERROR then

to

        aServer.Model.Root,fPort,(aHttpServerSecurity=secSSL),fDomainName,useHttpApiRegisteringURI=fHttpServerOptions)<>NO_ERROR then


Thanks!

#28 Re: mORMot Framework » Daily Log File » 2014-09-09 13:49:33

For those who are wondering, this feature has been added in.

Thanks Arnaud!

#30 Re: mORMot Framework » AutoFlushTimeOut has no effect when compiled app runs in non-C drive » 2014-09-07 16:34:15

I guess you could make use of the FindFirstChangeNotification API call to notify you of changes to the file.

I reckon you would most likely need to create a new descendant of TSynLogFile that
1. has an additional parameter to LoadFromMap to allow you to specify an Offset to P instead of always pointing to fMap.Buffer (chances are you would need to duplicate most of the code from TMemoryMapText.LoadFromMap as well as TSynLogFile.LoadFromMap, and strip out the parts that perform initialization and parts that always read from fLines[0], replacing it with fLines[fCount])
2. upon notification of file change, calculate the size difference and pass that in to a call to the new LoadFromMap

Or you could open a request ticket and request for Arnaud to add the Offset into TMemoryMapText.LoadFromMap and its descendants as well, and modify the code to support subsequent calls to LoadFromMap smile

#32 Re: mORMot Framework » Daily Log File » 2014-09-06 10:55:14

Rotation works too, if there is an event like OnArchive so that we can decide if we want to handle it (move it to a different folder, archive it with zip instead of SynLZ, etc). Would that be possible?

#33 mORMot Framework » Daily Log File » 2014-09-06 09:06:02

cheemeng
Replies: 8

Hi Arnaud,

What is the best way to create a daily log file using TSynLog?

I am thinking of creating a descendant of TSynLog with the following changes:

1. Filename will include current date, perhaps in yyyymmdd format.
2. If the filename exists when creating log for the first time, append to it, otherwise create a new file.
3. Start a background thread that checks for date change, and when that happens, recreate a new fWriterStream.

Or is there already some built-in mechanism?

The reason I am looking for a daily log file is to make it easier to search for logs based on dates. So if the user wants to look for logs on a particular date, he just needs to open the file that corresponds to the date he wants using a log viewer.

#34 Re: mORMot Framework » tcollection or tdynarray and select » 2014-09-05 16:36:00

My 2 cents, Arnaud... there are really quite a few scenarios I have encountered that require this time of 1-to-many relationships.

I do hope foreign keys and 1-to-many (with optional lazy loading) can rank higher in the wish list big_smile

#36 Re: mORMot Framework » Results from Stored Procedure (MS SQL) Returned as Base64? » 2014-09-05 07:49:27

Thanks Arnaud. I'll update this thread if I managed to get any positive breakthrough.

#37 Re: mORMot Framework » Switching Response Format (XML/JSON) » 2014-09-05 07:42:59

Hi Arnaud,

I would like to propose that the method JSONBufferToXML (SynCommons.pas, line 34345) accepts an optional ArrayName parameter that can be passed to AddJSONToXML. At the same time, perhaps the ArrayName can default to the Model's name, and may be set by a property of TSQLModel?

Thank you!

#38 Re: mORMot Framework » Results from Stored Procedure (MS SQL) Returned as Base64? » 2014-09-01 16:09:11

Arnaud, is there any other information that I could provide to help you look into this issue? Or should I just abandon this thread?

Thank you!

#39 Re: mORMot Framework » Results from Stored Procedure (MS SQL) Returned as Base64? » 2014-08-26 17:59:27

Lol... sorry about the connection string part... was getting sleepy (it's almost 2am here).

Now DataCompatibility is in the connection string, but no changes. Forcing the column type to ftUTF8 returns the same hex dump, and BinToHex of the Column as blob returns the same dump.

As for the fProps.Free error, I had no idea that a TSQLDBStatement instance would prevent the Conn Properties from being freed... oops big_smile

#40 Re: mORMot Framework » Results from Stored Procedure (MS SQL) Returned as Base64? » 2014-08-26 17:27:11

By the way, when I try to free my TOleDBMSSQL2012ConnectionProperties, I get this error at fProps.Free.

First chance exception at $75C5C42D. Exception class EOleDBException with message 'OLEDB Error 80040E05 - Object was open.'.

This only occurs if I execute a stored procedure. If I execute other SQL commands (SELECT, INSERT, UPDATE), I do not get this error. Here's some sample code:

    fProps := TOleDBMSSQLConnectionProperties.Create(fServer, fDatabase,
      fUser, fPassword);
    Data := fProps.Execute(fStatement, []);
    DataResult := Data.FetchAllAsJSON(True, nil, False, True);
    fProps.Free;

Happens when I use other versions of the MSSQL Connection Properties too.

#41 Re: mORMot Framework » Results from Stored Procedure (MS SQL) Returned as Base64? » 2014-08-26 17:02:49

I think it might be your Base64 converted to hex?

BinToHex(Data.ColumnBlob(0));

returns the same data

'44074100630063006F0075006E00740044114F007000650072006100740069006F006E0061006C00530074006100740075007300440A53007500620043006C007500730074006500720001014101024103470681020103410310E40400000200415381038101'

As for fConnectionString, I just stepped in the debugger, it is an empty string. So it is the same as fConnectionString := 'DataCompatibility=80;';

The full connection string will end up being 'Provider=SQLNCLI11;Data Source=.\sqlexpress;Initial Catalog=dummy;User Id=;Password=;Integrated Security=SSPI;Persist Security Info=False;'

#42 Re: mORMot Framework » Results from Stored Procedure (MS SQL) Returned as Base64? » 2014-08-26 16:30:58

Hi Arnaud,

Sorry for the late reply. There are no email notifications to the OP when a post is replied to sad

1. In any case, just to confirm whether my code is correct, I directly modified TOleDBMSSQL2012ConnectionProperties just to test things out:

procedure TOleDBMSSQL2012ConnectionProperties.SetInternalProperties;
begin
  if OSVersion>wVista then
    fProviderName := 'SQLNCLI11';
  fConnectionString := fConnectionString + 'DataTypeCompatibility=80;';
  inherited SetInternalProperties;
end;

This did not work. The column is still treated as an ftBlob. Tried the same with the 2008 driver. Client is using MSSQL 2008 by the way.

2. As for your second suggestion, I would like to avoid OpenQuery if possible (I'm worried about the part about data being passed twice... my resultset can sometimes be HUGE).

3. And here's a dump of what I got when I force the column type to ftUTF8 instead of ftBlob,

[{"XML_F52E2B61-18A1-11d1-B105-00805F49916B":"44074100630063006F0075006E00740044114F007000650072006100740069006F006E0061006C00530074006100740075007300440A53007500620043006C007500730074006500720001014101024103470681020103410310E40400000200415381038101"}]

while this is what I am supposed to get

<Account>
  <OperationalStatus>6</OperationalStatus>
  <SubCluster>AS</SubCluster>
</Account>

Thanks again!

#43 Re: mORMot Framework » Results from Stored Procedure (MS SQL) Returned as Base64? » 2014-08-23 12:43:59

Hi Arnaud,

As per my previous post, if I 'force' it as a column via sub-SELECT, then yes, it returns correctly as a string column using ColumnString or ColumnUTF8.

Otherwise, if I force the ColumnType to ftUTF8 in BindColumns, it will return the hex equivalent of the binary blob. The binary blob is not a real UTF-16, I suppose? Because if I force a conversion to UnicodeString using RawUnicodeToString(Data.ColumnBlob(0)), I get a mix of weird Asian characters especially where the angle brackets ("<" and ">") are.

#44 Re: mORMot Framework » Results from Stored Procedure (MS SQL) Returned as Base64? » 2014-08-22 15:12:49

Of course, if I do this

SELECT 
(SELECT	
  AccountCode, 
  CompanyName, 
  (
    SELECT Address1, Address2, PostCode
    FROM vwAddress vad
    WHERE vad.AccountID = va.AccountID
    FOR XML PATH ('Address'), ROOT('Addresses'), TYPE
  )
FROM VwAccount va
WHERE ClientID = @ClientID
FOR XML PATH ('Account')
)

then it works beautifully. But alas, I am not allowed to amend the code for their stored proc

#45 Re: mORMot Framework » Results from Stored Procedure (MS SQL) Returned as Base64? » 2014-08-22 15:07:11

Same result for both compatibility settings, and for both SQLNCLI11 and SQLNCLI10 drivers sad

I have a feeling I have to turn down this project...? Lol

#46 Re: mORMot Framework » Results from Stored Procedure (MS SQL) Returned as Base64? » 2014-08-22 14:47:09

It contains 128, which is DBTYPE_BYTES ... I guess that's why it is treated as BLOB?

#47 Re: mORMot Framework » Results from Stored Procedure (MS SQL) Returned as Base64? » 2014-08-22 14:17:54

Hi Arnaud,

Yes, I did try with the SQLNCLI11 provider. Also tried both of the DataTypeCompatibility values, neither work.

With ODBC, the statement executed, but .Step returns False?

#48 Re: mORMot Framework » Results from Stored Procedure (MS SQL) Returned as Base64? » 2014-08-22 12:26:06

In short, nope sad

It was already a miracle that the client agreed to let us take a peek at the source code for their stored proc :-/

#49 Re: mORMot Framework » Results from Stored Procedure (MS SQL) Returned as Base64? » 2014-08-22 11:34:59

Thanks for the reply!

Ord(Data.ColumnType(0)) returns 7, which maps to ftBlob, I believe.

There is no RawUTF8ToString function, I tried UTF8ToString, it output some binary characters.

RawUnicodeToString works partially, but 20% of the text got converted into some weird Asian characters smile

#50 mORMot Framework » Results from Stored Procedure (MS SQL) Returned as Base64? » 2014-08-22 09:50:16

cheemeng
Replies: 24

Hi Arnaud,

I have a stored procedure which returns an XML fragment due to the need to encapsulate a 1-to-Many result as one of the fields:

SELECT	
  AccountCode, 
  CompanyName, 
  (
    SELECT Address1, Address2, PostCode
    FROM vwAddress vad
    WHERE vad.AccountID = va.AccountID
    FOR XML PATH ('Address'), ROOT('Addresses'), TYPE
  )
FROM VwAccount va
WHERE ClientID = @ClientID
FOR XML PATH ('Account')

And will generate something like

<Account><AccountCode>621</AccountCode><CompanyName>Acme Inc</CompanyName><Addresses><Address><Address1>21 Jump St</Address1><Address2 /><PostCode>214012</PostCode></Address></Addresses></Account>

Here is the code I use to test the execution of the Stored Proc:

var
  Props: TSQLDBConnectionProperties;
  Data: ISQLDBRows;
  XML: RawUTF8;
begin
  Props := TOleDBMSSQLConnectionProperties.Create(edtServer.Text, edtDatabase.Text, edtUsername.Text, edtPassword.Text);
  Data := Props.Execute('Exec GetAccount ?', ['2928-12-34']);

  while Data.Step() do
  begin
    XML := Data.ColumnString(0);
//    Base64MagicDecode(XML);  <--- this does not work either, 
    Memo1.Lines.Add(XML);
  end;
  Props.Free;
end;

I tried using Base64MagicDecode because the dump starts with

ï¿°RAdBAGMAYwBvAHUAbgB0AEQLQQBjAGMAbwB1AG

but it returns an empty string.

Did I miss something here?


Thank you!

Board footer

Powered by FluxBB