Using FPC under linux x64, bug in UrlDecodeValue.

ComingNine · 2014-11-23 10:38:03

OS: CentOS v5.11 x64
CodeTyphon: v5.10 multiArch
Typhon: x86

Bug: For HTTP access to method-based services, when there is a single string parameter, the server cannot use urldecodevalue to get the value. InputUTF8 works correctly.

Console application code to reproduce:
http://www.rapidfileshare.net/b5oysncw5 … ar.gz.html
http://www40.zippyshare.com/v/9181464/file.html

Furthermore, could you help to change the line feed from #13 to #10 on Linux OS when echoing to the terminal ? When running the programs attached, the terminal displays twice the empty lines.

Last edited by ComingNine (2014-11-23 11:02:57)

ComingNine · 2014-11-23 11:02:39

There is another Bug in UrlDecodeXXX (UrlDecodeExtended)...

For HTTP access to method-based services, when there are multiple concurrent clients by running the following cmd,

for i in `seq 1 8`; do ./JobAnalyzerTcpIpClientLinux uclient.pas > $i.log & done

the server gives the following exception...

20141123 19462751  # srvr  	GET root/sum?A=210.5&B=1.5 ERROR=500 (Exception EOverflow: Floating point overflow)

Again, InputDouble works correctly...

Sample code:
http://www40.zippyshare.com/v/20221523/file.html

AOG · 2014-11-23 11:16:18

Could you please give me somewhat more info about your setup ?
mORMot only supports 32bit under Linux.
Your system is 64 bit. Do you crosscompile to 32 bit ? What libraries do you use to run 32 bit on your 64 bit system.
I need this (detailed) info to run your source under your setup. For correct analysis and bug-hunting.

ComingNine · 2014-11-23 11:37:01

On CentOS v5.11 x64, I built multiArch CodeTyphon v5.10. That is to say, both Typhon 64 and Typhon 32 IDE are available.

I use Typhon 32 IDE, which will use FPC 32.

Many thanks for your efforts !....

ComingNine · 2014-11-24 08:00:06

If you could not reproduce the bug, please let me know. I could either provide a compiled binary, or even the remote access of mine machine...

AOG · 2014-11-24 08:53:42

Your analysis is (nearly) correct ;-)
I cannot get CodeTyphon to install 32 bit on my 64 bit VM Arch Linux until now !
Sorry about this ... I am still trying !!

ComingNine · 2014-11-24 09:10:55

Thank you very much for your efforts !

When I install multiArch CodeTyphon v5.1 on CentOS v5.11 X64, the Big IDE cannot be built, but the Small IDE can be built, which is enough.

Furthermore, if there is linking failure, please try 'ln -s existing_library_name missing_library_name' ...

AOG · 2014-11-24 15:02:42

After a whole day of trying to get CodeTyphon 64/32 to work, I did not succeed and will stop.
It's a known problem on Arch. The linker does not look into /usr/lib32 for libraries.
I cannot solve this (multilib) problem. Hints are welcome.

If you want (my help), please try to reproduce this error on a pure 32bit system. This will make my life (MUCH) easier !
Or try to reproduce by cross-compiling from 64 to 32.
Thanks.

ComingNine · 2014-11-24 15:47:40

Thank you very much for your efforts !
If I test on "32bit CentOS", would it be fine for you ? If yes, I will try CentOS V5.11 X86.

AOG · 2014-11-24 17:27:16

Yes please !

ComingNine · 2014-11-25 11:49:25

I just checked the two .tar.gz files using CodeTyphon 5.1 on CentOS v5.11 x86.

Only change of the project settings is the library path "/usr/lib/gcc/i386-redhat-linux/4.1.2/" .
(When using Typhon 32 on CentOS x64, this value should be "/usr/lib/gcc/x86_64-redhat-linux/4.1.2/32".)

Same error applies...

Could you help to comment whether you could reproduce the errors on your x86 ?

Last edited by ComingNine (2014-11-30 02:05:00)

ComingNine · 2014-11-28 15:51:19

Dear AOG, could you help to comment whether you could reproduce the errors on your x86 ?

AOG · 2014-11-28 16:36:16

Sorry about me being not responsive ... !
I was (and still am) very busy with a mORMot application (system abstraction) that uses dataset lookup fields on TID fields.
And this still does not work 100% unfortunately. And I have to finish this first. I expect 2-3 more days of work.
So, I will have a look at your problem next week !!
Greetings, Alfred.

warleyalex · 2014-11-28 21:29:05

I know you use nearly all features of mORMot, on Linux. We have one big issue with FPC which prevents to use interface based services. This problem does not occurr with Kylix. Lazarus/Linux can load shared libraries created with Kylix.
I would like to test some code included at http://hallvards.blogspot.fr/2006/06/si … -rtti.html

Well, in my experiments, actually you can use Kylix shared library with Lazarus, you can call methods and functions, but unfortunately I always get an AV when I call DumpSimpleInterface method.
Any idea?

//libtest.dpr ----compiled with FPC/Lazarus/Linux

program libtest;

{$mode objfpc}{$H+}
{$MODE objfpc}
{$MODESWITCH AdvancedRecords}
{$inline on}
{$h+}

uses SysUtils, SynFPCTypInfo, TypInfo, Unit1;

function AddNum(Num1: Integer; Num2: Integer): Integer; cdecl; external 'mylib';
procedure DumpSimpleInterface(InterfaceTypeInfo: PTypeInfo); cdecl; external 'mylib';

type
  {$M-}
  IMyInterface = interface
    procedure Foo(A: integer);
    procedure Bar(const B: string);
    procedure Nada(const C: array of integer; D: TObject);
  end;
  IMyDispatchInterface = interface(IDispatch)
    ['{9BC5459B-6C31-4F5B-B733-DCA8FC8C1345}']
    procedure Foo; dispid 0;
  end;
  IMyDispInterface = dispinterface
    ['{8574E276-4671-49AC-B775-B299E6EF01C5}']
    procedure Bar;
  end;

var TypeData: PTypeData;
begin
      DumpSimpleInterface(TypeInfo(IMyInterface));
//    DumpSimpleInterface(TypeInfo(IMyDispatchInterface));
//    DumpSimpleInterface(TypeInfo(IMyDispInterface));
//    TypeData:=  SynFPCTypInfo.GetFPCTypeData(TypeInfo(IMyInterface));
//    DumpSimpleInterface(TypeInfo(TypeData));

  if AddNum(2, 3) = 5 then
  WriteLn('success')
  else
  WriteLn('fail');

end.

Project > Options for Project > Compiler Options > Compilation and Linking > Pass options to linker with -R ./

// mylib.dpr --- Compiled with Kylix

library mylib;

{.$mode objfpc}{$H+}

uses
  SysUtils, TypInfo;
 //... Insert the code above here

type
  PExtraInterfaceData = ^TExtraInterfaceData;
  TExtraInterfaceData = packed record
    MethodCount: Word;   { # methods }
  end;
  
function SkipPackedShortString(Value: PShortstring): pointer;
begin
  Result := Value;
  Inc(PChar(Result), SizeOf(Value^[0]) + Length(Value^));
end;  
  
procedure DumpSimpleInterface(InterfaceTypeInfo: PTypeInfo); cdecl;
var
  TypeData: PTypeData;
  ExtraData: PExtraInterfaceData;
  i: integer;
begin
  Assert(Assigned(InterfaceTypeInfo));
  Assert(InterfaceTypeInfo.Kind = tkInterface);
  TypeData := GetTypeData(InterfaceTypeInfo);
  ExtraData := SkipPackedShortString(@TypeData.IntfUnit);
  writeln('unit ', TypeData.IntfUnit, ';');
  writeln('type');
  write('  ', InterfaceTypeInfo.Name, ' = '); 
  if not (ifDispInterface in TypeData.IntfFlags) then
  begin
    write('interface');
    if Assigned(TypeData.IntfParent) then
      write(' (', TypeData.IntfParent^.Name, ')');
    writeln;
  end
  else  
    writeln('dispinterface');
  if ifHasGuid in TypeData.IntfFlags then
    writeln('    [''', GuidToString(TypeData.Guid), ''']');
  for i := 1 to ExtraData.MethodCount do  
    writeln('    procedure UnknownName',i,';');
  writeln('  end;');
  writeln;
end;

function AddNum(Num1: Integer; Num2: Integer): Integer; cdecl;
begin
  Result:= Num1 + Num2;
end;

exports
  AddNum, DumpSimpleInterface;
begin
end.

ab · 2014-11-28 22:42:51

FPC doesn't publish the needed rtti...

So typeinfo() returns invalid data...

ComingNine · 2014-11-29 12:08:35

AOG wrote:

Sorry about me being not responsive ... !
I was (and still am) very busy with a mORMot application (system abstraction) that uses dataset lookup fields on TID fields.
And this still does not work 100% unfortunately. And I have to finish this first. I expect 2-3 more days of work.
So, I will have a look at your problem next week !!
Greetings, Alfred.

Thank you very much for your efforts !...

warleyalex · 2014-11-29 23:13:39

ab wrote:

FPC doesn't publish the needed rtti...
So typeinfo() returns invalid data...

Thank a lot.

I had searched a lot on internet but not found the way to successfully implement in my test program.

I want to link a static library (.a) file in my Lazarus/Linux program.
This is a "rtl.a" file which is just a bunch of compressed .o static files such as
(TypInfo.o, Math.o, Variants.o, etc.). This static library (rtl.a file) is used by the old Delphi/Kylix.

I discovered that I can add a directive {$LINK rtl.a} in my lazarus/Linux program, and everything
is fine, It seems that link the object, my program is successfully compiled!

I have the crazy idea to to use an external function that resides in this static library (rtl.a).
I guess I could use one of those external functions that resides in a object file (TypInfo.o or Math.o) f.i,
but I couldn't find out a way to perform this.

I have to declare a function to use it, something like this:

function Max(const A, B: Integer): Integer; cdecl;
external name 'FVCOMMON_$$_MAX$LONGINT$LONGINT$$LONGINT';

Sadly I have no idea how to use/declare an external function, this is possible?

AOG · 2014-12-02 08:02:52

@ComingNine
I can confirm your findings: I get the same error on my system, with the latet mORMot, under Linux i386.
I will investigate further.

@warleyalex
It should be easily possible to use external functions !
But I do not have a rtl.a, so I cannot reproduce.

AOG · 2014-12-02 10:02:35

@ComingNine
Found the problem !
If you disable ILog (comment out ILog in your Sum method), then everything works as expected (at least on my system), also with extended !
I do not have a solution however.
I think we have to ask Ab, how and why ILog and extended influence each other (on Linux i386, but perhaps also on other systems).
Greetings, Alfred.

ComingNine · 2014-12-02 13:50:41

AOG wrote:

@ComingNine
Found the problem !
If you disable ILog (comment out ILog in your Sum method), then everything works as expected (at least on my system), also with extended !
I do not have a solution however.
I think we have to ask Ab, how and why ILog and extended influence each other (on Linux i386, but perhaps also on other systems).
Greetings, Alfred.

Thank you very much for your efforts !
Could you help to paste your test case here, to discover that both the "UrlDecodeValue" and the "UrlDecodeExtended" are faulty with ILog enabled ?

ab · 2014-12-03 10:07:36

I was not able to reproduce the issue.
... nor even clearly understand its exact scope, sadly.

I'm confused by this thread: there are several nested conversations, and I can not understand what the so called "bug" is.

AOG · 2014-12-03 12:31:25

@Ab.
I think ComingNine can clear things up !
I did have a quick look at the code below, and tried to get it to work on my system.
Commenting out the log-statements made everything run 100%, but I do not know why !

@ComingNine.
My setup: Oracle VM Arch Linux i386 with FPC 2.6.4 and Lazarus 1.2.6
userver.pas:

procedure TJobAnalyzerTcpIpServer.Sum(Context: TSQLRestServerURIContext);
var
  //ILog: ISynLog;
  A, B: extended;
begin
  //ILog := TSQLLog.Enter;
  if UrlDecodeNeedParameters(Context.Parameters, 'A,B') then
  begin
    //ILog.Log(sllInfo, 'A=%n, B=%n', [A, B]);
    while Context.Parameters <> nil do
    begin
      UrlDecodeExtended(Context.Parameters, 'A=', A);
      UrlDecodeExtended(Context.Parameters, 'B=', B, @Context.Parameters);
      //ILog.Log(sllInfo, 'A=%n, B=%n', [A, B]);
    end;
    Context.Results([A+B]);
    //ILog.Log(sllInfo, 'Context.Results([%n])', [A + B]);
  end
  else
    Context.Error('Missing Parameter');
end;

This did work on my system !

ComingNine · 2014-12-03 14:15:07

@AOG and @ab, please let me describe the problem, all of which are related with FPC under Linux, again:

Bug #1: UrlDecodeValue.
For HTTP access to method-based services, when there is a single string parameter, the server cannot use urldecodevalue to get the value. 'Context.Parameters' contains the correct value.
InputUTF8 works correctly.
Please download the sample project using the link below.

http://www40.zippyshare.com/v/9181464/file.html

Please compile and run server in one terminal, and compile and run client in another terminal, and check the log information from the server.

Bug #2: UrlDecodeExtended.
For HTTP access to method-based services, when there are multiple concurrent clients by running the client concurrently.
Please download the sample project using the link below.

http://www40.zippyshare.com/v/20221523/file.html

Please compile and run server in one terminal, and compile and run clients concurrently in another terminal using the following cmd,

for i in `seq 1 8`; do ./JobAnalyzerTcpIpClientLinux uclient.pas > $i.log & done

The server will give the following exception...

20141123 19462751  # srvr  	GET root/sum?A=210.5&B=1.5 ERROR=500 (Exception EOverflow: Floating point overflow)

If I switch UrlDecodeExtended to InputDouble, the exception goes away.

Request #3:
could you help to change the line feed from #13 to #10 on Linux OS when echoing to the terminal ? When running the programs above, the terminal displays twice the empty lines.

Last edited by ComingNine (2014-12-03 14:19:28)

ComingNine · 2015-09-22 09:25:00

@ab
@AOG

I have found the reason:

For the "UrlDecodeValue" case, the function need its second argument to be upper case, which I have clearly overlooked.

For the "UrlDecodeExtended" case, it should be noted first that the while-loop needs to be run multiple times. That is to say, one and only one variable gets assigned at one time. At the first time of the while-loop, the variable B could contain an extremely large or small number, and thus crashing the format function called by ILog.Log.

All in all, there is nothing wrong with mORMot ! Sorry for the confusion that I have brought. mORMot rocks !

Last edited by ComingNine (2015-09-22 09:25:26)

mORMot Open Source

#1 2014-11-23 10:38:03

Using FPC under linux x64, bug in UrlDecodeValue.

#2 2014-11-23 11:02:39

Re: Using FPC under linux x64, bug in UrlDecodeValue.

#3 2014-11-23 11:16:18

Re: Using FPC under linux x64, bug in UrlDecodeValue.

#4 2014-11-23 11:37:01

Re: Using FPC under linux x64, bug in UrlDecodeValue.

#5 2014-11-24 08:00:06

Re: Using FPC under linux x64, bug in UrlDecodeValue.

#6 2014-11-24 08:53:42

Re: Using FPC under linux x64, bug in UrlDecodeValue.

#7 2014-11-24 09:10:55

Re: Using FPC under linux x64, bug in UrlDecodeValue.

#8 2014-11-24 15:02:42

Re: Using FPC under linux x64, bug in UrlDecodeValue.

#9 2014-11-24 15:47:40

Re: Using FPC under linux x64, bug in UrlDecodeValue.

#10 2014-11-24 17:27:16

Re: Using FPC under linux x64, bug in UrlDecodeValue.

#11 2014-11-25 11:49:25

Re: Using FPC under linux x64, bug in UrlDecodeValue.

#12 2014-11-28 15:51:19

Re: Using FPC under linux x64, bug in UrlDecodeValue.

#13 2014-11-28 16:36:16

Re: Using FPC under linux x64, bug in UrlDecodeValue.

#14 2014-11-28 21:29:05

Re: Using FPC under linux x64, bug in UrlDecodeValue.

#15 2014-11-28 22:42:51

Re: Using FPC under linux x64, bug in UrlDecodeValue.

#16 2014-11-29 12:08:35

Re: Using FPC under linux x64, bug in UrlDecodeValue.

#17 2014-11-29 23:13:39

Re: Using FPC under linux x64, bug in UrlDecodeValue.

#18 2014-12-02 08:02:52

Re: Using FPC under linux x64, bug in UrlDecodeValue.

#19 2014-12-02 10:02:35

Re: Using FPC under linux x64, bug in UrlDecodeValue.

#20 2014-12-02 13:50:41

Re: Using FPC under linux x64, bug in UrlDecodeValue.

#21 2014-12-03 10:07:36

Re: Using FPC under linux x64, bug in UrlDecodeValue.

#22 2014-12-03 12:31:25

Re: Using FPC under linux x64, bug in UrlDecodeValue.

#23 2014-12-03 14:15:07

Re: Using FPC under linux x64, bug in UrlDecodeValue.

#24 2015-09-22 09:25:00

Re: Using FPC under linux x64, bug in UrlDecodeValue.

Board footer