#2 Re: mORMot 1 » UTF8 bug » 2015-01-10 08:26:20

Very interesting and confusing. Thanks! smile

EDIT:

Why do I get these warnings?

var
  Str: RawUTF8;
  A: RawUTF8;

function ExtractBetween(const Value, A, B: RawUTF8): RawUTF8;
var
  aPos, bPos: Integer;
begin
  result := '';
  aPos := SynCommons.Pos(A, Value);
  if aPos > 0 then
  begin
    aPos := aPos + Length(A);
    bPos := SynCommons.PosEx(B, Value, aPos);
    if bPos > 0 then
    begin
      result := Copy(Value, aPos, bPos - aPos);
    end;
  end;
end;

begin
  A := '中'; //[dcc32 Warning] Project3.dpr(32): W1062 Narrowing given wide string constant lost information
  Str := '中國哲學書電子化計劃';
  A := ExtractBetween(Str, '中國', '計劃');
  if A = '哲學書電子化' then
  WriteLn('Parsed Correctly.');
  ReadLn;
end.

#3 Re: mORMot 1 » UTF8 bug » 2015-01-10 07:37:50

But how else do you find true position of some chinese substring in a large text? smile

#4 mORMot 1 » UTF8 bug » 2015-01-10 00:26:23

louis_riviera
Replies: 8

Hi!

Seems a bug in SynCommons.pas ?

program Project3;

{$APPTYPE CONSOLE}

{$R *.res}

uses
  System.SysUtils, SynCommons;

var
  Str: RawUTF8;
  SubStr: RawUTF8;
  I: Integer;
begin
  Str := 'ийскf';
  SubStr := 'к';
  I := SynCommons.PosEx(SubStr, Str); // I := 7 ??? It should be 3
end.

EDIT:

Ah! It seems I need to div SizeOf(Char); smile

#5 Re: Low level and performance » Cookies » 2014-08-11 11:54:54

Ofcourse you can manipulate cookies! The problem is SynCrt didn't implement it... sadly.

Will need to make my own wrapper.

http://msdn.microsoft.com/en-us/library … s.85).aspx

#6 Low level and performance » Cookies » 2014-08-09 11:38:42

louis_riviera
Replies: 2

How to manage cookies with Synopse WinHTTP ? Is there something like TIdCookieManager for WinHTTP? smile

#7 Low level and performance » SetLength/Length » 2014-07-21 22:22:35

louis_riviera
Replies: 1

Does SynCommons.pas have fast routines for this? They aren't very optimized in XE6..

#8 Re: Low level and performance » Random() » 2014-07-16 10:20:12

Is better to use threadvar IMHO. Locks are slower, much slower.

#9 Re: Low level and performance » Random() » 2014-07-16 07:35:26

Random() is not thread safe at all. Because it is not a thread var. Plus 64 bit is a lot slower since its written in ASM.

#10 Low level and performance » Random() » 2014-07-10 16:23:44

louis_riviera
Replies: 4

Is there an alternative for this RTL random generator? It is not very fast or accurate.

An SSE optimized version would be nice but i cannot find any in SynCommons.pas

#11 Re: Other components » SynCommons.pas Int32ToUTF8 question » 2014-02-21 16:14:39

dwUtils.pas has a super fast version of IntToStr32/64 Unicode version ofcourse. smile

IMHO UTF8 is not worth it. Too many problems. You end up working AGAINST the compiler.

But if you want even more raw performance you should write it with ICC(Intel C++).

#12 Low level and performance » UTF8 -> String » 2014-02-14 20:00:17

louis_riviera
Replies: 1

Does SynCommons.pas have a faster routine than Delphi for PAnsiChar, AnsiString to String conversions? Converting PAnsiChar to String is very costly in Delphi XE5.

#13 Re: Low level and performance » FastMove » 2013-12-14 10:27:08

What is the license of SynCommons? I would like to extract all the low level RTL replacements and add it in to a more light unit. With your license ofcourse.

#14 Re: Low level and performance » FastMove » 2013-12-13 12:24:00

There is no need for syncommons optimized move. ScaleMM2 has added this for SSE3.

https://code.google.com/p/scalemm/sourc … n117&r=117

#15 Re: Low level and performance » FastMove » 2013-12-11 15:57:10

Actually is not the same. Your supports SSE2 MAX! That one supports from MMX - SSE3. Now cpus support SSE 4.1, 4.2 etc

So not the same.

#16 Low level and performance » FastMove » 2013-11-23 11:19:57

louis_riviera
Replies: 7

Hello!

Here is an MMX, SSE, SSE2, SSE3 optimized Move function but AFAIK it don't support x64.

Maybe possible to include in SynCommons.pas smile

http://alex.ciobanu.org/?p=39

#18 Re: mORMot 1 » TTextWriter » 2013-11-20 21:40:50

I agree but Eric's implementation works with UNICODE. And that will work with rest of components and that needs no to mess with conversions.

#19 Re: mORMot 1 » TTextWriter » 2013-11-20 20:53:00

Yes but easier to maintain IMHO... it is the future of Delphi like it or not. In 5 years it will be standard.

#20 Re: mORMot 1 » TTextWriter » 2013-11-20 20:29:47

Will framework use namespaces? smile

#21 Re: mORMot 1 » TTextWriter » 2013-11-20 17:59:46

Ah we found TWriteOnlyBlockStream. Which is about 10x faster than TTextWriter when using regular strings. It is about 20ms faster when TTextWriter is using UTF8.

#22 Re: mORMot 1 » TTextWriter » 2013-11-19 21:19:14

Yes only lazarus doesn't support it however what I remember, any Delphi version after Delphi 7 supports it. But that doesn't mean FPC will not support it in the future! big_smile

Syn.Common.pas
Syn.Compression.pas

and so on..

#23 Re: mORMot 1 » TTextWriter » 2013-11-19 20:05:01

Yes it would be very much great to split it in to namespaces much like newer versions do it..

System.Sysutils
System.Types

etc..

#24 Re: mORMot 1 » TTextWriter » 2013-11-19 17:57:35

IMHO better to use TMemoryStream to write / append and it will have similar performance and much better than TStringBuilder smile

For us SynCommons.pas is too complex / bloated.

#25 Re: mORMot 1 » TTextWriter » 2013-11-19 15:31:59

Yes but where to do the conversions for example? When interfacing?

#26 Re: mORMot 1 » TTextWriter » 2013-11-19 15:24:15

Agree but you cannot use any functions in new version because whole unit/project based on RAWUTF8 big_smile

#27 Re: mORMot 1 » TTextWriter » 2013-11-19 15:12:26

Only slightly by a couple of ms. Not much. But RawUTF8 string is alot faster. However you cannot use it with VCL/RTL in newer versions.
Due to constant conversions..

program Project6;

{$APPTYPE CONSOLE}

{$R *.res}

uses
  System.SysUtils,
  System.Diagnostics,
  SynCommons;

var
  TW: TTextWriter;
  SW: TStopwatch;
  StrA: String;
  StrB: RawUTF8;
  i, j: Integer;
begin
  j := 100000;
  StrA := 'test string';
  StrB := 'test string';
  SW := TStopwatch.Create;
  TW := TTextWriter.Create(nil);
  TW.CreateOwnedStream;

  // delphi UTF16
  SW.Start;
  for i := 0 to j do
  TW.AddString(StrA);
  SW.Stop;
  TW.Flush;
  TW.CancelAll;
  WriteLn('String: '+SW.ElapsedMilliseconds.ToString());
  SW.Reset;

  // RawUTF8
  SW.Start;
  for i := 0 to j do
  TW.AddString(StrB);
  SW.Stop;
  TW.Flush;
  TW.CancelAll;
  WriteLn('RawUTF8: '+SW.ElapsedMilliseconds.ToString());
  SW.Reset;

  // StringToUTF8
  SW.Start;
  for i := 0 to j do
  TW.AddString(StringToUTF8(StrA));
  SW.Stop;
  TW.Flush;
  TW.CancelAll;
  WriteLn('StringToUTF8: '+SW.ElapsedMilliseconds.ToString());
  Readln;
end.

#28 Re: mORMot 1 » TTextWriter » 2013-11-19 14:43:07

Yes but i think there is performance penalty regardless of implicit or explicit. String(RawUTF8) or RawUTF8(String)..

#29 Re: mORMot 1 » TTextWriter » 2013-11-19 12:51:00

Anyway {$WARN IMPLICIT_STRING_CAST OFF} is the solution for using Syncommons.pas from XE4. Nasty solution but it works.

#30 Re: mORMot 1 » TTextWriter » 2013-11-19 11:42:12

I will try to convert TTextWriter to use String without UTF8 fiasco smile

#31 Re: mORMot 1 » TTextWriter » 2013-11-19 09:29:10

@ab but how to store regular unicode string types in TTextWriter? Do we have to use UTF8ToString / StringToUTF8 ? This would be very very inefficient if used in a loop. IMHO SynCommons.pas is unusable because of this on newer versions.

It would be good to add something like this..

procedure TTextWriter.AddString(const S: String);
begin
..
end;

Reason?

[dcc32 Warning] unit.pas(76): W1057 Implicit string cast from 'Char' to 'RawUTF8'
[dcc32 Warning] unit.pas(79): W1057 Implicit string cast from 'RawUTF8' to 'string'

Or atleast some switch to turn RawUTF8 = String.

#32 Re: mORMot 1 » TTextWriter » 2013-11-18 13:22:41

TTextWriter needs to be renamed because there is an exact same class in System.Classes in XE4 Update 1!

http://docwiki.embarcadero.com/Librarie … iter.Write

#33 Re: mORMot 1 » TTextWriter » 2013-11-17 16:34:14

RTL/VCL needs to recompile to strip the RTTI out.

#34 Re: mORMot 1 » TTextWriter » 2013-11-17 15:56:37

I agree but 700kb is added to Console application using XE4 Update 1 - 32 bit. Just for using TTextWriter. I think SynLZ could be added with a compiler switch. smile

I doubt 700kb is only for TTextWriter respectively..

#35 Re: mORMot 1 » TTextWriter » 2013-11-17 11:56:32

More readable? It needs a simple "Result := Self" in the function. I am sure you can add this to the unit. smile
The problem I see with SynCommons.pas is that its too big. It adds 2mb to debug and 600kb to a console application just
by using TTextWriter. There seems to be a lot of dead code left !

program Project6;

{$APPTYPE CONSOLE}

{$R *.res}

uses
  System.SysUtils,
  SynCommons;

var
  TW: TTextWriter;

begin
  TW := TTextWriter.Create(nil);
  TW.CreateOwnedStream;
  try
    WriteLn(TW.AddString('Hello').AddString(' from').AddString(' Synopse forums!').Text);
  finally
    TW.Free;
  end;

  Readln;
end.

#36 mORMot 1 » TTextWriter » 2013-11-14 21:52:32

louis_riviera
Replies: 35

Hello!

Is it possible to add the TStringBuilder Append way of adding things?

For example now in TTextWriter there is AddString but you cannot do this

TStringBuilder.Append().Append().Append() .. etc

Thanks!

#37 Re: Low level and performance » Fast TStringList » 2013-10-27 11:15:28

Yes but i think an implementation based on THeapMemoryStream would be very very fast. Because TStringList has advantages over TStringBuilder but it consume a lot of memory!

#38 Low level and performance » Fast TStringList » 2013-10-26 21:41:41

louis_riviera
Replies: 22

Is there a fast implementation of TStringList like object? Based on TStringBuilder or DynArray?

There is ofcourse TRawUTF8List but its about 100ms slower than regular TStringList.

#39 Re: Low level and performance » How to write fast multi-thread Delphi applications » 2013-10-26 16:05:45

Yes but it would be nice if you could provide a 64 bit .obj file. smile

DISQLite3 supports 64 bit since ages! big_smile

EDIT: We decided to use UNIDAC.

#40 Re: Low level and performance » How to write fast multi-thread Delphi applications » 2013-10-26 15:27:53

Ah yes so we need to compile 64 bit our self to get .obj? I really don't like dll mess big_smile

#41 Re: Low level and performance » How to write fast multi-thread Delphi applications » 2013-10-26 15:12:57

Does SynCommons contain a fast dynamic array which dont use LOCK instructions?

Also is it possible to use sqlite3 with SynSQLite3 without dll? In 64 bit?

#45 Re: Low level and performance » How to write fast multi-thread Delphi applications » 2013-10-25 07:07:04

So only solution is to modify System.pas? But i guess how to recompile? They add these stupid locks everywhere.

With this lock isn't called.. LStrAsg is called and LOCK is skipped.

procedure Test(const A,B:Integer; var Return:RAWUTF8);
begin
  Return := 'A';
end;

Correct way of using?

#47 Re: Low level and performance » How to write fast multi-thread Delphi applications » 2013-10-24 15:43:09

I think this needs to be fixed on compiler side. Fiddling with code is waste of time and productivity. If you fix the string then you still other components that use it. So it is pretty much pointless. Must be fixed by Embercadero..

#48 Re: Low level and performance » How to write fast multi-thread Delphi applications » 2013-10-24 13:47:58

Oh I forgot to mention this was compiled with Delphi XE5. And analyzed with Dissasembly tool via Delphi Debugger.
I almost get the same result with function. So this solution doesn't really do anything..

I think this happens only when you use old old compilers such as D7. But not new.

function Test(const A, B, C: Integer):String;
begin
  Result := 'Test';
end;

Turns to:

TForm2.Test:
005B4910 55               push ebp
005B4911 8BEC             mov ebp,esp
005B4913 83C4F4           add esp,-$0c
005B4916 894DF4           mov [ebp-$0c],ecx
005B4919 8955F8           mov [ebp-$08],edx
005B491C 8945FC           mov [ebp-$04],eax
005B491F 8B4508           mov eax,[ebp+$08]
005B4922 BA40495B00       mov edx,$005b4940
005B4927 E8604DE5FF       call @UStrAsg
procedure Test(const A, B, C: Integer; var Return:String);
begin
  Return := 'Test';
end;

Turns to:

TForm2.Test:
005B4918 55                 push ebp
005B4919 8BEC              mov ebp,esp
005B491B 83C4F4           add esp,-$0c
005B491E 894DF4           mov [ebp-$0c],ecx
005B4921 8955F8           mov [ebp-$08],edx
005B4924 8945FC           mov [ebp-$04],eax
005B4927 8B4508           mov eax,[ebp+$08]
005B492A BA48495B00    mov edx,$005b4948
005B492F E8584DE5FF    call @UStrAsg

The more important question not raised here is.. What to use instead of "string" ??

#49 Re: Low level and performance » How to write fast multi-thread Delphi applications » 2013-10-24 12:40:12

The problem is that "var" also adds this "call @UStrAsg"

with this procedure Test(const A, B, C: Integer; var Return: String);

So what is real example on how to solve this?

#50 Re: mORMot 1 » Thread-safety of mORMot » 2013-10-03 16:19:19

Well hopefully mpv can provide some examples on how to use it. Specifically how to add new functions etc.

Board footer

Powered by FluxBB