#1 2013-11-14 21:52:32

louis_riviera
Member
Registered: 2013-09-23
Posts: 61

TTextWriter

Hello!

Is it possible to add the TStringBuilder Append way of adding things?

For example now in TTextWriter there is AddString but you cannot do this

TStringBuilder.Append().Append().Append() .. etc

Thanks!

Offline

#2 2013-11-15 15:25:17

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,182
Website

Re: TTextWriter

I'm not so convinced by the benefit of such a "fluent" interface here, since you will need to create a local variable to Free the instance.

IMHO fluent nested calls does make sense when working with interfaces (and auto-free - see our stubbing/mocking methods), or owned class (like mORMot interface-based services configuration), not with class instances.

You can use something like:

 with TTextWriter.CreateOwned do
 try
    Add(...);
    Add(...);
    SetText(aResultVariable);
 finally
   Free;
  end;

Which sound more readable (and secure) than the TStringBuilder syntax - see http://docwiki.embarcadero.com/CodeExam … t_(Delphi)
BTW, how awful is such sample code, when the TStringBuilder instance is not protected by a try/finally Free...

Offline

#3 2013-11-17 11:56:32

louis_riviera
Member
Registered: 2013-09-23
Posts: 61

Re: TTextWriter

More readable? It needs a simple "Result := Self" in the function. I am sure you can add this to the unit. smile
The problem I see with SynCommons.pas is that its too big. It adds 2mb to debug and 600kb to a console application just
by using TTextWriter. There seems to be a lot of dead code left !

program Project6;

{$APPTYPE CONSOLE}

{$R *.res}

uses
  System.SysUtils,
  SynCommons;

var
  TW: TTextWriter;

begin
  TW := TTextWriter.Create(nil);
  TW.CreateOwnedStream;
  try
    WriteLn(TW.AddString('Hello').AddString(' from').AddString(' Synopse forums!').Text);
  finally
    TW.Free;
  end;

  Readln;
end.

Last edited by louis_riviera (2013-11-17 12:37:17)

Offline

#4 2013-11-17 13:35:32

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,182
Website

Re: TTextWriter

Your code is just broken - you are using 2 constructors, the first one will make access violation since the inner stream is nil and the TW.CreateOwnedStream will do nothing but leak memory.

I've added TTextWriter.Add(const Values: array of const) method.
You could be able to write e.g.

with TTextWriter.CreateOwnedStream do
try
  Add(['Hello',' from',' Synopse forums!');
  Writeln(Text);
finally
  Free;
end;

See http://synopse.info/fossil/info/b55ead20df

Delphi compiler "smart linking" of the Delphi compiler would not add any dead code.
If you compile with "debug", it may add a lot of unused content.
If you compile with "release" (or use an older version of Delphi), you won't suffer from it.

Offline

#5 2013-11-17 15:34:19

mpv
Member
From: Ukraine
Registered: 2012-03-24
Posts: 1,534
Website

Re: TTextWriter

IMHO AB is right in this discussion. From my POV chaining is not very good pattern - the worst side of  chaining is debug - How to put breakpoint to the AddString(' from') of line below?

 WriteLn(TW.AddString('Hello').AddString(' from').AddString(' Synopse forums!').Text);

Chaining widely using in JavaScript because of unsafe javascript "with" construction, but in Delphi "with" is enough.

Offline

#6 2013-11-17 15:56:37

louis_riviera
Member
Registered: 2013-09-23
Posts: 61

Re: TTextWriter

I agree but 700kb is added to Console application using XE4 Update 1 - 32 bit. Just for using TTextWriter. I think SynLZ could be added with a compiler switch. smile

I doubt 700kb is only for TTextWriter respectively..

Last edited by louis_riviera (2013-11-17 15:57:10)

Offline

#7 2013-11-17 16:29:33

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,182
Website

Re: TTextWriter

This is a compiler issue, not a mORMot issue.

A blank Delphi XE4 console app is more than 1 MB!

A blank Delphi 7 console app with out LVCL units is about 15 KB.
With SynCommons it is 29 KB.
With TTextWriter it is 46 KB.

This is only one reason why I do not like latest compilers, and use Delphi 7 for such apps.

Offline

#8 2013-11-17 16:34:14

louis_riviera
Member
Registered: 2013-09-23
Posts: 61

Re: TTextWriter

RTL/VCL needs to recompile to strip the RTTI out.

Last edited by louis_riviera (2013-11-17 17:26:33)

Offline

#9 2013-11-18 13:22:41

louis_riviera
Member
Registered: 2013-09-23
Posts: 61

Re: TTextWriter

TTextWriter needs to be renamed because there is an exact same class in System.Classes in XE4 Update 1!

http://docwiki.embarcadero.com/Librarie … iter.Write

Last edited by louis_riviera (2013-11-18 13:23:27)

Offline

#10 2013-11-18 16:08:06

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,182
Website

Re: TTextWriter

Without any fluent interface.... pfffffffuiiiiii....

We did commit TTextWriter years before XE4 update 1!
sad

Offline

#11 2013-11-19 09:29:10

louis_riviera
Member
Registered: 2013-09-23
Posts: 61

Re: TTextWriter

@ab but how to store regular unicode string types in TTextWriter? Do we have to use UTF8ToString / StringToUTF8 ? This would be very very inefficient if used in a loop. IMHO SynCommons.pas is unusable because of this on newer versions.

It would be good to add something like this..

procedure TTextWriter.AddString(const S: String);
begin
..
end;

Reason?

[dcc32 Warning] unit.pas(76): W1057 Implicit string cast from 'Char' to 'RawUTF8'
[dcc32 Warning] unit.pas(79): W1057 Implicit string cast from 'RawUTF8' to 'string'

Or atleast some switch to turn RawUTF8 = String.

Last edited by louis_riviera (2013-11-19 11:35:50)

Offline

#12 2013-11-19 11:38:40

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,182
Website

Re: TTextWriter

It is worth saying that TTextWriter is not an alternative to TStringBuilder or Classes.TTextWriter.

TTextWriter is a class used internally by all our classes, which rely on UTF-8 from the ground up.
All units of mORMot rely on UTF-8 encoding for its Unicode storage.

You already have the following methods:

procedure TTextWriter.AddNoJSONEscapeString(const s: string);
procedure TTextWriter.AddJSONEscapeString(const s: string);

Offline

#13 2013-11-19 11:42:12

louis_riviera
Member
Registered: 2013-09-23
Posts: 61

Re: TTextWriter

I will try to convert TTextWriter to use String without UTF8 fiasco smile

Offline

#14 2013-11-19 12:51:00

louis_riviera
Member
Registered: 2013-09-23
Posts: 61

Re: TTextWriter

Anyway {$WARN IMPLICIT_STRING_CAST OFF} is the solution for using Syncommons.pas from XE4. Nasty solution but it works.

Offline

#15 2013-11-19 14:34:45

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,182
Website

Re: TTextWriter

louis_riviera wrote:

Anyway {$WARN IMPLICIT_STRING_CAST OFF} is the solution for using Syncommons.pas from XE4. Nasty solution but it works.

Yes, but you should NEVER turn such warnings OFF, since you may loose some Unicode content if your destination string is not UTF-8, but e.g. of a given charset.

In mORMot, we made all string type casts implicit, to ensure there is no such issue, nor unnoticed performance penalty.

It is IMHO a bad idea to trun this warning off.

Offline

#16 2013-11-19 14:43:07

louis_riviera
Member
Registered: 2013-09-23
Posts: 61

Re: TTextWriter

Yes but i think there is performance penalty regardless of implicit or explicit. String(RawUTF8) or RawUTF8(String)..

Offline

#17 2013-11-19 15:04:01

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,182
Website

Re: TTextWriter

Explicit StringToUTF8() via our optimized units will be faster than string() or the warning.

Offline

#18 2013-11-19 15:12:26

louis_riviera
Member
Registered: 2013-09-23
Posts: 61

Re: TTextWriter

Only slightly by a couple of ms. Not much. But RawUTF8 string is alot faster. However you cannot use it with VCL/RTL in newer versions.
Due to constant conversions..

program Project6;

{$APPTYPE CONSOLE}

{$R *.res}

uses
  System.SysUtils,
  System.Diagnostics,
  SynCommons;

var
  TW: TTextWriter;
  SW: TStopwatch;
  StrA: String;
  StrB: RawUTF8;
  i, j: Integer;
begin
  j := 100000;
  StrA := 'test string';
  StrB := 'test string';
  SW := TStopwatch.Create;
  TW := TTextWriter.Create(nil);
  TW.CreateOwnedStream;

  // delphi UTF16
  SW.Start;
  for i := 0 to j do
  TW.AddString(StrA);
  SW.Stop;
  TW.Flush;
  TW.CancelAll;
  WriteLn('String: '+SW.ElapsedMilliseconds.ToString());
  SW.Reset;

  // RawUTF8
  SW.Start;
  for i := 0 to j do
  TW.AddString(StrB);
  SW.Stop;
  TW.Flush;
  TW.CancelAll;
  WriteLn('RawUTF8: '+SW.ElapsedMilliseconds.ToString());
  SW.Reset;

  // StringToUTF8
  SW.Start;
  for i := 0 to j do
  TW.AddString(StringToUTF8(StrA));
  SW.Stop;
  TW.Flush;
  TW.CancelAll;
  WriteLn('StringToUTF8: '+SW.ElapsedMilliseconds.ToString());
  Readln;
end.

Last edited by louis_riviera (2013-11-19 15:12:56)

Offline

#19 2013-11-19 15:20:20

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,182
Website

Re: TTextWriter

louis_riviera wrote:

However you cannot use it with VCL/RTL in newer versions.
Due to constant conversions..

But you should not use it to produce "plain string" content.
As I already wrote, TTextWriter is not to be used as a TStringBuilder replacement!
It is used internally in our units, which are UTF-8 based, not UTF-16 based.

THIS IS BY DESIGN!
big_smile

Offline

#20 2013-11-19 15:24:15

louis_riviera
Member
Registered: 2013-09-23
Posts: 61

Re: TTextWriter

Agree but you cannot use any functions in new version because whole unit/project based on RAWUTF8 big_smile

Offline

#21 2013-11-19 15:26:06

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,182
Website

Re: TTextWriter

You have all needed RawUTF8 functions available in SynCommons.pas.
If some functions are missing, please ask for them.

Splitting your work using RawUTF8 within its business code - as with mORMot kernel - and plain String at VCL level is a good idea.
In short, RawUTF8 may help you separate your concerns, and force your code to be more designed.
You won't be tempted to mix logic and UI, for instance.

Offline

#22 2013-11-19 15:31:59

louis_riviera
Member
Registered: 2013-09-23
Posts: 61

Re: TTextWriter

Yes but where to do the conversions for example? When interfacing?

Offline

#23 2013-11-19 17:57:35

louis_riviera
Member
Registered: 2013-09-23
Posts: 61

Re: TTextWriter

IMHO better to use TMemoryStream to write / append and it will have similar performance and much better than TStringBuilder smile

For us SynCommons.pas is too complex / bloated.

Offline

#24 2013-11-19 19:25:24

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,182
Website

Re: TTextWriter

louis_riviera wrote:

For us SynCommons.pas is too complex / bloated.

No one did force you to use it.

It is the ground of our framework, and other units which could be used without mORMot (like SynGdiPlus or SynPdf).
It does not suffer from performance penalty problems, as the official Delphi RTL does version after version.

We found it pretty useful also, even for small programs, since it has nice structures and features (like dynamic array wrapper, or fast string process).

But it is indeed a huge unit, with a lot of features (like UTF-8 JSON low-level marshalling, or logging).
We may split this unit into several smaller peaces.

Offline

#25 2013-11-19 20:05:01

louis_riviera
Member
Registered: 2013-09-23
Posts: 61

Re: TTextWriter

Yes it would be very much great to split it in to namespaces much like newer versions do it..

System.Sysutils
System.Types

etc..

Last edited by louis_riviera (2013-11-19 20:05:36)

Offline

#26 2013-11-19 21:00:41

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,182
Website

Re: TTextWriter

Don't forget that mORMot is compatible with Delphi 6 (even Delphi 5 for the SynCommons and the SynDB part) up to the latest version.

Our reference is more http://www.freepascal.org - which sounds like a better warranty for the future.
We rely do not like the latest orientation of the Delphi compiler.
There are several proposals for namespaces - see http://wiki.freepascal.org/Namespaces - but AFAIK it is not implemented yet.

Units ARE namespaces since the beginning, right?
We tried to be consistent with out unit name: Syn* for all low-level common stuff, SynDB* for external database, mORMot* for framework units.

Offline

#27 2013-11-19 21:19:14

louis_riviera
Member
Registered: 2013-09-23
Posts: 61

Re: TTextWriter

Yes only lazarus doesn't support it however what I remember, any Delphi version after Delphi 7 supports it. But that doesn't mean FPC will not support it in the future! big_smile

Syn.Common.pas
Syn.Compression.pas

and so on..

Last edited by louis_riviera (2013-11-19 21:34:34)

Offline

#28 2013-11-20 17:59:46

louis_riviera
Member
Registered: 2013-09-23
Posts: 61

Re: TTextWriter

Ah we found TWriteOnlyBlockStream. Which is about 10x faster than TTextWriter when using regular strings. It is about 20ms faster when TTextWriter is using UTF8.

Offline

#29 2013-11-20 20:23:32

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,182
Website

Re: TTextWriter

Those two classes are not for the same purpose.
TWriteOnlyBlockStream, whatever its name states, does not write directly into a TStream, but into memory blocks, then at once into a TStream.

I have used TTextWriter to write GB of JSON content or log content, with very small memory use: it write to a TFileStream per chunk.
It won't be possible with TWriteOnlyBlockStream, which will fill all your available memory.

You can chose what fit best with your real need.
And don't trust benchmarks which measure the time to work a loop of fixed content which fits in the CPU L1 cache (like a "for" loop appending constant strings and integers).
I trust real benchmarks with real data, like our regression tests and our performance sample, which uses a database, marshall JSON to and from data, uses RTTI for filling and reading objects, parse an URI, check user rights, and so on...

My current challenge is to provide some code to http://www.techempower.com/benchmarks/
I’m adding MVC support to mORMot currently, using JavaScript BTW.
Here we will see how it works. In the real world… not with a naive and biaised benchmark.

And do not forget to compile your sample in RELEASE mode, not DEBUG mode: you need the compiler optimization to unleash TTextWriter power (which uses inlining in RELEASE mode only).
See the "update" note of Eric's article at http://www.delphitools.info/2013/11/07/ … -building/

Offline

#30 2013-11-20 20:29:47

louis_riviera
Member
Registered: 2013-09-23
Posts: 61

Re: TTextWriter

Will framework use namespaces? smile

Offline

#31 2013-11-20 20:31:07

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,182
Website

Re: TTextWriter

I do not see any reason yet.
We need to keep the backward compatibility with older version of Delphi, for existing projects, and lot of our users.

IMHO namespaces are just cosmetic, and "fashionable".

Offline

#32 2013-11-20 20:53:00

louis_riviera
Member
Registered: 2013-09-23
Posts: 61

Re: TTextWriter

Yes but easier to maintain IMHO... it is the future of Delphi like it or not. In 5 years it will be standard.

Last edited by louis_riviera (2013-11-20 20:53:23)

Offline

#33 2013-11-20 21:08:53

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,182
Website

Re: TTextWriter

So we will see in 5 years.
But I suppose I would have switched to FPC.

Offline

#34 2013-11-20 21:40:50

louis_riviera
Member
Registered: 2013-09-23
Posts: 61

Re: TTextWriter

I agree but Eric's implementation works with UNICODE. And that will work with rest of components and that needs no to mess with conversions.

Offline

#35 2013-11-20 23:57:09

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,182
Website

Re: TTextWriter

Utf8 is Unicode.

Offline

#36 2013-11-21 06:42:16

louis_riviera
Member
Registered: 2013-09-23
Posts: 61

Re: TTextWriter

Sorry I meant UTF16.

Offline

Board footer

Powered by FluxBB