TTextWriter

louis_riviera · 2013-11-14 21:52:32

Hello!

Is it possible to add the TStringBuilder Append way of adding things?

For example now in TTextWriter there is AddString but you cannot do this

TStringBuilder.Append().Append().Append() .. etc

Thanks!

ab · 2013-11-15 15:25:17

I'm not so convinced by the benefit of such a "fluent" interface here, since you will need to create a local variable to Free the instance.

IMHO fluent nested calls does make sense when working with interfaces (and auto-free - see our stubbing/mocking methods), or owned class (like mORMot interface-based services configuration), not with class instances.

You can use something like:

 with TTextWriter.CreateOwned do
 try
    Add(...);
    Add(...);
    SetText(aResultVariable);
 finally
   Free;
  end;

Which sound more readable (and secure) than the TStringBuilder syntax - see http://docwiki.embarcadero.com/CodeExam … t_(Delphi)
BTW, how awful is such sample code, when the TStringBuilder instance is not protected by a try/finally Free...

louis_riviera · 2013-11-17 11:56:32

More readable? It needs a simple "Result := Self" in the function. I am sure you can add this to the unit.
The problem I see with SynCommons.pas is that its too big. It adds 2mb to debug and 600kb to a console application just
by using TTextWriter. There seems to be a lot of dead code left !

program Project6;

{$APPTYPE CONSOLE}

{$R *.res}

uses
  System.SysUtils,
  SynCommons;

var
  TW: TTextWriter;

begin
  TW := TTextWriter.Create(nil);
  TW.CreateOwnedStream;
  try
    WriteLn(TW.AddString('Hello').AddString(' from').AddString(' Synopse forums!').Text);
  finally
    TW.Free;
  end;

  Readln;
end.

Last edited by louis_riviera (2013-11-17 12:37:17)

ab · 2013-11-17 13:35:32

Your code is just broken - you are using 2 constructors, the first one will make access violation since the inner stream is nil and the TW.CreateOwnedStream will do nothing but leak memory.

I've added TTextWriter.Add(const Values: array of const) method.
You could be able to write e.g.

with TTextWriter.CreateOwnedStream do
try
  Add(['Hello',' from',' Synopse forums!');
  Writeln(Text);
finally
  Free;
end;

See http://synopse.info/fossil/info/b55ead20df

Delphi compiler "smart linking" of the Delphi compiler would not add any dead code.
If you compile with "debug", it may add a lot of unused content.
If you compile with "release" (or use an older version of Delphi), you won't suffer from it.

mpv · 2013-11-17 15:34:19

IMHO AB is right in this discussion. From my POV chaining is not very good pattern - the worst side of chaining is debug - How to put breakpoint to the AddString(' from') of line below?

 WriteLn(TW.AddString('Hello').AddString(' from').AddString(' Synopse forums!').Text);

Chaining widely using in JavaScript because of unsafe javascript "with" construction, but in Delphi "with" is enough.

louis_riviera · 2013-11-17 15:56:37

I agree but 700kb is added to Console application using XE4 Update 1 - 32 bit. Just for using TTextWriter. I think SynLZ could be added with a compiler switch.

I doubt 700kb is only for TTextWriter respectively..

Last edited by louis_riviera (2013-11-17 15:57:10)

ab · 2013-11-17 16:29:33

This is a compiler issue, not a mORMot issue.

A blank Delphi XE4 console app is more than 1 MB!

A blank Delphi 7 console app with out LVCL units is about 15 KB.
With SynCommons it is 29 KB.
With TTextWriter it is 46 KB.

This is only one reason why I do not like latest compilers, and use Delphi 7 for such apps.

louis_riviera · 2013-11-17 16:34:14

RTL/VCL needs to recompile to strip the RTTI out.

Last edited by louis_riviera (2013-11-17 17:26:33)

louis_riviera · 2013-11-18 13:22:41

TTextWriter needs to be renamed because there is an exact same class in System.Classes in XE4 Update 1!

http://docwiki.embarcadero.com/Librarie … iter.Write

Last edited by louis_riviera (2013-11-18 13:23:27)

ab · 2013-11-18 16:08:06

Without any fluent interface.... pfffffffuiiiiii....

We did commit TTextWriter years before XE4 update 1!

louis_riviera · 2013-11-19 09:29:10

@ab but how to store regular unicode string types in TTextWriter? Do we have to use UTF8ToString / StringToUTF8 ? This would be very very inefficient if used in a loop. IMHO SynCommons.pas is unusable because of this on newer versions.

It would be good to add something like this..

procedure TTextWriter.AddString(const S: String);
begin
..
end;

Reason?

[dcc32 Warning] unit.pas(76): W1057 Implicit string cast from 'Char' to 'RawUTF8'
[dcc32 Warning] unit.pas(79): W1057 Implicit string cast from 'RawUTF8' to 'string'

Or atleast some switch to turn RawUTF8 = String.

Last edited by louis_riviera (2013-11-19 11:35:50)

ab · 2013-11-19 11:38:40

It is worth saying that TTextWriter is not an alternative to TStringBuilder or Classes.TTextWriter.

TTextWriter is a class used internally by all our classes, which rely on UTF-8 from the ground up.
All units of mORMot rely on UTF-8 encoding for its Unicode storage.

You already have the following methods:

procedure TTextWriter.AddNoJSONEscapeString(const s: string);
procedure TTextWriter.AddJSONEscapeString(const s: string);

louis_riviera · 2013-11-19 11:42:12

I will try to convert TTextWriter to use String without UTF8 fiasco

louis_riviera · 2013-11-19 12:51:00

Anyway {$WARN IMPLICIT_STRING_CAST OFF} is the solution for using Syncommons.pas from XE4. Nasty solution but it works.

ab · 2013-11-19 14:34:45

louis_riviera wrote:

Anyway {$WARN IMPLICIT_STRING_CAST OFF} is the solution for using Syncommons.pas from XE4. Nasty solution but it works.

Yes, but you should NEVER turn such warnings OFF, since you may loose some Unicode content if your destination string is not UTF-8, but e.g. of a given charset.

In mORMot, we made all string type casts implicit, to ensure there is no such issue, nor unnoticed performance penalty.

It is IMHO a bad idea to trun this warning off.

louis_riviera · 2013-11-19 14:43:07

Yes but i think there is performance penalty regardless of implicit or explicit. String(RawUTF8) or RawUTF8(String)..

ab · 2013-11-19 15:04:01

Explicit StringToUTF8() via our optimized units will be faster than string() or the warning.

louis_riviera · 2013-11-19 15:12:26

Only slightly by a couple of ms. Not much. But RawUTF8 string is alot faster. However you cannot use it with VCL/RTL in newer versions.
Due to constant conversions..

program Project6;

{$APPTYPE CONSOLE}

{$R *.res}

uses
  System.SysUtils,
  System.Diagnostics,
  SynCommons;

var
  TW: TTextWriter;
  SW: TStopwatch;
  StrA: String;
  StrB: RawUTF8;
  i, j: Integer;
begin
  j := 100000;
  StrA := 'test string';
  StrB := 'test string';
  SW := TStopwatch.Create;
  TW := TTextWriter.Create(nil);
  TW.CreateOwnedStream;

  // delphi UTF16
  SW.Start;
  for i := 0 to j do
  TW.AddString(StrA);
  SW.Stop;
  TW.Flush;
  TW.CancelAll;
  WriteLn('String: '+SW.ElapsedMilliseconds.ToString());
  SW.Reset;

  // RawUTF8
  SW.Start;
  for i := 0 to j do
  TW.AddString(StrB);
  SW.Stop;
  TW.Flush;
  TW.CancelAll;
  WriteLn('RawUTF8: '+SW.ElapsedMilliseconds.ToString());
  SW.Reset;

  // StringToUTF8
  SW.Start;
  for i := 0 to j do
  TW.AddString(StringToUTF8(StrA));
  SW.Stop;
  TW.Flush;
  TW.CancelAll;
  WriteLn('StringToUTF8: '+SW.ElapsedMilliseconds.ToString());
  Readln;
end.

Last edited by louis_riviera (2013-11-19 15:12:56)

ab · 2013-11-19 15:20:20

louis_riviera wrote:

However you cannot use it with VCL/RTL in newer versions.
Due to constant conversions..

But you should not use it to produce "plain string" content.
As I already wrote, TTextWriter is not to be used as a TStringBuilder replacement!
It is used internally in our units, which are UTF-8 based, not UTF-16 based.

THIS IS BY DESIGN!

louis_riviera · 2013-11-19 15:24:15

Agree but you cannot use any functions in new version because whole unit/project based on RAWUTF8

ab · 2013-11-19 15:26:06

You have all needed RawUTF8 functions available in SynCommons.pas.
If some functions are missing, please ask for them.

Splitting your work using RawUTF8 within its business code - as with mORMot kernel - and plain String at VCL level is a good idea.
In short, RawUTF8 may help you separate your concerns, and force your code to be more designed.
You won't be tempted to mix logic and UI, for instance.

louis_riviera · 2013-11-19 15:31:59

Yes but where to do the conversions for example? When interfacing?

louis_riviera · 2013-11-19 17:57:35

IMHO better to use TMemoryStream to write / append and it will have similar performance and much better than TStringBuilder

For us SynCommons.pas is too complex / bloated.

ab · 2013-11-19 19:25:24

louis_riviera wrote:

For us SynCommons.pas is too complex / bloated.

No one did force you to use it.

It is the ground of our framework, and other units which could be used without mORMot (like SynGdiPlus or SynPdf).
It does not suffer from performance penalty problems, as the official Delphi RTL does version after version.

We found it pretty useful also, even for small programs, since it has nice structures and features (like dynamic array wrapper, or fast string process).

But it is indeed a huge unit, with a lot of features (like UTF-8 JSON low-level marshalling, or logging).
We may split this unit into several smaller peaces.

louis_riviera · 2013-11-19 20:05:01

Yes it would be very much great to split it in to namespaces much like newer versions do it..

System.Sysutils
System.Types

etc..

Last edited by louis_riviera (2013-11-19 20:05:36)

ab · 2013-11-19 21:00:41

Don't forget that mORMot is compatible with Delphi 6 (even Delphi 5 for the SynCommons and the SynDB part) up to the latest version.

Our reference is more http://www.freepascal.org - which sounds like a better warranty for the future.
We rely do not like the latest orientation of the Delphi compiler.
There are several proposals for namespaces - see http://wiki.freepascal.org/Namespaces - but AFAIK it is not implemented yet.

Units ARE namespaces since the beginning, right?
We tried to be consistent with out unit name: Syn* for all low-level common stuff, SynDB* for external database, mORMot* for framework units.

louis_riviera · 2013-11-19 21:19:14

Yes only lazarus doesn't support it however what I remember, any Delphi version after Delphi 7 supports it. But that doesn't mean FPC will not support it in the future!

Syn.Common.pas
Syn.Compression.pas

and so on..

Last edited by louis_riviera (2013-11-19 21:34:34)

louis_riviera · 2013-11-20 17:59:46

Ah we found TWriteOnlyBlockStream. Which is about 10x faster than TTextWriter when using regular strings. It is about 20ms faster when TTextWriter is using UTF8.

ab · 2013-11-20 20:23:32

Those two classes are not for the same purpose.
TWriteOnlyBlockStream, whatever its name states, does not write directly into a TStream, but into memory blocks, then at once into a TStream.

I have used TTextWriter to write GB of JSON content or log content, with very small memory use: it write to a TFileStream per chunk.
It won't be possible with TWriteOnlyBlockStream, which will fill all your available memory.

You can chose what fit best with your real need.
And don't trust benchmarks which measure the time to work a loop of fixed content which fits in the CPU L1 cache (like a "for" loop appending constant strings and integers).
I trust real benchmarks with real data, like our regression tests and our performance sample, which uses a database, marshall JSON to and from data, uses RTTI for filling and reading objects, parse an URI, check user rights, and so on...

My current challenge is to provide some code to http://www.techempower.com/benchmarks/
I’m adding MVC support to mORMot currently, using JavaScript BTW.
Here we will see how it works. In the real world… not with a naive and biaised benchmark.

And do not forget to compile your sample in RELEASE mode, not DEBUG mode: you need the compiler optimization to unleash TTextWriter power (which uses inlining in RELEASE mode only).
See the "update" note of Eric's article at http://www.delphitools.info/2013/11/07/ … -building/

louis_riviera · 2013-11-20 20:29:47

Will framework use namespaces?

ab · 2013-11-20 20:31:07

I do not see any reason yet.
We need to keep the backward compatibility with older version of Delphi, for existing projects, and lot of our users.

IMHO namespaces are just cosmetic, and "fashionable".

louis_riviera · 2013-11-20 20:53:00

Yes but easier to maintain IMHO... it is the future of Delphi like it or not. In 5 years it will be standard.

Last edited by louis_riviera (2013-11-20 20:53:23)

ab · 2013-11-20 21:08:53

So we will see in 5 years.
But I suppose I would have switched to FPC.

louis_riviera · 2013-11-20 21:40:50

I agree but Eric's implementation works with UNICODE. And that will work with rest of components and that needs no to mess with conversions.

ab · 2013-11-20 23:57:09

Utf8 is Unicode.

louis_riviera · 2013-11-21 06:42:16

Sorry I meant UTF16.

mORMot Open Source

#1 2013-11-14 21:52:32

TTextWriter

#2 2013-11-15 15:25:17

Re: TTextWriter

#3 2013-11-17 11:56:32

Re: TTextWriter

#4 2013-11-17 13:35:32

Re: TTextWriter

#5 2013-11-17 15:34:19

Re: TTextWriter

#6 2013-11-17 15:56:37

Re: TTextWriter

#7 2013-11-17 16:29:33

Re: TTextWriter

#8 2013-11-17 16:34:14

Re: TTextWriter

#9 2013-11-18 13:22:41

Re: TTextWriter

#10 2013-11-18 16:08:06

Re: TTextWriter

#11 2013-11-19 09:29:10

Re: TTextWriter

#12 2013-11-19 11:38:40

Re: TTextWriter

#13 2013-11-19 11:42:12

Re: TTextWriter

#14 2013-11-19 12:51:00

Re: TTextWriter

#15 2013-11-19 14:34:45

Re: TTextWriter

#16 2013-11-19 14:43:07

Re: TTextWriter

#17 2013-11-19 15:04:01

Re: TTextWriter

#18 2013-11-19 15:12:26

Re: TTextWriter

#19 2013-11-19 15:20:20

Re: TTextWriter

#20 2013-11-19 15:24:15

Re: TTextWriter

#21 2013-11-19 15:26:06

Re: TTextWriter

#22 2013-11-19 15:31:59

Re: TTextWriter

#23 2013-11-19 17:57:35

Re: TTextWriter

#24 2013-11-19 19:25:24

Re: TTextWriter

#25 2013-11-19 20:05:01

Re: TTextWriter

#26 2013-11-19 21:00:41

Re: TTextWriter

#27 2013-11-19 21:19:14

Re: TTextWriter

#28 2013-11-20 17:59:46

Re: TTextWriter

#29 2013-11-20 20:23:32

Re: TTextWriter

#30 2013-11-20 20:29:47

Re: TTextWriter

#31 2013-11-20 20:31:07

Re: TTextWriter

#32 2013-11-20 20:53:00

Re: TTextWriter

#33 2013-11-20 21:08:53

Re: TTextWriter

#34 2013-11-20 21:40:50

Re: TTextWriter

#35 2013-11-20 23:57:09

Re: TTextWriter

#36 2013-11-21 06:42:16

Re: TTextWriter

Board footer