mORMot Open Source

Argh. I do need the NewPath line there. Otherwise the clipping mechanisme doesn't work for my other part of the PDF (for my signature where the clipping needs to be on another background pdf with pdftk).

So the question remains... why does the the clipping continue after an image (as shown in Image 2)?

You can load the meta.emf in emfexplorer.exe and see what happens there.
Also... rendering it on screen works correctly.
It's just creating a PDF from it, it hides everything after the image.

I'm not sure about Delphi 11 (I'm still on 10.2) but TJvRichEdit supports images. They have Ole callback stuff which is probably needed to render the images to the metastream.

(I currently do have a small problem with images, not showing text after an image, but I'll open a new topic for that)

No, components are installed via packages in Lazarus. Same as with Delphi. Only when you click Install the whole IDE (lazarus) gets re-compiled.

Now the mormot2ui package is made as an install (design-time) package which recompiles the IDE while this shouldn't be necessary because there is no components installed. So it can just be a non-design package (same as the mormot2).

It does work as it is now but "Install" wouldn't be needed I think.
Unless components are added.

BTW. In Delphi only the path is added to the project after which everything gets compiled automatically. This could also be done the same way with Lazarus or FPC. But a package could add that path automatically which is easier. But I think the package could also just contain all the needed paths without adding all the files specifically (but somehow that doesn't work correctly now). I will look into that.

Ah, yeah, I didn't look at that. I think FPC has its own fcl-pdf package anyway.

@loadymf. In addition... you might want to try mORMot 2 which includes src/ui/mormot.ui.pdf.pas with the latest changes to the PDF engine. It might give you more luck.

Wait??? Doesn't this line read bytes BEFORE the data structure if len is 12????

movups  xmm1, dqword ptr [data + len - 16] // no read after end of page

data is the first pointer, len is 12 so data + len - 16 is going to do data - 4 ?

So using a array of less than 128 bits (16 bytes) is really dangerous with this function

(that would be any bitmap with width < 6 pixels for 24bit pixels which seems to be consistent with my experience)

~~
I tried to test with this but I can't fool it.

var
  a: array[1..1] of byte;
  hash: THash128Rec;
begin
  DefaultHasher128(@hash, @a, 12);

You think that should give an AV.

~~
There would not be an immediate need to fix this, but we should be aware of it

(or a check could be added in drawbitmap and any other place where this hash is used)

And if someone needs a quick "fix".... this works too (just using another hasher128)

  DefaultHasher128 := @crc32c128;

Ok, then there is something going on with the row.

It's a 24bit bitmap (default)

row := PERROW[B.PixelFormat];    = 24 bits = 3 bytes

w = 3    ---> that should be 3 * 3 = 9 bytes per row ???

row := (((w * row) + 31) and (not 31)) shr 3; // inlined BytesPerScanLine

row = 12

for y := 0 to h - 1 do
  DefaultHasher128(@hash, B.{%H-}ScanLine[y], row);

So the row should be 9 bytes for a 3 pixel bitmap, wouldn't it?

The row value is 12. Seems 3 bytes too much

Crashed line is in _AesNiHashXmm0 for 64 bit.
But if row (len) is indeed 12 for a 3 pixel bitmap then that seems logical that it crashes there:

        movups  xmm1, dqword ptr [data + len - 16] // no read after end of page

I don't know if the scanline is supposed to be aligned at 32 bit but seeing that access violation it seems not.
(the access violation is always on the last scanline)

Thanks for the merge.

When looking for the word "clipping" here on the forum I came across this topic from MtwStark.
https://synopse.info/forum/viewtopic.php?id=4276

It seems (s)he made a lot (and I mean A LOT) of changes to SynPDF regarding the clipping-code.

I couldn't find a pull request (or even fixes on github).

I understand that those changes were a LOT to just merge with the existing code (and it wasn't even as fix for mORMot2) and was left behind because of this.

If I have any other fixes I'll try to keep them small and clearly understandable

Ok, here is another proof that the bitmap was drawn too small.

I added a FillRectangle of 1x1 at coordinates 0,0  1,1 and 4,6.

You see clearly that in the original drawing of bitmap with 2x4 pixels (drawn at 2,2), the lower right is not connected to the FillRectangle.

So, or all the BoxI() for FillRect etc. is not correct (but they seems to be places correctly) or the code for DrawBitmap is incorrect (too small).

After some further testing I will prepare a pull request for the ExtSelectClipRgn with the addition of corrected bottom/right in DrawBitmap.
(After that, when I get the time I'll see if I can tackle the EMR_SELECTCLIPPATH, which is completely missing now).

When corrected:

Grrr. translating these coordinates is really hard.

Still not perfect.
It's also not just adding or substracting a value because the ratio/proportions are also off when using BoxI().
I can't figure out what other translation then BoxI() should be done.

The larger the image, the more deviation.

(There is also a big bug in PDF Engine when dealing with bitmaps of size 7x7 (very small). It crashes, even without the clipping stuff.)

Yes, I saw that too when tinkering with the code for drawing.
But it is GDI+ that's passing on that info.

For clipping problem I opened a new topic:
https://synopse.info/forum/viewtopic.php?id=6694

I am about to create a pull request with a fix (but still need to learn how to do that).

But even with the fix for clipping, the dashed line is too big. There is no clipping involved in the dashed line from GDI+.
You can create an EMF file and examine it with emfexplorer (and then View>EMF as Text).
There is no clipping records there.

BTW. Can you share your code to create your clipping example. I would like to throw it though the clipping fix to see if that works correctly (especially at the edges).

The code is almost identical with the old code (with the check for iMode added).

But I will test this some more (we wouldn't want a bad patch ) and create a pull request with patch (once I know how to do that )

Hmmmm, Yeah, definitely not supported.

Size of region data from GDI is 208 bytes.

      EMR_EXTSELECTCLIPRGN:
      begin
        dum1 := PEMRExtSelectClipRgn(R)^.cbRgnData;        // <---- has a block of 208 bytes
        E.ExtSelectClipRgn(@PEMRExtSelectClipRgn(R)^.RgnData[0],
          PEMRExtSelectClipRgn(R)^.iMode);
      end;

But the function ExtSelectClipRgn() doesn't seem to do much with the data parameter??

procedure TPdfEnum.ExtSelectClipRgn(data: PRgnDataHeader; iMode: DWord);
begin
  try
    with DC[nDC] do
      case iMode of
        RGN_COPY:
          begin
            ClipRgn := MetaRgn; // where is the data parameter used??
            ClipRgnNull := false;
          end;
      end;
  except
    on e: Exception do
      ; // ignore any error (continue EMF enumeration)
  end;
end;

Any ideas?

Was this meant to be functional or is it still in development?

Did this work in the past ???
Seeing this post: https://synopse.info/forum/viewtopic.ph … 071#p20071

Edit:
The code from that old topic (which was once in SynPDF) seems to be working much better. (no anti-alias but that might be fixed.)
Why was that code removed?

I did see some downsides of my hack above.
When using larger width for pen for dahses it doesn't work anymore for flat caps (probably because EMR_POLYBEZIERTO16 isn't used in that case).
I have a more 'stable' hack for you now. But you would need to have a special unique color for the dashes:

      EMR_EXTCREATEPEN: // approx. - fast solution
        with PEMRExtCreatePen(R)^ do
          if ihPen - 1 < cardinal(length(E.Obj)) then
            with E.obj[ihPen - 1] do
            begin
              kind := OBJ_PEN;
              PenColor := elp.elpColor;
              PenWidth := elp.elpWidth;
              PenStyle := elp.elpPenStyle and (PS_STYLE_MASK or PS_ENDCAP_MASK);

              // this doesn't work because we don't have dashes
              // if pen.style and PS_STYLE_MASK = PS_DASH then PenWidth := 0;

              if elp.elpColor = 0 { r } + 255 { g } shl 8 + 0 { b } shl 16 then PenWidth := 0; // your color for dashes

            end;

This will only set the pen width to 0 for when that color is used.
It seems a lot more stable (but is still an ugly hack).
It will also work for flat caps

(If anyone else has any other ideas about how to detect if we are in dashed lines ?)

Result:
(You'll notice the blue line creeps to the left but that's because of the round caps, and is according to specification, using the round cap before start of line).

BTW. This is the original where you see the problem of GDI+ clearly.

And this was with flat caps:

You also notice the corners are slightly different from the EMF version
So that does seem to be a minor 'bug' in synpdf but when using pen=0 those corners are cut anyway.
But maybe that's the reason the 'bug' is there in GDI+ to provide those corners for flat caps

Yes. I found more references stating that GDI+ and the enhanced meta canvas is inaccurate below a width of 1.5 or 2.
I'm still investigating the problem a bit but it is hard.

As I understand it now... the Microsoft Print to PDF uses a complete different method of using the GDI+ for printing.
You have GDI-based printing and XPS-based printing. Microsoft Print to PDF does the latter.

That's probably also the reason the dashed line is a path instead of actual dashed line.
I think XPS-based printing always uses the ellipse version which is badly rounded.
But they did fixed the bug that the TEnhancedMetaFile has (i.e. the bigger line width which the ellipses are drawn).
So we can forget about that one.

You also have several flavors of GDI-printing. You can print via the print spooler (which uses EMF format) or directly to the printer.
I'm not sure how I can force the EMF method to see if my printer then also has the same problem (setting the processor didn't hold).
I think the printer version prints directly (via the dll's) but I'm not sure about that and how.

This is for printing Microsoft Print to PDF and the XPS writer in Windows:

procedure TForm1.btnPrintClick(Sender: TObject);
begin
  // Printer.PrinterIndex := Printer.Printers.IndexOf('Microsoft Print to PDF');
  // shows as technology = http://schemas.microsoft.com/xps/2005/06

  Printer.PrinterIndex := Printer.Printers.IndexOf('Postscript');
  // shows as technology = Postscript

  Printer.PrinterIndex := Printer.Printers.IndexOf('Microsoft XPS Document Writer');
  // shows as technology = http://schemas.microsoft.com/xps/2005/06

  Printer.BeginDoc;
  // Showmessage( GetTechnologyString( Printer.Canvas.Handle ) );
  Draw(Printer.Canvas.Handle);
  Printer.EndDoc;

  showmessage('done');

end;

This is the XPS Writer (and is the same as the Microsoft Print to PDF because it both uses XPS-based printing).

BTW. The used GetTechnologyString() looks like this. (the SetPostscriptIdentity doesn't seem to do much.)

const
  PSIDENT_GDICENTRIC = 0;
  PSIDENT_PSCENTRIC = 1;
  POSTSCRIPT_IDENTIFY = 4117;
  GETTECHNOLOGY = 20;

function GetTechnologyString(Handle: HDC): string;
var
  indata: DWord;
  outdata: array [0 .. 255] of Byte;
  S: string;
begin
  indata := GETTECHNOLOGY;
  if ExtEscape(Handle, QUERYESCSUPPORT, SizeOf(indata), @indata, 0, nil) > 0 then
  begin
    if ExtEscape(Handle, GETTECHNOLOGY, 0, nil, SizeOf(outdata), @outdata) > 0 then
    begin
      SetString(S, PAnsiChar(@outdata[0]), StrLen(PAnsiChar(@outdata[0])));
      Result := 'ok = ' + S;
    end
    else
        Result := 'gettechnology error';
  end
  else
      Result := 'query error';
end;

// doesn't do much ??
function SetPostscriptIdentity(Handle: HDC): string;
var
  indata: DWord;
  outdata: DWord;
begin
  indata := POSTSCRIPT_IDENTIFY;
  if ExtEscape(Handle, QUERYESCSUPPORT, SizeOf(indata), @indata, 0, nil) > 0 then
  begin
    indata := PSIDENT_GDICENTRIC;
    indata := PSIDENT_PSCENTRIC;
    if ExtEscape(Handle, POSTSCRIPT_IDENTIFY, SizeOf(indata), @indata, 0, nil) > 0 then
    begin
      Result := 'ok set POSTSCRIPT_IDENTIFY';
    end
    else
        Result := 'POSTSCRIPT_IDENTIFY error';
  end
  else
      Result := 'query error';
end;

Yes, that's from the Microsoft Print to PDF. That driver is really bad. It also creates very large PDF's and isn't optimized.
I usually avoid that printer.
(But I don't know why those segments are printed/interpreted correctly)

I have my own printer for my application. It creates a Postscript printer from which a catch the output to a .ps file and I throw that through ghostscript.
Somehow the .ps file from that Postscript printer is very small and the dashed line doesn't contain segments.

(you can do this yourself by creating a postscript printer and let it print to a port FILE:)

Attached is a Temp.zip with an example of a .ps file printed with the Print to PDF driver and one with Ghostscript.
https://file.io/sH9jya1SyMEj

BTW. Both drivers use the same base driver (PScript5.dll) so it must be something that the ghostscript.ppd is setting (because that's the only difference).
(Edit: actually, the print to pdf from Microsoft now uses some other driver I see)

Here is the resulting PDF from my ghostscript printer and the one from Microsoft Print to PDF.
https://file.io/4FeFlH109Psb

You can clearly see the difference when you zoom in. And the ghostscript one really has the dash (as you can see in the .ps versions).

From ghostscript.ppd.

*% == Basic Capabilities
*LanguageLevel: "3"
*ColorDevice: True
*DefaultColorSpace: RGB
*FileSystem: True
*Extensions: CMYK FileSystem Composite
*TTRasterizer: Type42
*FreeVM: "10000000"
*PrintPSErrors: True
*ContoneOnly: True
*% AccurateScreensSupport: True

Maybe the Postscript level but I haven't looked into that.

I think actually that the Enhanced Mata Canvas for sympdf is already EMF+.

Ths is from Winapi.GDIPOBJ in Delphi:

// The following methods are for playing an EMF+ to a graphics
// via the enumeration interface.  Each record of the EMF+ is
// sent to the callback (along with the callbackData).  Then
// the callback can invoke the Metafile::PlayRecord method
// to play the particular record.
function EnumerateMetafile(metafile: TGPMetafile; const destPoint: TGPPointF;
  callback: EnumerateMetafileProc; callbackData: Pointer = nil;
  imageAttributes: TGPImageAttributes = nil): TStatus; overload;

And mormot.ui.pdf.pas receives that in:

/// EMF enumeration callback function, called from GDI
// - draw most content on PDF canvas (do not render 100% GDI content yet)
function EnumEMFFunc(DC: HDC; var Table: THandleTable; R: PEnhMetaRecord;
  NumObjects: DWord; E: TPdfEnum): LongBool; stdcall;

I think the problem is that GDI+ thinks it has to do with a device which can't interpret postscript.

There is an option with POSTSCRIPT_IDENTIFY to set PSIDENT_GDICENTRIC or PSIDENT_PSCENTRIC for GDI centric or Postscript centric but that's for the device driver I think. (I don't even know if that would help)

I don't know how to influence GDI+ into thinking we have the dash-line capability (so it sends DASH instead of those ellipses).

Yes, but when GDI+ is going to pass on the records it stored, it does so via the EnumerateMetafile which is EMF+.
Synpdf itself works with Enhanced Meta Canvas with EMF+ structures doesn't it?

GDI+ does have the dash stored as real dash-structure (like passed to it). It gives it correctly to the printer driver (via the canvas) on Draw() command.
But for the TEnhancedMetaCanvas this is translated to ellipsed (which contain a bug that it draws the line length which is too big).

I whish I knew how the printerdriver did this. It also receives the GDI+ records on Draw() but seems to translate them much more efficient (postscript dash codes instead of ellipses).

BTW If I don't use Print To PDF from Microsoft but my own PDF writer (based on Ghostscript driver) I get a better result.

Now the COMPLETE PDF looks like this (only relevant print part).
This is including the solid line AND the dashed line.

%%Page: 1 1
%%BeginPageSetup
%%PageOrientation: Portrait
pdfStartPage
0 0 612 792 re W
%%EndPageSetup
[] 0 d
1 i
0 j
0 J
10 M
1 w
/DeviceGray {} cs
[0] sc
/DeviceGray {} CS
[0] SC
false op
false OP
{} settransfer
0 0 612 792 re
W
q
q
[0.12 0 0 0.12 0 0] cm
6 w
1 J
/DeviceRGB {} CS
[0 0 1] SC
180 3600 m
240 6360 l
1800 6300 l
S
[24 30] 0 d
/DeviceRGB {} CS
[0 1 0] SC
300 3600 m
360 6240 l
1800 6180 l
S
Q
Q
showpage
%%PageTrailer
pdfEndPage
%%Trailer
end

This is how it should be.
The [24 30] 0 d is the actual dashed line setup for pdf (as you could see in TPdfCanvas.SetDash()).
And the corresponding x l #10 after that prints the dashed line.

But I have no idea how to setup GDI+ to do exactly the same for synpdf.

I think it has to do with the fact synpdf uses a EMR/enhanced meta canvas and gdi will pass the drawing different from passing it to the printerdriver via canvas. But I'm not sure (as I said, I'm no expert).

Well, you should care if you want to fix this.

I'm not sure why GDI+ is drawing the dashes as ellipses but it's definitely doing that at a higher resolution.

1) You could draw this to another drawable canvas and after that copy that canvas to VCLCanvas and see if that helps (probably with loosing some resolution).

2) Second option is to find out why GDI+ is drawing ellipses and seeing if you can alter that behavior (maybe with setting the stroke color to transparent?).

3) Third option is to hack mormot.ui.pdf and adjust the line width when there is a dashed line. But I looked at the TPdfEnum.NeedPen and when it's called, it's not called with any dash style because GDI+ takes the drawing of dash completely for itself. So it's difficult to know if there is a dash being drawn. You could put a global boolean switch and hack the TPdfEnum.NeedPen, to pass 0.1 if that boolean is set (and set the boolean yourself when you want to draw a dash).

But you'll see it's not something the mormot.ui.pdf itself can do because from mormot's point of view, there is no dashed line being drawn, but multiple smaller ellipses, by GDI+.

No, actually it's something in GDI.

I had another quick look at the decompressed PDF it seems that there isn't really ONE dashed line (in terms of PDF line).

The line code looks like this and is correct:

/DeviceRGB {} CS
[0 0 1] SC
[] 0 d
1 J
0.75 w
22.5 467 m
30 812 l
225 804.5 l
S

And the dashed line looks like this with many repeats:

/DeviceRGB {} CS
[0 1 0] SC
[] 0 d
0 J
0.75 w
0.75 w
0.75 w
0.75 w
0.75 w
<many times>
/DeviceRGB {} cs
[0 1 0] sc
37.31 473.75 m
37.36 476.75 l
37.36 476.94 37.55 477.13 37.73 477.08 c
37.97 477.08 38.11 476.94 38.11 476.7 c
38.06 473.7 l
38.06 473.52 37.88 473.33 37.69 473.33 c
37.45 473.38 37.31 473.52 37.31 473.75 c
h
37.45 480.5 m
37.5 483.5 l
37.55 483.69 37.69 483.88 37.92 483.83 c
38.11 483.83 38.3 483.69 38.25 483.45 c
38.2 480.45 l
38.2 480.27 38.02 480.08 37.83 480.08 c
37.59 480.13 37.45 480.27 37.45 480.5 c
h
<last part repeated many times>

So it seems that GDI itself translate the dashed line itself to many multiple fragments (and calls to LineTo).

Above that code are many 0.75 w parts. (which stands for line width.

The kicker now.... I changed all the 0.75 w parts into 0.175 w to make the lines thinner.
The result is this:

Here you see clearly that it is GDI that is drawing the line.

When that is drawn on a canvas with 1 pixel resolution, you get the screen and printer result.
But when it is drawn to a canvas with much higher resolution, it's going to draw those circel thingies.

PS. I extracted the pdf with pdf2ps (from xpdf-tools) and looked at the resulting .ps file.
You can read the ps file afterwards with a postscript interpreter or with gimp (and setting a resolution of 1200 during import).

I know you can hack the pdf writer to write smaller lines (like I showed you earlier).
But it's definitely GDI that's drawing the dashed line itself.

Hi, a non-expert here...

For the actual setting of the pen and dash you can look in mormot.ui.pdf.pas at the procedure TPdfEnum.NeedPen.

But this only gets called when you actually save the file to pdf, not before (so the translation from gdi to pdf).

When putting a breakpoint there and watch for DC[nDC].pen.width you see that for both lines the value is 16 (in your example program).

Increasing the width to 100 in second breakpoint-loop seems to increase the width of the dashed line.
But lowering it under 16 to 8 for example doesn't seem to be possible.
Minimum width dashed line seems to be that value.

This is the result of setting the DC[nDC].pen.width to 1 for both lines.

And this for setting it to 100 for both lines.

Can you show your code or snippet?
For me it works fine with the latest source of mormot.ui.pdf.

PDF.VCLCanvas.Pen.Width := 1;
PDF.VCLCanvas.Pen.Style := psDash;
PDF.VCLCanvas.MoveTo(100, 600);
PDF.VCLCanvas.LineTo(300, 600);
PDF.VCLCanvas.Pen.Width := 4;
PDF.VCLCanvas.MoveTo(300, 600);
PDF.VCLCanvas.LineTo(600, 600);

I don't think that's (directly) possible with TPdfDocumentGDI.
You would think that it should work with TPdfDocumentGDI.RawPages but in TPdfDocumentGDI.AddPage there is this line at the top:

if (FCanvas<>nil) and (FCanvas.FPage<>nil) then
  TPdfPageGdi(FCanvas.FPage).FlushVCLCanvas;

That line flushes and frees TPdfPage.fVCLCurrentCanvas of the previous page.
That's the pdf.VCLCanvas you want to use for the page.
When going back to page one with TPdfDocumentGDI.RawPages the needed TPdfPage.VCLCanvas is already freed and destroyed so when accessing it again, it will be created again, new but empty.
That's why the page is overwritten.

You could try to comment out the above line. Then you get the result you want.
The TPdfPage.fVCLCurrentCanvas is kept after TPdfDocumentGDI.AddPage.
The TPdfPage.VCLCanvas is flushed in that case at TPdfDocumentGDI.SaveToStream() and not at AddPage.

But I'm not sure if there are any problems with it down the road.

Yes, I can confirm the UseUniscribe := true works in this case.
The UseFontFallBack were not needed for me (and seeing that the PDF already contains Arial I think they don't do much).

I don't now why UseUniscribe is needed when a fairly simple Arial font is used (I'm not that familiar with the inner workings of SynPDF).

But maybe it has something to do with the way everything is rendered into the WMF and after that the font-info becomes complex in which case there is a need for UseUniscribe (just guessing here).

BTW The reason some lines work fine and others not is in the fact that the space between words is wrong. And text which is combined is stretched. But text snippets that were placed on a certain position is fine.
The amount (Betrag) on the productline is on a specific position (so that works fine).
The amount  for subtotal and MWST probably have multiple spaces in front of them and that's because they are shifted too much to the right (the multiple spaces are all expanded).

(hey, and that's where that format('%11.2n) and format('%7.2n) comes in. The 11.2 has more spaces and each space is expanded to multiple spaces. 7.2 has less space and that's why that one seemed to fix some of the issues )

Maybe @ab can elaborate on the reason for the need for UseUniscribe.

We still haven't seen what exactly the problem is. Alignment problems? Or something else? "Doesn't render properly" isn't really a clear description.

That would suggest some alignment problems. If you use spaces you would need to use (right aligned or decimal aligned) tabs to always correctly align numbers.

These also don't align in text:
    145.12
133241.12
(and the first has 4 spaces, so is 7.2, second has 1 space so is also 7.2)

But I'm not sure why this would be correct in a WMF format (IF that's the actual problem of course).
Maybe it has to do with font substitution (in WMF one font and when rendered to PDF another font is chosen).

In that case it might be useful (especially for the developer) to have the sample meta.wmf to check where this goes wrong.

(It then can be loaded with aMeta.LoadFromFile('C:\temp\WMF\meta.wmf); and debugged properly.)