PDFEngine Clipping Implementation + Output stream optimizations

MtwStark · 2017-12-07 17:48:30

UPDATE 2022-06-09

I have released an updated version to current release (1.18.6395) for anyone with clipping problems.
It's not the final solution but it addresses many cases.
SynPDF_MTW_1.18.6395

UPDATE 2018-02-02

I have released a new version to solve problems about nested clipping (thanks to Tom for support and testing)
I have also added support for LineJoin

Hi all,
good news, I have finally made a somewhat big step ahead in clipping implementation.

It is a quite long explanation.. posting code.. functions.. would be difficult, so I share my entire SynPDF.pas
You can test output using test_libuemf_ref.emf and HRVPSDPrintForm.emf

I have problems sharing the reference emf, google drive thinks it contains a virus (but it doesn't) and warns the user before download

There are some differences from standard version, but I think that who is interested will find it useful.

I quote some people involved, hoping they will be notified and come to see:

Marsh wrote:

But I also noticed that clipping for EMF is not supported. So far I managed to get around with a hack, but now I'm looking for a proper solution.

Tom wrote:

But when converting a small EMF file to PDF I encountered a clipping problem, and I don't know how to solve it.

jeanmilost wrote:

But I recently tried the latest SynPdf version, and I noticed that the above code no longer works, as my page is drawn without clipping at all.

kalwados wrote:

with some small adjustments as mentioned where by MtwStark: Test for iMode == RGN_COPY. Now everything works fine, again.

Fritz wrote:

After investigate the clipping in SynPDF i must say there are a lot of problems with it. I will spend some more time to find a proper solution.

WBTeam wrote:

... the clipping issue is still not solved. I'm hoping for someone to take care of this, I'll try myself, but I'm no expert.

jkelley wrote:

I spent some time playing around with ExtSelectClipRgn to see if I could fix the problem, but unfortunately, I'm having trouble figuring out my way around the codebase.

davidheffernan wrote:

I'm seeing nothing at all in the generated PDF, or sometimes just a couple of parts of one of the metafiles.

I think that also davidheffernan problem with multiple metafiles, could be related to clipping, because in my HRVPSDPrintForm.emf test I had a similar problem..

Use it as you like, I got a great improvement with this, I hope that someone else will get it too.

Feel free to ask for explanations if something is not clear, any comment are welcome.

Essentially it is the standard version 1.18.4051 of 2017-12-06 (yesterday) with some differences:

In december 2015 I implemented my function to handle EMR_ARC, EMR_CHORD, EMR_PIE, EMR_ARCTO and I still have not replaced it with ProHolz version
In november 2016 I have integrated some code from JeanMilost about FontFallBackNames and KerningHHorzScale and some code from Niki about dynamic load of UNISCRIBE

All other differences are mine, you will find:

a new set of functions absI2X, absI2Y, absRectI, absBoxI to calc I2X coordinates with absolute reference to page origin (not using WorldTransform), used in ExtSelectClipRgn

a new procedure RectangleClockWise handy for special fill/clip operations with different winding rules

a new InPath flag in TPdfEnumState, to handle operations between EMR_BEGINPATH and EMR_ENDPATH

MetaRgn and ClipRgn in TPdfEnumState and also fInitMetaRgn in TPdfEnum are now TRect instead of TPdfBox to handle clipping objects in logical units

I have also defined the types for brush (TPdfEnumStateBrush) and font (TPdfEnumStateFont) outside TPdfEnumState

some properties and changed some property setter for some internal variable, in SetFillColor(), SetStrokeColor(), SetPenWidth() added test for invalid value -1 before to write to pdf, SetInLined() handles Stroke if needed;

procedure RestoreDC now handles the DC number parameter and (most important) clear the Clip Path if restored DC had a NULL Clip Path;

a new procedure ResetClipPath to clear the Clip Path on PDF and invalidate colors, styles and font, you can't enlarge Clip path on PDF, only reduce it, so the only solution is to restore the NULL Clip Path and apply a new one;

a new procedure InvalidateColorsAndStyle, just a wrapper for fFillColor := -1; fStrokeColor := -1; fPenStyle := -1; fPenWidth := -1;

a new procedure ExtCreatePen to handle a little more the EMR_EXTCREATEPEN record, added basic support for PenStyle = PS_USERSTYLE (emulated with PS_DASH), sets a default pen color black if brush style not in [BS_SOLID, BS_HATCHED] (should use patterns..), sets a default pen width = 1 if brush style in [BS_DIBPATTERN, BS_DIBPATTERNPT, BS_HATCHED, BS_PATTERN], sets pen style to PS_NULL if brush style = BS_NULL

procedure SetMetaRgn now clear the Clip Path on PDF and updates MetaRgn variable

Now let's see the clipping core functions:

IntersectClipRect: now updates the Clip Path on PDF intersecting the rect passed and updates the internal ClipRgn variable
ExcludeClipRect: Added support for EMR_EXCLUDECLIPRECT, updates the Clip Path on PDF excluding the rect passed but does not update the internal ClipRgn (we keep the largest bound rect)
SelectClipPath: Added support for EMR_SELECTCLIPPATH (mode RGN_AND), updates the Clip Path on PDF with the path plotted (but still not painted), but does not update the internal ClipRgn (because we have no info on path applied)
ExtSelectClipRgn: Added support for EMR_EXTSELECTCLIPRGN (modes RGN_COPY and RGN_AND), intersect or replace the Clip Path on PDF with the rect list passed and updates the internal ClipRgn variable

some changes in procedure TextOut:

excluded the test on white brush color for opaque background, because white is not transparent;
inverted the test on font background color, I think it was wrong;
forced the brush color to font background color for opaque background;
changed the dimensions of background rectangle because in some cases there was uncovered zones, I have tried to make it automatic but it can be improved and it should be tested with many other fonts styles and dimensions;
deleted some uselse commands for clip rectangle;
added silent paint (NewPath) after the Canvas.Clip command, but (WARNING!!) deleted it from FillRectangle();
deleted the entire block who save graphic state, change clipping, restore graphic state.. if nothing put marks on pdf between clip and restore.. all of this is useless

some changes to EnumEMFFunc function:

EMR_LINETO:
- set Moved to true instead of false because LineTo does move the current position on PDF
- tested the new property InLined for Stroke management

EMR_POLYGON, EMR_POLYLINE, EMR_POLYGON16, EMR_POLYLINE16:
- set Moved to true instead of false because LineTo does move the current position on PDF
- tested the new property InLined to avoid the paint (stroke or fill or silent)

EMR_POLYBEZIER, EMR_POLYBEZIER16, EMR_POLYBEZIERTO, EMR_POLYBEZIERTO16, EMR_POLYLINETO, EMR_POLYLINETO16, EMR_POLYDRAW, EMR_POLYDRAW16:
- set Moved to true instead of false because LineTo does move the current position on PDF

EMR_BEGINPATH, EMR_ENDPATH:
- management of new flag InPath true/false

EMR_FILLPATH:
- forced ClosePath before fill, unfortunately doesn't exists a direct PDF command for Close + Fill

EMR_STROKEPATH:
- made silent paint alternative to Stroke

EMR_STROKEANDFILLPATH:
- forced ClosePath before paint (stroke or fill or silent), unfortunately doesn't exists a direct PDF command for Close + Fill
- made silent paint alternative to Stroke and/or Fill

and finally a little optimization in TextRect()
- repaced MoveTo, LineTo, LineTo, LineTo and CloePath with Rectangle

Merry Christmas from Italy
MtwStark

P.S.
sorry for poor english or bad code..

Last edited by MtwStark (2022-07-07 08:50:50)

Marsh · 2018-01-17 09:24:30

Hi MtwStark,

Thanks for your fixes. I have used my workaround for years now, but I'll look into this when I get the chance.

Last edited by Marsh (2018-01-17 09:42:22)

Tom · 2018-01-30 07:36:40

Hello MtwStark

Thank you for your work! I tested your version of SynPDF.pas and it works with all my EMFs very well!!

Unfortunately I never could use the original SynPDF.pas for EMF export, because about 90% of my EMFs contain clippings.
As a workaround I converted all EMF to BMP to create the PDF file, with the result that all graphics where correct but the size of the PDF file was gigantic!
With your version I could reduce the size of my biggest PDF file by factor 25 and the quality is even better!

I just can encourage other users to test this unit and hopefully Arnaud will merge this genius clipping enhancements into the original SynPDF.pas

Best regards
Tom

Fritz · 2018-08-03 10:44:44

Hallo MtwStark,
sorry for the long delay. I have looked at your work and I like it very much. It seems you have fixed the clipping problems with SynPdf.
will do some more tests. I have done also some work for ModifyWorldTransform and co for my use case, try to put in in your work.

RalfS · 2022-06-09 17:38:15

Hi,
great Work. Please can you manage to merge it into the synopse main branch?

best regards!

ab · 2022-06-09 17:49:29

My concern is that almost every time I merge some pull request in SynPdf which pleases some users, there is a regression for others...

So I usually un-merge it a few days later...

jonjbar · 2022-06-10 07:52:43

ab wrote:

My concern is that almost every time I merge some pull request in SynPdf which pleases some users, there is a regression for others...
So I usually un-merge it a few days later...

Perhaps it would make sense to investigate some kind of PDF testing tool? I suspect, as an example, that it would be possible to use Chromium to produce images out of reference PDF documents, and some kind of image comparison algorithm to compare documents generated by nightly builds with those references.
Or perhaps a similar open source tool already exists in the JavaScript / Go / .net... ecosystem?
What do you think @ab ?

ab · 2022-06-10 09:33:13

Yes, automated regression test could be awesome.
But this is a lot of work, especially because comparing output may be tricky.

MtwStark · 2022-06-13 11:01:34

ab wrote:

My concern is that almost every time I merge some pull request in SynPdf which pleases some users, there is a regression for others...
So I usually un-merge it a few days later...

I know that it's not so simple to merge work from others in main trunk, especially when the proposed solution is only partial and not a full solution,
but I think that something could be useful to many other users.

I use this metafile to test source code test_libuemf_ref.emf and the differencies are not limited to clipping.
(It's not a virus although google drive warns about it, you can proceed with download)

Everyone can feel free to use my work for their needs, keeping credits to the original developer and a greeting message would be very appreciated.

mORMot Open Source

#1 2017-12-07 17:48:30