simple-pdf export test

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

simple-pdf export test

JordanBrown
Is anybody familiar with PDF export, and in particular with the
simple-pdf test?

Here's the setup...

I have Windows builds before and after my changes.

I have Ubuntu builds before and after my changes.

The test passes on both platforms when run against the "before" build.

It is expected that the test would fail with my changes, because they
change the spacing of OpenSCAD-rendered text.  I updated the reference
image on the Windows system based on the new actual result, and (of
course) the test now passes on the "after" Windows build.

I test the "after" build on Ubuntu with that revised reference image,
and it fails.

That's undesirable and unexpected, of course, but it's what tests do and
now I need to figure out why there's a difference.

Now the puzzle...

On Ubuntu, I compare the "before" reference image with the "before"
actual image, and there are considerable differences. The "Scale is to
calibrate" note along the bottom is significantly longer on the
before-actual image than it is on the before-reference image.  On the
before-reference image, that note extends to about 83 on that scale,
while on the before-actual image it extends to about 94.  And yet the
test passes.

On the other hand, I compare the "after" reference image (from the
Windows build) with the "after" actual image, and the differences are
basically the same as they are for before-reference vs before-actual...
but the test fails.  There do not appear to be *any* differences in the
OpenSCAD-rendered portion of the image, the stuff that I changed - only
in the labels along the axes.

The PDF files generated on Ubuntu are very different from the PDF files
generated on Windows, but there are no reference PDFs so I can't
directly say which is "right", if either is.

My only guess is that either Cairo or GhostScript is different enough
between the two platforms to produce different results, and the results
are different enough to pass some difference limit that wasn't being
passed with the original image.

Anybody have any ideas?  I can supply all of the images if that would be
helpful.

_______________________________________________
OpenSCAD mailing list
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: simple-pdf export test

JordanBrown

I should perhaps mention:

Windows

  • mingw-w64-x86_64-ghostscript 9.53.3-1
  • mingw-w64-x86_64-cairo 1.17.4-2

Ubuntu

  • ghostscript 9.53.3~dfsg-7
  • libcairo2 1.16.0-5ubuntu1




_______________________________________________
OpenSCAD mailing list
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: simple-pdf export test

Troberg
In reply to this post by JordanBrown
While we are on the subject of PDF expoer, I would love a "clean" export, pretty much the same as the svg export.

That would help me immensely, as I have a project where I frequently export a shitload of svg files, then has to convert them all to pdf before joining them to a single pdf. This process currently takes 15 hours, and as the number of files increase, the time will also. If I could skip the conversion stage, I would cut that time in half, as well as not locking up the computer during the process.

Sent from the OpenSCAD mailing list archive at Nabble.com.

_______________________________________________
OpenSCAD mailing list
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: simple-pdf export test

OpenSCAD mailing list-2
How are you doing the conversion?

Couldn't you just automate the conversion and concatenation using a tool such as pdftex?

If you'd send me a set of files incl. a PDF (or just the filenames) and a note on the order I'll gladly work up a file for this.

William




-----Original Message-----
From: Troberg <[hidden email]>
To: [hidden email]
Sent: Sat, Jun 12, 2021 6:40 am
Subject: [OpenSCAD] Re: simple-pdf export test

While we are on the subject of PDF expoer, I would love a "clean" export, pretty much the same as the svg export.

That would help me immensely, as I have a project where I frequently export a shitload of svg files, then has to convert them all to pdf before joining them to a single pdf. This process currently takes 15 hours, and as the number of files increase, the time will also. If I could skip the conversion stage, I would cut that time in half, as well as not locking up the computer during the process.

Sent from the OpenSCAD mailing list archive at Nabble.com.

_______________________________________________
OpenSCAD mailing list
To unsubscribe send an email to [hidden email]

_______________________________________________
OpenSCAD mailing list
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: simple-pdf export test

Troberg
OpenSCAD mailing list-2 wrote
How are you doing the conversion?
Couldn't you just automate the conversion and concatenation using a tool such as pdftex?
If you'd send me a set of files incl. a PDF (or just the filenames) and a note on the order I'll gladly work up a file for this.
I do it by batching InkScape.

I'll take a look at pdftex and see if that is workable. But, still, I like to have as short a tool chain as possible.

Sent from the OpenSCAD mailing list archive at Nabble.com.

_______________________________________________
OpenSCAD mailing list
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: simple-pdf export test

adrianv
I didn't really think too carefully about this when you posted it originally, but 15 hours!  You said you had a lot of files, but not how many you mean,   Is it fifty thousand files?   Because 15 hours seems extremely long.   Where is the time going?

I also don't understand the desire to keep the tool chain short.   Are you using Windows?  Under unix part of the point is that the tools do their one job and you use a lot of them if necessary.   I'd say you should use how ever many tools enable you to complete the task efficiently, and it sounds like your current setup isn't doing the job.  

I concat pdf files with pdftk but I'm not sure how that compares to using pdftex.  I just concated 12 copies of a 300 kB pdf and it runs instantly.  I can't imagine how big your pdf would be for the pdf concatenation step to be nontrivial in time.

Actually I looked up pdftex and it doesn't appear that it has a method for concatenating pdf files.  People talk of writing a TeX file that imports all the pdfs.  I suppose that could be a good solution if you really want to pack multiple pdf files onto one page.  I was assuming one page per pdf.  But using TeX to concatenate pdf files seems unlikely to be the best solution to the plain pdf concatenation problem.  It will surely be slower than pdftk.  

If inkscape is taking 7 hours to convert to pdf I'd consider whether there's a better alternative, unless you do have tends of thousands of files.   So perhaps:
https://itectec.com/superuser/how-to-convert-an-svg-to-a-pdf-on-linux/

Troberg wrote
OpenSCAD mailing list-2 wrote
> How are you doing the conversion?
> Couldn't you just automate the conversion and concatenation using a tool
> such as pdftex?
> If you'd send me a set of files incl. a PDF (or just the filenames) and a
> note on the order I'll gladly work up a file for this.

I do it by batching InkScape.

I'll take a look at pdftex and see if that is workable. But, still, I like
to have as short a tool chain as possible.



--
Sent from: http://forum.openscad.org/
_______________________________________________
OpenSCAD mailing list
To unsubscribe send an email to [hidden email]


Sent from the OpenSCAD mailing list archive at Nabble.com.

_______________________________________________
OpenSCAD mailing list
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: simple-pdf export test

OpenSCAD mailing list-2
pdftk is convenient for concatenation and basic operations --- pdftex is a lot more flexible and capable, offering the option of putting multiple graphics on a single page, or adding them one per page to the document.

It also has a shell-escape mechanism for converting graphics, and there is a package for directly importing an SVG:

https://www.ctan.org/tex-archive/graphics/svg

I've also found pdftex to be faster than pdftk for some operations.

Lastly, it might be worth looking into using lualatex.

William


-----Original Message-----
From: adrianv <[hidden email]>
To: [hidden email]
Sent: Sun, Jun 13, 2021 9:27 am
Subject: [OpenSCAD] Re: simple-pdf export test

I didn't really think too carefully about this when you posted it originally, but 15 hours!  You said you had a lot of files, but not how many you mean,   Is it fifty thousand files?   Because 15 hours seems extremely long.   Where is the time going?

I also don't understand the desire to keep the tool chain short.   Are you using Windows?  Under unix part of the point is that the tools do their one job and you use a lot of them if necessary.   I'd say you should use how ever many tools enable you to complete the task efficiently, and it sounds like your current setup isn't doing the job.  

I concat pdf files with pdftk but I'm not sure how that compares to using pdftex.  I just concated 12 copies of a 300 kB pdf and it runs instantly.  I can't imagine how big your pdf would be for the pdf concatenation step to be nontrivial in time.

Actually I looked up pdftex and it doesn't appear that it has a method for concatenating pdf files.  People talk of writing a TeX file that imports all the pdfs.  I suppose that could be a good solution if you really want to pack multiple pdf files onto one page.  I was assuming one page per pdf.  But using TeX to concatenate pdf files seems unlikely to be the best solution to the plain pdf concatenation problem.  It will surely be slower than pdftk.  

If inkscape is taking 7 hours to convert to pdf I'd consider whether there's a better alternative, unless you do have tends of thousands of files.   So perhaps:
https://itectec.com/superuser/how-to-convert-an-svg-to-a-pdf-on-linux/

Troberg wrote
OpenSCAD mailing list-2 wrote
> How are you doing the conversion?
> Couldn't you just automate the conversion and concatenation using a tool
> such as pdftex?
> If you'd send me a set of files incl. a PDF (or just the filenames) and a
> note on the order I'll gladly work up a file for this.

I do it by batching InkScape.

I'll take a look at pdftex and see if that is workable. But, still, I like
to have as short a tool chain as possible.



--
Sent from: http://forum.openscad.org/
_______________________________________________
OpenSCAD mailing list
To unsubscribe send an email to [hidden email]


Sent from the OpenSCAD mailing list archive at Nabble.com.
_______________________________________________
OpenSCAD mailing list
To unsubscribe send an email to [hidden email]

_______________________________________________
OpenSCAD mailing list
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: simple-pdf export test

JordanBrown
In reply to this post by JordanBrown
More data:

On both Windows and Ubuntu, I ran the test before and after my changes.

On both platforms, the differences between the before and after runs are exactly what I expect:  the OpenSCAD-rendered text is spaced differently.  The axes and their labels are exactly identical between the two runs.

So, minor mystery #1:  Why is the Ubuntu result different from the Windows result?  (My guesses: either the different versions of cairo, or different font availability.)
And bigger mystery #2:  Why does the test succeed on the "before" results on both platforms, and the Ubuntu run fails on the "after" runs when using the Windows image as the golden copy?

And of course that leads into the problem that I really need to solve:  how do I make the test happy?
Troberg hints at one possible answer:  change the PDF export to optionally include the axes and labels, and have the test use images that don't include them.

Anybody know?

_______________________________________________
OpenSCAD mailing list
To unsubscribe send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: simple-pdf export test

Troberg
In reply to this post by adrianv
adrianv wrote
I didn't really think too carefully about this when you posted it originally,
but 15 hours!  You said you had a lot of files, but not how many you mean,  
Is it fifty thousand files?   Because 15 hours seems extremely long.   Where
is the time going?
Around 5000. Most of the time is spend in rendering and converting to PDF, and most of that time is probably in loading OpenSCAD and InkScape for each file. I do some other stuff as well, such as sorting, concatenating and fixing file formats, but that's less than 5 minutes.

adrianv wrote
I also don't understand the desire to keep the tool chain short.   Are you
using Windows?  Under unix part of the point is that the tools do their one
job and you use a lot of them if necessary.   I'd say you should use how
ever many tools enable you to complete the task efficiently, and it sounds
like your current setup isn't doing the job.
Windows, my Linux machines don't have the hardware to do this anywhere near as fast. The reason I want the toolchain short is that one day, I'll run this on another machine, and it'll make it much easier to get it running again.

Sent from the OpenSCAD mailing list archive at Nabble.com.

_______________________________________________
OpenSCAD mailing list
To unsubscribe send an email to [hidden email]