Module caching

classic Classic list List threaded Threaded
30 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Module caching

nophead
I was under the impression that if a module was instantiated multiple times with the same parameters the results were cached and reused.

If I make a module that renders() the difference between a cube and a sphere with lots of vertices  and use it 100 times in a preview it take the same time to preview 100 as it does one. I.e. it only does the slow difference once and then draws the result 100 times.

However if I generate a spring using a helical sweep of a circle than it takes about 1 second to preview one and 100 seconds to preview 100.

So it seems I misunderstood the caching. It only seems to cache CGAL operations, not the actual instantiation of the module. It still needs to calculate the sweep each time despite all the springs being identical. Is this the case? I.e. it doesn't shortcut modules when the parameters don't change?

I am surprised it takes a second to generate all the vertices for the polyhedron. It is about 9000 facets generated with list comprehensions and transformations in an interpreted language, but even so, PCs are very fast.

image.png

Looks like I need to save the points and face lists myself and reuse them.

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: Module caching

Parkinbot
polyhedrons don't get cached. The cache mechanism would have to hash both
parameters.



--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: Module caching

nophead
So I rewrote my spring module as a function that returns a mesh (points and faces) so I can save it as a variable and then re-use it to draw many springs by simply calling polyhedron.

100 springs was taking 61 seconds when it was doing a sweep for each one. With only one sweep to generate the mesh and 100 calls of polygon it now takes 23 seconds. Why is polyhedron so slow? I expected it to by instantaneous in F5. All it has to do is process the lists into a polyset.

On Tue, 22 Jan 2019 at 23:35, Parkinbot <[hidden email]> wrote:
polyhedrons don't get cached. The cache mechanism would have to hash both
parameters.



--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: Module caching

Parkinbot
I guess I don't have to tell you you can "activate" the cache by importing an
STL of the spring as a work around.
Therefore I understand your question more like: Why isn't/can't polyhedron
be implemented to be faster.

Marius pointed out in a post some years ago that OpenSCAD has to sort out a
structure from the triag soup when importing a STL. So, even for F5 it
cannot just copy the soup into the GPU, but has to analyse it.  
While a STL contains triags only, polyhedron can be called with arbitrary
polygon faces. This for sure will create additional overhead.  
 



--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: Module caching

nophead
But OpenSCAD's Polyset class is just a polygon soup, so no better than STL and suffers all the same problems as far as I can see. You have to sort anything out to make one from a polyhedron, just convert the face lists to polygons by indexing the points. You are actually going from a more structured data format to a less complicated one. I think it is only if you want to use it with CGAL that a more complicated data structure is used, I.e. Nef_Polyhedron. For an F5 preview a PolySet should be all you need. It needs to be triangulated but I have already done that for all but the endcaps.

The code is here: https://github.com/openscad/openscad/blob/69bf5a55475102be43c2ddc698198e85821332b3/src/primitives.cc#L552 I don't see anything that would take any time on a modern PC that runs at GHz.

On Sat, 26 Jan 2019 at 13:08, Parkinbot <[hidden email]> wrote:
I guess I don't have to tell you you can "activate" the cache by importing an
STL of the spring as a work around.
Therefore I understand your question more like: Why isn't/can't polyhedron
be implemented to be faster.

Marius pointed out in a post some years ago that OpenSCAD has to sort out a
structure from the triag soup when importing a STL. So, even for F5 it
cannot just copy the soup into the GPU, but has to analyse it. 
While a STL contains triags only, polyhedron can be called with arbitrary
polygon faces. This for sure will create additional overhead. 




--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: Module caching

nophead
Is it perhaps that the lists are stored as strings when the polyhedron is instantiated and then parsed to vectors every time the geometry is created.

Would converting them to vectors of doubles in PrimitiveModule::instantiate make it faster or is that called 100 times as well?

On Sat, 26 Jan 2019 at 17:33, nop head <[hidden email]> wrote:
But OpenSCAD's Polyset class is just a polygon soup, so no better than STL and suffers all the same problems as far as I can see. You have to sort anything out to make one from a polyhedron, just convert the face lists to polygons by indexing the points. You are actually going from a more structured data format to a less complicated one. I think it is only if you want to use it with CGAL that a more complicated data structure is used, I.e. Nef_Polyhedron. For an F5 preview a PolySet should be all you need. It needs to be triangulated but I have already done that for all but the endcaps.

The code is here: https://github.com/openscad/openscad/blob/69bf5a55475102be43c2ddc698198e85821332b3/src/primitives.cc#L552 I don't see anything that would take any time on a modern PC that runs at GHz.

On Sat, 26 Jan 2019 at 13:08, Parkinbot <[hidden email]> wrote:
I guess I don't have to tell you you can "activate" the cache by importing an
STL of the spring as a work around.
Therefore I understand your question more like: Why isn't/can't polyhedron
be implemented to be faster.

Marius pointed out in a post some years ago that OpenSCAD has to sort out a
structure from the triag soup when importing a STL. So, even for F5 it
cannot just copy the soup into the GPU, but has to analyse it. 
While a STL contains triags only, polyhedron can be called with arbitrary
polygon faces. This for sure will create additional overhead. 




--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: Module caching

Parkinbot
nophead wrote
> Is it perhaps that the lists are stored as strings when the polyhedron is
> instantiated and then parsed to vectors every time the geometry is
> created.

I doubt that. But you are right, the code really looks like nothing very
time consuming is happening.
Is the triangulation of the faces done by Open GL? What happens after the
geometry has been created?



--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: Module caching

nophead
It spends most of the 22 seconds stuck at Compiling design (CSG Tree generation)...

After that it does all the rest in about 1 second, including a very fast progress bar.

Compiling design (CSG Products generation)...

Geometries in cache: 395

Geometry cache size in bytes: 20331104

CGAL Polyhedrons in cache: 0

CGAL cache size in bytes: 0

Compiling design (CSG Products normalization)...

Normalized CSG tree has 100 elements

Compile and preview finished.

Total rendering time: 0 hours, 0 minutes, 22 seconds


It would be a problem if I was designing a mattress with PLA springs but fortunately only a battery box with four, but it would be nice if polyhedrons were cached like other geometry seems to be. I clearly don't fully understand OpenSCADs caching system.


On Sun, 27 Jan 2019 at 22:33, Parkinbot <[hidden email]> wrote:
nophead wrote
> Is it perhaps that the lists are stored as strings when the polyhedron is
> instantiated and then parsed to vectors every time the geometry is
> created.

I doubt that. But you are right, the code really looks like nothing very
time consuming is happening.
Is the triangulation of the faces done by Open GL? What happens after the
geometry has been created?



--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: Module caching

thehans
Hi nophead, 

Could you post a sample script for others to reproduce the slowness you experience?

Thanks,
Hans

On Sun, Jan 27, 2019 at 4:59 PM nop head <[hidden email]> wrote:
It spends most of the 22 seconds stuck at Compiling design (CSG Tree generation)...

After that it does all the rest in about 1 second, including a very fast progress bar.

Compiling design (CSG Products generation)...

Geometries in cache: 395

Geometry cache size in bytes: 20331104

CGAL Polyhedrons in cache: 0

CGAL cache size in bytes: 0

Compiling design (CSG Products normalization)...

Normalized CSG tree has 100 elements

Compile and preview finished.

Total rendering time: 0 hours, 0 minutes, 22 seconds


It would be a problem if I was designing a mattress with PLA springs but fortunately only a battery box with four, but it would be nice if polyhedrons were cached like other geometry seems to be. I clearly don't fully understand OpenSCADs caching system.


On Sun, 27 Jan 2019 at 22:33, Parkinbot <[hidden email]> wrote:
nophead wrote
> Is it perhaps that the lists are stored as strings when the polyhedron is
> instantiated and then parsed to vectors every time the geometry is
> created.

I doubt that. But you are right, the code really looks like nothing very
time consuming is happening.
Is the triangulation of the faces done by Open GL? What happens after the
geometry has been created?



--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: Module caching

nophead

I captured the points and face lists generated by sweep and hard coded them to remove all of my library. Each spring is 4576 points, 9122 faces but they are all the same polyhedron repeated 100 times. Given that it doesn't cache polyhedra it has to process the same lists to a Polyset 100 times. Taking about 1/4 second to process 9000 faces seems very slow to me on a 64 bit 2.5GHz core I7 with 8 GB memory.

On Mon, 28 Jan 2019 at 02:23, Hans L <[hidden email]> wrote:
Hi nophead, 

Could you post a sample script for others to reproduce the slowness you experience?

Thanks,
Hans

On Sun, Jan 27, 2019 at 4:59 PM nop head <[hidden email]> wrote:
It spends most of the 22 seconds stuck at Compiling design (CSG Tree generation)...

After that it does all the rest in about 1 second, including a very fast progress bar.

Compiling design (CSG Products generation)...

Geometries in cache: 395

Geometry cache size in bytes: 20331104

CGAL Polyhedrons in cache: 0

CGAL cache size in bytes: 0

Compiling design (CSG Products normalization)...

Normalized CSG tree has 100 elements

Compile and preview finished.

Total rendering time: 0 hours, 0 minutes, 22 seconds


It would be a problem if I was designing a mattress with PLA springs but fortunately only a battery box with four, but it would be nice if polyhedrons were cached like other geometry seems to be. I clearly don't fully understand OpenSCADs caching system.


On Sun, 27 Jan 2019 at 22:33, Parkinbot <[hidden email]> wrote:
nophead wrote
> Is it perhaps that the lists are stored as strings when the polyhedron is
> instantiated and then parsed to vectors every time the geometry is
> created.

I doubt that. But you are right, the code really looks like nothing very
time consuming is happening.
Is the triangulation of the faces done by Open GL? What happens after the
geometry has been created?



--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: Module caching

thehans
Are you using a somewhat recent dev snapshot?  The CSG Tree generation and key storing got reworked significantly in the past year.

With the example you gave I get 7s preview and "Geometries in cache: 1". I see that yours is showing 395 geometries so either your posted example simplified away the cause of the issue, or something different between with the build you are running.

On Mon, Jan 28, 2019 at 2:59 AM nop head <[hidden email]> wrote:

I captured the points and face lists generated by sweep and hard coded them to remove all of my library. Each spring is 4576 points, 9122 faces but they are all the same polyhedron repeated 100 times. Given that it doesn't cache polyhedra it has to process the same lists to a Polyset 100 times. Taking about 1/4 second to process 9000 faces seems very slow to me on a 64 bit 2.5GHz core I7 with 8 GB memory.

On Mon, 28 Jan 2019 at 02:23, Hans L <[hidden email]> wrote:
Hi nophead, 

Could you post a sample script for others to reproduce the slowness you experience?

Thanks,
Hans

On Sun, Jan 27, 2019 at 4:59 PM nop head <[hidden email]> wrote:
It spends most of the 22 seconds stuck at Compiling design (CSG Tree generation)...

After that it does all the rest in about 1 second, including a very fast progress bar.

Compiling design (CSG Products generation)...

Geometries in cache: 395

Geometry cache size in bytes: 20331104

CGAL Polyhedrons in cache: 0

CGAL cache size in bytes: 0

Compiling design (CSG Products normalization)...

Normalized CSG tree has 100 elements

Compile and preview finished.

Total rendering time: 0 hours, 0 minutes, 22 seconds


It would be a problem if I was designing a mattress with PLA springs but fortunately only a battery box with four, but it would be nice if polyhedrons were cached like other geometry seems to be. I clearly don't fully understand OpenSCADs caching system.


On Sun, 27 Jan 2019 at 22:33, Parkinbot <[hidden email]> wrote:
nophead wrote
> Is it perhaps that the lists are stored as strings when the polyhedron is
> instantiated and then parsed to vectors every time the geometry is
> created.

I doubt that. But you are right, the code really looks like nothing very
time consuming is happening.
Is the triangulation of the faces done by Open GL? What happens after the
geometry has been created?



--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: Module caching

nophead
I am using a snapshot from yesterday.

I only get one geometry if that is the only file loaded. I had more MDI windows open when I posted yesterday.

You might have a machine that is three times faster than this laptop but even so 7 seconds seems slow to me. It has to read about 1 million faces from a list and convert them into Polysets which are just vectors of vectors of vertices. Perhaps I have just lost touch with how inefficient modern software is.

OpenSCAD 2019.01.27.ci1306
https://www.openscad.org/

Copyright (C) 2009-2019 The OpenSCAD Developers

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

Loaded design 'C:/Users/ChrisP/Documents/springs.csg'.

Compiling design (CSG Tree generation)...

Compiling design (CSG Products generation)...

Geometries in cache: 1

Geometry cache size in bytes: 657560

CGAL Polyhedrons in cache: 0

CGAL cache size in bytes: 0

Compiling design (CSG Products normalization)...

Normalized CSG tree has 100 elements

Compile and preview finished.

Total rendering time: 0 hours, 0 minutes, 21 seconds


On Mon, 28 Jan 2019 at 10:31, Hans L <[hidden email]> wrote:
Are you using a somewhat recent dev snapshot?  The CSG Tree generation and key storing got reworked significantly in the past year.

With the example you gave I get 7s preview and "Geometries in cache: 1". I see that yours is showing 395 geometries so either your posted example simplified away the cause of the issue, or something different between with the build you are running.

On Mon, Jan 28, 2019 at 2:59 AM nop head <[hidden email]> wrote:

I captured the points and face lists generated by sweep and hard coded them to remove all of my library. Each spring is 4576 points, 9122 faces but they are all the same polyhedron repeated 100 times. Given that it doesn't cache polyhedra it has to process the same lists to a Polyset 100 times. Taking about 1/4 second to process 9000 faces seems very slow to me on a 64 bit 2.5GHz core I7 with 8 GB memory.

On Mon, 28 Jan 2019 at 02:23, Hans L <[hidden email]> wrote:
Hi nophead, 

Could you post a sample script for others to reproduce the slowness you experience?

Thanks,
Hans

On Sun, Jan 27, 2019 at 4:59 PM nop head <[hidden email]> wrote:
It spends most of the 22 seconds stuck at Compiling design (CSG Tree generation)...

After that it does all the rest in about 1 second, including a very fast progress bar.

Compiling design (CSG Products generation)...

Geometries in cache: 395

Geometry cache size in bytes: 20331104

CGAL Polyhedrons in cache: 0

CGAL cache size in bytes: 0

Compiling design (CSG Products normalization)...

Normalized CSG tree has 100 elements

Compile and preview finished.

Total rendering time: 0 hours, 0 minutes, 22 seconds


It would be a problem if I was designing a mattress with PLA springs but fortunately only a battery box with four, but it would be nice if polyhedrons were cached like other geometry seems to be. I clearly don't fully understand OpenSCADs caching system.


On Sun, 27 Jan 2019 at 22:33, Parkinbot <[hidden email]> wrote:
nophead wrote
> Is it perhaps that the lists are stored as strings when the polyhedron is
> instantiated and then parsed to vectors every time the geometry is
> created.

I doubt that. But you are right, the code really looks like nothing very
time consuming is happening.
Is the triangulation of the faces done by Open GL? What happens after the
geometry has been created?



--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: Module caching

Parkinbot
In reply to this post by thehans
thehans wrote
> Are you using a somewhat recent dev snapshot?  The CSG Tree generation and
> key storing got reworked significantly in the past year.

I tried some older snaps with 100 copies of a sweep object, each containing
19002 facets and no buffering of the points and faces: one from 2016
(1'36"), one from 2017 (1'16"), one from 2018 (1'00") and a recent 2019
(1':00"). So there was *some* progress.
With buffering of points and faces the 2019 version needed 0'42".

As compiling an own fork of OpenSCAD is not recommended on Windows (too much
effort to set it up), I don't know the code well enough to be able to follow
what is happening in an F5 context. But I am curious.

Can anybody provide a sketch of how the execution goes on after
createGeometry()?



--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: Module caching

nophead
I think the majority of the time is taken in createGeometry, not after it it.

100 spheres with $fn set to 150, which gives  11250 vertices each, only takes 1 second, presumably because it is cached, or it is 20 times quicker to calculate vertices with trig than it is to read them from a list.

On Mon, 28 Jan 2019 at 14:40, Parkinbot <[hidden email]> wrote:
thehans wrote
> Are you using a somewhat recent dev snapshot?  The CSG Tree generation and
> key storing got reworked significantly in the past year.

I tried some older snaps with 100 copies of a sweep object, each containing
19002 facets and no buffering of the points and faces: one from 2016
(1'36"), one from 2017 (1'16"), one from 2018 (1'00") and a recent 2019
(1':00"). So there was *some* progress.
With buffering of points and faces the 2019 version needed 0'42".

As compiling an own fork of OpenSCAD is not recommended on Windows (too much
effort to set it up), I don't know the code well enough to be able to follow
what is happening in an F5 context. But I am curious.

Can anybody provide a sketch of how the execution goes on after
createGeometry()?



--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: Module caching

thehans
Well for reference my cpu is: Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz, I  wouldn't expect it to be 3x faster than a 2.5GHz core i7..

I am testing on Linux, not Windows, so maybe that is the main difference. I wonder if Windows snapshots are not getting the same -O2 compiler optimizations?

But anyways, the main thing about caching is that it doesn't simply cache a *call* to a module.  OpenSCAD first converts the entire script into csg format which is just a static representation of primitives, multmatrix transformations and boolean operations, so it has to evaluate all lists comprehensions, control structures etc, creating scope for each node etc; essentially building up the entire output that you see in .csg export, in memory (about 31.6MB of text for nopheads example, although in memory it should be a bit less due to whitespace/indentation stripping).  *Then* it caches Polysets for matching subtrees (substrings basically of any node and its children) from this .csg representation.

Compare the 31.6MB to the size of the .csg output of your sphere example, that's gonna be like a couple kb of text processing.

On Mon, Jan 28, 2019 at 11:01 AM nop head <[hidden email]> wrote:
I think the majority of the time is taken in createGeometry, not after it it.

100 spheres with $fn set to 150, which gives  11250 vertices each, only takes 1 second, presumably because it is cached, or it is 20 times quicker to calculate vertices with trig than it is to read them from a list.

On Mon, 28 Jan 2019 at 14:40, Parkinbot <[hidden email]> wrote:
thehans wrote
> Are you using a somewhat recent dev snapshot?  The CSG Tree generation and
> key storing got reworked significantly in the past year.

I tried some older snaps with 100 copies of a sweep object, each containing
19002 facets and no buffering of the points and faces: one from 2016
(1'36"), one from 2017 (1'16"), one from 2018 (1'00") and a recent 2019
(1':00"). So there was *some* progress.
With buffering of points and faces the 2019 version needed 0'42".

As compiling an own fork of OpenSCAD is not recommended on Windows (too much
effort to set it up), I don't know the code well enough to be able to follow
what is happening in an F5 context. But I am curious.

Can anybody provide a sketch of how the execution goes on after
createGeometry()?



--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: Module caching

nophead
It is only 31.6MB in the csg because the lists are text and are repeated over and over again. But at runtime there should only be one copy of the lists in binary. Does it really generate a textual csg representation in memory or is that just when you dump it? And does it actually make 100 copies of the lists by value? That might explain why it is so slow.

On Mon, 28 Jan 2019 at 18:08, Hans L <[hidden email]> wrote:
Well for reference my cpu is: Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz, I  wouldn't expect it to be 3x faster than a 2.5GHz core i7..

I am testing on Linux, not Windows, so maybe that is the main difference. I wonder if Windows snapshots are not getting the same -O2 compiler optimizations?

But anyways, the main thing about caching is that it doesn't simply cache a *call* to a module.  OpenSCAD first converts the entire script into csg format which is just a static representation of primitives, multmatrix transformations and boolean operations, so it has to evaluate all lists comprehensions, control structures etc, creating scope for each node etc; essentially building up the entire output that you see in .csg export, in memory (about 31.6MB of text for nopheads example, although in memory it should be a bit less due to whitespace/indentation stripping).  *Then* it caches Polysets for matching subtrees (substrings basically of any node and its children) from this .csg representation.

Compare the 31.6MB to the size of the .csg output of your sphere example, that's gonna be like a couple kb of text processing.

On Mon, Jan 28, 2019 at 11:01 AM nop head <[hidden email]> wrote:
I think the majority of the time is taken in createGeometry, not after it it.

100 spheres with $fn set to 150, which gives  11250 vertices each, only takes 1 second, presumably because it is cached, or it is 20 times quicker to calculate vertices with trig than it is to read them from a list.

On Mon, 28 Jan 2019 at 14:40, Parkinbot <[hidden email]> wrote:
thehans wrote
> Are you using a somewhat recent dev snapshot?  The CSG Tree generation and
> key storing got reworked significantly in the past year.

I tried some older snaps with 100 copies of a sweep object, each containing
19002 facets and no buffering of the points and faces: one from 2016
(1'36"), one from 2017 (1'16"), one from 2018 (1'00") and a recent 2019
(1':00"). So there was *some* progress.
With buffering of points and faces the 2019 version needed 0'42".

As compiling an own fork of OpenSCAD is not recommended on Windows (too much
effort to set it up), I don't know the code well enough to be able to follow
what is happening in an F5 context. But I am curious.

Can anybody provide a sketch of how the execution goes on after
createGeometry()?



--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: Module caching

thehans
Yes it really does.  The csg text representation of a node *is* its cache key.  And the csg format doesn't really do individual variable (list) caching.

On Mon, Jan 28, 2019 at 12:41 PM nop head <[hidden email]> wrote:
It is only 31.6MB in the csg because the lists are text and are repeated over and over again. But at runtime there should only be one copy of the lists in binary. Does it really generate a textual csg representation in memory or is that just when you dump it? And does it actually make 100 copies of the lists by value? That might explain why it is so slow.

On Mon, 28 Jan 2019 at 18:08, Hans L <[hidden email]> wrote:
Well for reference my cpu is: Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz, I  wouldn't expect it to be 3x faster than a 2.5GHz core i7..

I am testing on Linux, not Windows, so maybe that is the main difference. I wonder if Windows snapshots are not getting the same -O2 compiler optimizations?

But anyways, the main thing about caching is that it doesn't simply cache a *call* to a module.  OpenSCAD first converts the entire script into csg format which is just a static representation of primitives, multmatrix transformations and boolean operations, so it has to evaluate all lists comprehensions, control structures etc, creating scope for each node etc; essentially building up the entire output that you see in .csg export, in memory (about 31.6MB of text for nopheads example, although in memory it should be a bit less due to whitespace/indentation stripping).  *Then* it caches Polysets for matching subtrees (substrings basically of any node and its children) from this .csg representation.

Compare the 31.6MB to the size of the .csg output of your sphere example, that's gonna be like a couple kb of text processing.

On Mon, Jan 28, 2019 at 11:01 AM nop head <[hidden email]> wrote:
I think the majority of the time is taken in createGeometry, not after it it.

100 spheres with $fn set to 150, which gives  11250 vertices each, only takes 1 second, presumably because it is cached, or it is 20 times quicker to calculate vertices with trig than it is to read them from a list.

On Mon, 28 Jan 2019 at 14:40, Parkinbot <[hidden email]> wrote:
thehans wrote
> Are you using a somewhat recent dev snapshot?  The CSG Tree generation and
> key storing got reworked significantly in the past year.

I tried some older snaps with 100 copies of a sweep object, each containing
19002 facets and no buffering of the points and faces: one from 2016
(1'36"), one from 2017 (1'16"), one from 2018 (1'00") and a recent 2019
(1':00"). So there was *some* progress.
With buffering of points and faces the 2019 version needed 0'42".

As compiling an own fork of OpenSCAD is not recommended on Windows (too much
effort to set it up), I don't know the code well enough to be able to follow
what is happening in an F5 context. But I am curious.

Can anybody provide a sketch of how the execution goes on after
createGeometry()?



--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: Module caching

thehans
I did some profiling and found that its spending much of its time in PolysetUtils::tessellate_faces, because its re-tessellating each geometry after pulling from geometry cache.  Ideally it should tesselate only once *before* putting into the cache.  Pull request incoming. :)

On Mon, Jan 28, 2019 at 12:47 PM Hans L <[hidden email]> wrote:
Yes it really does.  The csg text representation of a node *is* its cache key.  And the csg format doesn't really do individual variable (list) caching.

On Mon, Jan 28, 2019 at 12:41 PM nop head <[hidden email]> wrote:
It is only 31.6MB in the csg because the lists are text and are repeated over and over again. But at runtime there should only be one copy of the lists in binary. Does it really generate a textual csg representation in memory or is that just when you dump it? And does it actually make 100 copies of the lists by value? That might explain why it is so slow.

On Mon, 28 Jan 2019 at 18:08, Hans L <[hidden email]> wrote:
Well for reference my cpu is: Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz, I  wouldn't expect it to be 3x faster than a 2.5GHz core i7..

I am testing on Linux, not Windows, so maybe that is the main difference. I wonder if Windows snapshots are not getting the same -O2 compiler optimizations?

But anyways, the main thing about caching is that it doesn't simply cache a *call* to a module.  OpenSCAD first converts the entire script into csg format which is just a static representation of primitives, multmatrix transformations and boolean operations, so it has to evaluate all lists comprehensions, control structures etc, creating scope for each node etc; essentially building up the entire output that you see in .csg export, in memory (about 31.6MB of text for nopheads example, although in memory it should be a bit less due to whitespace/indentation stripping).  *Then* it caches Polysets for matching subtrees (substrings basically of any node and its children) from this .csg representation.

Compare the 31.6MB to the size of the .csg output of your sphere example, that's gonna be like a couple kb of text processing.

On Mon, Jan 28, 2019 at 11:01 AM nop head <[hidden email]> wrote:
I think the majority of the time is taken in createGeometry, not after it it.

100 spheres with $fn set to 150, which gives  11250 vertices each, only takes 1 second, presumably because it is cached, or it is 20 times quicker to calculate vertices with trig than it is to read them from a list.

On Mon, 28 Jan 2019 at 14:40, Parkinbot <[hidden email]> wrote:
thehans wrote
> Are you using a somewhat recent dev snapshot?  The CSG Tree generation and
> key storing got reworked significantly in the past year.

I tried some older snaps with 100 copies of a sweep object, each containing
19002 facets and no buffering of the points and faces: one from 2016
(1'36"), one from 2017 (1'16"), one from 2018 (1'00") and a recent 2019
(1':00"). So there was *some* progress.
With buffering of points and faces the 2019 version needed 0'42".

As compiling an own fork of OpenSCAD is not recommended on Windows (too much
effort to set it up), I don't know the code well enough to be able to follow
what is happening in an F5 context. But I am curious.

Can anybody provide a sketch of how the execution goes on after
createGeometry()?



--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: Module caching

Parkinbot
Ah I see. Thanks for your information. That explains a lot and it looks like
values are indeed parsed from string representation as nophead had supposed.




--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: Module caching

nophead
I am totally confused because in the source code it is just passed as a variable that happens to be a list. Where does it get converted to back a string again and parsed? Or are OpenSCAD lists just vectors of strings?

And as for re-tessellating. How long does it take to tessellate something that is already mostly triangles? Should I manually tessellate the end caps with a triangle fan? The other 9000+ facets are already triangles.

On Mon, 28 Jan 2019 at 21:30, Parkinbot <[hidden email]> wrote:
Ah I see. Thanks for your information. That explains a lot and it looks like
values are indeed parsed from string representation as nophead had supposed.




--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
12