When modules are cached?

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

When modules are cached?

Ronaldo
I have a demanding operator that is called many many times, each time with a different non demanding purpose. I would expect that its operation was done only for the first time and cached since there is no children or parameter changes. However, the run time seems to indicate that the operation is executed at each call.

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: When modules are cached?

nophead
I think it is geometry that is cached with key from the module parameters of the primitive that creates it. Are you sure they are all the same? and the $ variables?

If the module has a lot of code that takes time to calculate, like a sweep, then that will be executed each time I think. It is only the geometry that isn't repeated.   

Is this F5 or F6?

What happens if you wrap it with a render?


On Mon, 28 Oct 2019 at 14:16, Ronaldo Persiano <[hidden email]> wrote:
I have a demanding operator that is called many many times, each time with a different non demanding purpose. I would expect that its operation was done only for the first time and cached since there is no children or parameter changes. However, the run time seems to indicate that the operation is executed at each call.
_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: When modules are cached?

Ronaldo

I think it is geometry that is cached with key from the module parameters of the primitive that creates it. Are you sure they are all the same? and the $ variables?

There is no $ variables and not even parameters in the operator module. It returns just a simple 2D rectangle so the remaining code should be fast.
 
If the module has a lot of code that takes time to calculate, like a sweep, then that will be executed each time I think. It is only the geometry that isn't repeated.

It is not my case. It involves linear_extrude and projection though. 
  Is this F5 or F6?

F5
What happens if you wrap it with a render?
 
No benefit. 

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: When modules are cached?

nophead
Projection is very slow because it uses CGAL. I wonder if 2D geometry gets cached because all the other 2D ops are very fast.

On Mon, 28 Oct 2019 at 15:59, Ronaldo Persiano <[hidden email]> wrote:

I think it is geometry that is cached with key from the module parameters of the primitive that creates it. Are you sure they are all the same? and the $ variables?

There is no $ variables and not even parameters in the operator module. It returns just a simple 2D rectangle so the remaining code should be fast.
 
If the module has a lot of code that takes time to calculate, like a sweep, then that will be executed each time I think. It is only the geometry that isn't repeated.

It is not my case. It involves linear_extrude and projection though. 
  Is this F5 or F6?

F5
What happens if you wrap it with a render?
 
No benefit. 
_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: When modules are cached?

Ronaldo
As illustration of my concerns, please look at the following code:

B();
A(50000) C();
 
module C() square(10,center=true);
 
module B() {
   projection()
    linear_extrude()
      intersection(){
        projection() rotate([10,20,30]) cube(1000,center=true);
        projection() rotate([20,30,10]) cube(1000,center=true);
        projection() rotate([30,10,20]) cube(1000,center=true);
        square(10,center=true);
      }
  }
 
module A(n=0)
  for(i=[for(i=0;i<n;i=i+1) i]) rotate(360*i/n) children();
    
B() is a very expensive way to construct the same geometry (a square) of C(). None of them have parameters. A() is just a multiple rotation of its children.

As is, this code runs in 4 sec in my machine. If I exchange the places of B() and C() in the main code and flush the cache, the code runs in 23 sec.
I would expect that both B() and C() were evaluated once and their cached geometry were used in both versions of the main code. If was that so, both main code would have the same runtime.

What objects are cached? When they are cached?

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: When modules are cached?

Ronaldo
I would expect that both B() and C() were evaluated once and their cached geometry were used in both versions of the main code. If was that so, both main code would have the same runtime.

Perhaps I have a wrong understanding on how cache is done or works. Any clue on that is welcome.

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: When modules are cached?

MichaelAtOz
Administrator
Congratulations, you discovered a  crash
<https://github.com/openscad/openscad/issues/3112>  . So do not do
Display_CSG_Products...

In the console B(); A() C(); F5 has:

Parsing design (AST generation)...
Compiling design (CSG Tree generation)...
Compiling design (CSG Products generation)...
Geometries in cache: 8
Geometry cache size in bytes: 2704
CGAL Polyhedrons in cache: 0
CGAL cache size in bytes: 0
Compiling design (CSG Products normalization)...
Normalized CSG tree has 50001 elements
Compile and preview finished.
Total rendering time: 0 hours, 0 minutes, 3 seconds

Then do Design/Display_CSG_Tree. Don't close the Window.

Then change to C(); A() B(); Clear_Cache/F5 which shows:

Compiling design (CSG Tree generation)...
Compiling design (CSG Products generation)...
Geometries in cache: 8
Geometry cache size in bytes: 2704
CGAL Polyhedrons in cache: 0
CGAL cache size in bytes: 0
Compiling design (CSG Products normalization)...
Normalized CSG tree has 50001 elements
Compile and preview finished.
Total rendering time: 0 hours, 0 minutes, 21 seconds

Then do Design/Display_CSG_Tree. Compare the two trees, (if your's is the
same, the scroll bars behave strangely until you scroll a bit).

Note the 50001 elements.
The first has a few more complex elements and 50,000 odd simpler elements.
The second 50,000 odd complex elements.

The tree needs to be parsed to display the image.
I don't think it is cache related.
There are --debug <something> command line options that could show cache
activity, I'd have to guess the right values...



-----
Admin - email* me if you need anything, or if I've done something stupid...

* click on my MichaelAtOz label, there is a link to email me.

Unless specifically shown otherwise above, my contribution is in the Public Domain; to the extent possible under law, I have waived all copyright and related or neighbouring rights to this work. Obviously inclusion of works of previous authors is not included in the above.

The TPP is no simple “trade agreement.”   Fight it! http://www.ourfairdeal.org/   time is running out!
--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Admin - email* me if you need anything, or if I've done something stupid...
* click on my MichaelAtOz label, there is a link to email me.

Unless specifically shown otherwise above, my contribution is in the Public Domain; to the extent possible under law, I have waived all copyright and related or neighbouring rights to this work.
Obviously inclusion of works of previous authors is not included in the above.
Reply | Threaded
Open this post in threaded view
|

Re: When modules are cached?

MichaelAtOz
Administrator
I wonder whether collapsing the redundant
group() {
        group() {
would improve it somewhat??



-----
Admin - email* me if you need anything, or if I've done something stupid...

* click on my MichaelAtOz label, there is a link to email me.

Unless specifically shown otherwise above, my contribution is in the Public Domain; to the extent possible under law, I have waived all copyright and related or neighbouring rights to this work. Obviously inclusion of works of previous authors is not included in the above.

The TPP is no simple “trade agreement.”   Fight it! http://www.ourfairdeal.org/   time is running out!
--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Admin - email* me if you need anything, or if I've done something stupid...
* click on my MichaelAtOz label, there is a link to email me.

Unless specifically shown otherwise above, my contribution is in the Public Domain; to the extent possible under law, I have waived all copyright and related or neighbouring rights to this work.
Obviously inclusion of works of previous authors is not included in the above.
Reply | Threaded
Open this post in threaded view
|

Re: When modules are cached?

Ronaldo
Thank you, Michael, for expending your time on that. I have not gotten any crash with version 2019.01-RC4. I guess it will depend on the available memory to display the CSG tree.

As we say in Portuguese,I have shot what I saw and hit what I haven't. Besides, the code I have presented was a failed attempt to illustrate with a small code the problem I am having in my original code. Here is a better illustration although not as simple.

I have devised the following code to find the bounding box of a 2D object:

module bbox()
  intersection() {
    ext = pow(2,16);
    hull() {
      translate([ ext,    0]) children();
      translate([-ext,    0]) children();
    }
    hull() {
      translate([    0, ext]) children();
      translate([    0,-ext]) children();
    }
  }

It returns the bounding box (a rectangle) of 2D objects with points away from the origin less than pow(2,15) (or something like that).

It is rather efficient even with complex 2D geometries:

n = 270;
text = S(n); 
font = "Palatino Linotype:style=Italic";

// just a complex 2D model
module T() resize([100,0]) text(text,font=font,valign="center",halign="center");

// creates a string with n capital S
function S(n,s="") = n<=0? s: S(n-1,str(s,"S"));

T();
bbx() T();

But things get a bit convoluted when bbx() is called many times even over the same object:

module complexbbx(p)
  intersection(){
    render() bbox() children();
    bbox() children();
    bbox() children();
    bbox() children();
    bbox() children();
    bbox() children();
    bbox() children();
    bbox() children();
    bbox() children();
  }  


  complexbbx([1,1]) T();

This last code requires 18 sec on my machine while a simple call to bbx() T() requires 2 sec.

In the next code,  bbox() children() is called just once but the intersection of 9 squares is still done:

module complexbbx2(p)
  intersection(){
    render() bbox() children();
    square([100,10],center=true);
    square([100,10],center=true);
    square([100,10],center=true);
    square([100,10],center=true);
    square([100,10],center=true);
    square([100,10],center=true);
    square([100,10],center=true);
    square([100,10],center=true);
  }  


Now,  complexbbx2([1,1]) T()  requires only 2 sec to run which shows 9 rectangle intersection is negligible when compared to bbox(). It also shows that the bbox() children() in  complexbbx() is not being cached but instead computed again and again.

So, the question in this   thread subject remains unanswered. Is there any way I could enforce that bbox() children() in  complexbbx() be cached?

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: When modules are cached?

nophead
I think it needs an answer from one of the developers but my guess is that 2D objects are not cached at all. They live in a world of their own with 64 bit fixed point numbers. Most 2D operations are blindingly fast compared to the very slow 3D CGAL stuff, so it would normally not give much speed increase to cache them. 

On Fri, 1 Nov 2019 at 18:13, Ronaldo Persiano <[hidden email]> wrote:
Thank you, Michael, for expending your time on that. I have not gotten any crash with version 2019.01-RC4. I guess it will depend on the available memory to display the CSG tree.

As we say in Portuguese,I have shot what I saw and hit what I haven't. Besides, the code I have presented was a failed attempt to illustrate with a small code the problem I am having in my original code. Here is a better illustration although not as simple.

I have devised the following code to find the bounding box of a 2D object:

module bbox()
  intersection() {
    ext = pow(2,16);
    hull() {
      translate([ ext,    0]) children();
      translate([-ext,    0]) children();
    }
    hull() {
      translate([    0, ext]) children();
      translate([    0,-ext]) children();
    }
  }

It returns the bounding box (a rectangle) of 2D objects with points away from the origin less than pow(2,15) (or something like that).

It is rather efficient even with complex 2D geometries:

n = 270;
text = S(n); 
font = "Palatino Linotype:style=Italic";

// just a complex 2D model
module T() resize([100,0]) text(text,font=font,valign="center",halign="center");

// creates a string with n capital S
function S(n,s="") = n<=0? s: S(n-1,str(s,"S"));

T();
bbx() T();

But things get a bit convoluted when bbx() is called many times even over the same object:

module complexbbx(p)
  intersection(){
    render() bbox() children();
    bbox() children();
    bbox() children();
    bbox() children();
    bbox() children();
    bbox() children();
    bbox() children();
    bbox() children();
    bbox() children();
  }  


  complexbbx([1,1]) T();

This last code requires 18 sec on my machine while a simple call to bbx() T() requires 2 sec.

In the next code,  bbox() children() is called just once but the intersection of 9 squares is still done:

module complexbbx2(p)
  intersection(){
    render() bbox() children();
    square([100,10],center=true);
    square([100,10],center=true);
    square([100,10],center=true);
    square([100,10],center=true);
    square([100,10],center=true);
    square([100,10],center=true);
    square([100,10],center=true);
    square([100,10],center=true);
  }  


Now,  complexbbx2([1,1]) T()  requires only 2 sec to run which shows 9 rectangle intersection is negligible when compared to bbox(). It also shows that the bbox() children() in  complexbbx() is not being cached but instead computed again and again.

So, the question in this   thread subject remains unanswered. Is there any way I could enforce that bbox() children() in  complexbbx() be cached?
_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: When modules are cached?

Ronaldo
I think it needs an answer from one of the developers but my guess is that 2D objects are not cached at all. They live in a world of their own with 64 bit fixed point numbers. Most 2D operations are blindingly fast compared to the very slow 3D CGAL stuff, so it would normally not give much speed increase to cache them. 
 
If so, I would expect that bbox0() children() in the following version would be cached but my tests show the opposite. T() as before.

module complexbbx(p)
  intersection(){
    render() projection() bbox0() children();
    projection() bbox0() children();
    projection() bbox0() children();
    projection() bbox0() children();
    projection() bbox0() children();
    projection() bbox0() children();
    projection() bbox0() children();
    projection() bbox0() children();
    projection() bbox0() children();
  }  

module bbox0()
  linear_extrude()
  intersection() {
    ext = pow(2,16);
    hull() {
      translate([ ext,    0]) children();
      translate([-ext,    0]) children();
    }
    hull() {
      translate([    0, ext]) children();
      translate([    0,-ext]) children();
    }
  }
 
projection() bbox0() T();
complexbbx([1,1]) T();


_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: When modules are cached?

nophead
Even if bbox was cached you are calling projection many times and it is the odd one out as it is a very slow CGAL operation but it produces a 2D results, so perhaps not cached.

On Fri, 1 Nov 2019 at 20:10, Ronaldo Persiano <[hidden email]> wrote:
I think it needs an answer from one of the developers but my guess is that 2D objects are not cached at all. They live in a world of their own with 64 bit fixed point numbers. Most 2D operations are blindingly fast compared to the very slow 3D CGAL stuff, so it would normally not give much speed increase to cache them. 
 
If so, I would expect that bbox0() children() in the following version would be cached but my tests show the opposite. T() as before.

module complexbbx(p)
  intersection(){
    render() projection() bbox0() children();
    projection() bbox0() children();
    projection() bbox0() children();
    projection() bbox0() children();
    projection() bbox0() children();
    projection() bbox0() children();
    projection() bbox0() children();
    projection() bbox0() children();
    projection() bbox0() children();
  }  

module bbox0()
  linear_extrude()
  intersection() {
    ext = pow(2,16);
    hull() {
      translate([ ext,    0]) children();
      translate([-ext,    0]) children();
    }
    hull() {
      translate([    0, ext]) children();
      translate([    0,-ext]) children();
    }
  }
 
projection() bbox0() T();
complexbbx([1,1]) T();

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: When modules are cached?

Ronaldo

Even if bbox was cached you are calling projection many times and it is the odd one out as it is a very slow CGAL operation but it produces a 2D results, so perhaps not cached.

I checked that the projection of a cube (like bbox0() is) and  the intersections have a negligible runtime in this case. The repetitive computation of bbox0() children() (a 3D object) is what is slowing the run.

A simpler and more dramatic case is the following:

module complexbbx3(p)
  projection()
  intersection(){
    render() children();
    children();
    children();
    children();
    children();
    children();
    children();
    children();
    children();
  }  

module bbox0()
  linear_extrude()
  intersection() {
    ext = pow(2,16);
    hull() {
      translate([ ext,    0]) children();
      translate([-ext,    0]) children();
    }
    hull() {
      translate([    0, ext]) children();
      translate([    0,-ext]) children();
    }
  } 
  
complexbbx3([1,1])  bbox0() T();

Here, bbox0() T() is a 3D solid, a cube.  complexbbx3(p) applied on a cube do just the intersection of 9 equal cubes and project it into a square. The intersection of 9 equal cubes and the (only one) projection have negligible runtime compared with the bbox0() operation. What the runtimes show is that the children of  complexbbx3() are evaluated 9 times. So, its children geometry (a 3D solid) is not cached.

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: When modules are cached?

nophead
What happens if you put render() linear_extrude() in bbox()?

On Fri, 1 Nov 2019 at 20:49, Ronaldo Persiano <[hidden email]> wrote:

Even if bbox was cached you are calling projection many times and it is the odd one out as it is a very slow CGAL operation but it produces a 2D results, so perhaps not cached.

I checked that the projection of a cube (like bbox0() is) and  the intersections have a negligible runtime in this case. The repetitive computation of bbox0() children() (a 3D object) is what is slowing the run.

A simpler and more dramatic case is the following:

module complexbbx3(p)
  projection()
  intersection(){
    render() children();
    children();
    children();
    children();
    children();
    children();
    children();
    children();
    children();
  }  

module bbox0()
  linear_extrude()
  intersection() {
    ext = pow(2,16);
    hull() {
      translate([ ext,    0]) children();
      translate([-ext,    0]) children();
    }
    hull() {
      translate([    0, ext]) children();
      translate([    0,-ext]) children();
    }
  } 
  
complexbbx3([1,1])  bbox0() T();

Here, bbox0() T() is a 3D solid, a cube.  complexbbx3(p) applied on a cube do just the intersection of 9 equal cubes and project it into a square. The intersection of 9 equal cubes and the (only one) projection have negligible runtime compared with the bbox0() operation. What the runtimes show is that the children of  complexbbx3() are evaluated 9 times. So, its children geometry (a 3D solid) is not cached.
_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: When modules are cached?

Ronaldo
What happens if you put render() linear_extrude() in bbox()?

Sorry but my last tests were wrong. It seems that the geometry of bbox0() T() is really cached. That is, it seems that only 3D geometry are cached, what is a shame.

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Change mailing to daily digest?

unmechanical
In reply to this post by Ronaldo
Hey, sorry if it says so somewhere, I haven't sen it - but can I change my subscription to this list somewhere to send daily digests instead of dozens of mails a day?
 

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: Change mailing to daily digest?

Michael Marx

http://lists.openscad.org/mailman/options/discuss_lists.openscad.org

 

There is a link in the first pinned post on the Forum.

 


From: Discuss [mailto:[hidden email]] On Behalf Of [hidden email]
Sent: Sat, 2 Nov 2019 09:12
To: [hidden email]
Subject: [OpenSCAD] Change mailing to daily digest?

 

Hey, sorry if it says so somewhere, I haven't sen it - but can I change my subscription to this list somewhere to send daily digests instead of dozens of mails a day?

 


Virus-free. www.avg.com

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org