# CPU and RAM Resources Allocations Increase for OpenSCAD

 Classic List Threaded
37 messages
12
Reply | Threaded
Open this post in threaded view
|

## Re: CPU and RAM Resources Allocations Increase for OpenSCAD

 Yes I know it can easily be optimised but OpenSCAD uses the CGAL library and that doesn't seem to optimise for speed. Also it does all the maths in rational notation, so simple operations take at least three orders of magnitude longer than I would expect. Coupled with the at least O(N) explosion and it gets impractically slow with large numbers of 3D vertices.On 21 May 2018 at 10:56, Rogier Wolff wrote:On Mon, May 21, 2018 at 10:16:37AM +0100, nop head wrote: > I think union time is proportional to the square of the number of facets of > the two items being unioned. With a "number" of spheres with 61738 facets I > don't think it will finish in a life time. > > The way to estimatet how long it will take is to run it with \$fn set to > small numbers and look at how the time increases with \$fn. Then you can > predict how long it will be for \$fn=250. It will be a gross underestimation > because it doesn't take into account swapping, etc. But I am sure it will > show you it is totally impractical to work with such values of \$fn in Weeeelllll.... This means to me that there is work to be done. Is unioning inherently quadratic? Well,  yes it is: You have to check for each pair of triangles if they intersect. Right? Wrong! For example, you can split up the working space is small cubes. Go through the list of triangles once, and those that are small and land in 1 or two of those cubes, you place in a "bin" for that cube. The big trangles and those that are unlucky go into a bin for "other". Now from a theoretical point-of-view, the algorithm will probably still be O(N^2) But from a practical point-of-view, in say the case where you are unioning a bunch of near-perfect spheres, you will be able to handle 90 or 99% of the items in O(N) time. This improves the "feasable dataset size" by a big margin. In practise, I work at low \$fn, until for the final render I increase it.         Roger. -- ** [hidden email] ** http://www.BitWizard.nl/ ** +31-15-2600998 ** **    Delftechpark 26 2628 XH  Delft, The Netherlands. KVK: 27239233    ** *-- BitWizard writes Linux device drivers for any device you may have! --* The plan was simple, like my brother-in-law Phil. But unlike Phil, this plan just might work. _______________________________________________ OpenSCAD mailing list [hidden email] http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org _______________________________________________ OpenSCAD mailing list [hidden email] http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

## Re: CPU and RAM Resources Allocations Increase for OpenSCAD

 In reply to this post by Still_learning On 20/05/18 16:58, Still_learning wrote: ```My laptop computer is currently crunching numbers for an OpenSCAD STL file which can be described as including a number of spheres, with an \$fn=250 setting.``` Ouch, I never go above \$fn=120 and even that can be very slow. I also don't use \$fn but instead use \$fs and \$fa. My logic is complex because I do the high quality from make files, but it boils down to - fn=60;     // or 120 for HQ \$fa=360/fn; fs=0.5;    // or 0.1 for HQ _______________________________________________ OpenSCAD mailing list [hidden email] http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

## Re: CPU and RAM Resources Allocations Increase for OpenSCAD

 In reply to this post by rew On 21. mai 2018 11:56, Rogier Wolff wrote: > Now from a theoretical point-of-view, the algorithm will > probably still be O(N^2) But from a practical point-of-view, in say > the case where you are unioning a bunch of near-perfect spheres, you > will be able to handle 90 or 99% of the items in O(N) time. This > improves the "feasable dataset size" by a big margin. > > In practise, I work at low \$fn, until for the final render I increase > it. There was a discussion about "unioning a bunch of near-perfect spheres" last year http://forum.openscad.org/Performance-test-manyballs-td21455.htmlIn comparison with OpenSCAD 2018.04.19.nightly on Linux Kubuntu with 4 CPUs and 32GB ram and referring to a case of only 7x7x7 intersecting spheres, booleans with https://github.com/arnholm/xcsg are now ~69x faster, as OpenSCAD only uses one CPU and xcsg using all 4 CPUs plus generating geodesic spheres (fewer triangles). Defining 16x16x16 intersecting spheres https://gist.github.com/arnholm/23739f7e0fb48f477aaf1b6bbeee968fand executing with xcsg I find booleans to now require about 130 seconds, then add about 60 seconds for fixing non-triangular faces afterwards. Maximum required memory is about 4GB, generating 1448960 triangles, ref. image in gist link. Carsten Arnholm _______________________________________________ OpenSCAD mailing list [hidden email] http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

## Re: CPU and RAM Resources Allocations Increase for OpenSCAD

 In reply to this post by Gadgetmind Thank you for the pointers. I have been running some tests and should be posting results later today to the original thread. On May 21, 2018 4:28 AM, "Gadgetmind" <[hidden email]> wrote: On 20/05/18 16:58, Still_learning wrote: ```My laptop computer is currently crunching numbers for an OpenSCAD STL file which can be described as including a number of spheres, with an \$fn=250 setting.``` Ouch, I never go above \$fn=120 and even that can be very slow. I also don't use \$fn but instead use \$fs and \$fa. My logic is complex because I do the high quality from make files, but it boils down to - fn=60;     // or 120 for HQ \$fa=360/fn; fs=0.5;    // or 0.1 for HQ _______________________________________________ OpenSCAD mailing list [hidden email] http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org The more I learn, the more I get curious about.
Reply | Threaded
Open this post in threaded view
|

## Re: CPU and RAM Resources Allocations Increase for OpenSCAD

 In reply to this post by Still_learning Thanks to all for the insights, comments and suggestions. As the initial file crunching was a test to gain some idea of the time involved for crunching the subsequent versions (which are even bigger and more complex), I killed the 2.5 day computation process to check my code and test some options. I looked over the coding and came across some possible bugaboos, and fixed them.  I will explain the errors and solution after further testing of alternatives. I ran additional tests on the same file with different \$fn= settings and here are the results: When  \$fn= was set to:             Crunch Time Was      25                                                6 minutes      50                                              25 minutes     100                                   1 hour 39 minutes     200                                   > 17 hours when processing stopped ----- The more I learn, the more I get curious about. -- Sent from: http://forum.openscad.org/_______________________________________________ OpenSCAD mailing list [hidden email] http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org The more I learn, the more I get curious about.
Reply | Threaded
Open this post in threaded view
|

## Re: CPU and RAM Resources Allocations Increase for OpenSCAD

 In reply to this post by Still_learning Other test was to determine if "nested" \$fn= commands had any impact on imported modules. Example: File_a.scad code contains general instruction of \$fn=10, and calls to module_b.scad (which contains an instruction of \$fn=20) Is there a compunding effect? No, not as far as I can tell. If the values above are reversed, the lower value is used and displayed when rendered.  This means that imported files are treated as completely separate entities.  Good to know. ----- The more I learn, the more I get curious about. -- Sent from: http://forum.openscad.org/_______________________________________________ OpenSCAD mailing list [hidden email] http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org The more I learn, the more I get curious about.
Reply | Threaded
Open this post in threaded view
|

## Re: CPU and RAM Resources Allocations Increase for OpenSCAD

 In reply to this post by jon_bondy On Sun, 20 May 2018 20:29:58 -0400 jon <[hidden email]> wrote: > There was a thread a while back about how performance was degraded > because one of the RAM caches was overflowing.  Getting dirt cheap RAM > is not always as good a deal as you might imagine I have no idea what you are talking about here. Caches are part of the processor not the RAM. And in fact generally speaking Xeons have bigger caches than client parts. The most important thing is to have enough RAM. People often get the idea that swap is a little bit slower than memory. The old analogy of swap being like your filing cabinet is (in a modern system) better replaced by swap being sending documents to and from a warehouse on the far side of town in a truck, and probably in the middle of rush hour. The scale of memory versus swap performance is *enormous* Alan _______________________________________________ OpenSCAD mailing list [hidden email] http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

## Re: CPU and RAM Resources Allocations Increase for OpenSCAD

 On 05/22/2018 03:10 PM, Alan Cox wrote: > On Sun, 20 May 2018 20:29:58 -0400 > jon <[hidden email]> wrote: > >> There was a thread a while back about how performance was degraded >> because one of the RAM caches was overflowing.  Getting dirt cheap RAM >> is not always as good a deal as you might imagine > > I have no idea what you are talking about here. > I think that was discussing the cache setting in OpenSCAD for already rendered sub-trees. If that value is set too low, this can cause performance impact even with more RAM available as identical sub-trees are calculated multiple times. ciao,    Torsten. _______________________________________________ OpenSCAD mailing list [hidden email] http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org -- Torsten
Reply | Threaded
Open this post in threaded view
|

## Re: CPU and RAM Resources Allocations Increase for OpenSCAD

 This is what I was recalling... http://forum.openscad.org/OpenSCAD-performance-mystery-td23248.htmlJon On 5/22/2018 2:47 PM, Torsten Paul wrote: > On 05/22/2018 03:10 PM, Alan Cox wrote: >> On Sun, 20 May 2018 20:29:58 -0400 >> jon <[hidden email]> wrote: >> >>> There was a thread a while back about how performance was degraded >>> because one of the RAM caches was overflowing.  Getting dirt cheap RAM >>> is not always as good a deal as you might imagine >> >> I have no idea what you are talking about here. >> > I think that was discussing the cache setting in OpenSCAD for > already rendered sub-trees. If that value is set too low, this > can cause performance impact even with more RAM available as > identical sub-trees are calculated multiple times. > > ciao, >   Torsten. > > _______________________________________________ > OpenSCAD mailing list > [hidden email] > http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org_______________________________________________ OpenSCAD mailing list [hidden email] http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

## Re: CPU and RAM Resources Allocations Increase for OpenSCAD

 I meant this discussion:    http://forum.openscad.org/Geometry-cache-thrashing-td23978.htmlwhich is relevant too, but as mentioned before "just" a configuration setting in OpenSCAD. Anyone feeling to collect all that information into something that can be added to the FAQ? ciao,    Torsten. _______________________________________________ OpenSCAD mailing list [hidden email] http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org -- Torsten
Reply | Threaded
Open this post in threaded view
|

## Re: CPU and RAM Resources Allocations Increase for OpenSCAD

 In reply to this post by nophead After checking my Preference settings, I find that both CGAL Cache and PolySet Cache sizes are set to 1000000000 (one billion) bytes, and that number does not accept alteration that increases the size. Is there some reason for that? ----- The more I learn, the more I get curious about. -- Sent from: http://forum.openscad.org/_______________________________________________ OpenSCAD mailing list [hidden email] http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org The more I learn, the more I get curious about.
Reply | Threaded
Open this post in threaded view
|

## Re: CPU and RAM Resources Allocations Increase for OpenSCAD

 Administrator My settings are changeable but less than yours, ~800M & ~400M. However I could only change it to 2.1G, I suspect that is unintended, something like a variable type issue. ----- Admin - PM me if you need anything, or if I've done something stupid... Unless specifically shown otherwise above, my contribution is in the Public Domain; to the extent possible under law, I have waived all copyright and related or neighbouring rights to this work. Obviously inclusion of works of previous authors is not included in the above. The TPP is no simple “trade agreement.”   Fight it! http://www.ourfairdeal.org/   time is running out! -- Sent from: http://forum.openscad.org/_______________________________________________ OpenSCAD mailing list [hidden email] http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org Admin - email* me if you need anything, or if I've done something stupid... * click on my MichaelAtOz label, there is a link to email me. Unless specifically shown otherwise above, my contribution is in the Public Domain; to the extent possible under law, I have waived all copyright and related or neighbouring rights to this work. Obviously inclusion of works of previous authors is not included in the above.
Reply | Threaded
Open this post in threaded view
|

## Re: CPU and RAM Resources Allocations Increase for OpenSCAD

 Administrator Look like it is a signed int limit 2,147,483,647 (it actually accepts ...648, but I'd advise against that). I'll raise an issue on this. ----- Admin - PM me if you need anything, or if I've done something stupid... Unless specifically shown otherwise above, my contribution is in the Public Domain; to the extent possible under law, I have waived all copyright and related or neighbouring rights to this work. Obviously inclusion of works of previous authors is not included in the above. The TPP is no simple “trade agreement.”   Fight it! http://www.ourfairdeal.org/   time is running out! -- Sent from: http://forum.openscad.org/_______________________________________________ OpenSCAD mailing list [hidden email] http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org Admin - email* me if you need anything, or if I've done something stupid... * click on my MichaelAtOz label, there is a link to email me. Unless specifically shown otherwise above, my contribution is in the Public Domain; to the extent possible under law, I have waived all copyright and related or neighbouring rights to this work. Obviously inclusion of works of previous authors is not included in the above.
Reply | Threaded
Open this post in threaded view
|

## Re: CPU and RAM Resources Allocations Increase for OpenSCAD

 Administrator Issue   raised. This could be a hard limit (or 4GB maybe) for 32 bit systems, and may need fanagling to get more on 64 bit. ----- Admin - PM me if you need anything, or if I've done something stupid... Unless specifically shown otherwise above, my contribution is in the Public Domain; to the extent possible under law, I have waived all copyright and related or neighbouring rights to this work. Obviously inclusion of works of previous authors is not included in the above. The TPP is no simple “trade agreement.”   Fight it! http://www.ourfairdeal.org/   time is running out! -- Sent from: http://forum.openscad.org/_______________________________________________ OpenSCAD mailing list [hidden email] http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org Admin - email* me if you need anything, or if I've done something stupid... * click on my MichaelAtOz label, there is a link to email me. Unless specifically shown otherwise above, my contribution is in the Public Domain; to the extent possible under law, I have waived all copyright and related or neighbouring rights to this work. Obviously inclusion of works of previous authors is not included in the above.
Reply | Threaded
Open this post in threaded view
|

## Re: CPU and RAM Resources Allocations Increase for OpenSCAD

 Administrator In reply to this post by Still_learning Also check the output of render. Rendering Polygon Mesh using CGAL... Geometries in cache: 862 Geometry cache size in bytes: 1612848 CGAL Polyhedrons in cache: 14 CGAL cache size in bytes: 10765200 Total rendering time: 0 hours, 0 minutes, 36 seconds Note Geometry cache=Polyset cache. That shows the actual cache used. If the use for expensive models is below the limit, there is no point increasing it. ----- Admin - PM me if you need anything, or if I've done something stupid... Unless specifically shown otherwise above, my contribution is in the Public Domain; to the extent possible under law, I have waived all copyright and related or neighbouring rights to this work. Obviously inclusion of works of previous authors is not included in the above. The TPP is no simple “trade agreement.”   Fight it! http://www.ourfairdeal.org/   time is running out! -- Sent from: http://forum.openscad.org/_______________________________________________ OpenSCAD mailing list [hidden email] http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org Admin - email* me if you need anything, or if I've done something stupid... * click on my MichaelAtOz label, there is a link to email me. Unless specifically shown otherwise above, my contribution is in the Public Domain; to the extent possible under law, I have waived all copyright and related or neighbouring rights to this work. Obviously inclusion of works of previous authors is not included in the above.
Reply | Threaded
Open this post in threaded view
|

## Re: CPU and RAM Resources Allocations Increase for OpenSCAD

 As a concluding note on this series of posts, I also increased the Advanced Preference setting for  "Turn off rendering"  from 200,000 elements to 20,000,000 elements. Following the collective recommendations to reduce \$fn= settings, I lowered them to 150 on spheres and 120 on cylinders, then started the render crunching. Result was 17 hrs 39 minutes to conclusion. Did not note balance of computational results. Are they logged anyplace? ----- The more I learn, the more I get curious about. -- Sent from: http://forum.openscad.org/_______________________________________________ OpenSCAD mailing list [hidden email] http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org The more I learn, the more I get curious about.
Reply | Threaded
Open this post in threaded view
|

## Re: CPU and RAM Resources Allocations Increase for OpenSCAD

 Administrator Still_learning wrote > Are they logged anyplace? Nope, may be something to consider tho. ----- Admin - PM me if you need anything, or if I've done something stupid... Unless specifically shown otherwise above, my contribution is in the Public Domain; to the extent possible under law, I have waived all copyright and related or neighbouring rights to this work. Obviously inclusion of works of previous authors is not included in the above. The TPP is no simple “trade agreement.”   Fight it! http://www.ourfairdeal.org/   time is running out! -- Sent from: http://forum.openscad.org/_______________________________________________ OpenSCAD mailing list [hidden email] http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org Admin - email* me if you need anything, or if I've done something stupid... * click on my MichaelAtOz label, there is a link to email me. Unless specifically shown otherwise above, my contribution is in the Public Domain; to the extent possible under law, I have waived all copyright and related or neighbouring rights to this work. Obviously inclusion of works of previous authors is not included in the above.
12