CPU and RAM Resources Allocations Increase for OpenSCAD

classic Classic list List threaded Threaded
37 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: CPU and RAM Resources Allocations Increase for OpenSCAD

nophead
Yes I know it can easily be optimised but OpenSCAD uses the CGAL library and that doesn't seem to optimise for speed. Also it does all the maths in rational notation, so simple operations take at least three orders of magnitude longer than I would expect. Coupled with the at least O(N) explosion and it gets impractically slow with large numbers of 3D vertices.

On 21 May 2018 at 10:56, Rogier Wolff <[hidden email]> wrote:
On Mon, May 21, 2018 at 10:16:37AM +0100, nop head wrote:
> I think union time is proportional to the square of the number of facets of
> the two items being unioned. With a "number" of spheres with 61738 facets I
> don't think it will finish in a life time.
>
> The way to estimatet how long it will take is to run it with $fn set to
> small numbers and look at how the time increases with $fn. Then you can
> predict how long it will be for $fn=250. It will be a gross underestimation
> because it doesn't take into account swapping, etc. But I am sure it will
> show you it is totally impractical to work with such values of $fn in

Weeeelllll....

This means to me that there is work to be done. Is unioning inherently
quadratic?

Well,  yes it is: You have to check for each pair of triangles if they
intersect. Right?

Wrong! For example, you can split up the working space is small
cubes. Go through the list of triangles once, and those that are small
and land in 1 or two of those cubes, you place in a "bin" for that
cube. The big trangles and those that are unlucky go into a bin for
"other". Now from a theoretical point-of-view, the algorithm will
probably still be O(N^2) But from a practical point-of-view, in say
the case where you are unioning a bunch of near-perfect spheres, you
will be able to handle 90 or 99% of the items in O(N) time. This
improves the "feasable dataset size" by a big margin.

In practise, I work at low $fn, until for the final render I increase
it.

        Roger.

--
** [hidden email] ** http://www.BitWizard.nl/ ** +31-15-2600998 **
**    Delftechpark 26 2628 XH  Delft, The Netherlands. KVK: 27239233    **
*-- BitWizard writes Linux device drivers for any device you may have! --*
The plan was simple, like my brother-in-law Phil. But unlike
Phil, this plan just might work.

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org


_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: CPU and RAM Resources Allocations Increase for OpenSCAD

Gadgetmind
In reply to this post by Still_learning
On 20/05/18 16:58, Still_learning wrote:
My laptop computer is currently crunching numbers for an OpenSCAD STL file
which can be described as including a number of spheres, with an $fn=250
setting.

Ouch, I never go above $fn=120 and even that can be very slow.

I also don't use $fn but instead use $fs and $fa.

My logic is complex because I do the high quality from make files, but it boils down to -

fn=60;     // or 120 for HQ

$fa=360/fn;

fs=0.5;    // or 0.1 for HQ



_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: CPU and RAM Resources Allocations Increase for OpenSCAD

cacb
In reply to this post by rew
On 21. mai 2018 11:56, Rogier Wolff wrote:
> Now from a theoretical point-of-view, the algorithm will
> probably still be O(N^2) But from a practical point-of-view, in say
> the case where you are unioning a bunch of near-perfect spheres, you
> will be able to handle 90 or 99% of the items in O(N) time. This
> improves the "feasable dataset size" by a big margin.
>
> In practise, I work at low $fn, until for the final render I increase
> it.

There was a discussion about "unioning a bunch of near-perfect spheres"
last year
http://forum.openscad.org/Performance-test-manyballs-td21455.html

In comparison with OpenSCAD 2018.04.19.nightly on Linux Kubuntu with 4
CPUs and 32GB ram and referring to a case of only 7x7x7 intersecting
spheres, booleans with https://github.com/arnholm/xcsg are now ~69x
faster, as OpenSCAD only uses one CPU and xcsg using all 4 CPUs plus
generating geodesic spheres (fewer triangles).

Defining 16x16x16 intersecting spheres
https://gist.github.com/arnholm/23739f7e0fb48f477aaf1b6bbeee968f
and executing with xcsg I find booleans to now require about 130
seconds, then add about 60 seconds for fixing non-triangular faces
afterwards. Maximum required memory is about 4GB, generating 1448960
triangles, ref. image in gist link.

Carsten Arnholm

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: CPU and RAM Resources Allocations Increase for OpenSCAD

Still_learning
In reply to this post by Gadgetmind

Thank you for the pointers.

I have been running some tests and should be posting results later today to the original thread.

On May 21, 2018 4:28 AM, "Gadgetmind" <[hidden email]> wrote:
On 20/05/18 16:58, Still_learning wrote:
My laptop computer is currently crunching numbers for an OpenSCAD STL file
which can be described as including a number of spheres, with an $fn=250
setting.

Ouch, I never go above $fn=120 and even that can be very slow.

I also don't use $fn but instead use $fs and $fa.

My logic is complex because I do the high quality from make files, but it boils down to -

fn=60;     // or 120 for HQ

$fa=360/fn;

fs=0.5;    // or 0.1 for HQ



_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
The more I learn, the more I get curious about.
Reply | Threaded
Open this post in threaded view
|

Re: CPU and RAM Resources Allocations Increase for OpenSCAD

Still_learning
In reply to this post by Still_learning
Thanks to all for the insights, comments and suggestions.

As the initial file crunching was a test to gain some idea of
the time involved for crunching the subsequent versions
(which are even bigger and more complex), I killed the 2.5
day computation process to check my code and test
some options.

I looked over the coding and came across some possible
bugaboos, and fixed them.  I will explain the errors and
solution after further testing of alternatives.

I ran additional tests on the same file with different $fn=
settings and here are the results:

When  $fn= was set to:             Crunch Time Was

     25                                                6 minutes
     50                                              25 minutes
    100                                   1 hour 39 minutes
    200                                   > 17 hours when processing stopped





-----
The more I learn, the more I get curious about.
--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
The more I learn, the more I get curious about.
Reply | Threaded
Open this post in threaded view
|

Re: CPU and RAM Resources Allocations Increase for OpenSCAD

Still_learning
In reply to this post by Still_learning
Other test was to determine if "nested" $fn= commands had any impact
on imported modules.

Example:

File_a.scad code contains general instruction of $fn=10, and calls to
module_b.scad (which contains an instruction of $fn=20)

Is there a compunding effect?

No, not as far as I can tell.

If the values above are reversed, the lower value is used and
displayed when rendered.  This means that imported files
are treated as completely separate entities.  Good to know.



-----
The more I learn, the more I get curious about.
--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
The more I learn, the more I get curious about.
Reply | Threaded
Open this post in threaded view
|

Re: CPU and RAM Resources Allocations Increase for OpenSCAD

Alan Cox-2
In reply to this post by jon_bondy
On Sun, 20 May 2018 20:29:58 -0400
jon <[hidden email]> wrote:

> There was a thread a while back about how performance was degraded
> because one of the RAM caches was overflowing.  Getting dirt cheap RAM
> is not always as good a deal as you might imagine

I have no idea what you are talking about here.

Caches are part of the processor not the RAM. And in fact generally
speaking Xeons have bigger caches than client parts.

The most important thing is to have enough RAM. People often get the idea
that swap is a little bit slower than memory. The old analogy of swap
being like your filing cabinet is (in a modern system) better replaced by
swap being sending documents to and from a warehouse on the far side of
town in a truck, and probably in the middle of rush hour.

The scale of memory versus swap performance is *enormous*

Alan

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
tp3
Reply | Threaded
Open this post in threaded view
|

Re: CPU and RAM Resources Allocations Increase for OpenSCAD

tp3
On 05/22/2018 03:10 PM, Alan Cox wrote:
> On Sun, 20 May 2018 20:29:58 -0400
> jon <[hidden email]> wrote:
>
>> There was a thread a while back about how performance was degraded
>> because one of the RAM caches was overflowing.  Getting dirt cheap RAM
>> is not always as good a deal as you might imagine
>
> I have no idea what you are talking about here.
>
I think that was discussing the cache setting in OpenSCAD for
already rendered sub-trees. If that value is set too low, this
can cause performance impact even with more RAM available as
identical sub-trees are calculated multiple times.

ciao,
   Torsten.

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
-- Torsten
Reply | Threaded
Open this post in threaded view
|

Re: CPU and RAM Resources Allocations Increase for OpenSCAD

jon_bondy
This is what I was recalling...

http://forum.openscad.org/OpenSCAD-performance-mystery-td23248.html

Jon


On 5/22/2018 2:47 PM, Torsten Paul wrote:

> On 05/22/2018 03:10 PM, Alan Cox wrote:
>> On Sun, 20 May 2018 20:29:58 -0400
>> jon <[hidden email]> wrote:
>>
>>> There was a thread a while back about how performance was degraded
>>> because one of the RAM caches was overflowing.  Getting dirt cheap RAM
>>> is not always as good a deal as you might imagine
>>
>> I have no idea what you are talking about here.
>>
> I think that was discussing the cache setting in OpenSCAD for
> already rendered sub-trees. If that value is set too low, this
> can cause performance impact even with more RAM available as
> identical sub-trees are calculated multiple times.
>
> ciao,
>   Torsten.
>
> _______________________________________________
> OpenSCAD mailing list
> [hidden email]
> http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org


_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
tp3
Reply | Threaded
Open this post in threaded view
|

Re: CPU and RAM Resources Allocations Increase for OpenSCAD

tp3
I meant this discussion:

   http://forum.openscad.org/Geometry-cache-thrashing-td23978.html

which is relevant too, but as mentioned before "just" a
configuration setting in OpenSCAD.

Anyone feeling to collect all that information into something
that can be added to the FAQ?

ciao,
   Torsten.

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
-- Torsten
Reply | Threaded
Open this post in threaded view
|

Re: CPU and RAM Resources Allocations Increase for OpenSCAD

Still_learning
In reply to this post by nophead
After checking my Preference settings, I find that both CGAL Cache and
PolySet Cache sizes are set to 1000000000 (one billion) bytes, and that
number does not accept alteration that increases the size.

Is there some reason for that?



-----
The more I learn, the more I get curious about.
--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
The more I learn, the more I get curious about.
Reply | Threaded
Open this post in threaded view
|

Re: CPU and RAM Resources Allocations Increase for OpenSCAD

MichaelAtOz
Administrator
My settings are changeable but less than yours, ~800M & ~400M.
However I could only change it to 2.1G, I suspect that is unintended,
something like a variable type issue.



-----
Admin - PM me if you need anything, or if I've done something stupid...

Unless specifically shown otherwise above, my contribution is in the Public Domain; to the extent possible under law, I have waived all copyright and related or neighbouring rights to this work. Obviously inclusion of works of previous authors is not included in the above.

The TPP is no simple “trade agreement.”   Fight it! http://www.ourfairdeal.org/   time is running out!
--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Admin - PM me if you need anything,
or if I've done something stupid...

Unless specifically shown otherwise above, my contribution is in the Public Domain; to the extent possible under law, I have waived all copyright and related or neighbouring rights to this work.
Obviously inclusion of works of previous authors is not included in the above.


The TPP is no simple “trade agreement.” Fight it! http://www.ourfairdeal.org/ time is running out!
Reply | Threaded
Open this post in threaded view
|

Re: CPU and RAM Resources Allocations Increase for OpenSCAD

MichaelAtOz
Administrator
Look like it is a signed int limit 2,147,483,647 (it actually accepts ...648,
but I'd advise against that).
I'll raise an issue on this.



-----
Admin - PM me if you need anything, or if I've done something stupid...

Unless specifically shown otherwise above, my contribution is in the Public Domain; to the extent possible under law, I have waived all copyright and related or neighbouring rights to this work. Obviously inclusion of works of previous authors is not included in the above.

The TPP is no simple “trade agreement.”   Fight it! http://www.ourfairdeal.org/   time is running out!
--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Admin - PM me if you need anything,
or if I've done something stupid...

Unless specifically shown otherwise above, my contribution is in the Public Domain; to the extent possible under law, I have waived all copyright and related or neighbouring rights to this work.
Obviously inclusion of works of previous authors is not included in the above.


The TPP is no simple “trade agreement.” Fight it! http://www.ourfairdeal.org/ time is running out!
Reply | Threaded
Open this post in threaded view
|

Re: CPU and RAM Resources Allocations Increase for OpenSCAD

MichaelAtOz
Administrator
Issue <https://github.com/openscad/openscad/issues/2401>   raised.
This could be a hard limit (or 4GB maybe) for 32 bit systems, and may need
fanagling to get more on 64 bit.



-----
Admin - PM me if you need anything, or if I've done something stupid...

Unless specifically shown otherwise above, my contribution is in the Public Domain; to the extent possible under law, I have waived all copyright and related or neighbouring rights to this work. Obviously inclusion of works of previous authors is not included in the above.

The TPP is no simple “trade agreement.”   Fight it! http://www.ourfairdeal.org/   time is running out!
--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Admin - PM me if you need anything,
or if I've done something stupid...

Unless specifically shown otherwise above, my contribution is in the Public Domain; to the extent possible under law, I have waived all copyright and related or neighbouring rights to this work.
Obviously inclusion of works of previous authors is not included in the above.


The TPP is no simple “trade agreement.” Fight it! http://www.ourfairdeal.org/ time is running out!
Reply | Threaded
Open this post in threaded view
|

Re: CPU and RAM Resources Allocations Increase for OpenSCAD

MichaelAtOz
Administrator
In reply to this post by Still_learning
Also check the output of render.

Rendering Polygon Mesh using CGAL...
Geometries in cache: 862
Geometry cache size in bytes: 1612848
CGAL Polyhedrons in cache: 14
CGAL cache size in bytes: 10765200
Total rendering time: 0 hours, 0 minutes, 36 seconds

Note Geometry cache=Polyset cache.

That shows the actual cache used. If the use for expensive models is below
the limit, there is no point increasing it.



-----
Admin - PM me if you need anything, or if I've done something stupid...

Unless specifically shown otherwise above, my contribution is in the Public Domain; to the extent possible under law, I have waived all copyright and related or neighbouring rights to this work. Obviously inclusion of works of previous authors is not included in the above.

The TPP is no simple “trade agreement.”   Fight it! http://www.ourfairdeal.org/   time is running out!
--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Admin - PM me if you need anything,
or if I've done something stupid...

Unless specifically shown otherwise above, my contribution is in the Public Domain; to the extent possible under law, I have waived all copyright and related or neighbouring rights to this work.
Obviously inclusion of works of previous authors is not included in the above.


The TPP is no simple “trade agreement.” Fight it! http://www.ourfairdeal.org/ time is running out!
Reply | Threaded
Open this post in threaded view
|

Re: CPU and RAM Resources Allocations Increase for OpenSCAD

Still_learning
As a concluding note on this series of posts, I also increased the Advanced
Preference setting for  "Turn off rendering"  from 200,000 elements to
20,000,000 elements.

Following the collective recommendations to reduce $fn= settings, I lowered
them to 150 on spheres and 120 on cylinders, then started the render
crunching.

Result was 17 hrs 39 minutes to conclusion.

Did not note balance of computational results.

Are they logged anyplace?





-----
The more I learn, the more I get curious about.
--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
The more I learn, the more I get curious about.
Reply | Threaded
Open this post in threaded view
|

Re: CPU and RAM Resources Allocations Increase for OpenSCAD

MichaelAtOz
Administrator
Still_learning wrote
> Are they logged anyplace?

Nope, may be something to consider tho.



-----
Admin - PM me if you need anything, or if I've done something stupid...

Unless specifically shown otherwise above, my contribution is in the Public Domain; to the extent possible under law, I have waived all copyright and related or neighbouring rights to this work. Obviously inclusion of works of previous authors is not included in the above.

The TPP is no simple “trade agreement.”   Fight it! http://www.ourfairdeal.org/   time is running out!
--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Admin - PM me if you need anything,
or if I've done something stupid...

Unless specifically shown otherwise above, my contribution is in the Public Domain; to the extent possible under law, I have waived all copyright and related or neighbouring rights to this work.
Obviously inclusion of works of previous authors is not included in the above.


The TPP is no simple “trade agreement.” Fight it! http://www.ourfairdeal.org/ time is running out!
12