Quantcast

Can you give me any rendering advice...?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Can you give me any rendering advice...?

Dan Zuras 3D

        Folks,

        I am trying to design an object using OpenSCAD.  It
        is not all that complicated but it does involve the
        union, intersection, & difference of spheres &
        cylinders.  My method of debugging it is to add an
        object or group of objects, render the result to see
        if I have everything right, & make adjustments based
        on what I see before I add the next object.

        The fly in the ointment is that CGAL increases the
        rendering time by a lot for each object I add.  And
        I am no where near the number of objects I need.

        Is there some lesser rendering mode I could use that
        takes substantially less time?  A wire mesh, perhaps?
        Anything that would allow me to finish the design.

        Thanks,

                            Dan Zuras

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Can you give me any rendering advice...?

Giles Bathgate-2
On Fri, 2010-12-10 at 03:56 -0800, Dan Zuras 3D wrote:

> Folks,
>
> I am trying to design an object using OpenSCAD.  It
> is not all that complicated but it does involve the
> union, intersection, & difference of spheres &
> cylinders.  My method of debugging it is to add an
> object or group of objects, render the result to see
> if I have everything right, & make adjustments based
> on what I see before I add the next object.
>
> The fly in the ointment is that CGAL increases the
> rendering time by a lot for each object I add.  And
> I am no where near the number of objects I need.
>
> Is there some lesser rendering mode I could use that
> takes substantially less time?  A wire mesh, perhaps?
> Anything that would allow me to finish the design.
>
> Thanks,

Have you tried
a) rendering using openCSG (f5)
b) reducing the number of facets $fn=10; at the top of your script

Regards

Giles.




Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Can you give me any rendering advice...?

kintel
Administrator
In reply to this post by Dan Zuras 3D
On Dec 10, 2010, at 12:56 PM, Dan Zuras 3D wrote:

> The fly in the ointment is that CGAL increases the
> rendering time by a lot for each object I add.  And
> I am no where near the number of objects I need.
>

I took a look at this and the reason why the F5 preview has issues is that the intersection operation causes the number of evaluated objects for OpenCSG to increase dramatically. We have implemented a safety measure where we don't render this preview if the number of objects after CSG normalization exceeds 1000. This could be made configurable in the GUI, but it would quickly become extremely slow to render the resulting CSG trees.
I'm not 100% familiar with the inner workings of the OpenCSG library, so it hard to say if it's possible to optimize this.

Also, the CGAL library used for geometric evaluation (F6) is extremely slow. This is a long-standing issue which I'm hoping to be able to address in the future, although it's not too trivial. I'm painfully aware of that this significantly limits the usefulness of OpenSCAD for large designs, or even certain types of smaller designs. In the next version of OpenSCAD, CGAL will be isolated into components which could be rewritten to other libraries, which would open for easier experimentation in case someone feels like picking up this issue.

 -Marius


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Can you give me any rendering advice...?

Dan Zuras 3D
> Subject: Re: [OpenSCAD] Can you give me any rendering advice...?
> From: Marius Kintel <[hidden email]>
> Date: Sat, 25 Dec 2010 22:57:09 +0100
> To: Dan Zuras 3D <[hidden email]>,
>  [hidden email]
>
> On Dec 10, 2010, at 12:56 PM, Dan Zuras 3D wrote:
>
> > The fly in the ointment is that CGAL increases the
> > rendering time by a lot for each object I add.  And
> > I am no where near the number of objects I need.
> >
>
> I took a look at this and the reason why the F5 preview has issues is =
> that the intersection operation causes the number of evaluated objects =
> for OpenCSG to increase dramatically. We have implemented a safety =

        Thank you for looking into the problem.
        How dramatically?

> measure where we don't render this preview if the number of objects =
> after CSG normalization exceeds 1000. This could be made configurable in =

        Will this mitigate the problem or merely
        cause OpenSCAD to crap out when it happens?

> the GUI, but it would quickly become extremely slow to render the =
> resulting CSG trees.
> I'm not 100% familiar with the inner workings of the OpenCSG library, so =
> it hard to say if it's possible to optimize this.
>
> Also, the CGAL library used for geometric evaluation (F6) is extremely =
> slow. This is a long-standing issue which I'm hoping to be able to =

        I believe I (& others) have run into variations
        of this issue before.  While I also have no idea
        how it happens I am well aware that it does.

> address in the future, although it's not too trivial. I'm painfully =
> aware of that this significantly limits the usefulness of OpenSCAD for =
> large designs, or even certain types of smaller designs. In the next =
> version of OpenSCAD, CGAL will be isolated into components which could =
> be rewritten to other libraries, which would open for easier =
> experimentation in case someone feels like picking up this issue.
>
>  -Marius

        This last approach is the only path to a real cure,
        IMHO.

        The fact that OpenCSG or CGAL uses aribtrary precision
        software floating-point rather than 64-bit hardware
        double precision is at the root of the problem.  The
        fact that their data structures grow to gobble up
        huge amounts of memory in a more than linear fashion
        suggests some algorithmic problem as well.

        Really, in an era of cheap computers with gigabytes
        of memory & gigaFLOPS of processing power available,
        one would expect to be able to render objects with
        millions of primitives in seconds.  Not be crippled
        in both time & space by the attempt to render (at
        all) a thing with mere thousands of primitives.

        I know now that the fault lies with CGAL or OpenCSG
        & not with anything you have written.  Still, the
        decision to use them to save programming time must
        be looked at again in the light of how much rendering
        time your users are spending.

        If, as you suggest, this can be done incrementally
        then the pain of doing it can be administered in
        small doses.

        And the promise is that OpenSCAD will become an
        extraordinary tool when you come out the other end.

        I look forward to that day.

        Yours,

                                Dan

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Can you give me any rendering advice...?

kintel
Administrator
On Dec 26, 2010, at 17:05 PM, Dan Zuras 3D wrote:
>
> Thank you for looking into the problem.
> How dramatically?
>
Very dramatically. Think everything freezes for minutes while rendering, without progress output.

>> measure where we don't render this preview if the number of objects =
>> after CSG normalization exceeds 1000. This could be made configurable in =
>
> Will this mitigate the problem or merely
> cause OpenSCAD to crap out when it happens?
>
It would cause a freeze while rendering. This freeze could be remedied by rendering off-screen while keeping a progress bar around.

>> Also, the CGAL library used for geometric evaluation (F6) is extremely =
>> slow. This is a long-standing issue which I'm hoping to be able to =
>
> I believe I (& others) have run into variations
> of this issue before.  While I also have no idea
> how it happens I am well aware that it does.
>
As you correctly pointed out, this happens because arbitrary precision floats are being used, which is just over the top computationally expensive.
This is combined with CGAL wanting to have a so-called 3D Nef polyhedron around for CSG operations to be available, which means we have to evaluate everything to a single resulting polyhedron, which ends up gobbling up even more CPU time as well as memory.
As you also pointed out, there might very well be memory leaks in there somewhere, or rather temporary accumulation of memory in some evaluation loop. Combined with the performance issues, these things are a bit hard to catch and haven't been a priority. Again, the latest refactoring job will make it possible to run a lot more of this functionality in isolation, which gives some hope.

CGAL is, in my experience, not stable when using normal or double precision floating point numbers, so we'd have to look elsewhere for CSG evaluation.
Unfortunately, in the world of geometric evaluation of CSG in 3D, this appears to be a common issue, and I'd appreciate pointers to and experience with other libraries which could help improving this. As an intermediate resort, I've been thinking about using or implementing a faster algorithm, and rather fall back to CGAL whenever that one fails. It would naturally have to fail in a predictable and detectable fashion.

 -Marius


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Can you give me any rendering advice...?

Dan Zuras 3D
> Subject: Re: [OpenSCAD] Can you give me any rendering advice...?
> From: Marius Kintel <[hidden email]>
> Date: Sun, 26 Dec 2010 18:08:17 +0100
> Cc: [hidden email]
> To: Dan Zuras 3D <[hidden email]>
>
> On Dec 26, 2010, at 17:05 PM, Dan Zuras 3D wrote:
> >
> > Thank you for looking into the problem.
> > How dramatically?
> >
> Very dramatically. Think everything freezes for minutes while rendering, =
> without progress output.

        This is consistent with my observations
        of the problem.  Perhaps one can profile
        the CGAL behavior & see where it spends
        an inordinate amount of time when this
        crops up.  Just a suggestion.

>
> >> measure where we don't render this preview if the number of objects ==
>
> >> after CSG normalization exceeds 1000. This could be made configurable =
> in =
> >
> > Will this mitigate the problem or merely
> > cause OpenSCAD to crap out when it happens?
> >
> It would cause a freeze while rendering. This freeze could be remedied =
> by rendering off-screen while keeping a progress bar around.

        Doesn't sound like much of an improvement.
        Does it have some diagnostic value perhaps?

>
> >> Also, the CGAL library used for geometric evaluation (F6) is =
> extremely =
> >> slow. This is a long-standing issue which I'm hoping to be able to =
> >
> > I believe I (& others) have run into variations
> > of this issue before.  While I also have no idea
> > how it happens I am well aware that it does.
> >
> As you correctly pointed out, this happens because arbitrary precision =
> floats are being used, which is just over the top computationally =
> expensive.

        Also over the top memory expensive.

> This is combined with CGAL wanting to have a so-called 3D Nef polyhedron =
> around for CSG operations to be available, which means we have to =
> evaluate everything to a single resulting polyhedron, which ends up =
> gobbling up even more CPU time as well as memory.
> As you also pointed out, there might very well be memory leaks in there =
> somewhere, or rather temporary accumulation of memory in some evaluation =
> loop. Combined with the performance issues, these things are a bit hard =
> to catch and haven't been a priority. Again, the latest refactoring job =
> will make it possible to run a lot more of this functionality in =
> isolation, which gives some hope.

        I have a possible clue for this as well.

        I run on Ubuntu 10.04 on a 2 CPU system
        with 4GB of memory.  And I routinely run
        the system monitor when I render things
        to keep track of CPU & memory usage.

        When the problem crops up & it dies or
        I must kill the job & try something else,
        I notice that a significant amount of
        the leaked memory is left over.  That
        is, Ubuntu has lost track of it as well.

        I can recover it with a reboot but it
        suggests that at least part of the
        problem lies within the kernal & how it
        manages funny malloc & realloc calls.

        So, in this case at least, the problem
        may lie as much with the kernel as with
        the rendering code that caused it.

        BTW, I believe that the nature of the
        ill treated memory calls are pointers
        that are bad enough to cause writes on
        the page link data.  This would cause
        both CGAL & the kernel to drop those
        pages from their free lists.  Bad all
        around.

>
> CGAL is, in my experience, not stable when using normal or double =
> precision floating point numbers, so we'd have to look elsewhere for CSG =
> evaluation.

        I believe this instability may be due to
        some ill formed interpretation of the
        issue of when two points are the same
        point.  As we are rendering images in
        fields with merely thousands of pixels
        in them, I think a selectable epsilon
        that determines when 2 points are to be
        considered coincident would fix the
        problem.

        That is, one might have a (user changable)
        parameter such that if |x - y| < epsilon
        then x & y are to be considered as rendered
        to the same point.

        One need not change the values of x & y
        to some common value like (x + y)/2.  But
        just regard all points within that epsilon
        as being rendered to the same point.

        If one were to set the value of epsilon
        to something on the order of a pixel size
        many of the traditional rastering effects
        would start to happen.  But if one were to
        set it to something on the order of 1/100th
        or 1/1000the of a pixel, I suspect everything
        would be OK.

        And at ~1000 pixels on a screen & ~1/1000th
        of a pixel for the epsilon, something like
        ~20 bits are needed.  This is far exceeded
        by the 53 bits of precision that a double
        precision number carries.  Even in the most
        extraordinary chains of roundoff error, two
        originally identical points will still be
        within such an epsilon when rendered.  And
        two points distinct enough to see will be
        easily outside that epsilon.

        It has been years since I wrote any graphics
        code of my own & this approach may be naive
        by modern standards but it seems a simple
        enough cure to me.

        If, perhaps, greatly different from what is
        currently being done.

> Unfortunately, in the world of geometric evaluation of CSG in 3D, this =
> appears to be a common issue, and I'd appreciate pointers to and =
> experience with other libraries which could help improving this. As an =
> intermediate resort, I've been thinking about using or implementing a =
> faster algorithm, and rather fall back to CGAL whenever that one fails. =
> It would naturally have to fail in a predictable and detectable fashion.
>
>  -Marius

        An algorithm that makes far less use of its
        memory resources would automatically get
        faster even if somewhat more expensive
        computationally.

        It is a reasonable place to start.

        Yours,

                                Dan

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Can you give me any rendering advice...?

Tim Schmidt
On Sun, Dec 26, 2010 at 12:50 PM, Dan Zuras 3D <[hidden email]> wrote:

>        I have a possible clue for this as well.
>
>        I run on Ubuntu 10.04 on a 2 CPU system
>        with 4GB of memory.  And I routinely run
>        the system monitor when I render things
>        to keep track of CPU & memory usage.
>
>        When the problem crops up & it dies or
>        I must kill the job & try something else,
>        I notice that a significant amount of
>        the leaked memory is left over.  That
>        is, Ubuntu has lost track of it as well.
>
>        I can recover it with a reboot but it
>        suggests that at least part of the
>        problem lies within the kernal & how it
>        manages funny malloc & realloc calls.

This should _never_ happen.  When a process is killed, all memory it
has allocated should be free'd.  A claim that that is not happening is
a _serious_ one.  Quadruple-checking is in order.  Are you sure you're
looking at the memory usage +/- buffers/cache as reported by 'free'?

If you are, and the kernel still _appears_ to be losing track of ram,
a pass through valgrind might be in order.

--tim

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Can you give me any rendering advice...?

Dan Zuras 3D
> Date: Sun, 26 Dec 2010 13:33:01 -0500
> Subject: Re: [OpenSCAD] Can you give me any rendering advice...?
> From: Tim Schmidt <[hidden email]>
> To: Dan Zuras 3D <[hidden email]>, [hidden email]
>
> On Sun, Dec 26, 2010 at 12:50 PM, Dan Zuras 3D <[hidden email]> wr=
> ote:
> > I have a possible clue for this as well.
> >
> > . . .
> >
> > That is, Ubuntu has lost track of it as well.
> >
> > I can recover it with a reboot but it
> > suggests that at least part of the
> > problem lies within the kernal & how it
> > manages funny malloc & realloc calls.
>
> This should _never_ happen.  When a process is killed, all memory it
> has allocated should be free'd.

        I am aware of this.

> A claim that that is not happening is
> a _serious_ one.  Quadruple-checking is in order.

        I am aware of this as well.

> Are you sure you're
> looking at the memory usage +/- buffers/cache as reported by 'free'?

        No.  I am merely looking at the memory usage
        line as reported by the system monitor.  So
        I am no more sure of it than the monitor is.

>
> If you are, and the kernel still _appears_ to be losing track of ram,
> a pass through valgrind might be in order.
>
> --tim

        Alas, I have no idea what valgrind is &, in
        any case, am eminently unqualified to debug
        kernel problems.

        Nor have I reported these observations to the
        Ubuntu people.  I felt that, given that I am
        using a lot of software outside their control
        (OpenSCAD, et al, as well as my own sources)
        that it would be difficult for me to package
        up something that allowed them to reproduce
        the problem.

        I keep an eye out for it, though.  And I keep
        hoping to run across an easy to characterize &
        reproduce instance of the leak.  But I haven't
        spotted one yet.

        As I mentioned in my previous post, I suspect
        the problem lies with malloc'ed pages that
        have been so abused by bad pointers as to
        have their page link data overwritten.  But
        I have no idea where or how this was done.

        If you know more about the internal workings
        of this software, than you may be in a better
        position to characterize the problem than I.

        I hope so.

                            Dan

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Can you give me any rendering advice...?

Elmo Mäntynen
In reply to this post by kintel
What about BRLcad? Any idea if they have it much better? Having been in
development for 30 years would suggest the code to be well optimised at
least. Code sharing should be possible. At least asking for advise on
the best algorithms etc. could be beneficial.

I still like the language and simplicity of OpenSCAD better though :)

On 12/25/2010 11:57 PM, Marius Kintel wrote:

> On Dec 10, 2010, at 12:56 PM, Dan Zuras 3D wrote:
>
>> The fly in the ointment is that CGAL increases the
>> rendering time by a lot for each object I add.  And
>> I am no where near the number of objects I need.
>>
>
> I took a look at this and the reason why the F5 preview has issues is that the intersection operation causes the number of evaluated objects for OpenCSG to increase dramatically. We have implemented a safety measure where we don't render this preview if the number of objects after CSG normalization exceeds 1000. This could be made configurable in the GUI, but it would quickly become extremely slow to render the resulting CSG trees.
> I'm not 100% familiar with the inner workings of the OpenCSG library, so it hard to say if it's possible to optimize this.
>
> Also, the CGAL library used for geometric evaluation (F6) is extremely slow. This is a long-standing issue which I'm hoping to be able to address in the future, although it's not too trivial. I'm painfully aware of that this significantly limits the usefulness of OpenSCAD for large designs, or even certain types of smaller designs. In the next version of OpenSCAD, CGAL will be isolated into components which could be rewritten to other libraries, which would open for easier experimentation in case someone feels like picking up this issue.
>
>   -Marius
>
> _______________________________________________
> OpenSCAD mailing list
> [hidden email]
> http://rocklinux.net/mailman/listinfo/openscad

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Can you give me any rendering advice...?

kintel
Administrator
On Jan 12, 2011, at 12:07 PM, Elmo wrote:

> What about BRLcad? Any idea if they have it much better?

BRLCad was evaluated prior to writing OpenSCAD. It turned out that it wasn't possible to create an object from a mesh (import STL or extrude a polygon) which subsequently was used in a CSG operation. This was a show-stopper for us, so we abandoned the idea of using BRLCad.
This might of course change if someone points out that we're wrong, or something changes in BRLCad.

 -Marius


Loading...