command line and multiple mesh export

classic Classic list List threaded Threaded
5 messages Options
rmi
Reply | Threaded
Open this post in threaded view
|

command line and multiple mesh export

rmi
Hi,

(sorry for the long introduction, but I think it helps to understand my
point)

I'm creating a plugin for Cura, such that Cura can open .scad files and
selectively (controlled
by carefully crafted comments) export meshes from it. The focus is on
improving the workflow for
parts that make use of multi extrusion. They require multiple meshes, often
produced from the same
.scad file, and so I find myself juggling with * and ! to control what will
be rendered; a
tiresome and error prone process.

The plugin would ease this process as it allows to
- export multiple meshes
- track slicer settings per mesh
- groups several meshes into a single part
- track changes to the slicer profile

To select which mesh is exported, the plugin writes a small that contains a
single statement and
includes the original .scad file, eg.

!whatever you want to render() #! to make sure that this only this will be
rendered
include <file.scad>                   # use <file.scad> wouldn't set the
global variables

The downside of this approach; it requires several invocations of openscad
which makes it slow as
the CGAL cache isn't preserved. I had several ideas how to solve, or at
least speed it up, and I'm
willing to try to implement it.

(A) cache persistance, eg. give a filename and size limit on the command
line to save the cache
between invocations. This has been discussed a couple of times in the past
and the general consens
sounded like 'would be great'. Forcing the CGAL cache to allocate it's
memroy from a memory mapped
file region could be the way to go. Some care has to be taken on reload, the
cache should be
treated like hostile user input. Integrating it into the UI shouldn't be to
difficult, eg. by
allowing the UI to construct a cache file name based on the loaded file.  

(B) specify several source/destination file tuples on the command line and
have them all exported in
one go, sharing the CGAL cache. I've seen discussion about adding an export
statement to the language,
which would create all kind of problems down the road, up to allow arbitrary
files written when files
are downloaded from thingiverse. That's why I'm suggesting a repeatable
command line argument, which
wouldn't have this issue.

(C) 'export' a hash of the normalized object tree, such that the plugin can
decide if a previous output
can be reused or not, basically pushing the caching issue one layer up.

What's your opinion?



--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: command line and multiple mesh export

jdawgaz
There would have to be a way to determine if the cache saved is valid or not, for the given .scad file.

It would seem to me that one way would be to put a timestamp in the cache that matches the modification timestamp of the .scad file. If the timestamps differ then the saved cache would be invalid.



--
Extra Ham Operator: K7AZJ
Registered Linux User: 275424
Raspberry Pi and Openscad developer

The most exciting phrase to hear in science - the one that heralds new discoveries - is not "Eureka!" but "That's funny...".
- Isaac. Asimov


On Wed, Nov 15, 2017 at 1:15 AM, rmi <[hidden email]> wrote:
Hi,

(sorry for the long introduction, but I think it helps to understand my
point)

I'm creating a plugin for Cura, such that Cura can open .scad files and
selectively (controlled
by carefully crafted comments) export meshes from it. The focus is on
improving the workflow for
parts that make use of multi extrusion. They require multiple meshes, often
produced from the same
.scad file, and so I find myself juggling with * and ! to control what will
be rendered; a
tiresome and error prone process.

The plugin would ease this process as it allows to
- export multiple meshes
- track slicer settings per mesh
- groups several meshes into a single part
- track changes to the slicer profile

To select which mesh is exported, the plugin writes a small that contains a
single statement and
includes the original .scad file, eg.

!whatever you want to render() #! to make sure that this only this will be
rendered
include <file.scad>                   # use <file.scad> wouldn't set the
global variables

The downside of this approach; it requires several invocations of openscad
which makes it slow as
the CGAL cache isn't preserved. I had several ideas how to solve, or at
least speed it up, and I'm
willing to try to implement it.

(A) cache persistance, eg. give a filename and size limit on the command
line to save the cache
between invocations. This has been discussed a couple of times in the past
and the general consens
sounded like 'would be great'. Forcing the CGAL cache to allocate it's
memroy from a memory mapped
file region could be the way to go. Some care has to be taken on reload, the
cache should be
treated like hostile user input. Integrating it into the UI shouldn't be to
difficult, eg. by
allowing the UI to construct a cache file name based on the loaded file.

(B) specify several source/destination file tuples on the command line and
have them all exported in
one go, sharing the CGAL cache. I've seen discussion about adding an export
statement to the language,
which would create all kind of problems down the road, up to allow arbitrary
files written when files
are downloaded from thingiverse. That's why I'm suggesting a repeatable
command line argument, which
wouldn't have this issue.

(C) 'export' a hash of the normalized object tree, such that the plugin can
decide if a previous output
can be reused or not, basically pushing the caching issue one layer up.

What's your opinion?



--
Sent from: http://forum.openscad.org/

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org


_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
rmi
Reply | Threaded
Open this post in threaded view
|

Re: command line and multiple mesh export

rmi

<[hidden email]>:
There would have to be a way to determine if the cache saved is valid or not, for the given .scad file.


Are you sure? Worst case your hit ratio would be 0, but the CGALCache will never return wrong information as the key is the normalized object tree.





_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
Reply | Threaded
Open this post in threaded view
|

Re: command line and multiple mesh export

jdawgaz
well, take for example: python.

I believe how it works, is that when a given python script file is run, it determines if there is a .pyc file for it, and if so, checks to see if it is up-to-date WRT to the .py file. And if so, uses the .pyc file. If not, then it re-creates the .pyc file and then runs.

That would be what I am referring to. Otherwise, how could you make sure your persisted cache is the same as the source it came from?

I know I would want it that way.



--
Extra Ham Operator: K7AZJ
Registered Linux User: 275424
Raspberry Pi and Openscad developer

The most exciting phrase to hear in science - the one that heralds new discoveries - is not "Eureka!" but "That's funny...".
- Isaac. Asimov


On Thu, Nov 16, 2017 at 2:35 AM, Roland Mieslinger <[hidden email]> wrote:

<[hidden email]>:
There would have to be a way to determine if the cache saved is valid or not, for the given .scad file.


Are you sure? Worst case your hit ratio would be 0, but the CGALCache will never return wrong information as the key is the normalized object tree.





_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org



_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org
rmi
Reply | Threaded
Open this post in threaded view
|

Re: command line and multiple mesh export

rmi
> well, take for example: python.
>
> I believe how it works, is that when a given python script file is run, it determines if there is
> a .pyc file for it, and if so, checks to see if it is up-to-date WRT to the .py file. And if so, uses
> the .pyc file. If not, then it re-creates the .pyc file and then runs.
 
well, taking a procederal languag (python) as an example for a functional (OpenSCAD) ... but
let's try. The difference is that python caches code (methods, functions) while the CGAL cache
maps OpenSCAD code to points, edges and surfaces (results). It's the same difference as in
'how to add two numbers' vs. '1 + 1 = 2'.
 
> That would be what I am referring to. Otherwise, how could you make sure your persisted
> cache is the same as the source it came from?
 
It doesn't matter. If you create the same model in two places, it is still the same model and
can be used; eg. have a design with multiple files, but each requires the same 20 tooth
bevel gear.
 
> I know I would want it that way.
Still?
 
 
btw. I have a prototype (nothing beats running code) working. Lot more work needed
- the cache file is hardcoded
- unused objects are always discarded
- not even started to touch the GUI
 
'Lazy' (or deferred) loading seems to be important, as deserializing CGAL_Nef_polyhedron
seems to be expensiv.
 
first execution, takes 36s
time ../src/openscad/openscad --render -o Extruder.stl Extruder.scad 
...
CGAL cache saved 51 entries.
real    0m36.602s
user    0m35.138s
sys    0m0.918s
 
second execution, takes 3s
time ../src/openscad/openscad --render -o Extruder.stl Extruder.scad 
CGAL lazy loaded %multmatrix([[1,0,0,0],[0,1,0,0],[0,0,1,: 
...
CGAL lazy loaded 52 entries
CGAL lazy cache load: %multmatrix([[1,0,0,0],[0,1,0,0],[0,0,1,
CGAL Cache insert: %multmatrix([[1,0,0,0],[0,1,0,0],[0,0,1, (3966872 bytes)
CGAL cache saved %multmatrix([[1,0,0,0],[0,1,0,0],[0,0,1, 
CGAL cache saved 1 entries.
real    0m2.951s
user    0m2.692s
sys    0m0.188s
 
--
Roland

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org