File caching

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

File caching

nophead
Does openscad cache modules at the file level?

The reason I ask is I had a syntax error in a file that I fixed. If I loaded the file it complied and rendered. If I opened my main file that used the file then it reported the syntax error that I had already fixed. Closing and reopening openscad fixed the problem.

I can't see why it would cache source text, but it explains a lot of the weird updating problems I have. E.g.

Getting an old version of a module in my assembly, even after opening the file with the module and seeing the new version.

A module using a old include path after I had changed it, until I reload openscad.


Reply | Threaded
Open this post in threaded view
|

Re: File caching

kintel
Administrator
On Feb 15, 2012, at 10:13 AM, nop head wrote:

> Does openscad cache modules at the file level?
>
Modules included with a use statement are cached as compiled modules with timestamps attached.
However, this caching doesn't come with dependency tracking, so multiple levels of use will easily break it.

I'm working on a rewrite of that, see issue https://github.com/openscad/openscad/issues/75.

As usual, if you can provide a recipe with concrete examples how to trigger the problem I can test for that, just in case the tests I do don't cover your usage scenario.

 -Marius


Reply | Threaded
Open this post in threaded view
|

Re: File caching

nophead
But how can a compiled module have a syntax error in it?

On 15 February 2012 10:24, Marius Kintel <[hidden email]> wrote:
On Feb 15, 2012, at 10:13 AM, nop head wrote:

> Does openscad cache modules at the file level?
>
Modules included with a use statement are cached as compiled modules with timestamps attached.
However, this caching doesn't come with dependency tracking, so multiple levels of use will easily break it.

I'm working on a rewrite of that, see issue https://github.com/openscad/openscad/issues/75.

As usual, if you can provide a recipe with concrete examples how to trigger the problem I can test for that, just in case the tests I do don't cover your usage scenario.

 -Marius

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://rocklinux.net/mailman/listinfo/openscad

Reply | Threaded
Open this post in threaded view
|

Re: File caching

kintel
Administrator
On Feb 15, 2012, at 11:34 AM, nop head wrote:

> But how can a compiled module have a syntax error in it?
>
If it has a syntax error, it cannot be compiled and thus shouldn't be cached.
Bugs happen though - our task to make sure we only fix each bug once, hence regression testing.

 -Marius


Reply | Threaded
Open this post in threaded view
|

Re: File caching

nophead


On 15 February 2012 10:38, Marius Kintel <[hidden email]> wrote:
On Feb 15, 2012, at 11:34 AM, nop head wrote:

> But how can a compiled module have a syntax error in it?
>
If it has a syntax error, it cannot be compiled and thus shouldn't be cached.
 
Yes but how can a syntax error be cached if it only caches compiled code? It is reporting as syntax error with a line number. Surely all that info is gone once it is in the compiled form?

Bugs happen though - our task to make sure we only fix each bug once, hence regression testing.

It is hard to see how this is a simple bug. It appears to remember the textual contents of a file but you say it only caches the compiled version.  Does the compiled version remember syntax errors and file include paths and try to reuse them?

This explains all my problems. Yes it doesn't track dependencies, but I can handle those myself by touching files, or simply loading them into openscad. The big problems I have is with it maintaining multiple copies of files, which is very confusing and it doesn't get fixed by flushing the cache.
 

 -Marius

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://rocklinux.net/mailman/listinfo/openscad

Reply | Threaded
Open this post in threaded view
|

Re: File caching

kintel
Administrator
On Feb 15, 2012, at 11:46 AM, nop head wrote:
>  
> Yes but how can a syntax error be cached if it only caches compiled code? It is reporting as syntax error with a line number. Surely all that info is gone once it is in the compiled form?
>
> It is hard to see how this is a simple bug. It appears to remember the textual contents of a file but you say it only caches the compiled version.  Does the compiled version remember syntax errors and file include paths and try to reuse them?
>
It's a bit messy - the actual output during a compilation is cached as a string together with the cached module. When reading the cache, this string is output to the console. I think this was an attempt to make caching transparent to the user. This was also done hierarchically, also for includes, so even simple bugs could give very confusing output.
I'll probably turn off error message caching and make the module caching and dependency tracking solid, then perhaps put it back later.

The primary issue since last release is that we didn't have a good test coverage for the reloading and caching, plus the duplicated code discussed recently. Both these have caused a number of issues to surface after internal refactoring and now we need to cough up the missing use-cases to make sure the current rewrite stays solid.

Again, anything which can be proven to break with actual test data is easy to keep track of, so whenever you have some time to help out constructing these cases, it would be truly appreciated.

 -Marius


Reply | Threaded
Open this post in threaded view
|

Re: File caching

nophead
On 15 February 2012 11:05, Marius Kintel <[hidden email]> wrote:
On Feb 15, 2012, at 11:46 AM, nop head wrote:
>
> Yes but how can a syntax error be cached if it only caches compiled code? It is reporting as syntax error with a line number. Surely all that info is gone once it is in the compiled form?
>
> It is hard to see how this is a simple bug. It appears to remember the textual contents of a file but you say it only caches the compiled version.  Does the compiled version remember syntax errors and file include paths and try to reuse them?
>
It's a bit messy - the actual output during a compilation is cached as a string together with the cached module. When reading the cache, this string is output to the console. I think this was an attempt to make caching transparent to the user. This was also done hierarchically, also for includes, so even simple bugs could give very confusing output.
I'll probably turn off error message caching and make the module caching and dependency tracking solid, then perhaps put it back later.

Yes it seems overly complex and bazaar. If the module fails to compile don't cache it and then no need to cache errors which seems like a very bad idea.
 

The primary issue since last release is that we didn't have a good test coverage for the reloading and caching, plus the duplicated code discussed recently. Both these have caused a number of issues to surface after internal refactoring and now we need to cough up the missing use-cases to make sure the current rewrite stays solid.

Again, anything which can be proven to break with actual test data is easy to keep track of, so whenever you have some time to help out constructing these cases, it would be truly appreciated.

The problem is these seem to be transitory errors which disappear when you reload openscad, so it would be hard to make a test case.  I have never been able to recreate them by repeating the same actions. I will try putting the syntax error back. It was a stray character that somehow got on the end of a file I wasn't working on. It didn't help that the line number was totally bogus.

One fundamental thing is that when you open a file and compile it, it should update any cached copy of that file. Having multiple copies of the same module and regurgitating old error messages leads to a totally confusing user experience.


 

 -Marius

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://rocklinux.net/mailman/listinfo/openscad