bits from debian packaging

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

bits from debian packaging

chrysn
hello openscad developers,

i'm trying to get the unit tests to work fully for the next debian
package release.

i had to work around several issues to get them to work out of tree, and
these are topics i'd like to mention or would prefer to get changed:

* throwntogethertest: imo this should go to the main openscad binary,
  the other programs in the same style already did. once this is done,
  parts of the other code in the tests/ directory can be dropped (eg
  csgtestcore). this is issue #424.

  the reason i'm mentioning again is that throwntogethertest has very
  subtle differences from the main binary with respect to include paths.
  i didn't bother drilling that down because it will go away with #424
  and can be easily worked around by explicitly setting OPENSCADPATH.

* some more tests are still built but not run or not needed any more
  (modulecachetest, cgalcachetest, csgtexttest). could they be excluded
  from the default build process?

* the --info option, generating sysinfo, became unfunctional when the
  no-testprograms branch became merged. i've moved the functionality
  into the openscad main binary (--buildinfo), and adapted the test
  suite to use that option instead. see the attached patch.

* virtualfb.sh -- the way this is invoked is error-prone, and uses
  heuristics (pid guessing). why is this mechanism used instead of a
  straight-forward xvfb-run? there was a discussion about xvfb-run not
  working on ubuntu long ago[1], but from the arguments that were passed
  back then, it seems to me that they just forgot to give '-s "-screen 0
  800x600x24"' (the software renderer won't work with the default 8bit
  color depth).

* there is a patch to tests/CMakeLists.txt still open from long ago:

  > -if(${CMAKE_SYSTEM_NAME} MATCHES "FreeBSD")
  > +if(${CMAKE_SYSTEM_NAME} MATCHES "^FreeBSD")
  >    # FreeBSD has an old flex in /usr/bin and a new flex in /usr/local/bin
  >    set(FLEX_EXECUTABLE /usr/local/bin/flex)

  could you apply that? it will solve the issue of openscad not
  compiling on debian kfreebsd.

* example024.scad: as opposed to all the other example files, this one
  has an explicit copyright note. that's ok, but it's not explicit
  enough -- CC-BY-SA is not a defined license, it would need a version.
  ideally, it would say something like

  > This work is licensed under a Creative Commons
  > Attribution-ShareAlike 3.0 Unported License
  > (http://creativecommons.org/licenses/by-sa/3.0/).

  or be relicensed under the same terms as the rest of the software.

it'll take some time for the openscad-mcad package to be ready and
uploaded, but things look much nicer by now.

best regards
chrysn

[1] http://forum.openscad.org/Headless-OpenSCAD-td5187.html

--
To use raw power is to make yourself infinitely vulnerable to greater powers.
  -- Bene Gesserit axiom

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://rocklinux.net/mailman/listinfo/openscad
http://openscad.org - https://flattr.com/thing/121566

0001-move-info-to-openscad-buildinfo.patch (7K) Download Attachment
signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: bits from debian packaging

donbright
"* virtualfb.sh -- the way this is invoked is error-prone, and uses
  heuristics (pid guessing). why is this mechanism used instead of a
  straight-forward xvfb-run? there was a discussion about xvfb-run not
  working on ubuntu long ago[1], but from the arguments that were passed
  back then, it seems to me that they just forgot to give '-s "-screen 0
  800x600x24"' (the software renderer won't work with the default 8bit
  color depth)."

1. xvfb-run is not installed and not available and/or broken on some systems, and it doesn't work with Xvnc. BSD typically doesn't have it. The xvfb-run shell code from debian is not very portable so I couldn't just copy/paste it.

2. Since I was running 'ctest' hundreds of times in many variations, I wanted to just type stuff like 'ctest -R cylinder.*' and have it 'just work', instead of  having to worry about typing 'xvfb-test -s "-screen 0 800x600x24" ctest -R cylinder.*'  plus extra per-system options every time I wanted to run a test. This also means that ctest needs to be able to read which DISPLAY the framebuffer startup script found and used to start Xvfb, and pass that info on to the ctest subprocesses - i didnt find an easy way to do that with xvfb-run. Obviously, running a separate xvfb-run instance for each of 400+ tests is not acceptable b/c it would be incredibly slow.

3. virtualfb.sh has been tested to work on dozens of systems. it is a kludge but if you google you will find that other projects use similar kludges because X itself does not provide a reliable, portable way to run a per-user software framebuffer.

Please feel free to rewrite the code to detect the presence of debian and use xvfb-run on debian systems if you feel that is a better solution.

-DB



On Thu, Aug 8, 2013 at 1:30 PM, chrysn <[hidden email]> wrote:
hello openscad developers,

i'm trying to get the unit tests to work fully for the next debian
package release.

i had to work around several issues to get them to work out of tree, and
these are topics i'd like to mention or would prefer to get changed:

* throwntogethertest: imo this should go to the main openscad binary,
  the other programs in the same style already did. once this is done,
  parts of the other code in the tests/ directory can be dropped (eg
  csgtestcore). this is issue #424.

  the reason i'm mentioning again is that throwntogethertest has very
  subtle differences from the main binary with respect to include paths.
  i didn't bother drilling that down because it will go away with #424
  and can be easily worked around by explicitly setting OPENSCADPATH.

* some more tests are still built but not run or not needed any more
  (modulecachetest, cgalcachetest, csgtexttest). could they be excluded
  from the default build process?

* the --info option, generating sysinfo, became unfunctional when the
  no-testprograms branch became merged. i've moved the functionality
  into the openscad main binary (--buildinfo), and adapted the test
  suite to use that option instead. see the attached patch.

* virtualfb.sh -- the way this is invoked is error-prone, and uses
  heuristics (pid guessing). why is this mechanism used instead of a
  straight-forward xvfb-run? there was a discussion about xvfb-run not
  working on ubuntu long ago[1], but from the arguments that were passed
  back then, it seems to me that they just forgot to give '-s "-screen 0
  800x600x24"' (the software renderer won't work with the default 8bit
  color depth).

* there is a patch to tests/CMakeLists.txt still open from long ago:

  > -if(${CMAKE_SYSTEM_NAME} MATCHES "FreeBSD")
  > +if(${CMAKE_SYSTEM_NAME} MATCHES "^FreeBSD")
  >    # FreeBSD has an old flex in /usr/bin and a new flex in /usr/local/bin
  >    set(FLEX_EXECUTABLE /usr/local/bin/flex)

  could you apply that? it will solve the issue of openscad not
  compiling on debian kfreebsd.

* example024.scad: as opposed to all the other example files, this one
  has an explicit copyright note. that's ok, but it's not explicit
  enough -- CC-BY-SA is not a defined license, it would need a version.
  ideally, it would say something like

  > This work is licensed under a Creative Commons
  > Attribution-ShareAlike 3.0 Unported License
  > (http://creativecommons.org/licenses/by-sa/3.0/).

  or be relicensed under the same terms as the rest of the software.

it'll take some time for the openscad-mcad package to be ready and
uploaded, but things look much nicer by now.

best regards
chrysn

[1] http://forum.openscad.org/Headless-OpenSCAD-td5187.html

--
To use raw power is to make yourself infinitely vulnerable to greater powers.
  -- Bene Gesserit axiom

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://rocklinux.net/mailman/listinfo/openscad
http://openscad.org - https://flattr.com/thing/121566


_______________________________________________
OpenSCAD mailing list
[hidden email]
http://rocklinux.net/mailman/listinfo/openscad
http://openscad.org - https://flattr.com/thing/121566
Reply | Threaded
Open this post in threaded view
|

Re: bits from debian packaging

donbright
I'd also like to note that if you run "xvfb-run -s blah blah blah ctest", then the virtualfb.sh should never be called. Ctest should detect the DISPLAY that xvfb-run sets, and pass that DISPLAY to it's subprocesses. So for the debian script, it should be sufficient to just run "xvfb-run -s blah blah blah ctest", and virtualfb.sh should, in that case, never come in to play.


On Thu, Aug 8, 2013 at 5:53 PM, Don Bright <[hidden email]> wrote:
"* virtualfb.sh -- the way this is invoked is error-prone, and uses
  heuristics (pid guessing). why is this mechanism used instead of a
  straight-forward xvfb-run? there was a discussion about xvfb-run not
  working on ubuntu long ago[1], but from the arguments that were passed
  back then, it seems to me that they just forgot to give '-s "-screen 0
  800x600x24"' (the software renderer won't work with the default 8bit
  color depth)."

1. xvfb-run is not installed and not available and/or broken on some systems, and it doesn't work with Xvnc. BSD typically doesn't have it. The xvfb-run shell code from debian is not very portable so I couldn't just copy/paste it.

2. Since I was running 'ctest' hundreds of times in many variations, I wanted to just type stuff like 'ctest -R cylinder.*' and have it 'just work', instead of  having to worry about typing 'xvfb-test -s "-screen 0 800x600x24" ctest -R cylinder.*'  plus extra per-system options every time I wanted to run a test. This also means that ctest needs to be able to read which DISPLAY the framebuffer startup script found and used to start Xvfb, and pass that info on to the ctest subprocesses - i didnt find an easy way to do that with xvfb-run. Obviously, running a separate xvfb-run instance for each of 400+ tests is not acceptable b/c it would be incredibly slow.

3. virtualfb.sh has been tested to work on dozens of systems. it is a kludge but if you google you will find that other projects use similar kludges because X itself does not provide a reliable, portable way to run a per-user software framebuffer.

Please feel free to rewrite the code to detect the presence of debian and use xvfb-run on debian systems if you feel that is a better solution.

-DB



On Thu, Aug 8, 2013 at 1:30 PM, chrysn <[hidden email]> wrote:
hello openscad developers,

i'm trying to get the unit tests to work fully for the next debian
package release.

i had to work around several issues to get them to work out of tree, and
these are topics i'd like to mention or would prefer to get changed:

* throwntogethertest: imo this should go to the main openscad binary,
  the other programs in the same style already did. once this is done,
  parts of the other code in the tests/ directory can be dropped (eg
  csgtestcore). this is issue #424.

  the reason i'm mentioning again is that throwntogethertest has very
  subtle differences from the main binary with respect to include paths.
  i didn't bother drilling that down because it will go away with #424
  and can be easily worked around by explicitly setting OPENSCADPATH.

* some more tests are still built but not run or not needed any more
  (modulecachetest, cgalcachetest, csgtexttest). could they be excluded
  from the default build process?

* the --info option, generating sysinfo, became unfunctional when the
  no-testprograms branch became merged. i've moved the functionality
  into the openscad main binary (--buildinfo), and adapted the test
  suite to use that option instead. see the attached patch.

* virtualfb.sh -- the way this is invoked is error-prone, and uses
  heuristics (pid guessing). why is this mechanism used instead of a
  straight-forward xvfb-run? there was a discussion about xvfb-run not
  working on ubuntu long ago[1], but from the arguments that were passed
  back then, it seems to me that they just forgot to give '-s "-screen 0
  800x600x24"' (the software renderer won't work with the default 8bit
  color depth).

* there is a patch to tests/CMakeLists.txt still open from long ago:

  > -if(${CMAKE_SYSTEM_NAME} MATCHES "FreeBSD")
  > +if(${CMAKE_SYSTEM_NAME} MATCHES "^FreeBSD")
  >    # FreeBSD has an old flex in /usr/bin and a new flex in /usr/local/bin
  >    set(FLEX_EXECUTABLE /usr/local/bin/flex)

  could you apply that? it will solve the issue of openscad not
  compiling on debian kfreebsd.

* example024.scad: as opposed to all the other example files, this one
  has an explicit copyright note. that's ok, but it's not explicit
  enough -- CC-BY-SA is not a defined license, it would need a version.
  ideally, it would say something like

  > This work is licensed under a Creative Commons
  > Attribution-ShareAlike 3.0 Unported License
  > (http://creativecommons.org/licenses/by-sa/3.0/).

  or be relicensed under the same terms as the rest of the software.

it'll take some time for the openscad-mcad package to be ready and
uploaded, but things look much nicer by now.

best regards
chrysn

[1] http://forum.openscad.org/Headless-OpenSCAD-td5187.html

--
To use raw power is to make yourself infinitely vulnerable to greater powers.
  -- Bene Gesserit axiom

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://rocklinux.net/mailman/listinfo/openscad
http://openscad.org - https://flattr.com/thing/121566



_______________________________________________
OpenSCAD mailing list
[hidden email]
http://rocklinux.net/mailman/listinfo/openscad
http://openscad.org - https://flattr.com/thing/121566
Reply | Threaded
Open this post in threaded view
|

Re: bits from debian packaging

donbright
By the way the DISPLAY detection code is in CTestCustom.template

It attempts to detect if ctest has been started under an already running X environment (like on a desktop or laptop, DISPLAY will usually be :0) ... and if Xvfb or Xvnc has already been started by a user or by xvfb-run, then DISPLAY will be set to something like :99 or whatnot. CTestCustom.template should detect this, and it will only start virtualfb.sh if it doesn't find DISPLAY already set. So, as I noted, under debian packaging, it should be possible to just run "xvfb-run -s blahblahblah ctest", and it should work without calling virtualfb.sh.


On Thu, Aug 8, 2013 at 5:59 PM, Don Bright <[hidden email]> wrote:
I'd also like to note that if you run "xvfb-run -s blah blah blah ctest", then the virtualfb.sh should never be called. Ctest should detect the DISPLAY that xvfb-run sets, and pass that DISPLAY to it's subprocesses. So for the debian script, it should be sufficient to just run "xvfb-run -s blah blah blah ctest", and virtualfb.sh should, in that case, never come in to play.


On Thu, Aug 8, 2013 at 5:53 PM, Don Bright <[hidden email]> wrote:
"* virtualfb.sh -- the way this is invoked is error-prone, and uses
  heuristics (pid guessing). why is this mechanism used instead of a
  straight-forward xvfb-run? there was a discussion about xvfb-run not
  working on ubuntu long ago[1], but from the arguments that were passed
  back then, it seems to me that they just forgot to give '-s "-screen 0
  800x600x24"' (the software renderer won't work with the default 8bit
  color depth)."

1. xvfb-run is not installed and not available and/or broken on some systems, and it doesn't work with Xvnc. BSD typically doesn't have it. The xvfb-run shell code from debian is not very portable so I couldn't just copy/paste it.

2. Since I was running 'ctest' hundreds of times in many variations, I wanted to just type stuff like 'ctest -R cylinder.*' and have it 'just work', instead of  having to worry about typing 'xvfb-test -s "-screen 0 800x600x24" ctest -R cylinder.*'  plus extra per-system options every time I wanted to run a test. This also means that ctest needs to be able to read which DISPLAY the framebuffer startup script found and used to start Xvfb, and pass that info on to the ctest subprocesses - i didnt find an easy way to do that with xvfb-run. Obviously, running a separate xvfb-run instance for each of 400+ tests is not acceptable b/c it would be incredibly slow.

3. virtualfb.sh has been tested to work on dozens of systems. it is a kludge but if you google you will find that other projects use similar kludges because X itself does not provide a reliable, portable way to run a per-user software framebuffer.

Please feel free to rewrite the code to detect the presence of debian and use xvfb-run on debian systems if you feel that is a better solution.

-DB



On Thu, Aug 8, 2013 at 1:30 PM, chrysn <[hidden email]> wrote:
hello openscad developers,

i'm trying to get the unit tests to work fully for the next debian
package release.

i had to work around several issues to get them to work out of tree, and
these are topics i'd like to mention or would prefer to get changed:

* throwntogethertest: imo this should go to the main openscad binary,
  the other programs in the same style already did. once this is done,
  parts of the other code in the tests/ directory can be dropped (eg
  csgtestcore). this is issue #424.

  the reason i'm mentioning again is that throwntogethertest has very
  subtle differences from the main binary with respect to include paths.
  i didn't bother drilling that down because it will go away with #424
  and can be easily worked around by explicitly setting OPENSCADPATH.

* some more tests are still built but not run or not needed any more
  (modulecachetest, cgalcachetest, csgtexttest). could they be excluded
  from the default build process?

* the --info option, generating sysinfo, became unfunctional when the
  no-testprograms branch became merged. i've moved the functionality
  into the openscad main binary (--buildinfo), and adapted the test
  suite to use that option instead. see the attached patch.

* virtualfb.sh -- the way this is invoked is error-prone, and uses
  heuristics (pid guessing). why is this mechanism used instead of a
  straight-forward xvfb-run? there was a discussion about xvfb-run not
  working on ubuntu long ago[1], but from the arguments that were passed
  back then, it seems to me that they just forgot to give '-s "-screen 0
  800x600x24"' (the software renderer won't work with the default 8bit
  color depth).

* there is a patch to tests/CMakeLists.txt still open from long ago:

  > -if(${CMAKE_SYSTEM_NAME} MATCHES "FreeBSD")
  > +if(${CMAKE_SYSTEM_NAME} MATCHES "^FreeBSD")
  >    # FreeBSD has an old flex in /usr/bin and a new flex in /usr/local/bin
  >    set(FLEX_EXECUTABLE /usr/local/bin/flex)

  could you apply that? it will solve the issue of openscad not
  compiling on debian kfreebsd.

* example024.scad: as opposed to all the other example files, this one
  has an explicit copyright note. that's ok, but it's not explicit
  enough -- CC-BY-SA is not a defined license, it would need a version.
  ideally, it would say something like

  > This work is licensed under a Creative Commons
  > Attribution-ShareAlike 3.0 Unported License
  > (http://creativecommons.org/licenses/by-sa/3.0/).

  or be relicensed under the same terms as the rest of the software.

it'll take some time for the openscad-mcad package to be ready and
uploaded, but things look much nicer by now.

best regards
chrysn

[1] http://forum.openscad.org/Headless-OpenSCAD-td5187.html

--
To use raw power is to make yourself infinitely vulnerable to greater powers.
  -- Bene Gesserit axiom

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://rocklinux.net/mailman/listinfo/openscad
http://openscad.org - https://flattr.com/thing/121566




_______________________________________________
OpenSCAD mailing list
[hidden email]
http://rocklinux.net/mailman/listinfo/openscad
http://openscad.org - https://flattr.com/thing/121566
Reply | Threaded
Open this post in threaded view
|

Re: bits from debian packaging

donbright
In reply to this post by chrysn
"
* the --info option, generating sysinfo, became unfunctional when the
  no-testprograms branch became merged. i've moved the functionality
  into the openscad main binary (--buildinfo), and adapted the test
  suite to use that option instead. see the attached patch."


It should not be called --buildinfo because some of that information is run-time info, such as the OpenGL settings being used. If you swap video cards, you will get a different readout with the same build of openscad. Also if you use a software framebuffer vs a hardware framebuffer, again, different readout.

-DB


_______________________________________________
OpenSCAD mailing list
[hidden email]
http://rocklinux.net/mailman/listinfo/openscad
http://openscad.org - https://flattr.com/thing/121566
Reply | Threaded
Open this post in threaded view
|

Re: bits from debian packaging

donbright
In reply to this post by chrysn
"* there is a patch to tests/CMakeLists.txt still open from long ago:

  > -if(${CMAKE_SYSTEM_NAME} MATCHES "FreeBSD")
  > +if(${CMAKE_SYSTEM_NAME} MATCHES "^FreeBSD")
  >    # FreeBSD has an old flex in /usr/bin and a new flex in /usr/local/bin
  >    set(FLEX_EXECUTABLE /usr/local/bin/flex)

  could you apply that? it will solve the issue of openscad not
  compiling on debian kfreebsd.
"

Applied to master. Thank you. 
-DB


_______________________________________________
OpenSCAD mailing list
[hidden email]
http://rocklinux.net/mailman/listinfo/openscad
http://openscad.org - https://flattr.com/thing/121566
Reply | Threaded
Open this post in threaded view
|

Re: bits from debian packaging

chrysn
In reply to this post by donbright
On Thu, Aug 08, 2013 at 05:53:21PM -0500, Don Bright wrote:
> 1. xvfb-run is not installed and not available [...]

oh, i missed that it's a debian specific addition. of course, this makes
it unusable everywhere else.

> 2. [...] having to worry about typing 'xvfb-test -s "-screen 0
> 800x600x24" ctest -R cylinder.*'

i'd rather have thought of something along the lines of ctest checking
the DISPLAY variable and re-executing itself under xvfb-run (provided
something like this can be done with ctest); otherwise there could be a
wrapper ./test like

    #!/bin/sh
    if [ -n "$DISPLAY"] then
      ctest $@
    else
      xvfb-run -s '-screen 0 800x600x24' ctest $@
    fi

but given the non-availability of xvfb-runs on other platforms, that
won't work anyway.

> Please feel free to rewrite the code to detect the presence of debian and
> use xvfb-run on debian systems if you feel that is a better solution.

the openscad-testrun script (for running the test suite out-of-tree on
installed systems) has provisions for using xvfb-run, but ctest can
still resort to virtualfb.sh too.


thanks for your reply
c

--
To use raw power is to make yourself infinitely vulnerable to greater powers.
  -- Bene Gesserit axiom

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://rocklinux.net/mailman/listinfo/openscad
http://openscad.org - https://flattr.com/thing/121566

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: bits from debian packaging

chrysn
In reply to this post by donbright
On Thu, Aug 08, 2013 at 06:06:18PM -0500, Don Bright wrote:
> It should not be called --buildinfo because some of that information is
> run-time info, such as the OpenGL settings being used. If you swap video
> cards, you will get a different readout with the same build of openscad.
> Also if you use a software framebuffer vs a hardware framebuffer, again,
> different readout.

i was hesitant to use --info because it might be assumed to give info
about a given scad file, but i just had a look at several man pages, and
--info is also used by binutils, ps and schroot in similar ways.

the new patch (see attached) calls the option --info and also updates
the man page, which i forgot in the first place.

best regards
chrysn

--
To use raw power is to make yourself infinitely vulnerable to greater powers.
  -- Bene Gesserit axiom

_______________________________________________
OpenSCAD mailing list
[hidden email]
http://rocklinux.net/mailman/listinfo/openscad
http://openscad.org - https://flattr.com/thing/121566

combined_info.patch (7K) Download Attachment
signature.asc (853 bytes) Download Attachment