View Issue Details

IDProjectCategoryView StatusLast Update
0004247Slicer4Core: Extensionspublic2017-08-21 10:37
ReporterpinterAssigned Tojcfr 
PriorityhighSeveritymajorReproducibilityN/A
Status assignedResolutionfixed 
PlatformWindowsOSOS Version
Product VersionSlicer 4.5.0-1 
Target VersionSlicer 4.6.0Fixed in VersionSlicer 4.7.0 
Summary0004247: Extensions do not get built on Windows if any test in the dependent extensions fail
Description

Which is undesired, because if an extension has 60 tests and one fails (may it be a widget test that fails because the module widget is 5 pixels wider than the set threshold), it does not mean it's not operational, and it doesn't make sense not to even start building the extensions that depend on it.

The extensions that motivated this issue are GelDosimetry and FilmDosimetry, both of which depend on SlicerRT. Sometimes one of the lengthy python self tests fail, or a widget test due to width.

TagsNo tags attached.

Activities

lassoan

lassoan

2016-09-11 10:42

developer   ~0014088

In Extensions\CMake\SlicerBlockBuildPackageAndUploadExtension.cmake this line causes the target build step to fail:

ctest_test(
    BUILD ${EXTENSION_SUPERBUILD_BINARY_DIR}/${EXTENSION_BUILD_SUBDIRECTORY}
    PARALLEL_LEVEL ${CTEST_PARALLEL_LEVEL})

Build step (in which ctest.exe returns with error code if ctest_test finds that a test fails):

...
"c:\Program Files\CMake\bin\ctest.exe" -C $(Configuration) -DCTEST_BUILD_CONFIGURATION:STRING=$(Configuration) -DCTEST_MODEL:STRING=Experimental -DSCRIPT_ARGS_FILE:FILEPATH=C:/D/N/E-0/SlicerRT-upload-command-args.cmake -S C:/D/N/Slicer-0/Extensions/CMake/SlicerBlockBuildPackageAndUploadExtension.cmake -VV
...

How ctest.exe knows that ctest_test failed (from parsing XML output file)? How it could be prevented to return with error code?

pinter

pinter

2016-09-13 10:45

developer   ~0014093

For reference: SlicerRT ticket created for the workaround that needs to be applied due to this issue
https://app.assembla.com/spaces/slicerrt/tickets/838-failing-slicerrt-tests-cause-dependent-extensions-not-to-build/details

jcfr

jcfr

2016-10-13 00:05

administrator   ~0014194

This should be fixed by

http://viewvc.slicer.org/viewvc.cgi/Slicer4?view=revision&revision=25440
http://viewvc.slicer.org/viewvc.cgi/Slicer4?view=revision&revision=25439
http://viewvc.slicer.org/viewvc.cgi/Slicer4?view=revision&revision=25438

For future reference the associated github topic as:

https://github.com/Slicer/Slicer/pull/605

pinter

pinter

2016-10-16 10:30

developer   ~0014197

I re-enabled the failing SlicerRT tests to see if this change fixes the issue and we get dependent extensions to build even with the failing tests.

Unfortunately the windows build for dependent tests still does not start if any test fails.

pinter

pinter

2016-12-23 16:57

developer   ~0014282

I'm increasing the priority of this issue, as no progress has been made over the lsat four months, and it considerably decreases the usability of the extension dependency mechanism.

jcfr

jcfr

2016-12-23 18:32

administrator   ~0014283

Thanks for the update.

I wonder how different is the SlicerRt extension configuration from the current tests.

The extension build system tests are added here:

https://github.com/Slicer/Slicer/blob/d7f501e925831b48393214c81a1533d316a810dd/Extensions/CMake/Testing/CMakeLists.txt#L65-L79

And for all test cases, extension B and C depend on extension A which has one failing test.
See https://github.com/Slicer/Slicer/blob/d7f501e925831b48393214c81a1533d316a810dd/Extensions/CMake/Testing/SlicerExtensionBuildSystemTest.py#L283-L289

A mock web server also confirm that all CDash and Midas queries are submitted as expected.

Is SlicerRt test crashing/hanging forever and preventing the dependent extensions from being built ?

pinter

pinter

2017-01-05 11:19

developer   ~0014285

Thanks for investigating! Okay, so the issue is not so trivial and seems SlicerRT specific.
Andras also wanted to create a minimal test case, but it worked for him as well, and we didn't know how to proceed. There must be something in how SlicerRT is configured, I agree.

I don't think that the any test hangs or crashes. If I have a module widget generic test fail because the module is wider than the threshold it's checked against, the dependent extensions don't build.

jcfr

jcfr

2017-08-10 12:56

administrator   ~0015034

See https://github.com/Slicer/Slicer/pull/768

jcfr

jcfr

2017-08-10 14:59

administrator   ~0015041

This should be fixed in 26223
See http://viewvc.slicer.org/viewvc.cgi/Slicer4?view=revision&revision=26223

pinter

pinter

2017-08-21 10:36

developer   ~0015066

Unfortunately nothing changed in this manner. I re-enabled the SlicerRT tests that failed or were prone to failing, and the dependent extension builds do not start on Windows, for example
http://slicer.cdash.org/index.php?project=Slicer4&filtercount=1&showfilters=1&field1=label&compare1=63&value1=GelDosimetryAnalysis

Related Changesets

Import 2017-06-07 23:51:09: master c226f731

2016-10-12 23:50:38

jcfr

Details Diff
BUG: Extension build system: Ignore extension CTest return code. Fixes 0004247

This commit ensures each "EXTENSION_UPLOAD_COMMAND" responsible to configure,
build, test and package any given extension is completely "sandboxed" by
using a wrapper script.

This is required because simply setting the RETURN_VALUE parameter
to the ctest_test() command is not enough to avoid the script from
exiting with error code.

#
# The following wrapper script is required to workaround issue 0004247
# and avoid the all extension build from failing if a test of
# extension fail.
#
# Note that as soon as CMake >= 3.6.7 is released, it should be possible
# to remove the wrapper script and simply specify CAPTURE_CMAKE_ERROR
# ctest_test option.
#
# See https://cmake.org/cmake/help/v3.7/command/ctest_test.html
#

Tested-by: Isaiah Norton <inorton@bwh.harvard.edu>
Tested-by: Johan Andruejol <johan.andruejol@kitware.com>
Tested-by: Nicole Aucoin <nicole@bwh.harvard.edu>
Tested-by: Steve Pieper <pieper@isomics.com>

Reported-by: Csaba Pinter <csaba.pinter@queensu.ca>
Reported-by: Andras Lasso <lasso@cs.queensu.ca>

Thanks: All of the above

git-svn-id: http://svn.slicer.org/Slicer4/trunk@25439 3bd1e089-480b-0410-8dfb-8563597acbee
mod - Extensions/CMake/SlicerBlockBuildPackageAndUploadExtensions.cmake Diff File

Import 2017-06-07 23:51:09: master 41df9c1a

2016-10-12 23:50:40

jcfr

Details Diff
BUG: Extension build system: Fix use of wrapper script on windows. Fixes 0004247

This commit ensures the build configuration is passed to the wrapped
command.

It fixes error like this one:


8: SetMakeCommand:C:\D\Support\CMake-3.5.0-rc3\bin\cmake.exe --build . --config "$(Configuration)"
8:
8: SetCTestConfiguration:MakeCommand:C:\D\Support\CMake-3.5.0-rc3\bin\cmake.exe --build . --config "$(Configuration)"
8:
8: Build project
8:
8: MakeCommand:C:\D\Support\CMake-3.5.0-rc3\bin\cmake.exe --build . --config "$(Configuration)"
8: Run command: "C:\D\Support\CMake-3.5.0-rc3\bin\cmake.exe" "--build" "." "--config" "$(Configuration)"
8:
8: Microsoft (R) Build Engine version 12.0.40629.0
8: [Microsoft .NET Framework, version 4.0.30319.34209]
8: Copyright (C) Microsoft Corporation. All rights reserved.
8: Build started 10/12/2016 10:57:46 PM.
8:
8: Project "C:\D\N\Slicer-1-build\Slicer-build\Extensions\CMake\Testing\build_with_upload-build\TestExtA-build\ALL_BUILD.vcxproj" on node 1 (default targets).
8: Project "C:\D\N\Slicer-1-build\Slicer-build\Extensions\CMake\Testing\build_with_upload-build\TestExtA-build\ALL_BUILD.vcxproj" (1) is building "C:\D\N\Slic
er-1-build\Slicer-build\Extensions\CMake\Testing\build_with_upload-build\TestExtA-build\ZERO_CHECK.vcxproj" (2) on node 1 (default targets).
8:
8:
8: C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V120\Microsoft.Cpp.Platform.targets(61,5): error MSB8013: This project doesn't contain the Configuration an
d Platform combination of $(Configuration)|x64. [C:\D\N\Slicer-1-build\Slicer-build\Extensions\CMake\Testing\build_with_upload-build\TestExtA-build\ZERO_CHECK.v
cxproj] [C:\D\N\Slicer-1-build\Slicer-build\Extensions\CMake\Testing\build_with_upload-build\TestExtA.vcxproj]
8:

git-svn-id: http://svn.slicer.org/Slicer4/trunk@25440 3bd1e089-480b-0410-8dfb-8563597acbee
mod - Extensions/CMake/SlicerBlockBuildPackageAndUploadExtensions.cmake Diff File
mod - Extensions/CMake/SlicerBlockUploadExtension.cmake Diff File

Issue History

Date Modified Username Field Change
2016-08-24 11:38 pinter New Issue
2016-08-24 11:38 pinter Status new => assigned
2016-08-24 11:38 pinter Assigned To => jcfr
2016-09-11 10:42 lassoan Note Added: 0014088
2016-09-13 10:45 pinter Note Added: 0014093
2016-10-12 11:45 jcfr Target Version Slicer 4.5.1 => Slicer 4.6.0
2016-10-13 00:05 jcfr Note Added: 0014194
2016-10-13 00:05 jcfr Status assigned => resolved
2016-10-13 00:05 jcfr Fixed in Version => Slicer 4.6.0
2016-10-13 00:05 jcfr Resolution open => fixed
2016-10-13 01:24 jcfr Product Version Slicer 4.5.1 => Slicer 4.5.0-1
2016-10-16 10:30 pinter Note Added: 0014197
2016-10-16 10:30 pinter Status resolved => confirmed
2016-12-23 16:57 pinter Note Added: 0014282
2016-12-23 16:57 pinter Priority normal => high
2016-12-23 18:32 jcfr Note Added: 0014283
2017-01-05 11:19 pinter Note Added: 0014285
2017-06-10 08:51 jcfr Changeset attached => Slicer master 41df9c1a
2017-06-10 08:51 jcfr Changeset attached => Slicer master c226f731
2017-08-10 12:56 jcfr Note Added: 0015034
2017-08-10 14:59 jcfr Status confirmed => resolved
2017-08-10 14:59 jcfr Fixed in Version Slicer 4.6.0 => Slicer 4.7.0
2017-08-10 14:59 jcfr Note Added: 0015041
2017-08-21 10:36 pinter Status resolved => assigned
2017-08-21 10:36 pinter Note Added: 0015066
2017-08-21 10:37 pinter Summary Extensions do not get built if any test in the dependent extensions fail => Extensions do not get built on Windows if any test in the dependent extensions fail