In Extensions\CMake\SlicerBlockBuildPackageAndUploadExtension.cmake this line causes the target build step to fail:

ctest_test(
    BUILD ${EXTENSION_SUPERBUILD_BINARY_DIR}/${EXTENSION_BUILD_SUBDIRECTORY}
    PARALLEL_LEVEL ${CTEST_PARALLEL_LEVEL})

Build step (in which ctest.exe returns with error code if ctest_test finds that a test fails):

...
"c:\Program Files\CMake\bin\ctest.exe" -C $(Configuration) -DCTEST_BUILD_CONFIGURATION:STRING=$(Configuration) -DCTEST_MODEL:STRING=Experimental -DSCRIPT_ARGS_FILE:FILEPATH=C:/D/N/E-0/SlicerRT-upload-command-args.cmake -S C:/D/N/Slicer-0/Extensions/CMake/SlicerBlockBuildPackageAndUploadExtension.cmake -VV
...

How ctest.exe knows that ctest_test failed (from parsing XML output file)? How it could be prevented to return with error code?

For reference: SlicerRT ticket created for the workaround that needs to be applied due to this issue
https://app.assembla.com/spaces/slicerrt/tickets/838-failing-slicerrt-tests-cause-dependent-extensions-not-to-build/details

This should be fixed by

http://viewvc.slicer.org/viewvc.cgi/Slicer4?view=revision&revision=25440
http://viewvc.slicer.org/viewvc.cgi/Slicer4?view=revision&revision=25439
http://viewvc.slicer.org/viewvc.cgi/Slicer4?view=revision&revision=25438

For future reference the associated github topic as:

https://github.com/Slicer/Slicer/pull/605

I re-enabled the failing SlicerRT tests to see if this change fixes the issue and we get dependent extensions to build even with the failing tests.

Unfortunately the windows build for dependent tests still does not start if any test fails.

I'm increasing the priority of this issue, as no progress has been made over the lsat four months, and it considerably decreases the usability of the extension dependency mechanism.

Thanks for the update.

I wonder how different is the SlicerRt extension configuration from the current tests.

The extension build system tests are added here:

https://github.com/Slicer/Slicer/blob/d7f501e925831b48393214c81a1533d316a810dd/Extensions/CMake/Testing/CMakeLists.txt#L65-L79

And for all test cases, extension B and C depend on extension A which has one failing test.
See https://github.com/Slicer/Slicer/blob/d7f501e925831b48393214c81a1533d316a810dd/Extensions/CMake/Testing/SlicerExtensionBuildSystemTest.py#L283-L289

A mock web server also confirm that all CDash and Midas queries are submitted as expected.

Is SlicerRt test crashing/hanging forever and preventing the dependent extensions from being built ?

Thanks for investigating! Okay, so the issue is not so trivial and seems SlicerRT specific.
Andras also wanted to create a minimal test case, but it worked for him as well, and we didn't know how to proceed. There must be something in how SlicerRT is configured, I agree.

I don't think that the any test hangs or crashes. If I have a module widget generic test fail because the module is wider than the threshold it's checked against, the dependent extensions don't build.

See https://github.com/Slicer/Slicer/pull/768

This should be fixed in 26223
See http://viewvc.slicer.org/viewvc.cgi/Slicer4?view=revision&revision=26223

Unfortunately nothing changed in this manner. I re-enabled the SlicerRT tests that failed or were prone to failing, and the dependent extension builds do not start on Windows, for example
http://slicer.cdash.org/index.php?project=Slicer4&filtercount=1&showfilters=1&field1=label&compare1=63&value1=GelDosimetryAnalysis

I believe this has been fixed in r26291
See http://viewvc.slicer.org/viewvc.cgi/Slicer4?view=revision&revision=26291

Looking at the dashboard of August 22nd, the GelDosimetry is configured, built and tested.

See http://slicer.cdash.org/index.php?project=Slicer4&date=2017-08-22&filtercount=1&showfilters=1&field1=label&compare1=63&value1=GelDosimetryAnalysis

This is great! Thanks a lot for fixing this.

Sorry I just saw this update somehow. I confirm that the dependent extensions build. Thanks a lot for the fix! Closing the issue.

I have to reopen this issue. I re-enabled the tests that fail (generic tests with module dependencies) or tend to fail (very long python self test), and the dependent extensions son't build on Windows. See this dashboard filtered to SlicerRT and its dependent extensions. You'll see that SlicerRT builds on Windows, but none of the dependencies build on Windows.
http://slicer.cdash.org/index.php?project=Slicer4&date=2017-11-21&filtercount=4&showfilters=1&filtercombine=or&field1=label&compare1=63&value1=SlicerRT&field2=label&compare2=63&value2=GelDosimetry&field3=label&compare3=63&value3=FilmDosimetry&field4=label&compare4=63&value4=SegmentRegistration

Another occurrence of this issue:
http://slicer.cdash.org/index.php?project=Slicer4&display=project&filtercombine=and&filtercombine=and&filtercombine=or&filtercombine=or&filtercombine=or&filtercombine=or&filtercombine=or&filtercombine=or&filtercount=4&showfilters=1&filtercombine=or&field1=label&compare1=63&value1=SlicerRT&field2=label&compare2=63&value2=GelDosimetry&field3=label&compare3=63&value3=FilmDosimetry&field4=label&compare4=63&value4=SegmentRegistration

I confirm that this issue still exists today, in latest master version of Slicer on (SlicerPreview dashboard)!

MatlabBridge extension did not get build on Windows, it was simply missing from the dashboard (but it was built successfully on linux and mac):

http://slicer.cdash.org/index.php?project=SlicerPreview&date=2018-12-08&filtercount=2&showfilters=1&filtercombine=or&field1=buildname&compare1=63&value1=SlicerOpenIGTLink&field2=buildname&compare2=63&value2=MatlabBridge

Then, we disabled failing tests of SlicerOpenIGTLink extension (no other change was made in Slicer core or any extensions) and as a result, MatlabBridge got built successfully and appeared on the dashboard:

http://slicer.cdash.org/index.php?project=SlicerPreview&date=2018-12-09&filtercount=2&showfilters=1&filtercombine=or&field1=buildname&compare1=63&value1=SlicerOpenIGTLink&field2=buildname&compare2=63&value2=MatlabBridge

True. I thought this was fixed but indeed, this week a test failed in SlicerRT for a day and that day the dependencies didn't build:
http://slicer.cdash.org/index.php?project=SlicerPreview&date=2018-12-05&filtercount=4&showfilters=1&filtercombine=or&field1=label&compare1=63&value1=SlicerRT&field2=label&compare2=63&value2=GelDosimetry&field3=label&compare3=63&value3=FilmDosimetry&field4=label&compare4=63&value4=SegmentRegistration

A similar error is happening with the Sequences extension, without any test failures @jcfr. Have retargeted this to 4.10.1

It's worth noting that none of the tests in Sequences are failing, although there is an error message in one (http://slicer.cdash.org/testDetails.php?test=9369351&build=1465199).

Extensions that are dependent on Sequences are still being built by factory.perklab (http://slicer.cdash.org/buildSummary.php?buildid=1464935, http://slicer.cdash.org/buildSummary.php?buildid=1464936)

I changed the error message to debug in Sequences (https://github.com/SlicerRt/Sequences/commit/9b07bf1307e1667f5cc904e8809c2b0f1feb0c32), and now all dependent extensions are building again.

Another occurrence (where we had to disable a test to allow dependent extensions to be built):
https://github.com/SlicerIGT/SlicerIGT/pull/189

Thanks for the note, I will have a look before cutting the 4.10.1 release.

For SlicerIGT test, I noticed that the dashboard had a pop asking to add a firewall rule for one of the test. I suspect this is what caused the blockage.

@pinter @arankin @lassoan Is it possible to isolate the exact test failure leading to the dependent extensions not being built ?

The most specific that we know so far is that if a test logs an error message (vtkErrorMacro and as far as I remember, even vtkWarningMacro) then dependent extensions are not even attempted to be built. Changing it to debug message (vtkDebugMacro) fixes the issue.

I was able to update the ExtensionBuildSystem test to reproduce the problem. Now working on a fix.
See https://github.com/Slicer/Slicer/compare/master...jcfr:4247-extension-build-system-windows-dependent-extension

This should now be fixed in r27944
See http://viewvc.slicer.org/viewvc.cgi/Slicer4?view=revision&revision=27944

To prevent extension test displaying string "ERROR:" from being considered
as build failure and causing the overall build of all extensions to fail,
this commit updates the execution of the build wrapper script to ensure
its standard output and error streams are logged to file.

Note that this was only observed when using Visual Studio generator on
Windows because in that case, there are no CTest launcher.
See https://cmake.org/cmake/help/latest/module/CTestUseLaunchers.html

This commit also improves the extension build system tests to avoid future
regressions.