View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0003177Slicer4Core: CLI infrastructurepublic2013-06-18 13:162013-06-20 11:16
Reporterhans_meine Assigned Tomillerjv  
PrioritynormalSeverityminorReproducibilityalways
Status assignedResolutionopen 
Product VersionSlicer 4.2.1 
Target VersionFixed in Version 
Summary0003177: CLI modules' XML output does not comply to specs
Description

I have written a parser in Python for the --xml output of the CLI modules that come with Slicer.

Below, you find a number of warnings, grouped by module, that indicate where there's a mismatch between their output and CTK/Libs/CommandLineModules/Core/Resources/ctkCmdLineModule.xsd, after which I modeled my parser.

Actually, this already assumes that https://github.com/jcfr/CTK/tree/279-tweak-cmdlinemodule-library-for-slicer-integration gets merged, in which 'transform', 'table', and 'measurement' elements are allowed again.

I think we should not assume that all needs to get fixed in the XML, e.g. I wonder if a 'step' within 'constraints' could be made optional in the .xsd instead.

Also, let me point out that OtsuThresholdImageFilter is the only module with an xmlns attribute:
<executable xmlns="http://slicer.org/module-description/1.0#&quot; version="1.0">
This made it quite hard for me, since it completely changes the DOM I see with Python's elementTree library. Thus, I vote for removing the namespace from the XML.

Additional Information

=== ACPCTransform ===
Element 'index' within 'executable' not parsed
Required element 'description' not found within 'parameters'

=== AffineRegistration ===
Element 'index' within 'executable' not parsed

=== BRAINSDemonWarp ===
Required element 'label' not found within 'boolean'
Required element 'label' not found within 'boolean'
Required element 'label' not found within 'integer'

=== BRAINSFit ===
Required element 'description' not found within 'parameters'
Required element 'description' not found within 'parameters'
Required element 'description' not found within 'parameters'
Required element 'description' not found within 'parameters'
Required element 'description' not found within 'parameters'
Required element 'description' not found within 'parameters'
Required element 'description' not found within 'parameters'
Required element 'label' not found within 'double'
Required element 'label' not found within 'double'
Required element 'label' not found within 'boolean'
Required element 'label' not found within 'boolean'
Required element 'description' not found within 'parameters'

=== BRAINSFitEZ ===
Required element 'description' not found within 'parameters'
Required element 'description' not found within 'parameters'
Required element 'label' not found within 'double'
Required element 'label' not found within 'double'
Required element 'label' not found within 'boolean'
Required element 'label' not found within 'boolean'

=== BRAINSResample ===
Required element 'description' not found within 'parameters'
Required element 'description' not found within 'parameters'

=== BRAINSROIAuto ===
Required element 'description' not found within 'parameters'

=== BRAINSTransformConvert ===
Required element 'description' not found within 'image'
Required element 'label' not found within 'string-enumeration'
Required element 'description' not found within 'image'

=== BSplineDeformableRegistration ===
Element 'index' within 'executable' not parsed

=== BSplineToDeformationField ===
Required element 'description' not found within 'transform'
Required element 'description' not found within 'image'
Required element 'description' not found within 'image'

=== CastScalarVolume ===
Required element 'description' not found within 'parameters'

=== CreateDICOMSeries ===
More than one <label> found within 'parameters' (using only first)

=== CurvatureAnisotropicDiffusion ===
Element 'index' within 'executable' not parsed
More than one <description> found within 'double' (using only first)
More than one <description> found within 'double' (using only first)

=== DiffusionWeightedVolumeMasking ===
Required element 'label' not found within 'parameters'
Required element 'description' not found within 'parameters'

=== DTIimport ===
Required element 'label' not found within 'parameters'
Required element 'description' not found within 'parameters'

=== DWIConvert ===
Required element 'label' not found within 'string-enumeration'

=== ExecutionModelTour ===
Required element 'name' not found within 'file'
Required element 'name' not found within 'file'
Required element 'name' not found within 'directory'
Required element 'name' not found within 'image'
Required element 'name' not found within 'image'
Required element 'name' not found within 'transform'
Required element 'name' not found within 'transform'
Required element 'description' not found within 'parameters'
Required element 'description' not found within 'parameters'
Required element 'description' not found within 'parameters'

=== ExpertAutomatedRegistration ===
Element 'index' within 'executable' not parsed

=== GaussianBlurImageFilter ===
Element 'index' within 'executable' not parsed

=== GradientAnisotropicDiffusion ===
Element 'index' within 'executable' not parsed

=== GrayscaleModelMaker ===
Required element 'step' not found within 'constraints'

=== LinearRegistration ===
Element 'index' within 'executable' not parsed

=== MedianImageFilter ===
Element 'index' within 'executable' not parsed

=== ModelMaker ===
Required element 'label' not found within 'table'
Required element 'step' not found within 'constraints'
Required element 'description' not found within 'parameters'

=== MultiResolutionAffineRegistration ===
Required element 'description' not found within 'parameters'

=== N4ITKBiasFieldCorrection ===
Element 'index' within 'executable' not parsed
Required element 'description' not found within 'parameters'

=== PETStandardUptakeValueComputation ===
Required element 'label' not found within 'table'

=== ResampleDTIVolume ===
Required element 'description' not found within 'transform'
Required element 'description' not found within 'parameters'
Required element 'description' not found within 'parameters'
Required element 'description' not found within 'parameters'
Required element 'description' not found within 'parameters'

=== ResampleScalarVectorDWIVolume ===
More than one <category> found within 'executable' (using only first)
Required element 'description' not found within 'transform'
Required element 'description' not found within 'parameters'
Required element 'description' not found within 'parameters'
Required element 'description' not found within 'parameters'

=== RigidRegistration ===
Element 'index' within 'executable' not parsed

=== SimpleRegionGrowingSegmentation ===
Required element 'step' not found within 'constraints'
Required element 'step' not found within 'constraints'

=== ThresholdScalarVolume ===
Required element 'description' not found within 'parameters'

=== VBRAINSDemonWarp ===
Required element 'label' not found within 'boolean'
Required element 'label' not found within 'boolean'
Required element 'label' not found within 'integer'

TagsSummer AHM 2013

Activities

millerjv

millerjv

2013-06-18 13:53

developer   ~0008780

There is no "official" xsd for the ModuleDescriptionParser. Several attempts have been made to retrofit an xsd but they have all been incomplete.

To me, it is not that the CLI modules do not conform to the xsd but that the xsd does not conform to the parser in ModuleDescriptionParser.
hans_meine

hans_meine

2013-06-18 14:06

reporter   ~0008781

Thanks, Jim. Nevertheless, many of the above warnings indicate improper XML descriptions.

E.g. I think every parameter should have a name (identifier), no? (OK, I could probably fall back to the longflag.)
Also, the "more than one <foo>" warnings are certainly relevant.

Otherwise, I agree – as I said, this is not a "the XML is wrong" report, but a "there's a mismatch" report. At the same time, I would not agree that this can be interpreted as "[just] the xsd is wrong". Both seem to need fixing.
hans_meine

hans_meine

2013-06-18 14:12

reporter   ~0008782

Also, the "ModelMaker" module has an "aggregate" attribute for the geometry parameter, which is not in the .xsd (not even in the above branched one).
hans_meine

hans_meine

2013-06-18 15:50

reporter   ~0008783

We should also document and check the format of field values, e.g. the file extensions being a comma-separated list of exensions starting with a dot, or the contributors being a comma-separated list, not e.g. freeform-text like "This tool was developed by Hans J. Johnson and Greg Harris." (seen with VBRAINSDemonWarp).

At least the latter is a requirement for MeVisLab, which currently complains about illegal characters in the author tag.

Furthermore, the description sometimes contains HTML, sometimes plain text with double newlines separating paragraphs.
hans_meine

hans_meine

2013-06-19 19:06

reporter   ~0008792

I also found that the flag/longflag sometimes start with '-' / '--', respectively, but sometimes just contain the flag name without prefix. It should be documented what is expected. I vote for the inclusion of dashes (or similar) in the XML, because that could eventually be used to use other syntax, like single-dash -longflags, or Windows-style /f flags. (It could even be used for programs that don't prefix their options at all.)
hans_meine

hans_meine

2013-06-20 11:16

reporter   ~0008797

I am not sure if I should file a separate issue about this, but another problem with the spec is this:

  &lt;!-- either 'flag' or 'longflag' (or both) or 'index' are required -->
  &lt;xsd:choice>
    &lt;xsd:annotation>
      &lt;xsd:documentation>You must either specify &quot;flag&quot; or &quot;longflag&quot; (or both) or &quot;index&quot;.&lt;/xsd:documentation>
    &lt;/xsd:annotation>
  ...

That is not correct for output parameters. At least the ExecutionModelTour specifies parameters with channel="output" which do not need index or flags; they are put into an ascii output file with key=value pairs. The key in these files is just the parameter name; no flag involved.

Another issue is that this obviously needs to be documented, because so far only Jim seemed to know about that. :-)

Issue History

Date Modified Username Field Change
2013-06-18 13:16 hans_meine New Issue
2013-06-18 13:16 hans_meine Status new => assigned
2013-06-18 13:16 hans_meine Assigned To => millerjv
2013-06-18 13:53 millerjv Note Added: 0008780
2013-06-18 14:06 hans_meine Note Added: 0008781
2013-06-18 14:12 hans_meine Note Added: 0008782
2013-06-18 15:50 hans_meine Note Added: 0008783
2013-06-19 19:06 hans_meine Note Added: 0008792
2013-06-19 21:54 jcfr Tag Attached: Summer AHM 2013
2013-06-20 11:16 hans_meine Note Added: 0008797