View Issue Details

IDProjectCategoryView StatusLast Update
0002249Slicer4Core: Scripting (Wrapping, Python)public2012-08-16 12:28
Reporterbmoloney Assigned Tojcfr  
PrioritynormalSeveritymajorReproducibilityalways
Status closedResolutionfixed 
Product VersionSlicer 4.1.0 
Target VersionSlicer 4.2.0Fixed in VersionSlicer 4.2.0 
Summary0002249: numpy.linalg.det does not work
Description

This occurs in my own build, but also the 4.1 binary release.

import numpy as np
test = np.eye(3)
np.linalg.det(test)
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/home/moloney/Software/slicer/Slicer-SuperBuild-Debug/python-build/lib/python2.6/site-packages/numpy/linalg/linalg.py", line 1503, in det
raise TypeError, "Illegal input to Fortran routine"
TypeError: Illegal input to Fortran routine

TagsNo tags attached.

Relationships

duplicate of 0002193 closedjcfr Python NumPy crash 
related to 0002276 closedjcfr Fix numpy build error where gfortran is installed 

Activities

bmoloney

bmoloney

2012-06-22 07:58

reporter   ~0004908

I also installed nose and tried running the numpy test suite. This seems to show multiple issues in numpy.linalg and actually crashes while running these tests.

The relevant output:

test_cdouble (test_linalg.TestCond2) ... ERROR
test_csingle (test_linalg.TestCond2) ... ERROR
test_double (test_linalg.TestCond2) ... ERROR
test_empty (test_linalg.TestCond2) ... ok
Check that matrix type is preserved. ... ERROR
Check that matrix type is preserved. ... ERROR
test_nonarray (test_linalg.TestCond2) ... ERROR
test_single (test_linalg.TestCond2) ... ERROR
error: [/home/moloney/Software/slicer/Slicer-SuperBuild-Debug/Slicer-build/bin/./SlicerApp-real] exit abnormally - Report the problem.

jcfr

jcfr

2012-06-30 23:21

administrator   ~0005017

Last edited: 2012-06-30 23:23

After investigating, it resumes it is pointless to compile CLAPACK and build numpy against it. Indeed, the modules "blas_lite.c" and "dlapack_lite.c" already bundle part of CLAPACK that are required by "linalg" numpy module.

While looking for the root cause of the problem, I also realized that CLAPACK can't be linked against numpy. The "f2c.h" bundled within clapack-3.2.1-CMAKE.tgz is different from the "f2c.h" available in "numpy/linalg" folder. For example, the typedefs used to specify "integer" or "uinteger" are different.

In clapack/INCLUDE/f2c.h => "integer" is defined as "long int"
In numpy/linalg/f2c.h => "integer" is defined as "int"

Within the python module "lapack_litemodule.c", we can read that the function signature used to specify the function to link against is of the form:

extern int foo_(char arg1, char arg2, int *n)

whereas in CLapack, the same function is defined as:

int foo_(char arg1, char arg2, integer *n) // integer being "long int"

This difference in type was causing the error reported in the bug.
While working on that issue, I also created a small project illustrating the problem. See https://github.com/jcfr/CFunclash#readme

This apply at least from numpy 1.4.1 to 1.6.2

The following topic resolved the issue on linux: https://github.com/jcfr/Slicer/tree/2249-fix-numpy-runtime-issue

Note: Will have to test it on both windows and macosx.

Associated commit: https://github.com/jcfr/Slicer/commit/fd21352c9212f64c5a1b40b2f3356e3a0f5dc17f

On linux 64bit (Ubuntu 10.04), the following numpy test suite run without
any errors:

=> nose version 1.1.2
=> gcc 4.4.3

numpy.test()
[...]
Ran 2483 tests in 10.194s

OK (KNOWNFAIL=4, SKIP=12)

Details (package by package)

np.linalg.test()
Running unit tests for numpy.linalg
NumPy version 1.4.1
[...]
Ran 116 tests in 0.203s

OK
<nose.result.TextTestResult run=116 errors=0 failures=0>

np.ma.test()
Running unit tests for numpy.ma
NumPy version 1.4.1
[...]
Ran 282 tests in 1.011s

OK
<nose.result.TextTestResult run=282 errors=0 failures=0>

np.polynomial.test()
Running unit tests for numpy.polynomial
NumPy version 1.4.1
[...]
Ran 86 tests in 0.324s

OK
<nose.result.TextTestResult run=86 errors=0 failures=0>

np.random.test()
Running unit tests for numpy.random
NumPy version 1.4.1
[...]
Ran 11 tests in 0.020s

OK
<nose.result.TextTestResult run=11 errors=0 failures=0>

np.fft.test()
Running unit tests for numpy.fft
NumPy version 1.4.1
[...]
Ran 5 tests in 0.023s

OK
<nose.result.TextTestResult run=5 errors=0 failures=0>

np.core.test()
Running unit tests for numpy.core
NumPy version 1.4.1
[...]
Ran 861 tests in 2.181s

OK (KNOWNFAIL=4, SKIP=11)
<nose.result.TextTestResult run=861 errors=0 failures=0>

np.linalg.test()
Running unit tests for numpy.linalg
NumPy version 1.4.1
[...]
Ran 116 tests in 0.184s

OK
<nose.result.TextTestResult run=116 errors=0 failures=0>

np.matrixlib.test()
Running unit tests for numpy.matrixlib
NumPy version 1.4.1
[...]
Ran 38 tests in 0.077s

OK
<nose.result.TextTestResult run=38 errors=0 failures=0>

np.lib.test()
Running unit tests for numpy.lib
NumPy version 1.4.1
[...]
Ran 994 tests in 6.313s

OK (SKIP=1)

numpy.oldnumeric.test()
[...]
Ran 2 tests in 0.020s

OK

numpy.testing.test()
[...]
Ran 69 tests in 0.128s

OK

jcfr

jcfr

2012-06-30 23:32

administrator   ~0005019

bmoloney> Would be great if you could try the topic I referenced in the previous note. Thanks

fedorov

fedorov

2012-07-01 14:03

developer   ~0005022

Tested with https://github.com/jcfr/Slicer/tree/2249-fix-numpy-runtime-issue.

I followed the steps in the original report, and there was no error.

import numpy as np
test=np.eye(3)
np.linalg.det(test)
1.0

fedorov

fedorov

2012-07-01 14:19

developer   ~0005024

I also tried to do the nose tests, but that did not work:

[fedorov@bear nose-1.1.2] pwd
/Users/fedorov/github/Slicer-Superbuild/python-build/nose-1.1.2
[fedorov@bear nose-1.1.2] ../bin/python

import numpy as np
np.test()
Running unit tests for numpy
NumPy version 1.4.1
NumPy is installed in /Users/fedorov/github/Slicer-Superbuild/python-build/lib/python2.6/site-packages/numpy
Python version 2.6.6 (r266:84292, Jul 1 2012, 17:09:31) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
nose version 1.1.2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/fedorov/github/Slicer-Superbuild/python-build/lib/python2.6/site-packages/numpy/testing/nosetester.py", line 335, in test
t = NumpyTestProgram(argv=argv, exit=False, plugins=plugins)
File "nose/core.py", line 118, in init
**extra_args)
File "/Users/fedorov/github/Slicer-Superbuild/python-build/lib/python2.6/unittest.py", line 816, in init
self.parseArgs(argv)
File "nose/core.py", line 135, in parseArgs
self.config.configure(argv, doc=self.usage())
File "nose/config.py", line 276, in configure
options, args = self._parseArgs(argv, cfg_files)
File "nose/config.py", line 264, in _parseArgs
return parser.parseArgsAndConfigFiles(argv[1:], cfg_files)
File "nose/config.py", line 132, in parseArgsAndConfigFiles
self._applyConfigurationToValues(self._parser, config, values)
File "nose/config.py", line 118, in _applyConfigurationToValues
name=name, filename=filename)
File "nose/config.py", line 261, in warn_sometimes
raise ConfigError(msg)
nose.config.ConfigError: Error reading config file 'setup.cfg': no such option 'with-doctest'

jcfr

jcfr

2012-07-01 14:34

administrator   ~0005025

Andriy> Edit ~/nose.cfg and remove the like having "with doc test". That should do it.

fedorov

fedorov

2012-07-01 14:41

developer   ~0005028

The file I had to edit was nose-1.1.2/setup.cfg

[fedorov@bear Slicer-Superbuild] ./python-build/bin/python
Python 2.6.6 (r266:84292, Jul 1 2012, 17:09:31)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import numpy as np
import nose
np.test()
Running unit tests for numpy
NumPy version 1.4.1
NumPy is installed in /Users/fedorov/github/Slicer-Superbuild/python-build/lib/python2.6/site-packages/numpy
Python version 2.6.6 (r266:84292, Jul 1 2012, 17:09:31) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
nose version 1.1.2
...

Ran 2483 tests in 4.686s

OK (KNOWNFAIL=4, SKIP=1)
<nose.result.TextTestResult run=2483 errors=0 failures=0>

np.linalg.test()
Running unit tests for numpy.linalg
NumPy version 1.4.1
NumPy is installed in /Users/fedorov/github/Slicer-Superbuild/python-build/lib/python2.6/site-packages/numpy
Python version 2.6.6 (r266:84292, Jul 1 2012, 17:09:31) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
nose version 1.1.2
S...................................................................................................................

Ran 116 tests in 0.109s

OK (SKIP=1)
<nose.result.TextTestResult run=116 errors=0 failures=0>

np.ma.test()
Running unit tests for numpy.ma
NumPy version 1.4.1
NumPy is installed in /Users/fedorov/github/Slicer-Superbuild/python-build/lib/python2.6/site-packages/numpy
Python version 2.6.6 (r266:84292, Jul 1 2012, 17:09:31) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
nose version 1.1.2
..........................................................................................................................................................................................................................................................................................

Ran 282 tests in 0.775s

OK
<nose.result.TextTestResult run=282 errors=0 failures=0>

np.polynomial.test()
Running unit tests for numpy.polynomial
NumPy version 1.4.1
NumPy is installed in /Users/fedorov/github/Slicer-Superbuild/python-build/lib/python2.6/site-packages/numpy
Python version 2.6.6 (r266:84292, Jul 1 2012, 17:09:31) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
nose version 1.1.2
......................................................................................

Ran 86 tests in 0.272s

OK
<nose.result.TextTestResult run=86 errors=0 failures=0>

np.random.test()
Running unit tests for numpy.random
NumPy version 1.4.1
NumPy is installed in /Users/fedorov/github/Slicer-Superbuild/python-build/lib/python2.6/site-packages/numpy
Python version 2.6.6 (r266:84292, Jul 1 2012, 17:09:31) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
nose version 1.1.2
...........

Ran 11 tests in 0.015s

OK
<nose.result.TextTestResult run=11 errors=0 failures=0>

np.core.test()
Running unit tests for numpy.core
NumPy version 1.4.1
NumPy is installed in /Users/fedorov/github/Slicer-Superbuild/python-build/lib/python2.6/site-packages/numpy
Python version 2.6.6 (r266:84292, Jul 1 2012, 17:09:31) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
nose version 1.1.2
...

Ran 861 tests in 1.247s

OK (KNOWNFAIL=4)
<nose.result.TextTestResult run=861 errors=0 failures=0>

np.matrixlib.test()
Running unit tests for numpy.matrixlib
NumPy version 1.4.1
NumPy is installed in /Users/fedorov/github/Slicer-Superbuild/python-build/lib/python2.6/site-packages/numpy
Python version 2.6.6 (r266:84292, Jul 1 2012, 17:09:31) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
nose version 1.1.2
......................................

Ran 38 tests in 0.062s

OK
<nose.result.TextTestResult run=38 errors=0 failures=0>

np.lib.test()
Running unit tests for numpy.lib
NumPy version 1.4.1
NumPy is installed in /Users/fedorov/github/Slicer-Superbuild/python-build/lib/python2.6/site-packages/numpy
Python version 2.6.6 (r266:84292, Jul 1 2012, 17:09:31) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
nose version 1.1.2
...

Ran 994 tests in 1.130s

OK
<nose.result.TextTestResult run=994 errors=0 failures=0>

np.oldnumeric.test()
Running unit tests for numpy.oldnumeric
NumPy version 1.4.1
NumPy is installed in /Users/fedorov/github/Slicer-Superbuild/python-build/lib/python2.6/site-packages/numpy
Python version 2.6.6 (r266:84292, Jul 1 2012, 17:09:31) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
nose version 1.1.2
..

Ran 2 tests in 0.017s

OK
<nose.result.TextTestResult run=2 errors=0 failures=0>

np.testing.test()
Running unit tests for numpy.testing
NumPy version 1.4.1
NumPy is installed in /Users/fedorov/github/Slicer-Superbuild/python-build/lib/python2.6/site-packages/numpy
Python version 2.6.6 (r266:84292, Jul 1 2012, 17:09:31) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
nose version 1.1.2
.....................................................................

Ran 69 tests in 0.103s

OK
<nose.result.TextTestResult run=69 errors=0 failures=0>

jcfr

jcfr

2012-07-01 18:29

administrator   ~0005029

Fixed in r20539
See http://viewvc.slicer.org/viewvc.cgi/Slicer4?view=revision&amp;revision=20539

jcfr

jcfr

2012-07-27 15:42

administrator   ~0005411

Dear issue reporter,

Good news :) Slicer developers SOLVED the problem you reported - YOU now need to VERIFY and CLOSE this issue.

bmoloney

bmoloney

2012-08-16 12:28

reporter   ~0005561

Sorry for my delayed response. I can confirm the issue is resolved now.

Thanks so much for the hard work!

Issue History

Date Modified Username Field Change
2012-06-22 05:48 bmoloney New Issue
2012-06-22 05:48 bmoloney Status new => assigned
2012-06-22 05:48 bmoloney Assigned To => jcfr
2012-06-22 07:58 bmoloney Note Added: 0004908
2012-06-30 23:21 jcfr Note Added: 0005017
2012-06-30 23:23 jcfr Note Edited: 0005017
2012-06-30 23:30 jcfr Relationship added related to 0002276
2012-06-30 23:32 jcfr Note Added: 0005019
2012-06-30 23:32 jcfr Status assigned => feedback
2012-06-30 23:39 jcfr Relationship added duplicate of 0002193
2012-07-01 14:03 fedorov Note Added: 0005022
2012-07-01 14:19 fedorov Note Added: 0005024
2012-07-01 14:34 jcfr Note Added: 0005025
2012-07-01 14:41 fedorov Note Added: 0005028
2012-07-01 18:29 jcfr Note Added: 0005029
2012-07-01 18:29 jcfr Status feedback => resolved
2012-07-01 18:29 jcfr Fixed in Version => Slicer 4.2.0 - Sept 1st 2012
2012-07-01 18:29 jcfr Resolution open => fixed
2012-07-01 18:30 jcfr Target Version => Slicer 4.2.0 - Sept 1st 2012
2012-07-27 15:42 jcfr Note Added: 0005411
2012-08-16 12:28 bmoloney Note Added: 0005561
2012-08-16 12:28 bmoloney Status resolved => closed