View Issue Details

IDProjectCategoryView StatusLast Update
0001806Slicer4Core: Scripting (Wrapping, Python)public2012-08-21 09:48
Reporterfedorov Assigned Tojcfr  
PrioritynormalSeverityminorReproducibilityalways
Status closedResolutionfixed 
Product Version 
Target VersionSlicer 4.1.0Fixed in VersionSlicer 4.1.0 
Summary0001806: Memory management in Python is confusing
Description

Steve Pieper via bwh.harvard.edu
Mar 11 (3 days ago)

to Julien, Andriy, Jean-Christophe
There are definitely some operations that are leading to memory leaks, but I haven't had a chance to look into it - I was planning to do this once I finish porting the editor to pure python.

But now that you mention it there seems to be something with the python/qt layer at work.

For example, the SampleData module does not generate a leak, even if I add something like self.image = vtk.vtkImageData() to the init. This reference gets cleaned up automatically.

However if I just enter the Endoscopy module and exit I get these leaks which seem to be related to the qSlicerNodeComboBox not releasing the vtkMRMLCameraNode:

vtkDebugLeaks has detected LEAKS!
Class "vtkPerspectiveTransform" has 2 instances still around.
Class "vtkObservation" has 1 instance still around.
Class "vtkMatrix4x4" has 14 instances still around.
Class "vtkSimpleTransform" has 5 instances still around.
Class "vtkObserverManager" has 1 instance still around.
Class "vtkEventBroker" has 1 instance still around.
Class "vtkOpenGLCamera" has 1 instance still around.
Class "vtkTransform" has 3 instances still around.
Class "vtkMRMLCameraNode" has 1 instance still around.
Class "vtkTimerLog" has 1 instance still around.
Class "vtkCommand or subclass" has 5 instances still around.

I'll try to look into this some time, but if anyone knows what's going on I'd be glad to hear it ;)

-Steve

On Sat, Mar 10, 2012 at 9:43 PM, Julien Finet <julien.finet@kitware.com> wrote:
Hi Andrei,

See below my answers

On Sat, Mar 10, 2012 at 8:23 PM, Andriy Fedorov <fedorov@bwh.harvard.edu> wrote:
Julien,

I looked into this issue

1) the reason tests were passing on my machine (I tested on mac and
linux) is when I run them locally, memory leaks are not treated as
errors, so the tests pass and you need -VV to see the leaks.
I forgot about it, but yes, it's one oddity of VTK, it has been fixed in VTK 5.10.

2) I realized I do not know what are the rules regarding memory
allocation in python to prevent memory leaks.

Example 1:

Just adding this line to init()

self.__logic = slicer.modulelogic.vtkSlicerMultiVolumeExplorerLogic()

leads to the following leaks:

301: Class "vtkObserverManager" has 2 instances still around.
301: Class "vtkSlicerMultiVolumeExplorerLogic" has 1 instance still around.
301: Class "vtkCommand or subclass" has 2 instances still around.

In this module, I do not use __logic, so this is easy to fix, but I
don't understand this.

Example 2:

To get rid of the remaining leaks in the generic tests, I have to
comment out the following lines from my setup():

self.chart = self.chartView.chart()
self.chartTable = vtk.vtkTable()
self.
xArray = vtk.vtkFloatArray()
self.yArray = vtk.vtkFloatArray()
self.
xArray.SetName('X')
self.yArray.SetName('Y')
self.
chartTable.AddColumn(self.xArray)
self.
chartTable.AddColumn(self.__yArray)

Why is this? Does this mean I need to manually manage references to
all VTK objects?

You earlier email about managing references deals with objects created
by factories ( http://massmail.spl.harvard.edu/public-archives/slicer-devel/2011/007513.html ),
and I do not recall any other guidelines about explicit reference
counting from python.

The bottom line is I need some guidelines regarding memory management
from python to fix these tests.

Interesting... Does that means that python can't instantiate VTK objects without making leaks ? I'm not enough aware of the python wrapping to give you an answer, I let the python experts answering you :-)

Julien.

On Thu, Mar 8, 2012 at 8:53 PM, Andriy Fedorov <fedorov@bwh.harvard.edu> wrote:

Yes, I have all these three flags set to ON. I do get leaks messages
when I close Slicer.

On Thu, Mar 8, 2012 at 7:11 PM, Julien Finet <julien.finet@kitware.com> wrote:

do you have Slicer_USE_VTK_DEBUG_LEAKS set to ON in the superbuild directory
?
and Slicer_USE_VTK_DEBUG_LEAKS and VTK_DEBUG_LEAKS to ON in the inner build
?
j.

On Thu, Mar 8, 2012 at 6:50 PM, Andriy Fedorov <fedorov@bwh.harvard.edu>
wrote:

Julien,

fyi, these tests pass on my mac, and I have timeout when I try to
access the links included in your email.

Test project /Users/fedorov/Slicer/Slicer4-SuperBuild/Slicer-build
Start 301: qSlicerMultiVolumeExplorerModuleGenericTest
1/3 Test 0000301: qSlicerMultiVolumeExplorerModuleGenericTest
................... Passed 8.77 sec
Start 302: qSlicerMultiVolumeExplorerModuleWidgetGenericTest
2/3 Test 0000302: qSlicerMultiVolumeExplorerModuleWidgetGenericTest
............. Passed 5.89 sec
Start 318: py_nomainwindow_qSlicerMultiVolumeImporterModuleGenericTest
3/3 Test 0000318: py_nomainwindow_qSlicerMultiVolumeImporterModuleGenericTest
... Passed 16.92 sec

100% tests passed, 0 tests failed out of 3

On Thu, Mar 8, 2012 at 6:38 PM, Andriy Fedorov <fedorov@bwh.harvard.edu>
wrote:

Yes, I should, I will try this week.

On Thu, Mar 8, 2012 at 5:09 PM, Julien Finet <julien.finet@kitware.com>
wrote:

Hi Andrei,

qSlicerMultiVolumeExplorerModuleGenericTest
http://slicer.cdash.org/testDetails.php?test=863239&amp;build=11504

and qSlicerMultiVolumeExplorerModuleWidgetGenericTest
http://slicer.cdash.org/testDetails.php?test=928324&amp;build=11504

are having memory leaks.

Can you eventually give a look ?

Thanks,
Julien.

TagsNo tags attached.

Activities

jcfr

jcfr

2012-03-21 00:51

administrator   ~0003898

Following r19669. Endoscopy module doesn't cause any leaks.
See http://viewvc.slicer.org/viewvc.cgi/Slicer4?view=revision&amp;revision=19669

fedorov

fedorov

2012-04-01 18:33

developer   ~0003941

I now have error in vtkMRMLScene.cxx, and the tests just crash.

Thread 0 Crashed: Dispatch queue: com.apple.main-thread
0 libSystem.B.dylib 0x00007fff87a9a0b6 kill + 10
1 libSystem.B.dylib 0x00007fff87b3a9f6 abort + 83
2 libSystem.B.dylib 0x00007fff87a52195 free + 128
3 libvtkCommon.5.10.dylib 0x0000000110fc9e39 vtkObjectBase::~vtkObjectBase() + 259 (vtkObjectBase.cxx:95)
4 libvtkCommon.5.10.dylib 0x0000000110fc9875 vtkObjectBase::UnRegisterInternal(vtkObjectBase, int) + 235 (vtkObjectBase.cxx:286)
5 libvtkCommon.5.10.dylib 0x0000000110ef2107 vtkCollection::UnRegister(vtkObjectBase
) + 45 (vtkCollection.cxx:361)
6 libMRMLCore.dylib 0x00000001082a1a2a vtkMRMLScene::RemoveNode(vtkMRMLNode) + 1496 (vtkMRMLScene.cxx:1373)
7 libMRMLCore.dylib 0x00000001082a1dee vtkMRMLScene::Clear(int) + 138 (vtkMRMLScene.cxx:480)
8 libMRMLCore.dylib 0x00000001082a1f15 vtkMRMLScene::SceneCallback(vtkObject
, unsigned long, void, void) + 69 (vtkMRMLScene.cxx:456)
9 libvtkCommon.5.10.dylib 0x0000000110eeb5c6 vtkCallbackCommand::Execute(vtkObject, unsigned long, void) + 68 (vtkCallbackCommand.cxx:43)
10 libvtkCommon.5.10.dylib 0x0000000110ec96a2 vtkSubjectHelper::InvokeEvent(unsigned long, void, vtkObject) + 1652 (vtkObject.cxx:605)
11 libvtkCommon.5.10.dylib 0x0000000110ec97ba vtkObject::InvokeEvent(unsigned long, void) + 58 (vtkObject.cxx:770)
12 libvtkCommon.5.10.dylib 0x0000000110fc872d vtkObject::UnRegisterInternal(vtkObjectBase
, int) + 771 (vtkObject.cxx:881)
13 libvtkCommon.5.10.dylib 0x0000000110fc963f vtkObjectBase::UnRegister(vtkObjectBase*) + 45 (vtkObjectBase.cxx:190)
14 libMRMLCore.dylib 0x000000010834d6f2 vtkObserverManager::SetObject(vtkObject*, vtkObject) + 826 (vtkObserverManager.cxx:107)
15 libMRMLLogic.dylib 0x000000010553c099 vtkMRMLAbstractLogic::SetMRMLSceneInternal(vtkMRMLScene) + 73 (vtkMRMLAbstractLogic.cxx:246)
16 libMRMLLogic.dylib 0x000000010553b403 vtkMRMLAbstractLogic::SetMRMLScene(vtkMRMLScene
) + 117 (vtkMRMLAbstractLogic.cxx:233)
17 libMRMLLogic.dylib 0x000000010553c803 vtkMRMLAbstractLogic::~vtkMRMLAbstractLogic() + 57 (vtkMRMLAbstractLogic.cxx:107)
18 libMRMLLogic.dylib 0x0000000105547419 vtkMRMLApplicationLogic::~vtkMRMLApplicationLogic() + 97 (vtkMRMLApplicationLogic.cxx:115)
19 libSlicerBaseLogic.dylib 0x00000001053e1531 vtkSlicerApplicationLogic::~vtkSlicerApplicationLogic() + 1237 (vtkSlicerApplicationLogic.cxx:282)
20 libvtkCommon.5.10.dylib 0x0000000110fc9875 vtkObjectBase::UnRegisterInternal(vtkObjectBase, int) + 235 (vtkObjectBase.cxx:286)
21 libvtkCommon.5.10.dylib 0x0000000110fc8746 vtkObject::UnRegisterInternal(vtkObjectBase
, int) + 796 (vtkObject.cxx:886)
22 libvtkCommon.5.10.dylib 0x0000000110fc963f vtkObjectBase::UnRegister(vtkObjectBase) + 45 (vtkObjectBase.cxx:190)
23 libvtkCommon.5.10.dylib 0x000000011101f81c vtkSmartPointerBase::~vtkSmartPointerBase() + 66 (vtkSmartPointerBase.cxx:64)
24 libqSlicerBaseQTGUI.dylib 0x0000000100524da1 vtkSmartPointer<vtkMRMLApplicationLogic>::~vtkSmartPointer() + 37 (vtkSmartPointer.h:27)
25 ...RMLDisplayableManager.dylib 0x0000000104794e4c vtkMRMLDisplayableManagerFactory::vtkInternal::~vtkInternal() + 42 (vtkMRMLDisplayableManagerFactory.cxx:45)
26 ...RMLDisplayableManager.dylib 0x0000000104793ff1 vtkMRMLDisplayableManagerFactory::~vtkMRMLDisplayableManagerFactory() + 71 (vtkMRMLDisplayableManagerFactory.cxx:79)
27 ...RMLDisplayableManager.dylib 0x00000001047957a5 vtkMRMLThreeDViewDisplayableManagerFactory::~vtkMRMLThreeDViewDisplayableManagerFactory() + 51 (vtkMRMLThreeDViewDisplayableManagerFactory.cxx:74)
28 libvtkCommon.5.10.dylib 0x0000000110fc9875 vtkObjectBase::UnRegisterInternal(vtkObjectBase
, int) + 235 (vtkObjectBase.cxx:286)
29 libvtkCommon.5.10.dylib 0x0000000110fc8746 vtkObject::UnRegisterInternal(vtkObjectBase, int) + 796 (vtkObject.cxx:886)
30 libvtkCommon.5.10.dylib 0x0000000110fc963f vtkObjectBase::UnRegister(vtkObjectBase
) + 45 (vtkObjectBase.cxx:190)
31 libvtkCommon.5.10.dylib 0x0000000110fc95b3 vtkObjectBase::Delete() + 37 (vtkObjectBase.cxx:135)
32 ...RMLDisplayableManager.dylib 0x000000010479551d vtkMRMLThreeDViewDisplayableManagerFactory::classFinalize() + 53 (vtkMRMLThreeDViewDisplayableManagerFactory.cxx:82)
33 ...RMLDisplayableManager.dylib 0x00000001047955d4 vtkMRMLThreeDViewDisplayableManagerFactoryInitialize::~vtkMRMLThreeDViewDisplayableManagerFactoryInitialize() + 62 (vtkMRMLThreeDViewDisplayableManagerFactory.cxx:82)
34 ...RMLDisplayableManager.dylib 0x00000001047a5d45
tcf_1 + 40 (vtkMRMLThreeDViewDisplayableManagerFactory.h:72)
35 libSystem.B.dylib 0x00007fff87a5e374 __cxa_finalize + 203
36 libSystem.B.dylib 0x00007fff87a5e28c exit + 18
37 MultiVolumeExplorerCxxTests 0x0000000100023dff start + 59

fedorov

fedorov

2012-04-01 18:34

developer   ~0003942

here's the error:

80: ERROR: In /Users/fedorov/Slicer/Slicer4/Libs/MRML/Core/vtkMRMLScene.cxx, line 1364
80: vtkMRMLScene (0x11292eb90): RemoveNode: class: vtkObjectBase name:Generic Warning: In /Users/fedorov/Slicer/Slicer4-SuperBuild/VTK/Common/vtkDebugLeaks.cxx, line 295

fedorov

fedorov

2012-04-01 18:42

developer   ~0003943

Sorry, forgot to mention: this is happening when I run tests for MultiVolumeExplorer.

jcfr

jcfr

2012-04-01 19:11

administrator   ~0003944

Will send you a pull request in the coming 5/10 minutes :)

jcfr

jcfr

2012-04-01 23:14

administrator   ~0003945

Please, consider integrating the following pull request: ttps://github.com/fedorov/MultiVolumeExplorer/pull/7

Make SURE to update External_MultiVolumeExplorer.cmake in Slicer code base.

fedorov

fedorov

2012-04-02 07:11

developer   ~0003947

Confirmed no memory leaks in the MultiVolume* generic tests.

Issue History

Date Modified Username Field Change
2012-03-13 14:00 fedorov New Issue
2012-03-13 14:00 fedorov Status new => assigned
2012-03-13 14:00 fedorov Assigned To => finetjul
2012-03-13 14:00 fedorov Assigned To finetjul => jcfr
2012-03-13 14:41 finetjul Category Base Code => Scripting (Wrapping, Python)
2012-03-21 00:48 jcfr Description Updated
2012-03-21 00:51 jcfr Note Added: 0003898
2012-04-01 18:33 fedorov Note Added: 0003941
2012-04-01 18:34 fedorov Note Added: 0003942
2012-04-01 18:34 fedorov Status assigned => feedback
2012-04-01 18:42 fedorov Note Added: 0003943
2012-04-01 19:11 jcfr Note Added: 0003944
2012-04-01 23:14 jcfr Note Added: 0003945
2012-04-01 23:15 jcfr Status feedback => resolved
2012-04-02 07:11 fedorov Note Added: 0003947
2012-04-02 07:11 fedorov Status resolved => closed
2012-04-02 07:11 fedorov Resolution open => fixed
2012-04-02 07:11 fedorov Fixed in Version => Slicer 4.1.0
2012-08-21 09:48 jcfr Target Version => Slicer 4.1.0