View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0004329 | Slicer4 | Core: Segmentation | public | 2017-01-23 18:27 | 2018-03-02 11:02 |
Reporter | matrimcauthon | Assigned To | lassoan | ||
Priority | urgent | Severity | crash | Reproducibility | always |
Status | closed | Resolution | fixed | ||
Platform | Mac OS | OS | MacOS | OS Version | 10.11.6 |
Product Version | |||||
Target Version | Fixed in Version | Slicer 4.7.0 | |||
Summary | 0004329: Slicer crashes when attempting to load large segmentations from nrrd files in 4.7 nightlies | ||||
Description | The new segmentions and segment editor are a fabulous improvement from previous sectioning methods. I am coming across a problem however where Slicer will not load the segment files back into memory. If i save each part of a certain (8gb nrrd volume based) assembly together Slicer crashes immediately when i try to load the seen or try to load the .seg.nrrd file. If however I save each each segment out to a separate segmentation and save each as its own .seg.nrrd file I can bring them all back in individually (and then add them to a complete segmentation to do further assembly edits with all parts included). The attached debug report was of loading 4 single "part" segmentation followed by trying to load the full segmentation of identical geometry where Slicer promptly crashes. I have been able to reproduce this several times with this data set so i am fairly confident its not just a corrupt .seg.nrrd. Thanks for the help! I am working on an urgent problem and this tool (while i know is still in nightly) has been a life saver processing my data. Please let me know if i can provide any other information. | ||||
Additional Information | [DEBUG][Qt] 23.01.2017 16:01:31 [] (unknown:0) - Session start time .......: 2017-01-23 16:01:31 | ||||
Tags | No tags attached. | ||||
Thanks for the report! |
|
I will see what I can do but almost everything we have is OUO. The unlimited release sets I keep are fairly simple/small and may not be able to reproduce this. While I work on that is there any better diagnostics i can give you from this set that would help narrow it down? Thanks! |
|
You can enter the following to the python interactor to print out the details of the segmentation you want to save
My guess is that the save operation crashes because when a 4D NRRD is saved, the "effective extents" (that minimize memory usage by only allocating the extent of the labelmap that actually contains data) are padded to be the largest so that a real 4D image can be saved. It might be too much memory to be allocated. Also this is why you can save them individually, because then the effective extents are preserved. As a workaround, you can save them as surfaces. To do that, go to the Segmentations module after you're finished segmenting, create the Closed surface representation in the Representations section if it doesn't exist yet, and click Make master. Now if you save the segmentation it will be saved to individual files. When you load it, you can convert back to labelmaps. I know that by doing this the data will lose some fidelity, but for example by setting the smoothing parameter (long click Create and then Advanced, or Update if it exists) to 0 you can minimize this loss. Let me know about the result of the print statement and how the surface saving works! In the meantime we'll discuss the ticket I linked to. |
|
I have been able to reproduce on a UUR deck that i can share. How would you like me to get it to you (its about 9gb)? |
|
hmmm, responding to the above i will try to see what the output is but I don't know that its a save issue. They save just fine, its loading where the crash occurs. |
|
It might be the very same issue though. Do you have access to any large file sharing services? How much is it if you compress to zip? |
|
I am compressing now and think I can send it with the Sandia file transfer tool (its the only official means open to me). I have to have an email address to send it to though. Thank you so much for the prompt responses with this. We get to work with Kitware quite often here with Paraview so its nice to have the common backing here too =). |
|
btw, i am happy to report this as another issue but is there a known bug where segments dissapear/are no longer visible? This happens often with large #s of segmentation quite frequently. The segment is still there and i can see it in 3d but it dissapears from the slice views. |
|
Could you try to increase the virtual memory size to be 4x larger and see if loading still crashes? For the segment disappearing problem - we haven't experienced this. Please enter another ticket and provide enough details so we can reproduce it. Thanks! |
|
My email address is csaba dot pinter at queensu dot ca. I didn't know about a disappearing segments bug so far. Can you add another issue for that one please? Thank you! |
|
ok, I am sending now. Its large so i expect it will take a bit to upload but it should send you an email when its done. For memory I don't think that is a problem. I have 128Gb on here and am only using 20-40Gb with Slicer. There is still 60Gb free at the moment. |
|
I see, so it's probably not the lack of memory on your side. Then probably my computer will crash for that reason, not the one you're having. Fingers crossed though. |
|
Files are now sent. Let me know if you don't see them. Here is hoping that it can open... (you can just try loading the segments themselves and should still be able to make it crash from what i have seen). |
|
I downloaded them but cannot uncompress, I get an error message that the zip file is invalid. I'm downloading again. If the segmentation is enough, can you please send that only? |
|
Sent. Sorry for the big files. This coffee cup is tame (we have a lot of high res scans at 256Gb range). Always end up pushing system and software size limits. |
|
Thanks for the segmentation file! I opened it with the latest version, and although I didn't crash, the resulting segmentation was empty. I will further investigate. |
|
It seems that the execution of line Did you try the surface conversion I suggested earlier? It may be a viable workaround for the time being. |
|
Hmmm, I'd really like to not save it out as surfaces (until i am completely done with modeling it). That is odd that it didn't crash for you as it can for me every time I open it. So there is no way to output them as an old style label map and pull that back in? When you say its internally handled, who would that be by? |
|
As I said, VTK. You can also try to export the segmentation in an old-style labelmap in the Segmentations module (Import/export section on the bottom), and save it as labelmap nrrd. When you load it, it will be a labelmap node not a segmentation, so if you want to keep editing it or use the segmentations feature (automatic conversion, simultaneous visualization, transformation, etc.), then you need to import it to an empty segmentation. You will lose data this way though! Most importantly, overlapping segment information will be lost, so you need to make sure there are no overlaps. Color, terminology, etc. metadata will be lost too. |
|
Oh Sorry, didn't realize vtk was independent like that. I am going to copy Kitware on this and see if they can help. That data loss would be unfortunate but would at least save voxel to voxel locations (I don't have overlaps at the moment). I have to have some reliable way to save out the work as I go before a crash makes me lose hours of =/. |
|
regrettably when I ask it to create the water tight models it crashes with the following log: [DEBUG][Qt] 24.01.2017 11:08:12 [] (unknown:0) - Session start time .......: 2017-01-24 11:08:12 |
|
Continuously crashes when i try to make water tight. I was able to turn it into a label map and then from there use the old model maker to get some geometry out though the output doesn't quite look as nice. Still i guess its something. Still definitely would love resolutions to the above as it would be good for the code and MUCH better for getting parts out of scans. Thanks for the continued help! |
|
What do you mean by trying to make water tight? When you click on "Create surface" button? |
|
Yes, it crashes on create surface as well under the representations section of segmentations. The above log is from one such occurrence. Thanks for the help! |
|
Fixed in rev 25668. The issue was that at two places in vtkNRRDReader, 32-bit variables were used for computing total number of bytes in the image. This resulted incorrect value for images >2GB. |
|
@matrimcauthon The loading issue is fixed now (fix will be available in tomorrow's nightly build). Report any further issues with loading large segmentations from nrrd files in this issue. If there is any other issue that you would like us to investigate, please enter a separate bug report for each. |
|
Thanks for the help @lassoan. CT segmenting is becoming a large focus here with huge dataset so I expect you will hear from me again soon. I will see if I can reproduce the closed surface bug with a UUR example and post that to another ticket. |
|
Date Modified | Username | Field | Change |
---|---|---|---|
2017-01-23 18:27 | matrimcauthon | New Issue | |
2017-01-23 18:27 | matrimcauthon | Status | new => assigned |
2017-01-23 18:27 | matrimcauthon | Assigned To | => pinter |
2017-01-23 18:41 | pinter | Note Added: 0014291 | |
2017-01-24 09:10 | matrimcauthon | Note Added: 0014293 | |
2017-01-24 10:18 | pinter | Note Added: 0014294 | |
2017-01-24 10:22 | matrimcauthon | Note Added: 0014295 | |
2017-01-24 10:25 | matrimcauthon | Note Added: 0014296 | |
2017-01-24 10:31 | pinter | Note Added: 0014297 | |
2017-01-24 10:38 | matrimcauthon | Note Added: 0014298 | |
2017-01-24 10:40 | matrimcauthon | Note Added: 0014299 | |
2017-01-24 10:47 | lassoan | Note Added: 0014300 | |
2017-01-24 10:48 | pinter | Note Added: 0014301 | |
2017-01-24 10:53 | matrimcauthon | Note Added: 0014302 | |
2017-01-24 10:58 | pinter | Note Added: 0014303 | |
2017-01-24 11:39 | matrimcauthon | Note Added: 0014304 | |
2017-01-24 11:53 | pinter | Note Added: 0014305 | |
2017-01-24 11:59 | matrimcauthon | Note Added: 0014306 | |
2017-01-24 12:12 | pinter | Note Added: 0014307 | |
2017-01-24 12:54 | pinter | Note Added: 0014308 | |
2017-01-24 13:13 | matrimcauthon | Note Added: 0014309 | |
2017-01-24 13:26 | pinter | Note Added: 0014310 | |
2017-01-24 13:33 | matrimcauthon | Note Added: 0014311 | |
2017-01-24 14:15 | matrimcauthon | Note Added: 0014312 | |
2017-01-24 17:38 | matrimcauthon | Note Added: 0014313 | |
2017-01-24 17:43 | lassoan | Note Added: 0014314 | |
2017-01-24 17:44 | matrimcauthon | Note Added: 0014315 | |
2017-01-24 18:42 | lassoan | Summary | Slicer crashes when attempting to load multi-segment seg.nrrd files in 4.7 nightlies => Slicer crashes when attempting to load large segmentations from nrrd files in 4.7 nightlies |
2017-01-24 18:44 | lassoan | Note Added: 0014316 | |
2017-01-24 18:44 | lassoan | Status | assigned => resolved |
2017-01-24 18:44 | lassoan | Fixed in Version | => Slicer 4.7.0 |
2017-01-24 18:44 | lassoan | Resolution | open => fixed |
2017-01-24 18:44 | lassoan | Assigned To | pinter => lassoan |
2017-01-24 18:46 | lassoan | Note Added: 0014317 | |
2017-01-25 09:34 | matrimcauthon | Note Added: 0014318 | |
2017-07-25 14:25 | pinter | Relationship added | related to 0004399 |
2018-03-02 11:02 | jcfr | Status | resolved => closed |