Task #11505 (closed)
Opened 6 years ago
Closed 6 years ago
Bug: Insight unresponsive when zipping downloaded files
| Reported by: | drussell-x | Owned by: | jburel |
|---|---|---|---|
| Priority: | major | Milestone: | 5.0.0-rc1 |
| Component: | Insight | Version: | 4.4.8 |
| Keywords: | n.a. | Cc: | jamoore |
| Resources: | n.a. | Referenced By: | n.a. |
| References: | n.a. | Remaining Time: | n.a. |
| Sprint: | OMERO 5 Beta 2 (1) |
Description
If an uploaded image has companion files then when it is downloaded in Insight these files are downloaded into a directory and then zipped together.
Insight is fine during the download itself, but once it starts to zip the files together, it hangs until the zipping is completed. So that is a bug in itself.
Also...
This is a CPU bound operation and is because zip compression of the image takes a long time. Even using zip command line utility takes nearly as long... unless you turn off compression in which case it's only about as long as it takes to make a copy of the data on disk.
zip -0 imageDir.zip -r imageDir
I'm not sure if most image formats really benefit from zip compression anyway, although I guess it's feasible that there is a huge amount of companion files which might.
Tested on 4.4.8p1 and develop.
Change History (8)
comment:1 Changed 6 years ago by jamoore
- Priority changed from minor to major
- Summary changed from Insight unresponsive when zipping downloaded files to Bug: Insight unresponsive when zipping downloaded files
- Type changed from Bug to Task
comment:2 Changed 6 years ago by jamoore
- Component set to Insight
- Milestone set to OMERO-4.4.x
- Owner set to jburel
- Version set to 4.4.8
comment:3 Changed 6 years ago by jburel
- Cc jamoore added
- Milestone changed from OMERO-4.4.x to 5.0.0-beta2
- We currently use java.util.zip
- I took the opportunity to look at 2 other zip libraries: apache.commons.compress and zip4j (easy to use).
- Tested with a folder containing 4 dicoms files and one ini file
-rwx------ 1 jburel staff 387976 6 Aug 2012 CR-MONO1-10-chest.dcm -rwx------@ 1 jburel staff 135944 6 Aug 2012 abdominal.dcm -rwx------@ 1 jburel staff 262624 6 Aug 2012 ankle.dcm -rwx------@ 1 jburel staff 528642 6 Aug 2012 brain.dcm -rwx------ 1 jburel staff 14790 13 Oct 2009 test_setup.ini
- Result to compress with no compression flag (time in ms):
java.util.zip: 61 commons.compress:106 zp4j:333 command line: 8
Do you have a sample folder or a multi-images e.g. lei will be enough for more performance testing.
comment:4 Changed 6 years ago by jburel
Tested with a lei file (540 files)
java.util.zip gives us the best results of the libraries.
Action:
- Adjust parameters when zipping folder.
- create the zip asynch.
comment:5 Changed 6 years ago by jamoore
Perhaps an attempt at calling zip via ProcessBuilder and fall back to the Java implementation?
comment:6 Changed 6 years ago by jburel
yes, that was part of the plan too. click update too quickly.
comment:7 Changed 6 years ago by jburel
- Sprint set to OMERO 5 Beta 2 (1)
- Status changed from new to accepted
comment:8 Changed 6 years ago by jburel
- Resolution set to fixed
- Status changed from accepted to closed
Moving to task "Bug:"