Task #6701 (closed)
Opened 13 years ago
Closed 13 years ago
Bug: exponential slow down without setWriteSequentially in ExporterI
| Reported by: | jamoore | Owned by: | jamoore |
|---|---|---|---|
| Priority: | critical | Milestone: | OMERO-Beta4.3.2 |
| Component: | Performance | Version: | n.a. |
| Keywords: | n.a. | Cc: | a.herbert@…, cxallan, jburel |
| Resources: | n.a. | Referenced By: | n.a. |
| References: | n.a. | Remaining Time: | 0.0d |
| Sprint: | n.a. |
Description (last modified by jmoore)
See: https://www.openmicroscopy.org/community/posting.php?mode=reply&f=6&t=814
From Alex:
I did a lot of reading of the code in the process and noticed that the sequential flag was not set on the BioFormats ImageWriter?. This means that each time a new plane is written to the TIFF file, BioFormats reads the entire file to locate the positions of all the current IFD entries (see method loci.formats.tiff.TiffSaver?.writeImageIFD(...)). If the current plane being written is less than the total count of IFD entries then the correct IFD entry is extracted for use.
Since the planes are written using incremental plane numbers I do not think this condition will ever be met. Thus it is safe to set the sequential flag to true.
Change History (5)
comment:1 Changed 13 years ago by jmoore
- Description modified (diff)
comment:2 Changed 13 years ago by mlinkert
- Owner changed from mlinkert-x to jmoore
comment:3 Changed 13 years ago by jmoore
Thanks for passing it along, Melissa. As long as your understanding of "sequential" and that of Alex's matches, then I'll go about turning this on for release.
comment:4 Changed 13 years ago by jmoore <josh@…>
(In [31ed3ad3c22f9c9fe0018f8aa6918d5269d88584/ome.git] on branch develop) setWriteSequentially in ExporterI (See #6701)
comment:5 Changed 13 years ago by jmoore <josh@…>
- Remaining Time set to 0
- Resolution set to fixed
- Status changed from new to closed
(In [6e09514503d87f9b680be6f88afc665ec1be514c/ome.git] on branch develop) Merge branch 'feature/export-bugs' into develop (Fix #6520, #6701, #6713)
This is all completely expected behavior, and the reason why the setWriteSequentially method was added in the first place. Given that ExporterI is only exporting OME-TIFFs and doing so sequentially (right?) it should be safe to just add a call to setWriteSequentially(true) before calling setId, as Alex suggests.
Re-assigning to Josh, since this is an ExporterI issue. I've also filed #6721 to investigate improving the performance of writing TIFF planes without the sequential flag set (but I would not consider that a bug).