Warning: Can't synchronize with repository "(default)" (/home/git/ome.git does not appear to be a Git repository.). Look in the Trac log for more information.
Notice: In order to edit this ticket you need to be either: a Product Owner, The owner or the reporter of the ticket, or, in case of a Task not yet assigned, a team_member"

Task #6701 (closed)

Opened 10 years ago

Closed 10 years ago

Bug: exponential slow down without setWriteSequentially in ExporterI

Reported by: jamoore Owned by: jamoore
Priority: critical Milestone: OMERO-Beta4.3.2
Component: Performance Version: n.a.
Keywords: n.a. Cc: a.herbert@…, cxallan, jburel
Resources: n.a. Referenced By: n.a.
References: n.a. Remaining Time: 0.0d
Sprint: n.a.

Description (last modified by jmoore)

See: https://www.openmicroscopy.org/community/posting.php?mode=reply&f=6&t=814

From Alex:

I did a lot of reading of the code in the process and noticed that the sequential flag was not set on the BioFormats ImageWriter?. This means that each time a new plane is written to the TIFF file, BioFormats reads the entire file to locate the positions of all the current IFD entries (see method loci.formats.tiff.TiffSaver?.writeImageIFD(...)). If the current plane being written is less than the total count of IFD entries then the correct IFD entry is extracted for use.

Since the planes are written using incremental plane numbers I do not think this condition will ever be met. Thus it is safe to set the sequential flag to true.

Change History (5)

comment:1 Changed 10 years ago by jmoore

  • Description modified (diff)

comment:2 Changed 10 years ago by mlinkert

  • Owner changed from mlinkert-x to jmoore

This is all completely expected behavior, and the reason why the setWriteSequentially method was added in the first place. Given that ExporterI is only exporting OME-TIFFs and doing so sequentially (right?) it should be safe to just add a call to setWriteSequentially(true) before calling setId, as Alex suggests.

Re-assigning to Josh, since this is an ExporterI issue. I've also filed #6721 to investigate improving the performance of writing TIFF planes without the sequential flag set (but I would not consider that a bug).

comment:3 Changed 10 years ago by jmoore

Thanks for passing it along, Melissa. As long as your understanding of "sequential" and that of Alex's matches, then I'll go about turning this on for release.

comment:4 Changed 10 years ago by jmoore <josh@…>

(In [31ed3ad3c22f9c9fe0018f8aa6918d5269d88584/ome.git] on branch develop) setWriteSequentially in ExporterI (See #6701)

comment:5 Changed 10 years ago by jmoore <josh@…>

  • Remaining Time set to 0
  • Resolution set to fixed
  • Status changed from new to closed

(In [6e09514503d87f9b680be6f88afc665ec1be514c/ome.git] on branch develop) Merge branch 'feature/export-bugs' into develop (Fix #6520, #6701, #6713)

Note: See TracTickets for help on using tickets. You may also have a look at Agilo extensions to the ticket.

1.3.13-PRO © 2008-2011 Agilo Software all rights reserved (this page was served in: 0.67358 sec.)

We're Hiring!