Notice: In order to edit this ticket you need to be either: a Product Owner, The owner or the reporter of the ticket, or, in case of a Task not yet assigned, a team_member"

Task #6010 (closed)

Opened 10 years ago

Closed 9 years ago

Last modified 9 years ago

RFE: Speed up TIFF and BigTIFF file parsing

Reported by: mlinkert Owned by: mlinkert
Priority: critical Milestone: OMERO-Beta4.3.2
Component: Bio-Formats Version: n.a.
Keywords: n.a. Cc: christian.tischer@…, gbonamy@…
Resources: n.a. Referenced By: n.a.
References: n.a. Remaining Time: 0.0d
Sprint: 2011-09-01 (4)

Description

We now have several reports of setId on large or numerous TIFF and BigTIFF files taking far too long. One report suggests over an hour just to call setId on a ~9 GB BigTIFF file; other reports mention that grouping many single-plane TIFFs takes a very long time.

I don't know at the moment how much we can do to improve setId's performance in either case, but it is very definitely worth investigating. Obviously, improvements to setId's performance should not come at the price of degraded openBytes performance.

Change History (14)

comment:1 Changed 10 years ago by mlinkert

  • Cc christian.tischer@… added

comment:2 Changed 10 years ago by mlinkert

  • Cc gbonamy@… added

comment:3 Changed 9 years ago by mlinkert

  • Sprint set to 2011-09-01 (4)

comment:4 Changed 9 years ago by mlinkert

  • Status changed from new to accepted

comment:5 Changed 9 years ago by mlinkert

  • Resolution set to fixed
  • Status changed from accepted to closed

This has been fixed, but as Hudson is still unhappy the builds do not reflect it. The relevant commits will show up here once Hudson is happy again.

comment:6 Changed 9 years ago by Melissa Linkert <melissa@…>

(In [87da7ae0bb4bd7702e43c8ad815dced9f45d0d6e/bioformats.git]) Speed up OMETiffReader's isThisType implementation.

Previously, the first call to isThisType(...) would take ~1800 ms and
subsequent calls took ~40 ms, even if the file was a non-OME TIFF. Now, the
same calls take ~3 ms with a non-OME TIFF.

See #6010.

comment:7 Changed 9 years ago by Melissa Linkert <melissa@…>

(In [a1d4923d8bd637fcf04d3f4b0ff1a8d9bfafb3df/bioformats.git]) Speed up OMETiffReader's isThisType implementation.

Previously, the first call to isThisType(...) would take ~1800 ms and
subsequent calls took ~40 ms, even if the file was a non-OME TIFF. Now, the
same calls take ~3 ms with a non-OME TIFF.

See #6010.

comment:8 Changed 9 years ago by Melissa Linkert <melissa@…>

  • Remaining Time set to 0

(In [9933b76381c8ce8aafe2f9caf52d393248829566/bioformats.git]) Reduce the number of times that the IFDs must be fetched.

This should cut TIFF file parsing time roughly in half.

Closes #6010.

comment:9 Changed 9 years ago by Melissa Linkert <melissa@…>

(In [87da7ae0bb4bd7702e43c8ad815dced9f45d0d6e/bioformats.git]) Speed up OMETiffReader's isThisType implementation.

Previously, the first call to isThisType(...) would take ~1800 ms and
subsequent calls took ~40 ms, even if the file was a non-OME TIFF. Now, the
same calls take ~3 ms with a non-OME TIFF.

See #6010.

comment:10 Changed 9 years ago by Melissa Linkert <melissa@…>

(In [9933b76381c8ce8aafe2f9caf52d393248829566/bioformats.git]) Reduce the number of times that the IFDs must be fetched.

This should cut TIFF file parsing time roughly in half.

Closes #6010.

comment:11 Changed 9 years ago by Melissa Linkert <melissa@…>

(In [fdb4e3fc0611222a22c30c93d9bd3b8098ce6071/bioformats.git]) Speed up OMETiffReader's isThisType implementation.

Previously, the first call to isThisType(...) would take ~1800 ms and
subsequent calls took ~40 ms, even if the file was a non-OME TIFF. Now, the
same calls take ~3 ms with a non-OME TIFF.

See #6010.
(cherry picked from commit 46c5fa00ff5e795b4fa5e12f0805f9ec94ffd842)

comment:12 Changed 9 years ago by Melissa Linkert <melissa@…>

(In [fdb4e3fc0611222a22c30c93d9bd3b8098ce6071/bioformats.git]) Speed up OMETiffReader's isThisType implementation.

Previously, the first call to isThisType(...) would take ~1800 ms and
subsequent calls took ~40 ms, even if the file was a non-OME TIFF. Now, the
same calls take ~3 ms with a non-OME TIFF.

See #6010.
(cherry picked from commit 46c5fa00ff5e795b4fa5e12f0805f9ec94ffd842)

comment:13 Changed 9 years ago by Melissa Linkert <melissa@…>

(In [265907e03611413744f6a383a29208abf12b987b/bioformats.git]) Reduce the number of times that the IFDs must be fetched.

This should cut TIFF file parsing time roughly in half.

Closes #6010.
(cherry picked from commit 9933b76381c8ce8aafe2f9caf52d393248829566)

comment:14 Changed 9 years ago by Melissa Linkert <melissa@…>

(In [265907e03611413744f6a383a29208abf12b987b/bioformats.git]) Reduce the number of times that the IFDs must be fetched.

This should cut TIFF file parsing time roughly in half.

Closes #6010.
(cherry picked from commit 9933b76381c8ce8aafe2f9caf52d393248829566)

Note: See TracTickets for help on using tickets. You may also have a look at Agilo extensions to the ticket.

1.3.13-PRO © 2008-2011 Agilo Software all rights reserved (this page was served in: 0.77614 sec.)

We're Hiring!