Task #6010 (closed)
RFE: Speed up TIFF and BigTIFF file parsing
| Reported by: | mlinkert | Owned by: | mlinkert |
|---|---|---|---|
| Priority: | critical | Milestone: | OMERO-Beta4.3.2 |
| Component: | Bio-Formats | Version: | n.a. |
| Keywords: | n.a. | Cc: | christian.tischer@…, gbonamy@… |
| Resources: | n.a. | Referenced By: | n.a. |
| References: | n.a. | Remaining Time: | 0.0d |
| Sprint: | 2011-09-01 (4) |
Description
We now have several reports of setId on large or numerous TIFF and BigTIFF files taking far too long. One report suggests over an hour just to call setId on a ~9 GB BigTIFF file; other reports mention that grouping many single-plane TIFFs takes a very long time.
I don't know at the moment how much we can do to improve setId's performance in either case, but it is very definitely worth investigating. Obviously, improvements to setId's performance should not come at the price of degraded openBytes performance.
Change History (14)
comment:1 Changed 13 years ago by mlinkert
- Cc christian.tischer@… added
comment:2 Changed 13 years ago by mlinkert
- Cc gbonamy@… added
comment:3 Changed 13 years ago by mlinkert
- Sprint set to 2011-09-01 (4)
comment:4 Changed 13 years ago by mlinkert
- Status changed from new to accepted
comment:5 Changed 13 years ago by mlinkert
- Resolution set to fixed
- Status changed from accepted to closed
comment:6 Changed 13 years ago by Melissa Linkert <melissa@…>
(In [87da7ae0bb4bd7702e43c8ad815dced9f45d0d6e/bioformats.git]) Speed up OMETiffReader's isThisType implementation.
Previously, the first call to isThisType(...) would take ~1800 ms and
subsequent calls took ~40 ms, even if the file was a non-OME TIFF. Now, the
same calls take ~3 ms with a non-OME TIFF.
See #6010.
comment:7 Changed 13 years ago by Melissa Linkert <melissa@…>
(In [a1d4923d8bd637fcf04d3f4b0ff1a8d9bfafb3df/bioformats.git]) Speed up OMETiffReader's isThisType implementation.
Previously, the first call to isThisType(...) would take ~1800 ms and
subsequent calls took ~40 ms, even if the file was a non-OME TIFF. Now, the
same calls take ~3 ms with a non-OME TIFF.
See #6010.
comment:8 Changed 13 years ago by Melissa Linkert <melissa@…>
- Remaining Time set to 0
(In [9933b76381c8ce8aafe2f9caf52d393248829566/bioformats.git]) Reduce the number of times that the IFDs must be fetched.
This should cut TIFF file parsing time roughly in half.
Closes #6010.
comment:9 Changed 13 years ago by Melissa Linkert <melissa@…>
(In [87da7ae0bb4bd7702e43c8ad815dced9f45d0d6e/bioformats.git]) Speed up OMETiffReader's isThisType implementation.
Previously, the first call to isThisType(...) would take ~1800 ms and
subsequent calls took ~40 ms, even if the file was a non-OME TIFF. Now, the
same calls take ~3 ms with a non-OME TIFF.
See #6010.
comment:10 Changed 13 years ago by Melissa Linkert <melissa@…>
(In [9933b76381c8ce8aafe2f9caf52d393248829566/bioformats.git]) Reduce the number of times that the IFDs must be fetched.
This should cut TIFF file parsing time roughly in half.
Closes #6010.
comment:11 Changed 13 years ago by Melissa Linkert <melissa@…>
(In [fdb4e3fc0611222a22c30c93d9bd3b8098ce6071/bioformats.git]) Speed up OMETiffReader's isThisType implementation.
Previously, the first call to isThisType(...) would take ~1800 ms and
subsequent calls took ~40 ms, even if the file was a non-OME TIFF. Now, the
same calls take ~3 ms with a non-OME TIFF.
See #6010.
(cherry picked from commit 46c5fa00ff5e795b4fa5e12f0805f9ec94ffd842)
comment:12 Changed 13 years ago by Melissa Linkert <melissa@…>
(In [fdb4e3fc0611222a22c30c93d9bd3b8098ce6071/bioformats.git]) Speed up OMETiffReader's isThisType implementation.
Previously, the first call to isThisType(...) would take ~1800 ms and
subsequent calls took ~40 ms, even if the file was a non-OME TIFF. Now, the
same calls take ~3 ms with a non-OME TIFF.
See #6010.
(cherry picked from commit 46c5fa00ff5e795b4fa5e12f0805f9ec94ffd842)
comment:13 Changed 13 years ago by Melissa Linkert <melissa@…>
(In [265907e03611413744f6a383a29208abf12b987b/bioformats.git]) Reduce the number of times that the IFDs must be fetched.
This should cut TIFF file parsing time roughly in half.
Closes #6010.
(cherry picked from commit 9933b76381c8ce8aafe2f9caf52d393248829566)
comment:14 Changed 13 years ago by Melissa Linkert <melissa@…>
(In [265907e03611413744f6a383a29208abf12b987b/bioformats.git]) Reduce the number of times that the IFDs must be fetched.
This should cut TIFF file parsing time roughly in half.
Closes #6010.
(cherry picked from commit 9933b76381c8ce8aafe2f9caf52d393248829566)
This has been fixed, but as Hudson is still unhappy the builds do not reflect it. The relevant commits will show up here once Hudson is happy again.