Bug: C++ OMETIFFWriter fails when writing multiple series

[rleigh]

On Linux with libtiff 4.0.3 and GCC 5.2, this patch:

--- metadata-formatwriter.cpp	2016-02-03 14:23:27.835441000 +0000
+++ metadata-formatwriter2.cpp	2016-02-03 14:25:55.367334000 +0000
@@ -91,6 +91,7 @@
     core->bitsPerPixel = 12U;
     core->dimensionOrder = DimensionOrder::XYZTC;
     seriesList.push_back(core);
+    seriesList.push_back(core);
 
     fillMetadata(*meta, seriesList);
 
@@ -104,7 +105,7 @@
                  std::ostream& stream)
   {
     // Total number of images (series)
-    dimension_size_type ic = 1;
+    dimension_size_type ic = 2;
     stream << "Image count: " << ic << '\n';
 
     // Loop over images

causes the following stacktrace. This is a result of a failed libtiff TIFFGetField operation being caught by the RAII wrapper (it's not available when writing). We cache these field values in the IFD wrapper when setting up the IFD for writing to allow the wrapper methods to function when the underlying IFD structure doesn't permit anything but setting, but they haven't been set here for some reason. There may be a failure to set up the IFD correctly following a series switch. It's definitely working when writing multiple planes within a series, but there may be a special case here which needs handling, likely in OMETIFFWriter itself.

(gdb) break metadata-formatwriter2.cpp:193
Breakpoint 1 at 0x421fab: file /home/rleigh/code/bioformats/docs/sphinx/developers/cpp/examples/metadata-formatwriter2.cpp, line 193.
(gdb) run test2.tiff
Starting program: /home/rleigh/code/bioformats/testbuild/cpp/examples/metadata-formatwriter2 test2.tiff
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1, main (argc=2, argv=0x7fffffffe548)
    at /home/rleigh/code/bioformats/docs/sphinx/developers/cpp/examples/metadata-formatwriter2.cpp:195
195               writePixelData(*writer, std::cout);
(gdb) catch throw
Catchpoint 2 (throw)
(gdb) c
Continuing.
Image count: 2
        Plane count: 1
PixelBuffer PixelType is uint16
VariantPixelBuffer PixelType is uint16
Catchpoint 2 (exception thrown), 0x00007ffff52a5bcd in __cxa_throw ()
   from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
(gdb) bt
#0  0x00007ffff52a5bcd in __cxa_throw ()
   from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#1  0x00007ffff7a4551c in ome::bioformats::tiff::Sentry::error (
    this=0x7fffffffd6b0, message=...)
    at /home/rleigh/code/bioformats/cpp/lib/ome/bioformats/tiff/Sentry.cpp:182
#2  0x00007ffff7a4559a in ome::bioformats::tiff::Sentry::error (
    this=0x7fffffffd6b0)
    at /home/rleigh/code/bioformats/cpp/lib/ome/bioformats/tiff/Sentry.cpp:188
#3  0x00007ffff7a0723e in ome::bioformats::tiff::IFD::getRawField (
    this=0x654fc0, tag=322)
    at /home/rleigh/code/bioformats/cpp/lib/ome/bioformats/tiff/IFD.cpp:763
#4  0x00007ffff79f8570 in ome::bioformats::tiff::(anonymous namespace)::generic_get1<unsigned int> (ifd=..., tag=322, 
    value=@0x7fffffffd960: 4294957456)
    at /home/rleigh/code/bioformats/cpp/lib/ome/bioformats/tiff/Field.cpp:360
#5  0x00007ffff79f4150 in ome::bioformats::tiff::Field<ome::bioformats::tiff::UInt32Tag1>::get (this=0x7fffffffd970, 
    value=@0x7fffffffd960: 4294957456)
    at /home/rleigh/code/bioformats/cpp/lib/ome/bioformats/tiff/Field.cpp:1299
#6  0x00007ffff7a075b8 in ome::bioformats::tiff::IFD::getTileType (
    this=0x654fc0)
    at /home/rleigh/code/bioformats/cpp/lib/ome/bioformats/tiff/IFD.cpp:800
#7  0x00007ffff7a07ce2 in ome::bioformats::tiff::IFD::getTileWidth (
    this=0x654fc0)
    at /home/rleigh/code/bioformats/cpp/lib/ome/bioformats/tiff/IFD.cpp:899
#8  0x00007ffff7a49796 in ome::bioformats::tiff::TileInfo::Impl::Impl (
    this=0x6546d0, ifd=...)
    at /home/rleigh/code/bioformats/cpp/lib/ome/bioformats/tiff/TileInfo.cpp:113
#9  0x00007ffff7a48d93 in ome::bioformats::tiff::TileInfo::TileInfo (
    this=0x7fffffffdbc0, ifd=...)
    at /home/rleigh/code/bioformats/cpp/lib/ome/bioformats/tiff/TileInfo.cpp:177
#10 0x00007ffff7a07838 in ome::bioformats::tiff::IFD::getTileInfo (
---Type <return> to continue, or q <return> to quit---
    this=0x654fc0)
    at /home/rleigh/code/bioformats/cpp/lib/ome/bioformats/tiff/IFD.cpp:835
#11 0x00007ffff7a0aafd in ome::bioformats::tiff::IFD::writeImage (
    this=0x654fc0, source=..., x=0, y=0, w=512, h=512)
    at /home/rleigh/code/bioformats/cpp/lib/ome/bioformats/tiff/IFD.cpp:1371
#12 0x00007ffff79e1ef4 in ome::bioformats::out::OMETIFFWriter::saveBytes
    (this=0x64e7d0, plane=0, buf=..., x=0, y=0, w=512, h=512)
    at /home/rleigh/code/bioformats/cpp/lib/ome/bioformats/out/OMETIFFWriter.cpp:681
#13 0x00007ffff79b789e in ome::bioformats::detail::FormatWriter::saveBytes (this=0x64e7d0, plane=0, buf=...)
    at /home/rleigh/code/bioformats/cpp/lib/ome/bioformats/detail/FormatWriter.cpp:177
#14 0x0000000000421d4e in (anonymous namespace)::writePixelData (
    writer=..., stream=...)
    at /home/rleigh/code/bioformats/docs/sphinx/developers/cpp/examples/metadata-formatwriter2.cpp:160
#15 0x0000000000421fc4 in main (argc=2, argv=0x7fffffffe548)
    at /home/rleigh/code/bioformats/docs/sphinx/developers/cpp/examples/metadata-formatwriter2.cpp:195

[rleigh]

The fault is triggered by

                    omeMeta->setUUIDFileName(relative.generic_string(), series, nextPlane);

in OMETIFFWriter.cpp due to a metadatastore/model inconsistency. There's also a possible issue with the use of fileOnly in the close call resulting in a "double close" in the dtors which could result in nextIFD() being called twice, resulting in a corrupt/empty IFD at the end of the file. The actual assert is legitimate and looks like a logic error either in the OMETIFFWriter or in the metadata in the C++ example which triggers the inconsistency.

[rleigh]

Fixed on develop: ​https://github.com/openmicroscopy/bioformats/pull/2230

[rleigh]

This was a combination of two factors:

• Using incorrect indexes in the OMETIFFWriter when setting UUID metadata; the comments indicated that this was a source of some confusion during the initial port; it's now using the same indexing method as for all other plane-specific metadata, and this appears to be generating correct OME-XML (which validates and can be processed by the Java implementation).
• Use of fileOnly during close which could lead to writing out an uninitialised/corrupt IFD; this is now properly closed. Note however, that if close() throws an exception e.g. if invalid metadata is encountered when creating the XML block to set, then the file will potentially be left open and it will be re-closed on destruction which would lead to the same thing happening. That's something which warrants further investigation and discussion. We might be best noting the failure, fully completing the close and then throwing again after the base close() completes.

[rleigh]

Fixed for 5.1: ​https://github.com/openmicroscopy/bioformats/pull/2231

[rleigh]

Additional commit pushed to both PRs to address the issue of exceptions being raised during the close() call of writers. It now catches and cleans up to the extent possible, and tries to leave the writers in a consistent state such that it can be called idempotently without any undesirable effects. The state may not allow safe re-use of the writer again via setId (though the writers should not be used in this manner really).