Task #7319 (closed)
Opened 12 years ago
Closed 12 years ago
Bug: Bio-Formats LIF Timestamps
| Reported by: | wmoore | Owned by: | wmoore |
|---|---|---|---|
| Priority: | major | Milestone: | OMERO-4.4 |
| Component: | Bio-Formats | Version: | n.a. |
| Keywords: | n.a. | Cc: | |
| Resources: | n.a. | Referenced By: | n.a. |
| References: | n.a. | Remaining Time: | n.a. |
| Sprint: | 2012-03-13 (10) |
Description
As reported:
http://www.openmicroscopy.org/community/viewtopic.php?f=6&t=909
It seems that Bio-Formats is no-longer reading Original Metadata such as
Fluo exo/Image007 TimeStamp|HighInteger 1 29837616 Fluo exo/Image007 TimeStamp|HighInteger 10 29837617 Fluo exo/Image007 TimeStamp|HighInteger 100 29837626 Fluo exo/Image007 TimeStamp|HighInteger 101 29837626 Fluo exo/Image007 TimeStamp|HighInteger 102 29837626 ..etc
This is sampled from Bio-Formats (ImageJ) output Build # r5543 (Oct 2009) lif/Fabrice/Rosier.liff.
These do not appear now (using http://hudson.openmicroscopy.org.uk/job/BIOFORMATS-trunk/)
Also noticed that attribute values previously appearing like
Fluo exo/Image007 RelTimeStamp|Time 96 1468.01121166407 Fluo exo/Image007 RelTimeStamp|Time 97 1509.93370271929 Fluo exo/Image007 RelTimeStamp|Time 98 1551.85614438726 Fluo exo/Image007 RelTimeStamp|Time 99 1593.77857275096 Fluo exo/Image007 TimeStamp|HighInteger 1 29837616
now appear like
Series005 RelTimeStamp|Time 34 1468.01121166407 Series005 RelTimeStamp|Time 35 1509.93370271929 Series005 RelTimeStamp|Time 36 1551.85614438726 Series005 RelTimeStamp|Time 37 1593.77857275096 Series005 RelTimeStamp|Time 38 1635.70103076801
Change History (16)
comment:1 Changed 12 years ago by mlinkert
comment:2 Changed 12 years ago by wmoore
Image007
I think the original problem from the forum is that the Acquisition Date is not correct. When I open the Rosier.liff file in ImageJ I see
<Image ID="Image:0" Name="Fluo exo/Image007">
<AcquiredDate>2011-12-01T15:13:28</AcquiredDate>
which is now.
The metadata xml output from the LAF viewer says
<Data>
<Image>
<ImageDescription>
<StartTime>05/02/2007 14:17:44.195</StartTime>
<EndTime>05/02/2007 14:17:44.195</EndTime>
Delta-Ts for that image on ome.xml are
<Plane DeltaT="35.9138398391756" TheC="0" TheT="0" TheZ="0"/> <Plane DeltaT="1.2815158664195E10" TheC="1" TheT="0" TheZ="0"/> <Plane DeltaT="1.2815158664195E10" TheC="2" TheT="0" TheZ="0"/> <Plane DeltaT="1.2815158664195E10" TheC="3" TheT="0" TheZ="0"/> <Plane DeltaT="1.2815158664195E10" TheC="4" TheT="0" TheZ="0"/>
which seems wrong.
Looks like the first of these comes from
<RelTimeStamp Time="35.9138398391756" Frame="2" />
but that is the only one of those elements (there isn't one for each frame) and it doesn't appear in the transformed xml in a browser, so we should probably ignore it. Not sure where the 1.281515866 comes from?
The DeltaT info we want appears to be
<TimeStampList FirstTimeStampDate="05/02/2007" FirstTimeStampTime="14:17:44" FirstTimeStampMiliSeconds="195">
<TimeStamp HighInteger="29837616" LowInteger="1731343664" RelativeTime="0" Date="05/02/2007" Time="14:17:44" MiliSeconds="195" />
<TimeStamp HighInteger="29837616" LowInteger="1731343664" RelativeTime="0" Date="05/02/2007" Time="14:17:44" MiliSeconds="195" />
<TimeStamp HighInteger="29837616" LowInteger="1731343664" RelativeTime="0" Date="05/02/2007" Time="14:17:44" MiliSeconds="195" />
<TimeStamp HighInteger="29837616" LowInteger="1731343664" RelativeTime="0" Date="05/02/2007" Time="14:17:44" MiliSeconds="195" />
<TimeStamp HighInteger="29837616" LowInteger="1731343664" RelativeTime="0" Date="05/02/2007" Time="14:17:44" MiliSeconds="195" />
</TimeStampList>
This would suggest that all frames were acquired at the same time.
Series009
Looking at timestamps in Series009, it looks like there is a better correspondance between these two sections
<TimeStampList FirstTimeStampDate="05/02/2007" FirstTimeStampTime="14:23:49" FirstTimeStampMiliSeconds="142">
<TimeStamp HighInteger="29837617" LowInteger="1085846368" RelativeTime="0" Date="05/02/2007" Time="14:23:49" MiliSeconds="142" />
<TimeStamp HighInteger="29837617" LowInteger="1094286368" RelativeTime="0.844" Date="05/02/2007" Time="14:23:49" MiliSeconds="986" />
<TimeStamp HighInteger="29837617" LowInteger="1102726368" RelativeTime="1.688" Date="05/02/2007" Time="14:23:50" MiliSeconds="830" />
<TimeStamp HighInteger="29837617" LowInteger="1111166368" RelativeTime="2.532" Date="05/02/2007" Time="14:23:51" MiliSeconds="674" />
...
and:
<RelativeTime>
<RelTimeStamp Time="2.4317886456308" Frame="0" />
<RelTimeStamp Time="3.27584178378979" Frame="1" />
<RelTimeStamp Time="4.1198663735348" Frame="2" />
<RelTimeStamp Time="4.96395655963558" Frame="3" />
And so BF gets the correct times when it reads the <RelativeTime> element
<Plane DeltaT="2.4317886456308" TheC="0" TheT="0" TheZ="0"/> <Plane DeltaT="3.27584178378979" TheC="0" TheT="1" TheZ="0"/> ...
Series005 also seems to follow the pattern of Series009 (everything OK).
Maybe the solution is to use <RelativeTime> data unless the number of <RelTimeStamp> elements is less than the number of planes?
Cheers,
Will.
comment:3 Changed 12 years ago by Melissa Linkert <melissa@…>
(In [e02b4d22060b7391e09d1fae1e4014ed9f626d35/bioformats.git]) Fix timestamps that were set to the acquired date
See #7319.
comment:4 Changed 12 years ago by Melissa Linkert <melissa@…>
(In [e02b4d22060b7391e09d1fae1e4014ed9f626d35/bioformats.git]) Fix timestamps that were set to the acquired date
See #7319.
comment:5 Changed 12 years ago by mlinkert
OK, I think the AcquiredDate? and DeltaT values are sorted with the above commit. Will, when you have a chance could you please double-check the latest trunk build in ImageJ and see if the results are what you expect?
comment:6 Changed 12 years ago by mlinkert
- Owner changed from mlinkert-x to wmoore
comment:7 Changed 12 years ago by wmoore
- Sprint set to 2012-01-17 (6)
comment:8 Changed 12 years ago by wmoore
- Status changed from new to accepted
comment:9 Changed 12 years ago by wmoore
- Owner changed from wmoore to mlinkert-x
The Delta-Ts in Image007 are now fixed, but the Acquisition Date is still not working.
It still displays "now":
<Image ID="Image:0" Name="Fluo exo/Image007">
<AcquiredDate>2012-01-04T13:50:06</AcquiredDate>
For this image it should be 05/02/2007 14:17:44.195 (see above).
comment:10 Changed 12 years ago by mlinkert
- Owner changed from mlinkert-x to wmoore
This should be fixed on this branch: http://github.com/melissalinkert/bioformats/tree/default-date-cleanup
Could you please try building loci_tools.jar from that branch and see if it works in ImageJ ('ant tools && cp artifacts/loci_tools.jar /path/to/ImageJ/plugins' in the repository root)?
comment:11 Changed 12 years ago by wmoore
- Resolution set to fixed
- Status changed from accepted to closed
Looking good:
<Image ID="Image:0" Name="Fluo exo/Image007">
<AcquiredDate>2007-02-05T14:17:44</AcquiredDate>
Closing...
comment:12 Changed 12 years ago by Melissa Linkert <melissa@…>
(In [f8c4b7683e9c0e34069d6ea1f7d0ea6486f84afc/bioformats.git] on branch develop) Only reset the metadata when initializing a new file
Previously, the MetadataStore? associated with a ChannelFiller? would be
reset with every call to setId, even if the file name passed to setId
was equivalent to the currently initialized file name.
This caused the ImageJ import plugin to show incorrect image acquisition
dates, since MetadataTools?.populatePixels now sets a default image
acquisition date and the plugin generates multiple setId calls per file.
See #7319.
comment:13 Changed 12 years ago by wmoore
- Resolution fixed deleted
- Status changed from closed to reopened
Using sprint10-bug-fixes branch, I am still getting some AcquiredDate? values as "now" (2012-02-28T23:51:55)
E.g. Martin's sample files.lif
jrs-macbookpro-25107:bioformats will$ java -Xmx512m loci.formats.tools.ImageInfo -nopix -omexml-only -no-sas -xmlspaces 4 /Users/will/Documents/biology-data/Test-Import-Images/Martin/Leica-SP5/sample\ files.lif | grep "<AcquiredDate>"
<AcquiredDate>2009-06-09T15:06:13</AcquiredDate>
<AcquiredDate>2012-02-28T23:51:55</AcquiredDate>
<AcquiredDate>2009-04-24T10:28:44</AcquiredDate>
<AcquiredDate>2012-02-28T23:51:55</AcquiredDate>
<AcquiredDate>2009-04-24T10:43:08</AcquiredDate>
<AcquiredDate>2012-02-28T23:51:55</AcquiredDate>
<AcquiredDate>2009-04-24T10:53:23</AcquiredDate>
<AcquiredDate>2012-02-28T23:51:55</AcquiredDate>
<AcquiredDate>2009-04-24T11:06:29</AcquiredDate>
<AcquiredDate>2009-04-24T11:10:39</AcquiredDate>
<AcquiredDate>2009-04-24T11:24:52</AcquiredDate>
<AcquiredDate>2009-04-24T11:26:01</AcquiredDate>
<AcquiredDate>2009-02-18T13:03:45</AcquiredDate>
<AcquiredDate>2012-02-28T23:51:55</AcquiredDate>
comment:14 Changed 12 years ago by jburel
- Sprint changed from 2012-01-17 (6) to 2012-03-13 (10)
comment:15 Changed 12 years ago by mlinkert
Sorry, missed your comment completely.
This should be sorted out with https://github.com/melissalinkert/bioformats/commit/4867101e1eb47c426296a35672b8e38acee5d187. Note that now we are just using the file's "last modified time" (2009-06-11T18:44 for me) instead of "now". There really is no acquisition time stored in the file for those Images (as they are snapshots, not original data).
comment:16 Changed 12 years ago by wmoore
- Resolution set to fixed
- Status changed from reopened to closed
This sounds fine. omexml_repo updated accordingly: https://github.com/will-moore/openmicroscopy/commit/9d58934065bec438016ff2969912a98c15cf2063
Closing...
It is expected that the timestamps will not be present in the original metadata, per [00ed258ab0cffd3c809f5cca1dd61bc4f488b422/bioformats.git]. I'm inclined to leave thing as they are, since the timestamps are still parsed an placed in Plane.DeltaT (and we do tell people that original metadata is not an API). Happy to discuss, though.
Maybe you have different Images open? The "Series005" and "Fluo exo/Image007" are two of the Image names, which would appear if you had that Image and at least one other one open. I just tried opening both of those Images together, and saw that the correct original metadata keys were in place. If that doesn't match what you see, then I'll investigate further.