Bug: liff EmissionFilters

[wmoore]

I just tried importing a lif file and get incorrect excitation wavelengths.
E.g. Martin/Leica?-SP5/sample files.lif [series 02 - XYZChBF]

even more incorrect than reported http://www.openmicroscopy.org/community/viewtopic.php?f=4&t=743&p=2702

[wmoore]

Ooops - just noticed that this was supposed to be lif files....

[wmoore]

Sample files are under squig/apache_repo/3859

[wmoore]

Hi Melissa,

Are you likely to get this fixed by 4.3.2? The user who submitted the bug was asking about it again (see forum above). I will be away after this week, but still contactable if you need help with the metadata.
Cheers, Will.

[mlinkert]

Yes, I should be able to get this sorted out before 4.3.2 - just scheduled it for sprint 4. I'll reply to the forum post in just a minute.

[jburel]

Moved from sprint 2011-08-18 (3)

[Melissa Linkert <melissa@…>]

(In [17412525c3bd15afd50cf8fd9b53e0277f0e3597/bioformats.git]) Fix how laser wavelengths are linked to channels.

See #6189. The channel names are still wrong.

[Melissa Linkert <melissa@…>]

(In [34e4b6c8c97d874ebf650a7d71b1e52570ca59f6/bioformats.git]) Fix channel name population.

If more than the required number of names are present, use the last n
names instead of the first n names.

See #6189.

[Melissa Linkert <melissa@…>]

(In [780a2105ff4268d5dccf877149b44cf733c8a256/bioformats.git]) Final touches on detector/laser/channel linkages.

Closes #6189.

[Melissa Linkert <melissa@…>]

(In [17412525c3bd15afd50cf8fd9b53e0277f0e3597/bioformats.git]) Fix how laser wavelengths are linked to channels.

See #6189. The channel names are still wrong.

[Melissa Linkert <melissa@…>]

(In [34e4b6c8c97d874ebf650a7d71b1e52570ca59f6/bioformats.git]) Fix channel name population.

If more than the required number of names are present, use the last n
names instead of the first n names.

See #6189.

[Melissa Linkert <melissa@…>]

(In [780a2105ff4268d5dccf877149b44cf733c8a256/bioformats.git]) Final touches on detector/laser/channel linkages.

Closes #6189.

[Melissa Linkert <melissa@…>]

(In [6c01b29a50f65e916b1d6c0a916ab68f1298df0b/bioformats.git]) Fix how laser wavelengths are linked to channels.

See #6189. The channel names are still wrong.
(cherry picked from commit 17412525c3bd15afd50cf8fd9b53e0277f0e3597)

[Melissa Linkert <melissa@…>]

(In [cea120a0e48fd65c0e211f2ba73281ecae97fd6e/bioformats.git]) Fix channel name population.

If more than the required number of names are present, use the last n
names instead of the first n names.

See #6189.
(cherry picked from commit 34e4b6c8c97d874ebf650a7d71b1e52570ca59f6)

[Melissa Linkert <melissa@…>]

(In [e8f6a9b4c7d337886d59c7f7bd081ba3845eca4c/bioformats.git]) Final touches on detector/laser/channel linkages.

Closes #6189.
(cherry picked from commit 780a2105ff4268d5dccf877149b44cf733c8a256)

[Melissa Linkert <melissa@…>]

(In [6c01b29a50f65e916b1d6c0a916ab68f1298df0b/bioformats.git]) Fix how laser wavelengths are linked to channels.

See #6189. The channel names are still wrong.
(cherry picked from commit 17412525c3bd15afd50cf8fd9b53e0277f0e3597)

[Melissa Linkert <melissa@…>]

(In [cea120a0e48fd65c0e211f2ba73281ecae97fd6e/bioformats.git]) Fix channel name population.

If more than the required number of names are present, use the last n
names instead of the first n names.

See #6189.
(cherry picked from commit 34e4b6c8c97d874ebf650a7d71b1e52570ca59f6)

[Melissa Linkert <melissa@…>]

(In [e8f6a9b4c7d337886d59c7f7bd081ba3845eca4c/bioformats.git]) Final touches on detector/laser/channel linkages.

Closes #6189.
(cherry picked from commit 780a2105ff4268d5dccf877149b44cf733c8a256)

[wmoore]

Unfortunately this doesn't seem to be fixed for me. There is still a problem with the linking of filters.

E.g. for the image: liff/Fabrice/Rosier.lif [Fluo exo/Image007]
I expect Channel 1 to be like this: (more details in e-mail below)

Detector (Channel 1): Offset, Gain, Model "PMT 1", Type "PMT" etc.
SecondaryEmissionFilter: (Use Channel 1 MultiBand) CutIn: 406,
CutOut 479 (NB. Need to round to nearest int, not lowest)
Laser: Wavelength: 405, Attenuation: 0.7499.
Channel Name: "Leica/DAPI", Fluor "DAPI", ExWave: 405

Currently this Channel is linked to Filter:

"SP Mirror Channel 4", CutIn: 641, CutOut: 797.

You already have the correct Detector for this Channel "PMT1" so you can probably create the emission filter from the 'Band' attribute of <Detector />

Band="(406nm - 479nm)"

[mlinkert]

Pushing this to OME-5.0, since a lot of progress has been made on the channel/detector linkages and we are past the point in the 4.3.2 cycle where Bio-Formats changes are allowed.

[Melissa Linkert <melissa@…>]

(In [5c4398ea5e79fe0c0e575d9f453997b29fb71718/bioformats.git]) Temporarily disable channel -> Filter linkages.

See #6189.

Since the channel -> Filter linkages are largely incorrect, we'll just
turn them off for now. For 4.4, we'll review and re-enable the linkages once
they are working again.

[Melissa Linkert <melissa@…>]

(In [20473f48959c6b2e5857551597cabcb01eac557d/bioformats.git]) Temporarily disable channel -> Filter linkages.

See #6189.

Since the channel -> Filter linkages are largely incorrect, we'll just
turn them off for now. For 4.4, we'll review and re-enable the linkages once
they are working again.
(cherry picked from commit 5c4398ea5e79fe0c0e575d9f453997b29fb71718)

[Melissa Linkert <melissa@…>]

(In [5c4398ea5e79fe0c0e575d9f453997b29fb71718/bioformats.git]) Temporarily disable channel -> Filter linkages.

See #6189.

Since the channel -> Filter linkages are largely incorrect, we'll just
turn them off for now. For 4.4, we'll review and re-enable the linkages once
they are working again.

[Melissa Linkert <melissa@…>]

(In [20473f48959c6b2e5857551597cabcb01eac557d/bioformats.git]) Temporarily disable channel -> Filter linkages.

See #6189.

Since the channel -> Filter linkages are largely incorrect, we'll just
turn them off for now. For 4.4, we'll review and re-enable the linkages once
they are working again.
(cherry picked from commit 5c4398ea5e79fe0c0e575d9f453997b29fb71718)

[mlinkert]

I think this is sorted with this branch: ​http://github.com/melissalinkert/bioformats/tree/6189-lif-emission-filters

Will, could you please try building loci_tools.jar from that branch, and see if the emission filter linkages show up correctly for you?

[wmoore]

Having checked a couple of channels, the wavelengths look good. Closing...

[wmoore]

CutIn? & CutOut? rounding error, Offset also incorrect

[wmoore]

It seems that the CutIn and CutOut wavelengths are not being rounded to the nearest integer.
E.g. In Rosier 007 - Channel 3

We are currently reading:

<Filter ID="Filter:0:2" Model="SP Mirror Channel 3">
    <TransmittanceRange CutIn="564" CutOut="641"/>

The data should be:

• CutIn: 565
• CutOut: 642

Rounded from this data in the PFF

<FilterSettingRecord ObjectName="SP Mirror Channel 3" ClassName="CSpectrophotometerUnit" Attribute="Wavelength" Description="SP Mirror Channel 3 (left)" Data="0" Variant="564.8" VariantType="5">
<FilterSettingRecord ObjectName="SP Mirror Channel 3" ClassName="CSpectrophotometerUnit" Attribute="Wavelength" Description="SP Mirror Channel 3 (right)" Data="1" Variant="641.9" VariantType="5">

Screen-shot of Insight vv PFF metadata:
​https://trac.openmicroscopy.org.uk/ome/attachment/ticket/6189/Screen%20shot%202012-01-25%20at%2014.04.53.png

[mlinkert]

Will: Branch has been updated to fix the CutIn? and CutOut? rounding:

​http://github.com/melissalinkert/bioformats/tree/6189-lif-emission-filters

The Detector.Offset problem is covered with #7897, right? So it doesn't need to be tested as part of this ticket?

[wmoore]

Great - Rounding is fixed! Offset is #7897 yes. Closing...

[wmoore]

Apologies for this...
I just tried importing lif christophe/014C1Z.lif and the filters are not created/linked correctly.

I've just been looking at the LifReader? code but can't work out what the bug is.
This file is different from the one we were using above, in that it only has a <Detector> and <MultiBand?> per <ATLConfocalSettingDefinition>
But ALL <Detector> elements have Channel="1" in the file. Is this the problem?

The correct wavelengths are:

PMT 1   (650nm - 706nm)	Active, Gain: 820, Offset: -4
PMT 1   (540nm - 615nm)	Active, Gain: 740, Offset: -12
PMT 1   (495nm - 550nm)	Active, Gain: 740, Offset: -8
PMT 1   (415nm - 470nm)	Active, Gain: 850, Offset: -4

These appear to be in the PFF metadata as

<DetectorList>
    <Detector Channel="1" IsActive="1" IsReferenceUnitActivatedForCorrection="0" Gain="849.996185244526" Offset="-12" Type="PMT 1" OffsetShow="-12" GainShow="850" ActiveShow="Active" Band="(415nm - 470nm)" LutName="Green" />
</DetectorList>
etc (one DetectorList for each channel - each with one 'Active' detector)

<Spectro>
  <MultiBand Channel="1" LeftWorld="649.856115107914" RightWorld="706.474820143885" DyeName="Leica/Cy5" />
</Spectro>

[jburel]

Moved from sprint 2012-02-28 (9)

[wmoore]

I have attempted to edit ome.xml from Martin's "sample files.lif" to add the correct Emission Filter links.
​https://github.com/will-moore/openmicroscopy/commit/12656d78d1e7725b736de54a70fab04c71ddf313
This is on branch omexml_repo. The original ome.xml (before the commit above) was generated from BF with the 6189-lif-emission-filters branch merged in.

[wmoore]

Renaming to reflect the current status of this ticket. This and #7897 are the only outstanding LIF bugs. Currently BF is not reading any Emission Filters for LIFs (all LightPath? elements are empty):

$ java -Xmx512m loci.formats.tools.ImageInfo -nopix -omexml-only -no-sas -xmlspaces 4 /Users/will/Documents/biology-data/Test-Import-Images/Martin/Leica-SP5/sample\ files.lif | grep "LightPath"
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>
                <LightPath/>

[mlinkert]

Some progress made here: ​https://github.com/melissalinkert/bioformats/commit/35db21770a9645a57a6992536c04dad0026ba7d2 (note that that's on sprint10-bug-fixes; the 6189-lif-emission-filters branch is no longer valid).

There are still some discrepancies in the linkages, but as far as I can tell the Channels are linked to Filters with the correct CutIn? and CutOut? values. If you could have a look quickly to make sure things are on the right track before I keep working on it, I'd appreciate it.

[jburel]

Moved from sprint 2012-03-13 (10)

[wmoore]

Using BF like this:

The <LightPath?>s are looking much better. Some differences:

Below, - is what is correct, + is what I'm getting from BF.

Here, PMTs 1, 2 and 4 are active, so the last Channel should link to the 4th filter (Filter:0:3)

             <Channel Color="16711935" ID="Channel:0:2" PinholeSize="95.5652364435265" SamplesPerPixel="1">
                 <LightPath>
-                    <EmissionFilterRef ID="Filter:0:3"/>
+                    <EmissionFilterRef ID="Filter:0:2"/>
                 </LightPath>

In this Image (6), only a single PMT is Active (PMT 4) so we need to link to the corresponding filter (6:3)

             <Channel Color="-16776961" ID="Channel:6:0" PinholeSize="106.08964270380099" SamplesPerPixel="1">
                 <LightPath>
-                    <EmissionFilterRef ID="Filter:6:3"/>
+                    <EmissionFilterRef ID="Filter:6:0"/>
                 </LightPath>

In Image 8, the filters that you're linking to at the moment have the same CutIn? and CutOut? as the ones I'm linking to. The only difference is that they don't have the Model names, E.g. <Filter ID="Filter:8:0" Model="SP Mirror Channel 1"> so we'd lose that info from the Channel.

-                    <EmissionFilterRef ID="Filter:8:0"/>
+                    <EmissionFilterRef ID="Filter:8:4"/>
...
-                    <EmissionFilterRef ID="Filter:8:1"/>
+                    <EmissionFilterRef ID="Filter:8:5"/>
...
-                    <EmissionFilterRef ID="Filter:8:2"/>
+                    <EmissionFilterRef ID="Filter:8:6"/>
...
-                    <EmissionFilterRef ID="Filter:8:3"/>
+                    <EmissionFilterRef ID="Filter:8:7"/>

The exact same thing is happening for Images 9, 10, 11.

Image 12 has the same problem as Image 0

             <Channel Color="-16776961" ID="Channel:12:2" PinholeSize="67.8860479790053" SamplesPerPixel="1">
                 <LightPath>
-                    <EmissionFilterRef ID="Filter:12:3"/>
+                    <EmissionFilterRef ID="Filter:12:2"/>
                 </LightPath>

Those are the only problems I see now.

I made a small update ​https://github.com/will-moore/openmicroscopy/commit/ce0b42986a9a45d711347531dce42dccfdd6989a

I also rebased onto develop recently and did push -f so you'll have to reset.

Thanks,

Will.

[jburel]

Moved from sprint 2012-03-27 (11)

[mlinkert]

Should now be fixed with ​https://github.com/melissalinkert/bioformats/commit/b7286224d83c58159e5bca42b3203661afea1393 (on my sprint12-bug-fixes branch).

Only difference that I can see is the linkages for Image:4; in checking against Leica's software, I think that what Bio-Formats produces with the above commit is correct.

[jburel]

Moved from sprint 2012-04-10 (12)

[wmoore]

Hi Melissa,

In the Leica metdata for series 03 - XZChRefl (when renderend with xsl in a browser) I see these are the two active channels (one per scan)

• PMT 4 (550nm - 800nm) Active, Gain: 362, Offset: 0
• PMT 4 (650nm - 800nm) Active, Gain: 402, Offset: 0

In the ome xml we have these filters for Image 4:

        <Filter ID="Filter:4:4">
            <TransmittanceRange CutIn="380" CutOut="439"/>
        </Filter>
        <Filter ID="Filter:4:5">
            <TransmittanceRange CutIn="550" CutOut="800"/>
        </Filter>
        <Filter ID="Filter:4:6">
            <TransmittanceRange CutIn="650" CutOut="800"/>
        </Filter>

So I think we should be linking like this:

                 <LightSourceSettings Attenuation="0.7999756" ID="LightSource:4:0"/>
                 <DetectorSettings Gain="362.058442053864" ID="Detector:4:0" Offset="0.0"/>
                 <LightPath>
                     <EmissionFilterRef ID="Filter:4:5"/>
                 </LightPath>
             </Channel>
             <Channel Color="-16776961" ExcitationWavelength="633" ID="Channel:4:1" Name="" PinholeSize="106.08964270380099" SamplesPerPixel="1">
                 <LightSourceSettings Attenuation="0.5599707" ID="LightSource:4:7"/>
                 <DetectorSettings Gain="402.265964751659" ID="Detector:4:1" Offset="0.0"/>
                 <LightPath>
                     <EmissionFilterRef ID="Filter:4:6"/>
                 </LightPath>

The diff between this and what BF is giving me just now is

                 <LightSourceSettings Attenuation="0.7999756" ID="LightSource:4:0"/>
                 <DetectorSettings Gain="362.058442053864" ID="Detector:4:0" Offset="0.0"/>
                 <LightPath>
-                    <EmissionFilterRef ID="Filter:4:5"/>
+                    <EmissionFilterRef ID="Filter:4:4"/>
                 </LightPath>
             </Channel>
             <Channel Color="-16776961" ExcitationWavelength="633" ID="Channel:4:1" Name="" PinholeSize="106.08964270380099" SamplesPerPixel="1">
                 <LightSourceSettings Attenuation="0.5599707" ID="LightSource:4:7"/>
                 <DetectorSettings Gain="402.265964751659" ID="Detector:4:1" Offset="0.0"/>
                 <LightPath>
-                    <EmissionFilterRef ID="Filter:4:6"/>
+                    <EmissionFilterRef ID="Filter:4:5"/>
                 </LightPath>

Does this correlate with what you're seeing?

[mlinkert]

Yep, that does match what I was seeing - I was just a bit confused, as opening the file in LAS AF gave me the impression that what Bio-Formats was producing was correct.

In any case, should really be fixed here: ​https://github.com/melissalinkert/bioformats/commit/d929bed58ec23d0205f00e95a5b3065ef91a5729 (sprint12-bug-fixes)

[wmoore]

Yey! Fixed ;)

The only remaing LIF ticket (only diff between BF and my omexml_repo) is #8168.

Closing...