Task #12447 (closed)
BUG: Duplicate Wells on Plate import
| Reported by: | ajpatterson | Owned by: | mlinkert |
|---|---|---|---|
| Priority: | major | Milestone: | 5.1.0-m3 |
| Component: | Import | Version: | 5.0.2 |
| Keywords: | n.a. | Cc: | ux@…, rleigh, mlinkert, khgillen, cxallan, sbesson, imunro-x |
| Resources: | n.a. | Referenced By: | n.a. |
| References: | n.a. | Remaining Time: | 0.0d |
| Sprint: | n.a. |
Description
This bug has appeared with an artificial OME-TIFF dataset, but might be present in other SPW formats.
A plate is saved across multiple files - e.g.
Well-A-1.ome.tif Well-A-2.ome.tif Well-B-1.ome.tif Well-B-2.ome.tif
If when you import the dataset by selecting one of the files (e.g. Well-B-1.ome.tif) but one of the files is missing (e.g. Well-A-1.ome.tif) then instead of Well A1 being blank it will have a duplicate of the data of Well B1. With real cell data this would be very hard to spot.
The resultant plate will still have all 4 well populated and I have attached a screenshot.
--
NOTE: All the images are also being inverted but this would seem to be unrelated. I have inverted the screenshot to make it possible to read the well numbers.
Attachments (4)
Change History (20)
Changed 5 years ago by ajpatterson
comment:1 Changed 5 years ago by pwalczysko
- Cc ux@… rleigh mlinkert added
comment:2 Changed 5 years ago by pwalczysko
I have tried to import Cellworx plate from test_images_good. First I imported the plate as it is, with first well being B02. Then I deleted all the …scan.log and .pnl files with B02 in them. I also deleted both of the .tiffs in the TIFF folder with B02 in their name. After import of a plate distorted in this way, Insight still shows a well B02, but the image inside it has no data - seems just a 2 channel image with 2 black images. There was no false data duplication in this case, all the other wells seem to have some valid data in them.
But probably the test here is not as easy as this, but not sure how to perform it better - had a look at bd-pathway, but the data there are organised in a manner that there is a subfolders for each particular well. This does not seem to fit the data structure described here.
Any idea how to test this properly please @mlinkert ?
comment:3 Changed 5 years ago by mlinkert
Probably the best thing to do is to get the original dataset from Andrew, as this is likely an issue specific to OME-TIFF.
Changed 5 years ago by ajpatterson
Changed 5 years ago by ajpatterson
comment:4 Changed 5 years ago by mlinkert
- Owner changed from pwalczysko to mlinkert
comment:5 Changed 5 years ago by mlinkert
- Resolution set to fixed
- Status changed from new to closed
Fixed with: https://github.com/openmicroscopy/bioformats/pull/1384
As noted in the PR, the "inversion" issue is a result of the pixel type being set to int8 in the OME-XML stored in the files; this is not a reader issue and so is not addressed.
comment:6 Changed 5 years ago by Melissa Linkert <melissa@…>
- Remaining Time set to 0
(In [5c1d25536202cda9dc74eb36187dfc5ac9da46aa/bioformats.git] on branch develop) OME-TIFF: return blank planes if the corresponding file does not exist
Fixes #12447.
comment:7 Changed 4 years ago by pwalczysko
- Cc changed from ux@openmicroscopy.org.uk, rleigh, mlinkert, to ux@openmicroscopy.org.uk, rleigh, mlinkert
Re-attaching the two files again (delivered locally by ajpatterson) so that they are here. For trac fixing purposes, see the files attached in 2014 please.
comment:8 Changed 4 years ago by pwalczysko
- Cc khgillen cxallan added
comment:9 Changed 4 years ago by pwalczysko
- Cc sbesson added
As noted in jabber, there might be a problem with attachments getting lost as a result of trac migration. This has been
comment:10 Changed 4 years ago by pwalczysko
highlighted here, by the two attachments made in 2014 gone missing.
comment:11 Changed 4 years ago by pwalczysko
I just think I will not reattach here, but rather to the trello card, https://trello.com/c/RTlxoIdn/22-plate-binary-only-samples
comment:12 Changed 4 years ago by pwalczysko
After retest, I do not have any duplication of any well anymore as highlighted in the ticket. The missing well has a presence though, (the well B1, second row in first column, see screenshot, you can find the well on trout/merge user-4, read-only-1 group, Screen ID 652 and 654. Screenshot attached.
ajpatterson mlinkert is this expected ?
Changed 4 years ago by pwalczysko
comment:13 Changed 4 years ago by pwalczysko
On a second thought, I would not think this is acceptable with any normal plate - see the plate ID 903 in the same group/user - this is how it shoiuld look like, the field should be completely missing. On the other hand, the pixel values seem to be all zero. imunro-x - your opinion ?
comment:14 Changed 4 years ago by pwalczysko
- Cc imunro-x added
comment:15 Changed 4 years ago by imunro-x
Probably this won't affect us as the current plan is to have all wells in a single file. My expectation, as a user, would be to find only the wells corresponding to the imported files populated - but i guess zero pixels might be acceptable as second best.
comment:16 Changed 4 years ago by mlinkert
That is all completely expected. The issue is that well B1 and all planes associated with it are defined in the metadata - removing the B1 file removes the pixels, but not the metadata. So you see black planes to show that the well was acquired, but the pixel data could not be found.
If the B1 well were not defined in the OME-XML, then nothing would appear in the B1 space in Insight/Web? (not even black pixels).
Inverted screenshot showing duplicated image data.