BUG: Unknown format exception

[rkferguson]

Problem importing SBIG image M81 M/40s.

Could see it in file finder window, but name had scissors symbol instead of / in it - see screenshot. Selected fine and then when clicked add to queue - nothing happened.

To check, I removed the / from the name in the Finder and it then added to queue and started to imported but gave unknown format exception error message below.

Tested both differently named images with Mac version of Insight and both queued but had import failure for both.

loci.formats.UnknownFormatException: Unknown file format: Y:\data_repo\test_images_good\sbig\M81 M40s copy
     at loci.formats.ImageReader.getReader(ImageReader.java:192)
     at loci.formats.ImageReader.setId(ImageReader.java:727)
     at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:529)
     at loci.formats.ChannelFiller.setId(ChannelFiller.java:263)
     at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:529)
     at loci.formats.ChannelSeparator.setId(ChannelSeparator.java:274)
     at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:529)
     at ome.formats.importer.ImportLibrary.open(ImportLibrary.java:311)
     at ome.formats.importer.ImportLibrary.importImage(ImportLibrary.java:487)
     at org.openmicroscopy.shoola.env.data.OMEROGateway.importImage(OMEROGateway.java:6324)
     at org.openmicroscopy.shoola.env.data.OmeroImageServiceImpl.importFile(OmeroImageServiceImpl.java:1248)
     at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter.importFile(ImagesImporter.java:85)
     at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter.access$000(ImagesImporter.java:54)
     at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter$1.doCall(ImagesImporter.java:110)
     at org.openmicroscopy.shoola.env.data.views.BatchCall.doStep(BatchCall.java:144)
     at org.openmicroscopy.shoola.util.concur.tasks.CompositeTask.doStep(CompositeTask.java:226)
     at org.openmicroscopy.shoola.env.data.views.CompositeBatchCall.doStep(CompositeBatchCall.java:126)
     at org.openmicroscopy.shoola.util.concur.tasks.ExecCommand.exec(ExecCommand.java:165)
     at org.openmicroscopy.shoola.util.concur.tasks.ExecCommand.run(ExecCommand.java:276)
     at org.openmicroscopy.shoola.util.concur.tasks.AsyncProcessor$Runner.run(AsyncProcessor.java:91)
     at java.lang.Thread.run(Unknown Source)

     at org.openmicroscopy.shoola.env.data.OMEROGateway.importImage(OMEROGateway.java:6373)
     at org.openmicroscopy.shoola.env.data.OmeroImageServiceImpl.importFile(OmeroImageServiceImpl.java:1248)
     at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter.importFile(ImagesImporter.java:85)
     at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter.access$000(ImagesImporter.java:54)
     at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter$1.doCall(ImagesImporter.java:110)
     at org.openmicroscopy.shoola.env.data.views.BatchCall.doStep(BatchCall.java:144)
     at org.openmicroscopy.shoola.util.concur.tasks.CompositeTask.doStep(CompositeTask.java:226)
     at org.openmicroscopy.shoola.env.data.views.CompositeBatchCall.doStep(CompositeBatchCall.java:126)
     at org.openmicroscopy.shoola.util.concur.tasks.ExecCommand.exec(ExecCommand.java:165)
     at org.openmicroscopy.shoola.util.concur.tasks.ExecCommand.run(ExecCommand.java:276)
     at org.openmicroscopy.shoola.util.concur.tasks.AsyncProcessor$Runner.run(AsyncProcessor.java:91)
     at java.lang.Thread.run(Unknown Source)
Caused by: loci.formats.UnknownFormatException: Unknown file format: Y:\data_repo\test_images_good\sbig\M81 M40s copy
     at loci.formats.ImageReader.getReader(ImageReader.java:192)
     at loci.formats.ImageReader.setId(ImageReader.java:727)
     at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:529)
     at loci.formats.ChannelFiller.setId(ChannelFiller.java:263)
     at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:529)
     at loci.formats.ChannelSeparator.setId(ChannelSeparator.java:274)
     at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:529)
     at ome.formats.importer.ImportLibrary.open(ImportLibrary.java:311)
     at ome.formats.importer.ImportLibrary.importImage(ImportLibrary.java:487)
     at org.openmicroscopy.shoola.env.data.OMEROGateway.importImage(OMEROGateway.java:6324)
     ... 11 more

[mlinkert]

Somehow copying the file didn't go quite right, as the two files were not equal:

mlinkert-x@necromancer /ome/data_repo/test_images_good/sbig $ md5sum *
ea8e665500c2a3a7ac3263fd793114c3  M81 M40s copy
f0d98451235181af57f29b453fb8fa9c  M81 M:40s

I took the liberty of fixing "M81 M40s copy" so that it now has an identical md5sum, and so both files do import for me using merge-blue build 196. Gus, can you verify that the fixed file imports for you as well?

[rkferguson]

Retesting on Windows 7 with today's build OMERO.insight-4.4.1-559-5aa58b8-ice33-b169-win:

Checked in OSX Finder - both original - M81 M/40s - and - M81 M40s copy - can be seen.

In File Finder in Insight the sbig folder in squig > test_images_good appeared empty.

Went ahead and tried importing the sbig folder as was. Insight elected to put it in screens. Went ahead - only the copy of the data - M81 M40s copy - showed as imported, the oriinal image did not appear. Refreshed data manager and could not find image anywhere in either screens or projects. Went back to importer results tab and no destination displayed. Double clicked on image and it opened in an image viewer window from the importer window.

Went back to select data to import tag, and could now see both images in the sbig folder. Selected them both and it allowed me to choose project as destination this time -imported them. Again only - M81 M40s copy - imported, with no sign of original. This time in expected location in projects tab of data manager.

Screenshots available if you want them.

[mlinkert]

The results of the first test are exactly what I'd expect if the "Files of type" box was accidentally set to "All supported HCS types", so that's not a bug in my opinion. Note that if Insight guesses incorrectly you can switch back to "Project/Dataset?" by clicking the button in the upper-left.

I haven't been able to duplicate the issue of only one file importing, though. In all of my tests, both files are recognized and imported. I do wonder if this isn't some sort of subtle connection issue, given that you have reported other import problems that came down to trouble connecting to squig.

Can you please try copying the 'sbig' folder to a local disk and importing from there? If that also doesn't work, then please check that the md5sums of both of the local files are f0d98451235181af57f29b453fb8fa9c and that the file names are "M81 M:40s" and "M81 M40s copy" (there should be no '/' in either name).

[rkferguson]

On LAN at Dundee - no network issues.

Viewing sbig folder on squig - the name definitely has a / in it: M81 M/40s - see: 9597 Screenshot.jpg

Tired copying folder from squig to local disc - got error - see: 9597 Screenshot 2.jpg

When looked at what had copied over - it was only M81 M/40s - M81 M/40s copy had not come over - see: 9597 Screenshot 3.jpg

Copied over M81 M/40s copy separately - worked see: 9597 Screenshot 4.jpg

Tried to change M81 M/40s to M81 M:40s - not allowed error message: see screenshot 9597 Screenshot 5.jpg

Md5 checksums do not match:

ls25532:~ rkferguson$ md5 /Users/rkferguson/Desktop/sbig/M81\ M:40s MD5 (/Users/rkferguson/Desktop/sbig/M81 M:40s) = a34c339ffe3f82df01f0b0a3c926e9e8

ls25532:~ rkferguson$ md5 /Users/rkferguson/Desktop/sbig/M81\ M40s\ copy MD5 (/Users/rkferguson/Desktop/sbig/M81 M40s copy) = f0d98451235181af57f29b453fb8fa9c

Importing tried with todays build: OMERO.insight-4.4.1-567-f247308-dirty-ice33-b171-mac on Mac OS 10.6

Can see both files in importer see: 9597 Screenshot 6.jpg

Imported whole sbig folder: only M81 M/40s copy shown - see: 9597 Screenshot 7.jpg

M81 M/40s copy - successfully imported see: 9597 Screenshot 8.jpg

When tried to import M81 M:40s separately - failed see: 9597 Screenshot 9.jpg

Insight does not appear to have written to the log file for this session.

Screenshots attached.

Error message from import fail:

loci.formats.UnknownFormatException: Unknown file format: /Users/rkferguson/Desktop/sbig/M81 M:40s
     at loci.formats.ImageReader.getReader(ImageReader.java:192)
     at loci.formats.ImageReader.setId(ImageReader.java:727)
     at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:529)
     at loci.formats.ChannelFiller.setId(ChannelFiller.java:263)
     at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:529)
     at loci.formats.ChannelSeparator.setId(ChannelSeparator.java:274)
     at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:529)
     at ome.formats.importer.ImportLibrary.open(ImportLibrary.java:313)
     at ome.formats.importer.ImportLibrary.importImage(ImportLibrary.java:489)
     at org.openmicroscopy.shoola.env.data.OMEROGateway.importImage(OMEROGateway.java:6400)
     at org.openmicroscopy.shoola.env.data.OmeroImageServiceImpl.importFile(OmeroImageServiceImpl.java:1250)
     at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter.importFile(ImagesImporter.java:85)
     at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter.access$000(ImagesImporter.java:54)
     at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter$1.doCall(ImagesImporter.java:110)
     at org.openmicroscopy.shoola.env.data.views.BatchCall.doStep(BatchCall.java:144)
     at org.openmicroscopy.shoola.util.concur.tasks.CompositeTask.doStep(CompositeTask.java:226)
     at org.openmicroscopy.shoola.env.data.views.CompositeBatchCall.doStep(CompositeBatchCall.java:126)
     at org.openmicroscopy.shoola.util.concur.tasks.ExecCommand.exec(ExecCommand.java:165)
     at org.openmicroscopy.shoola.util.concur.tasks.ExecCommand.run(ExecCommand.java:276)
     at org.openmicroscopy.shoola.util.concur.tasks.AsyncProcessor$Runner.run(AsyncProcessor.java:91)
     at java.lang.Thread.run(Thread.java:680)

     at org.openmicroscopy.shoola.env.data.OMEROGateway.importImage(OMEROGateway.java:6449)
     at org.openmicroscopy.shoola.env.data.OmeroImageServiceImpl.importFile(OmeroImageServiceImpl.java:1250)
     at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter.importFile(ImagesImporter.java:85)
     at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter.access$000(ImagesImporter.java:54)
     at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter$1.doCall(ImagesImporter.java:110)
     at org.openmicroscopy.shoola.env.data.views.BatchCall.doStep(BatchCall.java:144)
     at org.openmicroscopy.shoola.util.concur.tasks.CompositeTask.doStep(CompositeTask.java:226)
     at org.openmicroscopy.shoola.env.data.views.CompositeBatchCall.doStep(CompositeBatchCall.java:126)
     at org.openmicroscopy.shoola.util.concur.tasks.ExecCommand.exec(ExecCommand.java:165)
     at org.openmicroscopy.shoola.util.concur.tasks.ExecCommand.run(ExecCommand.java:276)
     at org.openmicroscopy.shoola.util.concur.tasks.AsyncProcessor$Runner.run(AsyncProcessor.java:91)
     at java.lang.Thread.run(Thread.java:680)
Caused by: loci.formats.UnknownFormatException: Unknown file format: /Users/rkferguson/Desktop/sbig/M81 M:40s
     at loci.formats.ImageReader.getReader(ImageReader.java:192)
     at loci.formats.ImageReader.setId(ImageReader.java:727)
     at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:529)
     at loci.formats.ChannelFiller.setId(ChannelFiller.java:263)
     at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:529)
     at loci.formats.ChannelSeparator.setId(ChannelSeparator.java:274)
     at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:529)
     at ome.formats.importer.ImportLibrary.open(ImportLibrary.java:313)
     at ome.formats.importer.ImportLibrary.importImage(ImportLibrary.java:489)
     at org.openmicroscopy.shoola.env.data.OMEROGateway.importImage(OMEROGateway.java:6400)
     ... 11 more

[rkferguson]

Changing file name from M81 M:40s to M81 M_40s resulted in return of md5 checksum to same as copy.

Import was then successful. Appears : is a reserved character in path names.