BUG: error on import - pyramid already exists

[pwalczysko]

On 08-05-13 working on Howe.
User-4.
Imported test_images_good/svs/alexandra folder.
Import went through apparently fine, although very quickly.
All thumbs had placeholder clocks.
After this, imported test_images_good/trestle/CMU-1 folder.
The import failed. The error (from Insight importer window) see below:

omero.ResourceError

   serverStackTrace = "ome.conditions.ResourceError: Destination '/OMERO/Pixels/Dir-059/59879_pyramid' already exists Please check server log.

                           at ome.services.RawPixelsBean.closePixelBuffer(RawPixelsBean.java:248)

                           at ome.services.RawPixelsBean.setPixelsId(RawPixelsBean.java:258)

                           at sun.reflect.GeneratedMethodAccessor929.invoke(Unknown Source)

                           at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

                           at java.lang.reflect.Method.invoke(Method.java:592)

                           at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)

                           at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)

                           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)

                           at ome.security.basic.EventHandler.invoke(EventHandler.java:154)

                           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(Ref

Insight log attached.
With help of @bpindelski and @jamoore, had a bits of Blitz.log.

2013-05-08 11:15:28,819 INFO  [                ome.io.nio.PixelsService] (l.Server-1) Missing pyramid:/OMERO/Pixels/Dir-059/59867_pyramid
2013-05-08 11:15:28,819 WARN  [         ome.logic.RenderingSettingsImpl] (l.Server-1) MissingPyramidException, not resetting settings for Image:60417
2013-05-08 11:15:28,820 INFO  [             ome.io.nio.FilePathResolver] (l.Server-1) Metadata only file, resulting path: /OMERO/Files/Dir-054/54821
2013-05-08 11:15:28,820 INFO  [  ome.services.pixeldata.PixelDataThread] (l.Server-1) Received: ome.io.messages.MissingPyramidMessage[source=ome.io.nio.PixelsService@6680f297]

The Insight UI was not showing any progress on the waiting thumbnails of the svs imported in the first round. @jamoore advised after investigation that this is because of the following image in the database.

omero=# select * from originalfile where id = 48918;
-[ RECORD 1 ]-----------------------------------------
id          | 48918
atime       |
ctime       |
permissions | -56
mimetype    | TiffDelegate
mtime       |
name        | foo-1048576x256.tif
path        | /Users/simon/Desktop/
sha1        | 2da0069dfb149d1b26f0b84a26810a6ab8de7317
size        | 1610615153
version     |
creation_id | 716028
external_id |
group_id    | 8
owner_id    | 5
update_id   | 716029
repo        |
params      |

The image was evidently the cause of the problem - after it was deleted by @jamoore, the svs images thumbnails started to appear and the rendering of the imported svs was successful. Also further subsequent import of the svs/alexandra folder was successsfull.
The suggested course of action was to repeat the workflow - for this the original .tiff file is needed from @spli . Could you please retrieve it Simon ?

[pwalczysko]

The path to the image in question is squig: ome/team/simon/test_images/foo-1048576x256.tif.

[mlinkert]

Picking this up for further investigation.

[mlinkert]

I haven't been able to reproduce the import error so far. It seems like there are two separate issues: one is that the svs/alexandra pyramids did not start generating right away (due to foo-1048576x256.tif having been imported and being in the middle of pyramid generation), and the other is that the trestle/CMU-1 import failed. The former is what I would expect to have happen, and the fact that deleting foo-1048576x256.tif causes the SVS pyramids to start generating also is what I would expect.

So far the only reason I can think of for the trestle/CMU-1 import failing is that somehow the /OMERO/Pixels directory wasn't properly cleaned up at some point. That still needs further investigation though.

[mlinkert]

I've only been able to duplicate the import of trestle/CMU-1/CMU-1.tif failing by doing the following:

1. drop and recreate the database and wipe out /OMERO
2. import trestle/CMU-1/CMU-1.tif
3. import tif/4kx4k.tif
4. wait for all pyramids to finish generating
5. drop and recreate the database (leaving /OMERO intact)
6. import trestle/CMU-1/CMU-1.tif

The stack trace is different now though (using the current dev_4_4 merge branch):

   serverStackTrace = "ome.conditions.ApiUsageException: In read-only mode!
                          at ome.io.bioformats.BfPyramidPixelBuffer.setTile(BfPyramidPixelBuffer.java:480)
                          at ome.services.RawPixelsBean.setTile(RawPixelsBean.java:792)
                          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
                          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                          at java.lang.reflect.Method.invoke(Method.java:601)
                          at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
                          at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
                          at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
                          at ome.security.basic.EventHandler.invoke(EventHandler.java:154)
                          at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                          at ome.tools.hibernate.SessionHandler.doStateful(SessionHandler.java:182)
                          at ome.tools.hibernate.SessionHandler.invoke(SessionHandler.java:166)
                          at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                          at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:108)
                          at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                          at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:241)
                          at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                          at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:116)
                          at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                          at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
                          at $Proxy90.setTile(Unknown Source)

As far as I can tell, with the current merge branch there is no way that the original stack trace could occur (based upon the exception message at least).

I'd be inclined to close this as not being a bug, since an exception when trying to overwrite a pyramid is expected, and this seems to only occur with dirty /OMERO/Pixels/ directories. Happy to hear other suggestions for things to test before closing, though.

[mlinkert]

Anyone have any further thoughts on this, or can I close it?

[jamoore]

No vetoes here.

[pwalczysko]

No vetoes here.