BUG: Integer overflow in ROMIO-QA4806

[omero-qa]

​https://www.openmicroscopy.org/qa2/qa2/qa/feedback/4806/

omero.ApiUsageException
    serverStackTrace = "ome.conditions.ApiUsageException: Converting Long 2910846976 to Integer is an overflow.
                        	at ome.io.nio.RomioPixelBuffer.safeLongToInteger(RomioPixelBuffer.java:121)
                        	at ome.io.nio.RomioPixelBuffer.getTimepoint(RomioPixelBuffer.java:598)
                        	at ome.io.nio.RomioPixelBuffer.calculateMessageDigest(RomioPixelBuffer.java:764)
                        	at ome.services.RawPixelsBean.save(RawPixelsBean.java:181)
                        	at ome.services.RawPixelsBean.close(RawPixelsBean.java:209)
                        	at sun.reflect.GeneratedMethodAccessor572.invoke(Unknown Source)
                        	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                        	at java.lang.reflect.Method.invoke(Method.java:601)
                        	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
                        	at ome.security.basic.EventHandler.invoke(EventHandler.java:154)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                        	at ome.tools.hibernate.SessionHandler.doStateful(SessionHandler.java:182)
                        	at ome.tools.hibernate.SessionHandler.invoke(SessionHandler.java:166)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                        	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:108)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                        	at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:241)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                        	at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:116)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                        	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
                        	at $Proxy88.close(Unknown Source)
                        	at sun.reflect.GeneratedMethodAccessor572.invoke(Unknown Source)
                        	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                        	at java.lang.reflect.Method.invoke(Method.java:601)
                        	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
                        	at ome.security.basic.BasicSecurityWiring.invoke(BasicSecurityWiring.java:98)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                        	at ome.services.blitz.fire.AopContextInitializer.invoke(AopContextInitializer.java:43)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                        	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
                        	at $Proxy88.close(Unknown Source)
                        	at ome.services.blitz.impl.AbstractAmdServant.close_async(AbstractAmdServant.java:241)
                        	at sun.reflect.GeneratedMethodAccessor624.invoke(Unknown Source)
                        	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                        	at java.lang.reflect.Method.invoke(Method.java:601)
                        	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
                        	at omero.cmd.CallContext.invoke(CallContext.java:59)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                        	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
                        	at $Proxy89.close_async(Unknown Source)
                        	at omero.api._RawPixelsStoreTie.close_async(_RawPixelsStoreTie.java:339)
                        	at omero.api._StatefulServiceInterfaceDisp.___close(_StatefulServiceInterfaceDisp.java:191)
                        	at omero.api._RawPixelsStoreDisp.__dispatch(_RawPixelsStoreDisp.java:1444)
                        	at IceInternal.Incoming.invoke(Incoming.java:159)
                        	at Ice.ConnectionI.invokeAll(ConnectionI.java:2357)
                        	at Ice.ConnectionI.dispatch(ConnectionI.java:1208)
                        	at Ice.ConnectionI.message(ConnectionI.java:1163)
                        	at IceInternal.ThreadPool.run(ThreadPool.java:302)
                        	at IceInternal.ThreadPool.access$300(ThreadPool.java:12)
                        	at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:643)
                        	at java.lang.Thread.run(Thread.java:722)
                        "
    serverExceptionClass = "ome.conditions.ApiUsageException"
    message = "Converting Long 2910846976 to Integer is an overflow."
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
	at java.lang.reflect.Constructor.newInstance(Unknown Source)
	at java.lang.Class.newInstance0(Unknown Source)
	at java.lang.Class.newInstance(Unknown Source)
	at IceInternal.BasicStream$DynamicUserExceptionFactory.createAndThrow(BasicStream.java:2243)
	at IceInternal.BasicStream.throwException(BasicStream.java:1632)
	at IceInternal.Outgoing.throwUserException(Outgoing.java:442)
	at omero.api._RawPixelsStoreDelM.close(_RawPixelsStoreDelM.java:1901)
	at omero.api.RawPixelsStorePrxHelper.close(RawPixelsStorePrxHelper.java:2642)
	at omero.api.RawPixelsStorePrxHelper.close(RawPixelsStorePrxHelper.java:2614)
	at ome.formats.OMEROMetadataStoreClient.finalizePixelStore(OMEROMetadataStoreClient.java:2484)
	at ome.formats.importer.ImportLibrary.importImage(ImportLibrary.java:593)
	at org.openmicroscopy.shoola.env.data.OMEROGateway.importImage(OMEROGateway.java:6335)
	at org.openmicroscopy.shoola.env.data.OmeroImageServiceImpl.importFile(OmeroImageServiceImpl.java:1212)
	at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter.importFile(ImagesImporter.java:85)
	at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter.access$000(ImagesImporter.java:54)
	at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter$1.doCall(ImagesImporter.java:110)
	at org.openmicroscopy.shoola.env.data.views.BatchCall.doStep(BatchCall.java:144)
	at org.openmicroscopy.shoola.util.concur.tasks.CompositeTask.doStep(CompositeTask.java:226)
	at org.openmicroscopy.shoola.env.data.views.CompositeBatchCall.doStep(CompositeBatchCall.java:126)
	at org.openmicroscopy.shoola.util.concur.tasks.ExecCommand.exec(ExecCommand.java:165)
	at org.openmicroscopy.shoola.util.concur.tasks.ExecCommand.run(ExecCommand.java:276)
	at org.openmicroscopy.shoola.util.concur.tasks.AsyncProcessor$Runner.run(AsyncProcessor.java:91)
	at java.lang.Thread.run(Unknown Source)

[jmoore]

Also ​http://qa.openmicroscopy.org.uk/qa/feedback/4846

[mlinkert]

Can be reproduced on dev_4_4 by running:

touch "ticket-10073&sizeX=1024&sizeY=1024&sizeZ=2100&sizeC=1&sizeT=1.fake"

and then importing the resulting .fake file. The same file imports and displays correctly on develop.

[mlinkert]

I would think that calculating the hash by looping over getPlane(z, c, t) instead of getTimepoint(t) would be sufficient to fix this, but I don't know enough about how RomioPixelBuffer? is used to know if there are any negative implications to doing that.

[jmoore]

Melissa, I would think inlining the loop can't possibly hurt (other than possibly a slight slow down). We could catch the overflow and only fallback to plane looping if the timepoint is too large.

CC Mark & Blazej since we'll soon be working more on the hashing code anyway.

[Chris Allan <callan@…>]

(In [1aa3775ae092729be5757efd4484136c54f85190/ome.git] on branch develop) Calculate hash digests on a per plane basis (Fixes #10073)

There are some edge cases, such as electron microscopy, where the stack
size is huge. Often 1000's of planes, which causes the byte size of a
timepoint to be larger than 2.1GB. As the hash digest algoritms we use
are CPU limited anyway there is very little justification for reading
the entire timepoint anyway. Calculating it on a per plane basis is
much safer and should have no performance implications at all.

[jean-marie burel <j.burel@…>]

(In [b51da7d66fa37238a18353baeb264b2949399b05/ome.git] on branch develop) Merge pull request #678 from chris-allan/10073-timepoint-overflow-develop

Calculate hash digests on a per plane basis (Fixes #10073)