Notice: In order to edit this ticket you need to be either: a Product Owner, The owner or the reporter of the ticket, or, in case of a Task not yet assigned, a team_member"

Task #12485 (closed)

Opened 5 years ago

Closed 5 years ago

Bug: reduce the number of companion files reported by VolocityReader

Reported by: mlinkert Owned by: mlinkert
Priority: critical Milestone: 5.1.0-m3
Component: Bio-Formats Version: 5.0.2
Keywords: n.a. Cc: johnbillheimer@…, jamoore, mtbcarroll
Resources: n.a. Referenced By: n.a.
References: n.a. Remaining Time: n.a.
Sprint: n.a.

Description

See QA 9407. Large Volocity datasets throw Ice MemoryLimitExceptions? on import, as the "companion" binary data files can be very large. Adjusting getSeriesUsedFiles to identify most/all files as pixels files should fix this.

Change History (7)

comment:1 Changed 5 years ago by mlinkert

  • Cc jamoore added

Full stack trace on import with 5.0.3-rc1:

2014-07-22 11:10:12,041 1002091    [l.Client-4] ERROR     ome.formats.importer.cli.ErrorHandler - INTERNAL_EXCEPTION: /home/melissa/data/volocity/qa-9407/Imaging.mvd2
java.lang.RuntimeException: Failure response on import!
Category: ::omero::grid::ImportRequest
Name: import-request-failure
Parameters: {message=, stacktrace=Ice.MemoryLimitException
    reason = "requested 67108865 bytes, maximum allowed is 67108864 bytes (see Ice.MessageSizeMax)"
	at IceInternal.Ex.throwMemoryLimitException(Ex.java:23)
	at IceInternal.BasicStream.expand(BasicStream.java:2049)
	at IceInternal.BasicStream.writeInt(BasicStream.java:888)
	at IceInternal.BasicStream.writeObject(BasicStream.java:1362)
	at omero.model.OriginalFile.__write(OriginalFile.java:1477)
	at IceInternal.BasicStream.writeInstance(BasicStream.java:1721)
	at IceInternal.BasicStream.writePendingObjects(BasicStream.java:1639)
	at omero.api._IUpdateDelM.saveCollection(_IUpdateDelM.java:348)
	at omero.api.IUpdatePrxHelper.saveCollection(IUpdatePrxHelper.java:994)
	at omero.api.IUpdatePrxHelper.saveCollection(IUpdatePrxHelper.java:967)
	at ome.formats.OMEROMetadataStoreClient.attachCompanionFilesToImage(OMEROMetadataStoreClient.java:794)
	at ome.services.blitz.repo.ManagedImportRequestI.importMetadata(ManagedImportRequestI.java:514)
	at ome.services.blitz.repo.ManagedImportRequestI.step(ManagedImportRequestI.java:379)
	at omero.cmd.HandleI.steps(HandleI.java:436)
	at omero.cmd.HandleI$1.doWork(HandleI.java:365)
	at omero.cmd.HandleI$1.doWork(HandleI.java:361)
	at sun.reflect.GeneratedMethodAccessor239.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
	at ome.services.util.Executor$Impl$Interceptor.invoke(Executor.java:576)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at ome.security.basic.EventHandler.invoke(EventHandler.java:154)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:111)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:108)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:241)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:116)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
	at com.sun.proxy.$Proxy69.doWork(Unknown Source)
	at ome.services.util.Executor$Impl.execute(Executor.java:457)
	at omero.cmd.HandleI.run(HandleI.java:359)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at ome.services.util.Executor$Impl$1.call(Executor.java:498)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)
}

	at ome.formats.importer.ImportLibrary$ImportCallback.onFinished(ImportLibrary.java:627)
	at omero.cmd.CmdCallbackI.finished(CmdCallbackI.java:301)
	at omero.cmd._CmdCallbackDisp.___finished(_CmdCallbackDisp.java:130)
	at omero.cmd._CmdCallbackDisp.__dispatch(_CmdCallbackDisp.java:157)
	at IceInternal.Incoming.invoke(Incoming.java:159)
	at Ice.ConnectionI.invokeAll(ConnectionI.java:2357)
	at Ice.ConnectionI.dispatch(ConnectionI.java:1208)
	at Ice.ConnectionI.message(ConnectionI.java:1163)
	at IceInternal.ThreadPool.run(ThreadPool.java:302)
	at IceInternal.ThreadPool.access$300(ThreadPool.java:12)
	at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:643)
	at java.lang.Thread.run(Thread.java:744)

java.lang.RuntimeException: Failure response on import!
Category: ::omero::grid::ImportRequest
Name: import-request-failure
Parameters: {message=, stacktrace=Ice.MemoryLimitException
    reason = "requested 67108865 bytes, maximum allowed is 67108864 bytes (see Ice.MessageSizeMax)"
	at IceInternal.Ex.throwMemoryLimitException(Ex.java:23)
	at IceInternal.BasicStream.expand(BasicStream.java:2049)
	at IceInternal.BasicStream.writeInt(BasicStream.java:888)
	at IceInternal.BasicStream.writeObject(BasicStream.java:1362)
	at omero.model.OriginalFile.__write(OriginalFile.java:1477)
	at IceInternal.BasicStream.writeInstance(BasicStream.java:1721)
	at IceInternal.BasicStream.writePendingObjects(BasicStream.java:1639)
	at omero.api._IUpdateDelM.saveCollection(_IUpdateDelM.java:348)
	at omero.api.IUpdatePrxHelper.saveCollection(IUpdatePrxHelper.java:994)
	at omero.api.IUpdatePrxHelper.saveCollection(IUpdatePrxHelper.java:967)
	at ome.formats.OMEROMetadataStoreClient.attachCompanionFilesToImage(OMEROMetadataStoreClient.java:794)
	at ome.services.blitz.repo.ManagedImportRequestI.importMetadata(ManagedImportRequestI.java:514)
	at ome.services.blitz.repo.ManagedImportRequestI.step(ManagedImportRequestI.java:379)
	at omero.cmd.HandleI.steps(HandleI.java:436)
	at omero.cmd.HandleI$1.doWork(HandleI.java:365)
	at omero.cmd.HandleI$1.doWork(HandleI.java:361)
	at sun.reflect.GeneratedMethodAccessor239.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
	at ome.services.util.Executor$Impl$Interceptor.invoke(Executor.java:576)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at ome.security.basic.EventHandler.invoke(EventHandler.java:154)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:111)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:108)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:241)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:116)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
	at com.sun.proxy.$Proxy69.doWork(Unknown Source)
	at ome.services.util.Executor$Impl.execute(Executor.java:457)
	at omero.cmd.HandleI.run(HandleI.java:359)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at ome.services.util.Executor$Impl$1.call(Executor.java:498)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)
}

	at ome.formats.importer.ImportLibrary$ImportCallback.onFinished(ImportLibrary.java:627) ~[blitz.jar:na]
	at omero.cmd.CmdCallbackI.finished(CmdCallbackI.java:301) [blitz.jar:na]
	at omero.cmd._CmdCallbackDisp.___finished(_CmdCallbackDisp.java:130) [blitz.jar:na]
	at omero.cmd._CmdCallbackDisp.__dispatch(_CmdCallbackDisp.java:157) [blitz.jar:na]
	at IceInternal.Incoming.invoke(Incoming.java:159) [ice.jar:na]
	at Ice.ConnectionI.invokeAll(ConnectionI.java:2357) [ice.jar:na]
	at Ice.ConnectionI.dispatch(ConnectionI.java:1208) [ice.jar:na]
	at Ice.ConnectionI.message(ConnectionI.java:1163) [ice.jar:na]
	at IceInternal.ThreadPool.run(ThreadPool.java:302) [ice.jar:na]
	at IceInternal.ThreadPool.access$300(ThreadPool.java:12) [ice.jar:na]
	at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:643) [ice.jar:na]
	at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]

comment:2 Changed 5 years ago by jamoore

  • Cc mtbcarroll added
  • Priority changed from major to critical

Looking at that stacktrace, this is a matter of the number of files as opposed to their size, right? (Do you know how many?) If so, I'd think:

  • short-term solution (5.0) : save the annotations links in batches
  • middle-term solution (5.1): evaluate not creating these annotations. With FS, it's really unnecessary.

comment:3 Changed 5 years ago by mlinkert

Should be 994 files, ranging from 512 bytes to 61MB - guess I assumed size was the issue based upon the size of the largest files.

comment:4 Changed 5 years ago by jamoore

​https://github.com/openmicroscopy/openmicroscopy/pull/2853 has been merged which should correct the most immediate error for 5.0.3. The Volocity fix itself and the middle-term removal of these companion files are still outstanding.

comment:5 Changed 5 years ago by jmoore <josh@…>

(In [e1589b08186a99a882a7a6145aeb09b472f5d39c/ome.git] on branch develop) Add batching to attachCompanionFilesToImage (See #12485)

comment:6 Changed 5 years ago by Josh Moore <josh@…>

(In [4652484245472e6d31e497ee320d14c318e86dd9/ome.git] on branch develop) Merge pull request #2867 from joshmoore/rebased/develop/12485-companion-loop

Add batching to attachCompanionFilesToImage (See #12485) (rebased onto develop)

comment:7 Changed 5 years ago by mlinkert

  • Resolution set to fixed
  • Status changed from new to closed

PR opened to address companion file classification: ​https://github.com/openmicroscopy/bioformats/pull/1267

#12519 opened to address the broader issue of companion attachment creation.

Note: See TracTickets for help on using tickets. You may also have a look at Agilo extensions to the ticket.

1.3.13-PRO © 2008-2011 Agilo Software all rights reserved (this page was served in: 0.89850 sec.)

© 2000-2014 University of Dundee & Open Microscopy Environment. Creative Commons Attribution 3.0 Unported License
OME source code is available under the GNU General public license or through commercial license from Glencoe Software

We're Hiring!