Bug #1002 (closed)
StackOverflowException on opening a large dataset
Reported by: | jamoore | Owned by: | jamoore |
---|---|---|---|
Priority: | major | Cc: | jburel, cxallan |
Sprint: | n.a. | ||
Total Remaining Time: | n.a. |
Description
From Jean-Marie:
That's the error message returned when Michael opens a dataset with
more than 15000 images.
Have you seen the overflow error before?
I tried to increase memory in JAVA_OPTS before running server but
still same problem?
java.lang.RuntimeException: java.lang.StackOverflowError at org.jboss.ejb3.interceptor.InvocationContextImpl.proceed (InvocationContextImpl.java:174) at ome.tools.spring.AOPAdapter.invokeJoinpoint (AOPAdapter.java:53) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed (ReflectiveMethodInvocation.java:149) at ome.security.basic.EventHandler.invoke(EventHandler.java: 112) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed (ReflectiveMethodInvocation.java:171) at org.springframework.orm.hibernate3.HibernateInterceptor.invoke (HibernateInterceptor.java:111) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed (ReflectiveMethodInvocation.java:171) at org.springframework.transaction.interceptor.TransactionInterceptor.invok e(TransactionInterceptor.java:106) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed (ReflectiveMethodInvocation.java:171) at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke (ProxyCleanupFilter.java:169) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed (ReflectiveMethodInvocation.java:171) at ome.services.util.ServiceHandler.invoke (ServiceHandler.java:92) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed (ReflectiveMethodInvocation.java:171) at org.springframework.aop.interceptor.AbstractTraceInterceptor.invoke (AbstractTraceInterceptor.java:113) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed (ReflectiveMethodInvocation.java:171) at ome.security.basic.BasicSecurityWiring.invoke (BasicSecurityWiring.java:79) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed (ReflectiveMethodInvocation.java:171) at ome.services.util.OmeroAroundInvoke.call (OmeroAroundInvoke.java:161) at ome.services.util.OmeroAroundInvoke.loginAndSpringWrap (OmeroAroundInvoke.java:134) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.jboss.ejb3.interceptor.InvocationContextImpl.proceed (InvocationContextImpl.java:118) at org.jboss.ejb3.interceptor.EJB3InterceptorsInterceptor.invoke (EJB3InterceptorsInterceptor.java:63) at org.jboss.aop.joinpoint.MethodInvocation.invokeNext (MethodInvocation.java:101) at org.jboss.ejb3.entity.TransactionScopedEntityManagerInterceptor.invoke (TransactionScopedEntityManagerInterceptor.java:54) at org.jboss.aop.joinpoint.MethodInvocation.invokeNext (MethodInvocation.java:101) at org.jboss.ejb3.AllowedOperationsInterceptor.invoke (AllowedOperationsInterceptor.java:47) at org.jboss.aop.joinpoint.MethodInvocation.invokeNext (MethodInvocation.java:101) at org.jboss.ejb3.tx.BMTInterceptor.handleStateless (BMTInterceptor.java:71) at org.jboss.ejb3.tx.BMTInterceptor.invoke (BMTInterceptor.java:131) at org.jboss.aop.joinpoint.MethodInvocation.invokeNext (MethodInvocation.java:101) at org.jboss.aspects.tx.TxPropagationInterceptor.invoke (TxPropagationInterceptor.java:76) at org.jboss.aop.joinpoint.MethodInvocation.invokeNext (MethodInvocation.java:101) at org.jboss.ejb3.stateless.StatelessInstanceInterceptor.invoke (StatelessInstanceInterceptor.java:62) at org.jboss.aop.joinpoint.MethodInvocation.invokeNext (MethodInvocation.java:101) at org.jboss.aspects.security.AuthenticationInterceptor.invoke (AuthenticationInterceptor.java:77) at org.jboss.ejb3.security.Ejb3AuthenticationInterceptor.invoke (Ejb3AuthenticationInterceptor.java:106) at org.jboss.aop.joinpoint.MethodInvocation.invokeNext (MethodInvocation.java:101) at org.jboss.ejb3.ENCPropagationInterceptor.invoke (ENCPropagationInterceptor.java:46) at org.jboss.aop.joinpoint.MethodInvocation.invokeNext (MethodInvocation.java:101) at org.jboss.ejb3.asynchronous.AsynchronousInterceptor.invoke (AsynchronousInterceptor.java:106) at org.jboss.aop.joinpoint.MethodInvocation.invokeNext (MethodInvocation.java:101) at org.jboss.ejb3.stateless.StatelessContainer.dynamicInvoke (StatelessContainer.java:278) at org.jboss.aop.Dispatcher.invoke(Dispatcher.java:106) at org.jboss.aspects.remoting.AOPRemotingInvocationHandler.invoke (AOPRemotingInvocationHandler.java:82) at org.jboss.remoting.ServerInvoker.invoke (ServerInvoker.java:734) at org.jboss.remoting.transport.socket.ServerThread.processInvocation (ServerThread.java:560) at org.jboss.remoting.transport.socket.ServerThread.dorun (ServerThread.java:383) at org.jboss.remoting.transport.socket.ServerThread.run (ServerThread.java:165) Caused by: java.lang.StackOverflowError at org.hibernate.hql.ast.util.NodeTraverser.visitDepthFirst (NodeTraverser.java:40)
Attachments (1)
Change History (11)
comment:1 Changed 16 years ago by jmoore
- Status changed from new to assigned
comment:2 Changed 16 years ago by jmoore
Perhaps: HHH-1985?
comment:3 Changed 16 years ago by jburel
method call is loadContainerHierarchy
with root node Dataset.class, passing the dataset id
and requesting the leaves
comment:4 Changed 16 years ago by jmoore
- Cc callan added
From Chris:
This is really a programming error rather than a memory allocation
one, the stack size can be set with -Xss (defaults to 256K on most
environments I think). Not sure the best way to handle this J-M, we
can always increase the stack size but then 20,000 images will fail,
or 25,000. The only way I can think of doing this "properly" is to
request the images in batches and not all at one time.
Do we add auto-batching on the server-side? Or throw an exception when idSet.size() > 5000 (or similar) ? This goes along with all the throttling tickets floating around:
comment:5 Changed 16 years ago by cxallan
- Version 3.0-M1 deleted
Just been reading http://gregluck.com/blog/archives/2004/12/limits_to_threa_1.html and playing with thread-test.jar which is from that blog post. It would appear that the Mac OS X JVM really sucks in these cases (256KB default stack size as opposed to Linux's 512KB) and we'd probably never see them on our Linux machines.
Not sure what to say other than: if you've got lots of images in the same dataset don't run your server on Mac OS X. :)
comment:6 Changed 16 years ago by cxallan
Been doing tests with:
$ java -cp thread-test.jar com.wotif.jaguar.threading.ThreadManualTest 20000
comment:7 Changed 16 years ago by jmoore
- Type changed from User Story to defect
comment:8 Changed 16 years ago by jburel
Need the possibility in PojosOptions? to retrieve group of images
e.g. 0-500 then 501- x etc.
comment:9 Changed 16 years ago by jmoore
- Resolution set to fixed
- Status changed from assigned to closed
r2563, r2574 and r2575 permit pagination on getImages. Jean-Marie, try that out, and let's open new tickets for any other methods.
Note: I was never able to reproduce the StackOverflowException even with datasets as large as 25,000 images on my Mac 10.4. So it's very fickle. We'll have to take this into account when we start to implement throttling. Not sure how I would catch this particular issue before the StackOverflowException other than to check the size of each collection manually.
comment:10 Changed 16 years ago by jmoore
- Milestone changed from 3.0-Beta4 to 3.0-Beta3.1
This bit would imply that it's due to the HQL. Could you give me that?