Warning: Can't synchronize with repository "(default)" (/home/git/ome.git does not appear to be a Git repository.). Look in the Trac log for more information.
Notice: In order to edit this ticket you need to be either: a Product Owner, The owner or the reporter of the ticket, or, in case of a Task not yet assigned, a team_member"

Task #6639 (closed)

Opened 13 years ago

Closed 12 years ago

Last modified 12 years ago

Bug: NoSuchElementException in log

Reported by: jburel Owned by: jamoore
Priority: major Milestone: OMERO-4.4
Component: Services Version: n.a.
Keywords: n.a. Cc: jamoore
Resources: n.a. Referenced By: n.a.
References: n.a. Remaining Time: 0.0d
Sprint: 2012-01-17 (6)

Description

found in blitz log

 c0:a8:1:4:-59ca51f9:1322692bb73:-7febomero.api.ThumbnailStore=ome.services.blitz.impl.ThumbnailStoreI@b306b3
java.lang.RuntimeException: omero.InternalException
    serverStackTrace = "java.util.NoSuchElementException
                        	at java.util.LinkedList.remove(LinkedList.java:644)
                        	at java.util.LinkedList.removeLast(LinkedList.java:144)
                        	at ome.security.basic.CurrentDetails.logout(CurrentDetails.java:180)
                        	at ome.security.basic.BasicSecurityWiring.logout(BasicSecurityWiring.java:110)
                        	at ome.security.basic.BasicSecurityWiring.invoke(BasicSecurityWiring.java:85)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                        	at ome.services.blitz.fire.AopContextInitializer.invoke(AopContextInitializer.java:43)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                        	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
                        	at $Proxy83.close(Unknown Source)
                        	at ome.services.blitz.impl.AbstractAmdServant.close_async(AbstractAmdServant.java:227)
                        	at ome.services.blitz.impl.AbstractAmdServant.close(AbstractAmdServant.java:198)
                        	at ome.services.blitz.impl.ServiceFactoryI.doDestroy(ServiceFactoryI.java:790)
                        	at ome.services.blitz.impl.ServiceFactoryI.destroy(ServiceFactoryI.java:703)
                        	at Glacier2._SessionDisp.___destroy(_SessionDisp.java:82)
                        	at omero.api._ServiceFactoryDisp.__dispatch(_ServiceFactoryDisp.java:1263)
                        	at IceInternal.Incoming.invoke(Incoming.java:159)
                        	at Ice.ConnectionI.invokeAll(ConnectionI.java:2037)
                        	at Ice.ConnectionI.message(ConnectionI.java:972)
                        	at IceInternal.ThreadPool.run(ThreadPool.java:577)
                        	at IceInternal.ThreadPool.access$100(ThreadPool.java:12)
                        	at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:971)
                        "
    serverExceptionClass = "java.util.NoSuchElementException"
    message = (null)
	at ome.services.blitz.impl.AbstractAmdServant$1.ice_exception(AbstractAmdServant.java:191)
	at ome.services.blitz.impl.AbstractAmdServant.close_async(AbstractAmdServant.java:255)
	at ome.services.blitz.impl.AbstractAmdServant.close(AbstractAmdServant.java:198)
	at ome.services.blitz.impl.ServiceFactoryI.doDestroy(ServiceFactoryI.java:790)
	at ome.services.blitz.impl.ServiceFactoryI.destroy(ServiceFactoryI.java:703)
	at Glacier2._SessionDisp.___destroy(_SessionDisp.java:82)

Change History (7)

comment:1 Changed 13 years ago by jburel

  • Milestone changed from Unscheduled to OMERO-Beta4.3.2

comment:2 Changed 13 years ago by jburel

  • Milestone changed from OMERO-Beta4.3.2 to OME-5.0

comment:3 Changed 12 years ago by jmoore

  • Component changed from General to Services
  • Owner set to jmoore
  • Priority changed from minor to major
  • Remaining Time set to 1.5
  • Summary changed from Bug: exception in log to Bug: NoSuchElementException in log

comment:4 Changed 12 years ago by jmoore

  • Remaining Time changed from 1.5 to 0.5
  • Sprint set to 2012-01-17 (6)
  • Status changed from new to accepted

comment:5 Changed 12 years ago by jmoore

  • Remaining Time changed from 0.5 to 0.1

To test set the timeouts very short:

omero.sessions.sync_force=300000
omero.sessions.sync_interval=60000
omero.sessions.timeout=15000

Create a thumbnail store, wait just a bit, and then close the session:

In [1]: client.sf.createThumbnailStore()
Out[1]: 23da0f01-e5d2-4ddb-ae29-7b6724bfc44f/e7f0d906-f208-4707-93dd-15a71ab8ede3omero.api.ThumbnailStore -t:tcp -h 192.168.178.32 -p 56333:tcp -h 192.168.178.21 -p 56333

In [2]: client.closeSession()

The steps that happen on the server are:

  • ServiceFactoryI.destroy catches an exception on SessionManagerImpl.detach
  • ServiceFactoryI.doDestroy is called
  • CloseableServant.close() is called (calls .close_async())
  • preClose is called.
  • StatefulServiceInterface.close() is called.
  • BasicSecurityWiring.login throws a SessionTimeoutException (i.e. no one is logged in)
  • During the finally block, logout() is called which throws the NoSuchElementException

comment:6 Changed 12 years ago by jmoore

  • Remaining Time changed from 0.1 to 0
  • Resolution set to fixed
  • Status changed from accepted to closed

Solving this by adding a new BasicEventContext subclass -- CloseOnNoSessionContext. When this is detected, most normal handling will be skipped. This shouldn't be a problem because the close() method in StatefulServiceInterface is mostly a marker method, i.e. handlers detect that that is the current method and therefore do proper clean up, but no actual logic should be in the close method itself.

Rather than the long stack trace, this now prints at the WARN level:

2012-01-05 15:31:51,313 WARN  [  ome.security.basic.BasicSecurityWiring] (l.Server-3) SessionTimeoutException on close:ff41408d-8b3a-422c-9fb8-aa4597d193d8

since this possibly points to a client developer error and/or a client crash, since services are not being closed before the session timeouts.

The fix has been pushed to my sprint6-server-backlog branch.

comment:7 Changed 12 years ago by jmoore <josh@…>

(In [605cd6c282405260e114bdb21e830ea360a18573/ome.git] on branch develop) Add CloseOnNoSessionContext? for handling close (Fix #6639)

Note: See TracTickets for help on using tickets. You may also have a look at Agilo extensions to the ticket.

1.3.13-PRO © 2008-2011 Agilo Software all rights reserved (this page was served in: 0.73520 sec.)

We're Hiring!