Blitz and OmeroPy Early Timeout

[cxallan]

The timeout test case output:

callan@warlock /usr/local/OMERO3/dist/lib $ PYTHONPATH=. ../../timeout_test 
Sleeping for 0 seconds...
Ping at: 2007-06-26 12:04:25.307867
Sleeping for 15 seconds...
Ping at: 2007-06-26 12:04:40.487151
Sleeping for 30 seconds...
Ping at: 2007-06-26 12:05:10.567213
Sleeping for 60 seconds...
Ping at: 2007-06-26 12:06:10.643380
Sleeping for 120 seconds...
Ping at: 2007-06-26 12:08:10.715622
Traceback (most recent call last):
  File "../../timeout_test", line 25, in ?
    ping(tstore, 120)
  File "../../timeout_test", line 13, in ping
    tstore.setPixelsId(PIXELS_ID)
  File "OMERO_API_ice.py", line 1323, in setPixelsId
    return _M_omero.api.ThumbnailStore._op_setPixelsId.invoke(self, (pixelsId, ), _ctx)
Ice.ConnectionLostException: Ice.ConnectionLostException:
recv() returned zero

The glacier2.config:

callan@warlock /usr/local/OMERO3/etc $ cat glacier2.config 
#
#   $Id$
#
#   Copyright 2007 Glencoe Software, Inc. All rights reserved.
#   Use is subject to license terms supplied in LICENSE.txt
#
#   See the glacier2_example.config file for reference documentation
#

#
# DO NOT CHANGE. Name to lookup on the blitz server.
#
Glacier2.InstanceName=OMEROGlacier2

#
# Host configurations. Only change the ip addresses, not the ports.
#  If blitz and the router are colocated, only Client.Endpoints needs 
#  to be changed.
#
Glacier2.Client.Endpoints=tcp -p 9998 -h 127.0.0.1
Glacier2.Server.Endpoints=tcp -h 127.0.0.1
Glacier2.SessionManager=Manager:tcp -h 127.0.0.1 -p 9999
Glacier2.PermissionsVerifier=Verifier:tcp -h 127.0.0.1 -p 9999

#
# Performance configurations. 
#   Especially useful is session timeout (effects license releases).
#
Glacier2.SessionTimeout=600
Ice.ThreadPerConnection.StackSize=262144

# TRACING
Glacier2.Client.Trace.Request=1
Glacier2.Server.Trace.Request=1
Glacier2.Client.Trace.Override=1
Glacier2.Server.Trace.Override=1
Glacier2.Client.Trace.Reject=1
Glacier2.Trace.Session=1
Glacier2.Trace.RoutingTable=1

Relevant sections of the Glacier2 router output (with tracing enabled as above):

[ 06/26/07 12:04:25.301 Glacier2: created session
  id = chris
  category = f4m*V*\|KR$%#1mR,g'R
  local address = 127.0.0.1:9998
  remote address = 127.0.0.1:37612 ]
[ 06/26/07 12:04:25.301 Glacier2: adding proxy to routing table:
  "chris/86:24:41:11:2fdb889f:11367af1cf1:-7ffa" -t:tcp -h 134.36.65.17 -p 9999 ]
[ 06/26/07 12:04:25.302 Glacier2: routing (buffered)
  proxy = "chris/86:24:41:11:2fdb889f:11367af1cf1:-7ffa" -t:tcp -h 134.36.65.17 -p 9999
  operation = ice_isA
  context =  ]
[ 06/26/07 12:04:25.302 Glacier2: routing (buffered)
  proxy = "chris/86:24:41:11:2fdb889f:11367af1cf1:-7ffa" -t:tcp -h 134.36.65.17 -p 9999
  operation = createThumbnailStore
  context =  ]
[ 06/26/07 12:04:25.308 Glacier2: adding proxy to routing table:
  "86:24:41:11:2fdb889f:11367af1cf1:-7ffa/86:24:41:11:2fdb889f:11367af1cf1:-7ff9ThumbnailStore" -t:tcp -h 134.36.65.17 -p 9999 ]
[ 06/26/07 12:04:25.308 Glacier2: routing (buffered)
  proxy = "86:24:41:11:2fdb889f:11367af1cf1:-7ffa/86:24:41:11:2fdb889f:11367af1cf1:-7ff9ThumbnailStore" -t:tcp -h 134.36.65.17 -p 9999
  operation = setPixelsId
  context =  ]
[ 06/26/07 12:04:25.418 Glacier2: routing (buffered)
  proxy = "86:24:41:11:2fdb889f:11367af1cf1:-7ffa/86:24:41:11:2fdb889f:11367af1cf1:-7ff9ThumbnailStore" -t:tcp -h 134.36.65.17 -p 9999
  operation = resetDefaults
  context =  ]
[ 06/26/07 12:04:40.487 Glacier2: routing (buffered)
  proxy = "86:24:41:11:2fdb889f:11367af1cf1:-7ffa/86:24:41:11:2fdb889f:11367af1cf1:-7ff9ThumbnailStore" -t:tcp -h 134.36.65.17 -p 9999
  operation = setPixelsId
  context =  ]
[ 06/26/07 12:04:40.498 Glacier2: routing (buffered)
  proxy = "86:24:41:11:2fdb889f:11367af1cf1:-7ffa/86:24:41:11:2fdb889f:11367af1cf1:-7ff9ThumbnailStore" -t:tcp -h 134.36.65.17 -p 9999
  operation = resetDefaults
  context =  ]
[ 06/26/07 12:05:10.567 Glacier2: routing (buffered)
  proxy = "86:24:41:11:2fdb889f:11367af1cf1:-7ffa/86:24:41:11:2fdb889f:11367af1cf1:-7ff9ThumbnailStore" -t:tcp -h 134.36.65.17 -p 9999
  operation = setPixelsId
  context =  ]
[ 06/26/07 12:05:10.578 Glacier2: routing (buffered)
  proxy = "86:24:41:11:2fdb889f:11367af1cf1:-7ffa/86:24:41:11:2fdb889f:11367af1cf1:-7ff9ThumbnailStore" -t:tcp -h 134.36.65.17 -p 9999
  operation = resetDefaults
  context =  ]
[ 06/26/07 12:06:10.643 Glacier2: routing (buffered)
  proxy = "86:24:41:11:2fdb889f:11367af1cf1:-7ffa/86:24:41:11:2fdb889f:11367af1cf1:-7ff9ThumbnailStore" -t:tcp -h 134.36.65.17 -p 9999
  operation = setPixelsId
  context =  ]
[ 06/26/07 12:06:10.654 Glacier2: routing (buffered)
  proxy = "86:24:41:11:2fdb889f:11367af1cf1:-7ffa/86:24:41:11:2fdb889f:11367af1cf1:-7ff9ThumbnailStore" -t:tcp -h 134.36.65.17 -p 9999
  operation = resetDefaults
  context =  ]
[ 06/26/07 12:08:10.716 Glacier2: rejecting request. no session is associated with the connection.
  identity: 86:24:41:11:2fdb889f:11367af1cf1:-7ffa/86:24:41:11:2fdb889f:11367af1cf1:-7ff9ThumbnailStore ]
[ 06/26/07 12:11:27.606 Glacier2: expiring session
  id = chris
  category = pJbk@R%pc`D)XD6G&~ib
  local address = 127.0.0.1:9998
  remote address = 127.0.0.1:35675 ]

[callan]

Timeout test case

[cxallan]

From the Ice manual:

The value of the Ice.Default.Router property is a proxy whose endpoints
must match those in Glacier2.Client.Endpoints.
The property Ice.ACM.Client governs the behavior of active connection
management (ACM, see Section 36.4), which conserves resources by periodically
closing idle outgoing connections. This feature must be disabled in a client that
uses a Glacier2 router, otherwise ACM might transparently close a client’s
connection to a router and thereby terminate the router session prematurely. ACM
is enabled by default, and therefore must be disabled by setting this property to
zero.
For clients that use AMI, we set Ice.MonitorConnections so that AMI
timeouts operate correctly. This property, if not defined, uses the value of
Ice.ACM.Client by default. Since we are disabling ACM by setting
Ice.ACM.Client to zero, we must specify an explicit value for Ice.MonitorConnections.
In this example, we use a value of 60 seconds.
Finally, setting Ice.RetryIntervals to -1 disables automatic retries,
which are not useful for proxies configured to use a Glacier2 router.

So, to ensure that this never happens to a connection which is active, you must ensure the following are in your ice.config file or options:

Ice.ACM.Client=0
Ice.MonitorConnections=60
Ice.RetryIntervals=-1