Bug: Gateway closes session on passed client

[jamoore]

When using:

import omero
from omero.gateway import BlitzGateway

c = omero.client()

try:
   g = BlitzGateway(c)
   # ... do stuff
finally:
    c.closeSession()

I was surprised to get the warning:

WARNING:omero.client:Ignoring error in client.__del__:<class 'Glacier2.SessionNotExistException'>

This doesn't look to be caused by garbage collection, but not creating the BlitzGateway object makes it go away.

[jmoore]

At Ola's suggestion, passing it to someone specific (Will) for review. Pass it along if you think it appropriate.

[wmoore]

Hmmm - I see the same warning when using the CLI sometimes (usually after timeout when I have to log in again).

Here's a recent example:

$ omero script replace 7801 omero/export_scripts/Make_Movie.py 
Previous session expired for will on localhost:4064
Server: [localhost]
Username: [will]will
Password:
Created session b72a4b84-0a70-471b-841e-489dd54ada0a (will@localhost:4064). Idle timeout: 10.0 min. Current group: Swedlow lab
WARNING:omero.client:Ignoring error in client.__del__:<class 'Glacier2.SessionNotExistException'>

[wmoore]

Also I get this when using client_obj = client.

I'm afraid I really have no idea what the WARNING means or what is going on here. Passing to Carlos...

>>> import omero
>>> from omero.gateway import BlitzGateway
>>> c = omero.client()
>>> conn = BlitzGateway(client_obj=c)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/will/Desktop/OMERO/dist/lib/python/omero/gateway/__init__.py", line 1270, in __init__
    self._createProxies()
  File "/Users/will/Desktop/OMERO/dist/lib/python/omero/gateway/__init__.py", line 1454, in _createProxies
    self._proxies['admin'] = ProxyObjectWrapper(self, 'getAdminService')
  File "/Users/will/Desktop/OMERO/dist/lib/python/omero/gateway/__init__.py", line 3163, in __init__
    self._resyncConn(conn)
  File "/Users/will/Desktop/OMERO/dist/lib/python/omero/gateway/__init__.py", line 3232, in _resyncConn
    self._sf = conn.c.sf
  File "/Users/will/Desktop/OMERO/dist/lib/python/omero/clients.py", line 931, in __getattr__
    return self.getSession()
  File "/Users/will/Desktop/OMERO/dist/lib/python/omero/clients.py", line 355, in getSession
    raise omero.ClientError("No session available")
omero.ClientError: No session available
>>> 
WARNING:omero.client:Ignoring error in client.__del__:<class 'Glacier2.SessionNotExistException'>

[jmoore]

Check the __del__ in omero.clients.BaseClient. You can add a except block for SNEE. What this is saying is that the client/session was not properly closed, then eventually garbage collection (some time later) tries to get rid of the object (by calling __del__) but the session has already timed out. It would be much better to free up those server resources earlier AND free up client file handles (this is likely related to the problems of "too many files" during testing)

[cneves]

Replying to wmoore:

Also I get this when using client_obj = client.

I'm afraid I really have no idea what the WARNING means or what is going on here. Passing to Carlos...

>>> import omero
>>> from omero.gateway import BlitzGateway
>>> c = omero.client()
>>> conn = BlitzGateway(client_obj=c)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/will/Desktop/OMERO/dist/lib/python/omero/gateway/__init__.py", line 1270, in __init__
    self._createProxies()
  File "/Users/will/Desktop/OMERO/dist/lib/python/omero/gateway/__init__.py", line 1454, in _createProxies
    self._proxies['admin'] = ProxyObjectWrapper(self, 'getAdminService')
  File "/Users/will/Desktop/OMERO/dist/lib/python/omero/gateway/__init__.py", line 3163, in __init__
    self._resyncConn(conn)
  File "/Users/will/Desktop/OMERO/dist/lib/python/omero/gateway/__init__.py", line 3232, in _resyncConn
    self._sf = conn.c.sf
  File "/Users/will/Desktop/OMERO/dist/lib/python/omero/clients.py", line 931, in __getattr__
    return self.getSession()
  File "/Users/will/Desktop/OMERO/dist/lib/python/omero/clients.py", line 355, in getSession
    raise omero.ClientError("No session available")
omero.ClientError: No session available
>>> 
WARNING:omero.client:Ignoring error in client.__del__:<class 'Glacier2.SessionNotExistException'>

I'm not sure if the ClientError? above is being reported as unexpected or if it is just the logged warning. Just to be completely clear, the omero.ClientError? is expected.

[cneves]

The fact sessions are kept open is by design, as there are multiple workers sharing sessions but no IPC to advertise when sessions are purposely closed. This can be changed, but the complexity to do so correctly seems too expensive for the gain.

The log entry is being filtered out with the commit ​https://github.com/cneves/openmicroscopy/commit/a58c0fe19fb6bce69a1b0532345c521150e72880 which branch has not yet been subject to a PR as it is still incomplete. As this is for the moment just a nuisance I'll hold off on making a separate branch just for this.

[jmoore]

Carlos, I don't understand why the Glacier2 exception can't be caught either. It'd be nice to have that fixed properly, because we catch it in other places so I filed another bug for that (#9076). But for the moment, I assume that the warning has been taken care of, but unless it's happened somewhere else the underlying problem of the client being closed too late was not fixed. Any thoughts on that? (And I'm not talking about the session being closed server-side, just about the gateway holding onto the BaseClient? and therefore the Communicator instances too long)

[jmoore]

After a discussion, we're assuming the remaining issue was not gateway related. If it re-appears, I'll re-open.