BUG: Error after group perms upgrade

[pwalczysko]

Login to eel merge as "owner".
Go to admin panel (using "User Settings" menu in top right to get there)
Note: you are an owner of almost all groups but NOT an admin.

Edit private-1 group.
Add a user to the group as a member.
Add another user to the group as an owner.
Save - all works fine.

Now go back to the group and upgrade this private group to read-only-1 and hit Save.
Error occurs

Traceback (most recent call last):

  File "/opt/hudson/workspace/OMERO-DEV-merge-deploy/src/dist/lib/python/django/core/handlers/base.py", line 114, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)

  File "/opt/hudson/workspace/OMERO-DEV-merge-deploy/src/dist/lib/python/omeroweb/decorators.py", line 471, in wrapped
    retval = f(request, *args, **kwargs)

  File "/opt/hudson/workspace/OMERO-DEV-merge-deploy/src/dist/lib/python/omeroweb/decorators.py", line 521, in wrapper
    context = f(request, *args, **kwargs)

  File "/opt/hudson/workspace/OMERO-DEV-merge-deploy/src/dist/lib/python/omeroweb/webadmin/views.py", line 823, in manage_group_owner
    conn.updatePermissions(group, perm)

  File "/opt/hudson/workspace/OMERO-DEV-merge-deploy/src/dist/lib/python/omeroweb/webclient/webclient_gateway.py", line 1459, in updatePermissions
    cb = self.c.submit(command)

  File "/opt/hudson/workspace/OMERO-DEV-merge-deploy/src/dist/lib/python/omero/clients.py", line 887, in submit
    closehandle=True)

  File "/opt/hudson/workspace/OMERO-DEV-merge-deploy/src/dist/lib/python/omero/clients.py", line 906, in waitOnCmd
    callback.loop(loops, ms)  # Throw LockTimeout

  File "/opt/hudson/workspace/OMERO-DEV-merge-deploy/src/dist/lib/python/omero/callbacks.py", line 256, in loop
    5000L, int(waited))

LockTimeout: exception ::omero::LockTimeout
{
    serverStackTrace = None
    serverExceptionClass = None
    message = Command unfinished after 5.0 seconds
    backOff = 5000
    seconds = 5
}


<WSGIRequest
path:/merge/webadmin/group_owner/save/3/,
GET:<QueryDict: {}>,
POST:<QueryDict: {u'csrfmiddlewaretoken': [u'JGIoFDNvpxf6kPa6LZXjr2YAntGLtphT'], u'owners': [u'4', u'2'], u'members': [u'3', u'4', u'10', u'5', u'8', u'6', u'9', u'7', u'2', u'13'], u'permissions': [u'1']}>,
COOKIES:{'_ga': 'GA1.2.483423307.1416590819',
 'csrftoken': 'JGIoFDNvpxf6kPa6LZXjr2YAntGLtphT',
 'phpbb3_a9mfv_k': '6b796a78233f1705',
 'phpbb3_a9mfv_sid': '9ee0fd5b5754a1c7ec9a086325230bad',
 'phpbb3_a9mfv_u': '2759',
 'sessionid_merge': '5h74mkjt6wqjnd1dq6isb2l7to7z4ijs',
 'style_cookie': 'null'},
META:{'CONTENT_LENGTH': '186',
 'CONTENT_TYPE': 'application/x-www-form-urlencoded',
 u'CSRF_COOKIE': u'JGIoFDNvpxf6kPa6LZXjr2YAntGLtphT',
 'HTTPS': 'on',
 'HTTP_ACCEPT': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
 'HTTP_ACCEPT_ENCODING': 'gzip, deflate',
 'HTTP_ACCEPT_LANGUAGE': 'en-GB,en;q=0.5',
 'HTTP_CONNECTION': 'keep-alive',
 'HTTP_CONTENT_LENGTH': '186',
 'HTTP_CONTENT_TYPE': 'application/x-www-form-urlencoded',
 'HTTP_COOKIE': 'phpbb3_a9mfv_u=2759; phpbb3_a9mfv_k=6b796a78233f1705; phpbb3_a9mfv_sid=9ee0fd5b5754a1c7ec9a086325230bad; style_cookie=null; _ga=GA1.2.483423307.1416590819; csrftoken=JGIoFDNvpxf6kPa6LZXjr2YAntGLtphT; sessionid_merge=5h74mkjt6wqjnd1dq6isb2l7to7z4ijs',
 'HTTP_HOST': 'eel.openmicroscopy.org',
 'HTTP_REFERER': 'https://eel.openmicroscopy.org/merge/webadmin/group_owner/edit/3/',
 'HTTP_USER_AGENT': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:33.0) Gecko/20100101 Firefox/33.0',
 'PATH_INFO': u'/webadmin/group_owner/save/3/',
 'QUERY_STRING': '',
 'REQUEST_METHOD': 'POST',
 'SCRIPT_INFO': '/merge',
 'SCRIPT_NAME': u'/merge',
 'SERVER_NAME': 'eel.openmicroscopy.org',
 'SERVER_PORT': '443',
 'SERVER_PROTOCOL': 'HTTP/1.1',
 'wsgi.errors': <flup.server.fcgi_base.TeeOutputStream object at 0x4895810>,
 'wsgi.input': <flup.server.fcgi_base.InputStream object at 0x4c07850>,
 'wsgi.multiprocess': True,
 'wsgi.multithread': False,
 'wsgi.run_once': False,
 'wsgi.url_scheme': 'https',
 'wsgi.version': (1, 0)}>

[pwalczysko]

Note that on eel merge, this error is repeatable, when just trying to upgrade the private-1 group as an owner.

[pwalczysko]

Further workflow which causes problem is admin upgrading a read-annotate group to read write

 File "/opt/hudson/workspace/OMERO-DEV-merge-deploy/src/dist/lib/python/omeroweb/webadmin/views.py", line 726, in manage_group
    description)

  File "/opt/hudson/workspace/OMERO-DEV-merge-deploy/src/dist/lib/python/omeroweb/webclient/webclient_gateway.py", line 1386, in updateGroup
    self.updatePermissions(group, permissions)

  File "/opt/hudson/workspace/OMERO-DEV-merge-deploy/src/dist/lib/python/omeroweb/webclient/webclient_gateway.py", line 1459, in updatePermissions
    cb = self.c.submit(command)

  File "/opt/hudson/workspace/OMERO-DEV-merge-deploy/src/dist/lib/python/omero/clients.py", line 887, in submit
    closehandle=True)

  File "/opt/hudson/workspace/OMERO-DEV-merge-deploy/src/dist/lib/python/omero/clients.py", line 906, in waitOnCmd
    callback.loop(loops, ms)  # Throw LockTimeout

[wmoore]

Just tried a bunch of edits running locally and saw this Lock Timout a couple of times.
Then tried it on eel a load of times as Owner and as Admin but didn't see any problems at-all.

[jamoore]

Yup, it's definitely a timing issue, but one which will simply have to be handled by all client usages of client.submit(). Either you will have to block indefinitely which I should be avoided when possible or you will have to handle possibly LockTimeout exceptions. The default for client.submit() is 5 seconds, so anytime that the call takes longer than 5 seconds to return (and depending on the load of the server it could happen almost anywhere) you will get a LockTimeout?.

[wmoore]

Fixed Ola: ​https://github.com/openmicroscopy/openmicroscopy/pull/3853