Warning: Can't synchronize with repository "(default)" (/home/git/ome.git does not appear to be a Git repository.). Look in the Trac log for more information.
Notice: In order to edit this ticket you need to be either: a Product Owner, The owner or the reporter of the ticket, or, in case of a Task not yet assigned, a team_member"

User Story #10178 (closed)

Opened 9 years ago

Closed 8 years ago

FS/Web stateful service interaction

Reported by: jamoore Owned by: wmoore
Priority: blocker Milestone: 5.0.0-beta1
Component: Web Keywords: FS
Cc: web-team@…, fs@… Story Points: n.a.
Sprint: n.a. Importance: n.a.
Total Remaining Time: n.a. Estimated Remaining Time: n.a.

Description (last modified by wmoore)

In 4.4, web methods were all converted to use a

   svc = openStatefulService()
   try:
      svc.doWork();
   finally:
      svc.close()

idiom. In other words, no stateful service lives longer than a single http requestl. With Bio-Formats now backing many of the services (renderingengine, rawpixelsservice, etc), this means that each method call (getPlane(), renderJpeg()) is performing a full setIdinvocation. For some formats, this can take several seconds.

Melissa is working on reducing setId times, and eventually the Bio-Formats state can be cached server-side in order to further speed up the individual lookups, but there may still be the need for properly handling state from the web client.

A first step is likely to evaluate the user impact of the slow down (especially when several users are accessing the web simultaneously). #10183

If the Blitz Gateway is to re-use existing stateful services (E.g. rendering engine already initialised on a particular imageId) then we have to support several steps

  • Once we have joined a session...
  • Identify stateful services that exist on the server
  • Determine whether any of these are initialised with the correct ImageId?
  • Recreate the client-side proxy
  • Update the state (E.g. turn channels on/off etc) and get output (Render plane)
  • Leave the service open.
  • The server would need to purge or limit the number of these open stateful services.

NB: If multiple web workers are simultaneously following the above steps, I imagine it's possible that they could both try to work with the same stateful service, causing all sorts of problems!

Change History (10)

comment:1 Changed 9 years ago by wmoore

Josh: would it help to have a performance testing web page, which tries to render multiple planes from the same (or different) images and logs the time taken? This could be part of webtest and included in dev_4_4 and develop, then we can compare times etc.

E.g. sequentially render all the planes in a specified image / images.

comment:2 Changed 9 years ago by jmoore

Will: a test like that we could have on both branches might be ideal. Something like ApacheBench

comment:3 Changed 9 years ago by wmoore

  • Sprint set to 2013-02-12 (5)

comment:4 Changed 9 years ago by wmoore

  • Sprint 2013-02-12 (5) deleted

comment:5 Changed 9 years ago by agilo

  • Status changed from new to accepted

Updated status, related task in progress

comment:6 follow-up: Changed 9 years ago by wmoore

  • Description modified (diff)

Josh / Chris / Carlos - any idea how tricky are the steps I've outlined above? How bad is the risk of 2 workers accessing the same stateful service etc?

comment:7 in reply to: ↑ 6 Changed 9 years ago by jmoore

The server can definitely limit the number of stateful sessions per user/group combination. And then throw a special exception saying, "clean up your mess" ?

As for having multiple workers work on the save SF-service, to some extent your protected since the invocations are synchronized (can only happen in serial). So 2 simultaneous actions shouldn't cause a problem. However, if there are two conceptually different actions, (e.g. setPixelsId(4) and setPixelsId(6)) then things will get wonky. It might suffice to lay down rules about what is permissible under these circumstances. Perhaps even passing a {"shared":"true"} with the context map to tell the server to prevent certain actions. Alternatively, we provide a way to "lock" a service in the webclient so only one worker can get it at a time.

comment:8 Changed 8 years ago by jburel

  • Keywords FS added
  • Milestone changed from OMERO-4.5 to OMERO-5

comment:9 Changed 8 years ago by wmoore

If we have multiple workers simultaneously accessing a pool of active Rendering Engines, creating client-side proxies, setting rendering settings and then rendering, we have the potential for 2 client-side processes to be accessing the same server-side Rendering Engine?

E.g. These methods are called in sequence when we render a 3 Channel image.

setActive
setChannelWindow
setRGBA
setActive
setChannelWindow
setRGBA
setActive
setChannelWindow
setRGBA
setModel
setCompressionLevel
renderCompressed

Multiple workers generating a Z-stack or T-series (all same channels on/off) would be unlikely to cause problems, but if different workers wanted to render with different channels / rendering settings, they could compete with each other and get out of sync (maybe)?

comment:10 Changed 8 years ago by wmoore

  • Resolution set to wontfix
  • Status changed from accepted to closed

Not going to try and re-use stateful services in web, without more server-side support. See https://github.com/openmicroscopy/openmicroscopy/pull/718

Note: See TracTickets for help on using tickets. You may also have a look at Agilo extensions to the ticket.

1.3.13-PRO © 2008-2011 Agilo Software all rights reserved (this page was served in: 0.102267 sec.)

We're Hiring!