Warning: Can't synchronize with repository "(default)" (/home/git/ome.git does not appear to be a Git repository.). Look in the Trac log for more information.
Notice: In order to edit this ticket you need to be either: a Product Owner, The owner or the reporter of the ticket, or, in case of a Task not yet assigned, a team_member"

Task #10661 (closed)

Opened 8 years ago

Closed 6 years ago

Last modified 6 years ago

Allow dynamically generated script params / Don't cache params

Reported by: spli Owned by:
Priority: major Milestone: 5.1.3
Component: Scripting Version: 5.1.1
Keywords: n.a. Cc: analysis@…, pwalczysko
Resources: n.a. Referenced By: n.a.
References: n.a. Remaining Time: 0.0d
Sprint: n.a.

Description

Could be controversial, thought I'd bring it up for discussion.

Use case

I've got several trained PyCHRM classifiers. I want to use one of them to run some predictions on a new dataset with features already calculated. I can select the dataset (or project or image) and it's ID will autopopulate the IDs parameter field, however to select the classifier I need to manually enter some sort of identifier. For the February PI/dev meeting classifiers were attached to projects, so the Project ID could be manually obtained and entered, though this is inconvenient.

With the proposed Classifier XMLAnnotations #10569 the IDs will only be obtainable from the Attachments (or equivalent) pane in Insight.

Removing the caching of script parameters would make this possible, e.g.
Pycharm_Predict.py:L281/Pycharm_Predict.py:L296

Change History (12)

comment:1 Changed 8 years ago by jamoore

There's a certain overhead in re-calculating the script params each time. We could add a flag to the cached params saying "this is dynamic" and then if the server detects the dynamic-flag, it could re-execute only such scripts?

comment:2 Changed 8 years ago by spli

A dynamic parameter flag sounds good to me.

comment:3 Changed 8 years ago by jamoore

The only thing in the model at the moment that would permit this would be "namespaces"

comment:4 Changed 8 years ago by crueden-x

For what it's worth, ImageJ2 supports dynamic modules. If OMERO gained this functionality, it would be straightforward to then allow such dynamic ImageJ modules with the "this is dynamic" flag (if any) set.

comment:5 Changed 8 years ago by spli

Seb S recently asked whether this was possible.

comment:6 Changed 8 years ago by ssimard-x

Description of the intended use case: populate script.List values from a client query, with parameters extracted from the currently selected IDs.

Last edited 8 years ago by ssimard-x (previous) (diff)

comment:7 Changed 6 years ago by jamoore

  • Cc pwalczysko added
  • Milestone changed from 5.x to 5.1.2
  • Version set to 5.1.1

Currently looking into this due to a weird cached-script issue.

comment:8 Changed 6 years ago by jamoore

PR opened: https://github.com/openmicroscopy/openmicroscopy/pull/3790

Discussions on just what namespace to be used, etc. to take place there.

comment:9 Changed 6 years ago by jamoore

  • Resolution set to fixed
  • Status changed from new to closed

comment:10 Changed 6 years ago by jamoore

  • Milestone changed from 5.1.2 to 5.1.3

comment:11 Changed 6 years ago by jmoore <josh@…>

  • Remaining Time set to 0

(In [5923de4ae9fa0659e4bdcab3fff67797cf895a60/ome.git] on branch develop) Add ParamsCache? for in-memory JobParams? (Fix #10661)

Rather than store JobParams? instance in the database,
the ParamsCache? reads all of the scripts on startup &
saves them in a Guava LoadingCache?. If some script is
not found in the cache, an attempt is made to create
the params.

comment:12 Changed 6 years ago by jmoore <josh@…>

(In [9946644bc41334e5b29c5ff35136c5b404c0702f/ome.git] on branch develop) Add NSDYNAMIC namespace for jobs (see #10661)

A script with the namespace NSDYNAMIC will no longer
have its JobParams? object cached by the server. This
will allow scripts to determine their params at run-
time at the cost of slower startups, e.g.:

`
test/integration/test_scripts.py::TestScripts::testDynamic

[2.2918, 2.2784, 3.4278] <-- Dynamic
[0.0006, 0.0005, 0.0007] <-- Cached

`

Note: See TracTickets for help on using tickets. You may also have a look at Agilo extensions to the ticket.

1.3.13-PRO © 2008-2011 Agilo Software all rights reserved (this page was served in: 0.69178 sec.)

We're Hiring!