Task #10661 (closed)
Allow dynamically generated script params / Don't cache params
| Reported by: | spli | Owned by: | |
|---|---|---|---|
| Priority: | major | Milestone: | 5.1.3 |
| Component: | Scripting | Version: | 5.1.1 |
| Keywords: | n.a. | Cc: | analysis@…, pwalczysko |
| Resources: | n.a. | Referenced By: | n.a. |
| References: | n.a. | Remaining Time: | 0.0d |
| Sprint: | n.a. |
Description
Could be controversial, thought I'd bring it up for discussion.
Use case
I've got several trained PyCHRM classifiers. I want to use one of them to run some predictions on a new dataset with features already calculated. I can select the dataset (or project or image) and it's ID will autopopulate the IDs parameter field, however to select the classifier I need to manually enter some sort of identifier. For the February PI/dev meeting classifiers were attached to projects, so the Project ID could be manually obtained and entered, though this is inconvenient.
With the proposed Classifier XMLAnnotations #10569 the IDs will only be obtainable from the Attachments (or equivalent) pane in Insight.
Removing the caching of script parameters would make this possible, e.g.
Pycharm_Predict.py:L281/Pycharm_Predict.py:L296
Change History (12)
comment:1 Changed 11 years ago by jamoore
comment:2 Changed 11 years ago by spli
A dynamic parameter flag sounds good to me.
comment:3 Changed 11 years ago by jamoore
The only thing in the model at the moment that would permit this would be "namespaces"
comment:4 Changed 11 years ago by crueden-x
For what it's worth, ImageJ2 supports dynamic modules. If OMERO gained this functionality, it would be straightforward to then allow such dynamic ImageJ modules with the "this is dynamic" flag (if any) set.
comment:5 Changed 11 years ago by spli
Seb S recently asked whether this was possible.
comment:6 Changed 11 years ago by ssimard-x
Description of the intented use case: populate script.List values from a client query, with parameters extracted from the currently selected IDs.
comment:7 Changed 9 years ago by jamoore
- Cc pwalczysko added
- Milestone changed from 5.x to 5.1.2
- Version set to 5.1.1
Currently looking into this due to a weird cached-script issue.
comment:8 Changed 9 years ago by jamoore
PR opened: https://github.com/openmicroscopy/openmicroscopy/pull/3790
Discussions on just what namespace to be used, etc. to take place there.
comment:9 Changed 9 years ago by jamoore
- Resolution set to fixed
- Status changed from new to closed
comment:10 Changed 9 years ago by jamoore
- Milestone changed from 5.1.2 to 5.1.3
comment:11 Changed 9 years ago by jmoore <josh@…>
- Remaining Time set to 0
(In [5923de4ae9fa0659e4bdcab3fff67797cf895a60/ome.git] on branch develop) Add ParamsCache? for in-memory JobParams? (Fix #10661)
Rather than store JobParams? instance in the database,
the ParamsCache? reads all of the scripts on startup &
saves them in a Guava LoadingCache?. If some script is
not found in the cache, an attempt is made to create
the params.
comment:12 Changed 9 years ago by jmoore <josh@…>
(In [9946644bc41334e5b29c5ff35136c5b404c0702f/ome.git] on branch develop) Add NSDYNAMIC namespace for jobs (see #10661)
A script with the namespace NSDYNAMIC will no longer
have its JobParams? object cached by the server. This
will allow scripts to determine their params at run-
time at the cost of slower startups, e.g.:
`
test/integration/test_scripts.py::TestScripts::testDynamic
[2.2918, 2.2784, 3.4278] <-- Dynamic
[0.0006, 0.0005, 0.0007] <-- Cached
`
There's a certain overhead in re-calculating the script params each time. We could add a flag to the cached params saying "this is dynamic" and then if the server detects the dynamic-flag, it could re-execute only such scripts?