User Story #918 (closed)
ImageJ plugged into scripting engine
Reported by: | jamoore | Owned by: | crueden-x |
---|---|---|---|
Priority: | critical | Milestone: | 5.0.2 |
Component: | Scripting | Keywords: | paris2008, paris2011 |
Cc: | crueden-x, jburel, wmoore | Story Points: | n.a. |
Sprint: | n.a. | Importance: | n.a. |
Total Remaining Time: | n.a. | Estimated Remaining Time: | n.a. |
Description (last modified by jmoore)
2011 Discussion
The current plan is to make use of pre/post plugins in ImageJ2 for parsing out the parameters and filling in the input values.
2008 Discussion
– macro? Aware of OMERO? – Start with ImageJ talking to OMERO – More: grab jar from web and drop it into a directory – OMERO downloader done; still working on uploader – Can we get help from ImageJ community? Wayne. – Jason calls Sebastian.
Change History (19)
comment:1 Changed 13 years ago by jmoore
- Cc crueden-x curtis jburel wmoore added
- Description modified (diff)
- Keywords paris2011 added
- Owner changed from jrswedlow to jmoore
- Priority changed from minor to critical
- Status changed from new to accepted
comment:3 Changed 12 years ago by crueden-x
- Owner changed from jamoore to crueden-x
comment:4 Changed 12 years ago by crueden-x
- Cc ctrueden curtis removed
comment:5 Changed 11 years ago by crueden-x
A couple of weeks ago, I spent most of the week digging in to the OMERO scripting framework. I bookmarked a few places in the source code (using comments) but then it sat on my hard drive for the next week or two. I have finally pushed it to a new branch, along with a WIP commit whose message expresses my current thoughts, which for posterity I duplicate here as well:
These are some of the key places defining how OMERO scripting works, and what the ImageJ scripting support will potentially need to touch.
My biggest concern right now is that the entire scripting infrastructure is Python-specific, and has a one-to-one mapping between original files and scripts. Both of those assumptions are untrue for server-side ImageJ execution: a JAR file might provide one or a multitude of ImageJ commands which should be exposed via the OMERO scripting service.
As a consequence, the method ScriptRepoHelper#loadAll?(boolean) which returns a List<OriginalFile?> cannot really be hacked to include ImageJ scripts, because OriginalFile? is not a proper metadata-rich object for actually describing an ImageJ command.
What makes the most sense to me would be to create a new ICE interface with fields that describe each script in slightly more detail. This would include an OriginalFile? but also other metadata to facilitate execution of ImageJ and potentially other scripting paradigms as well.
Unfortunately, due to ImageJ deadlines, it will be a while longer before I can pursue any details of the approach above. In the meantime, comments welcome!
comment:6 Changed 11 years ago by crueden-x
I am going to try a much easier approach, which is what jmoore suggested last time we spoke in person: wrap each ImageJ command in its own Python script, which executes it using a NativeWrapper-based approach.
Steps
- Create a native wrapper script that executes a single chosen ImageJ command.
- Once that is working, generalize the script to pull out boilerplate into a common ImageJScript superclass.
- Write some utility code to autogenerate Python wrappers for all ImageJ commands on the Java classpath.
JAR files will need to live on the server in a hardcoded (relative?) location for now.
Advantages
- Reuses OMERO's existing scripting framework.
- Isolates ImageJ execution from the OMERO server itself, in case of bugs, etc.
Disadvantages
- Going Java (OMERO server) -> Python (scripting framework) -> Java again (ImageJ script) seems very convoluted to me.
- Logic to autogenerate ImageJ commands from uploaded JAR file(s) needs to live somewhere and get integrated for users.
comment:7 Changed 11 years ago by crueden-x
This morning I talked to Lee Kamentsky about using the CellProfiler?-ImageJ integration (an in-process solution) for the Python scripts to invoke ImageJ. The good news is that the integration is a quite general Python-Java bridge with higher-level utility access and ImageJ2-specific usage. The bad news is that it is not completely well-separated from the CellProfiler? core right now, so there would be some effort to split it out (which Lee plans to do eventually anyway). I chatted about this with leekamentsky and dscho on #imagejdev.
Josh Moore also later chimed in, stating that he is in favor of a solution that does not require jumping back and forth from Python. We are planning to chat about how best to accomplish that tomorrow.
comment:8 Changed 11 years ago by crueden-x
After talking to Josh, we decided on a more direct Java-Java interoperability approach using Ice. This will make it much easier to translate ImageJ's image data structures back and forth from OMERO/Ice image data structures, since all the code for doing so will be Java-based. At various points, the OMERO server will spawn an ImageJ process with three main functions:
- Generate a list of stub files on disk (which can be in whatever format ImageJ likes) from which ImageJ can later reconstruct metadata about each available command/script. The vital pieces of information for the stub are: command class, input presets, and version number.
- Parse a stub file to produce metadata about that command/script and report it to OMERO (presumably via Ice...?)
- Execute a command as specified by a particular stub file, receiving the inputs from the OMERO server via Ice, and sending the outputs to the OMERO server via Ice afterwards.
I created an imagej-omero repository which will house all this code.
comment:9 Changed 11 years ago by crueden-x
First cut at a skeleton implementation is here.
comment:10 Changed 11 years ago by crueden-x
As of last night, I completed an initial version that does the ImageJ half of things; i.e.: it implements the three features discussed above (generate, parse and execute), providing a simple command line interface to specify which operation(s) to perform. It generates the stubs into an "ij-stubs" folder (configurability of the location to be added later). It can be told to parse a particular stub, from which it will determine the correct ModuleInfo and report its details (but not yet feed them to OMERO). It can also be given a stub for execution, and it will execute it, though no communication with OMERO is in place yet.
Still to be done in ij-omero-server:
- Add a dependency to omero_client library. Use it to feed parse results to OMERO, to request input parameter values from OMERO, and to feed output parameter values back to OMERO.
- In order to successfully request and feed parameter values to and from OMERO, a conversion between OMERO and ImageJ data structures will need to be implemented.
- NEXT STEP: I would like some guidance on the API to use for communication with OMERO: which services to call, which Ice interfaces are relevant, etc. And it is not clear to me whether we need to invent any new ones.
To be done on the OMERO side:
- Add ImageJ awareness, so that an ImageJ installation on the server side (or one on another server) can be detected and/or configured. OMERO needs to launch an ij-omero-server process and tell it to refresh the stubs. It also needs to query the metadata for each stub at some point, for storage into the OMERO DB. Once that is done, the available ImageJ commands will be available for use from OMERO clients. Lastly, it needs to be able to spawn a new ij-omero-server process whenever execution of an ImageJ command is desired.
(Technical side-note: we actually operate on ImageJ Modules rather than ImageJ Commands because a Command is simply one type of Module. There may be others. So we use the ImageJ ModuleService to do the heavy lifting. As long as the Module's ModuleInfo implements ImageJ's Identifiable interface, it can be represented as a stub on disk for later reference by OMERO.)
joshmoore: Please speak up if anything I've said above is incorrect!
comment:11 Changed 11 years ago by jamoore
PR opened with support for Jython: https://github.com/openmicroscopy/openmicroscopy/pull/1324
Either Curtis, you can take that forward and add specific IJ support, or we can workout what changes you need (and file endings you would like supported) and add them on gh-1324.
comment:12 Changed 11 years ago by crueden-x
Using PR #1324 as a starting point, it is now possible to call ImageJ commands as OMERO scripts. The required glue code lives in the ij-omero-server Java library. Each ImageJ command is wrapped as a Jython script.
The following ImageJ tutorial commands are tested and working:
- HelloWorld: a basic example with one string input, and one string output.
- WidgetDemo: an example exercising many different parameter types, providing a good illustration of how type conversion works going back and forth between ImageJ and OMERO.
To try it out, follow these steps:
- Build OMERO from joshmoore's jy-scripts branch.
- Download ImageJ2 and unpack into $OMERO_HOME/dist/lib (it will create a subfolder called ImageJ.app).
- Download the ij-omero-server interoperability library into $OMERO_HOME/dist/lib/ImageJ.app/jars.
- Download the simple-command and widget-demo tutorial plugins into $OMERO_HOME/dist/lib/ImageJ.app/plugins.
- Download the latest pre-built standalone version of Jython.
- Create a jython launch script on your path:
#!/bin/sh export OMERO_HOME="$HOME/code/ome/openmicroscopy/dist" java -cp "$HOME/bin/jython-standalone-2.5.3.jar:$OMERO_HOME/lib/ImageJ.app/jars/*:$OMERO_HOME/lib/ImageJ.app/plugins/*:$CLASSPATH" org.python.util.jython $@
- Run jython and verify the following lines execute without error:
import imagej.ImageJ import HelloWorld import WidgetDemo
- Download HelloWorld.jy into $OMERO_HOME/lib/scripts/imagej (create the folder).
- omero admin start (if you haven't already)
- omero script list
You should see HelloWorld.jy as an available option.
- omero script params $(omero script list | grep HelloWorld | sed 's/|.*//')
- omero script launch $(omero script list | grep HelloWorld | sed 's/|.*//')
- To test WidgetDemo or any other ImageJ2 command, just copy HelloWorld.jy to e.g. WidgetDemo.jy and make the relevant edits to call the desired command instead.
The next step is to support image parameters (i.e., pixels IDs). That work is pending on the scifio-format branch. The goal is to implement an "OMERO" format for SCIFIO which supports transparent pixels access to and from OMERO via the SCIFIO API (and hence via the ImgLib2 API).
comment:13 Changed 11 years ago by crueden-x
The ImageJ-OMERO code has been cleaned up and refactored. The above instructions have been added to the README. See:
I also implemented the SCIFIO format for accessing OMERO pixels, both read and write:
There are now ImageJ commands for working with OMERO pixels from within ImageJ, too.
Unfortunately, there are still bugs and nothing related to images fully works yet.
comment:14 Changed 11 years ago by crueden-x
Downloading from OMERO to ImageJ is now working. So you can now connect to OMERO from the ImageJ application, and visualize a given image, with cells downloaded on demand from the server thanks to the SCIFIOCellImg functionality.
However, the server-side ij-omero module still cannot do it as part of a script execution, likely due to time performance hangups. Will keep investigating.
comment:15 Changed 11 years ago by crueden-x
Performance issue was largely addressed in commit ef4b63be. In my tests, executing the HelloWorld and WidgetDemo examples improved from >60s to <20s each. And the ComputeStats example (which takes an image as input) now also works in under 20s with a small single-plane image. So the OMERO -> ImageJ direction is working for images!
Previously, though the ImageJ command itself was finished executing, OMERO was waiting for sessions to be closed before finishing script execution. The new ScriptRunner.main code explicitly disposes the ImageJ context then shuts down the JVM, which addresses the issue.
Next up: uploading images back to OMERO from ImageJ. Main challenge there is that SCIFIO's ImgSaver only supports writing certain types of Img objects. It needs to be expanded to handle any Img including SCIFIOCellImg.
comment:16 Changed 11 years ago by Josh Moore <josh@…>
(In [c6cccddf8a7b72faf7d563c45ade2423562d6b52/ome.git] on branch develop) Merge pull request #1324 from joshmoore/jy-scripts
Jython scripts (See #918)
comment:17 Changed 10 years ago by crueden-x
The issues relating to ImageJ-OMERO are now all filed in its issue tracker.
What is left in this story are only tasks relating to the OMERO side of things. Should I still own this? Should we split them to a separate story?
comment:18 Changed 10 years ago by jamoore
- Resolution set to fixed
- Status changed from accepted to closed
I've relinked all existing tickets under #1343 and scheduled it for late-summer/fall. Closing. Thanks, Curtis!
comment:19 Changed 10 years ago by jamoore
- Milestone changed from Unscheduled to 5.0.2
(Should have adjusted the milestone)
Changing from Jason ownership for evaluation. We might want to move this out of the #1856 requirement so that it's visible in the top-level tickets report.