Requirement #4625 (new)
Opened 13 years ago
Last modified 10 years ago
Safe Haven data storage & analysis — at Initial Version
Reported by: | jamoore | Owned by: | |
---|---|---|---|
Priority: | critical | Milestone: | OMERO-Beta4.3 |
Component: | n.a. | Keywords: | n.a. |
Cc: | jburel, cxallan, jrswedlow | Business Value: | n.a. |
Total Story Points: | n.a. | Roif: | n.a. |
Mandatory Story Points: | n.a. |
Description
The initial data to be used for the first demo(s) will be some representative
subset of the GoDARTs data provided by Andy and Alison (step 0). This will
approximate the project sets exported to "research data centers". This data
will get loaded into OMERO.tables (step 1) via a command-line script. Some work
has been done on a generic loader, and may be re-usable for this task.
Otherwise, a custom script will be written.
Once the data is in OMERO, another command-line script will be written to
export the OMERO.tables data to a CSV file (step 2). This represents the
current state of the researchers' workflow: easy to implement on the OMERO side
but does not add any security to the system. The script should include
functionality for choosing columns and filtering the exported data. The
usability of this script should be validated by the researchers. Other options
may need to be added: exporting to TSV, XLS; more advanced querying; etc.
The next steps will work to add security constraints to full export to support
the Safe Haven requirements. Any API methods which will be used by partial
export will have auditing added (step 3) so that it is clear which researchers
have accessed what data. Further, classes of authorization will be added to
each column in the data set (step 4). Levels may include (from least to most
secure): full access, aggregations, aggregations without outliers,
correlations, absolute subset, admin-only access. With column security in
place, full export can be disabled, leaving only partial data export (step 5).
At this point, the researchers should again be asked for user-feedback to
determine what features must be added to make this modified workflow still
viable for them.
The final step (6) for the initial phase is then to allow researchers to
submit a script for execution on the entire data set.