Alternative Storage MongoDB

[dzmacdonald]

This page is the plan of work and tasks required to further test MongoDB as an alternative storage mechanism for ROI.

For a reason behind our decision to choose mongo see AlternativeStorage.

As of the presentation on Tuesday 16th February, it was decided to create a test bed for MongoDB to see how it would perform in storing ROI from HCS, and too see what structures could be put in place to alieviate the difficulties described in the presentation.
Timeline ¶

• Investigated import of CSV file from Incell data to create ROI and measurement in MongoDB (Est. 1 day/Act. 2 days)

o Time to import 110K measurements and ROI: 17 seconds. Code : insertData.py
o Searching and querying time seems fast, even on non-indexed fields, can be slow when searching non-indexed data for non-existing fields.

Usage ¶

Create a test system that can be populated with roi from an HCS dataset and see what storage mechanisms are required to make it work for MongoDB.

Breakdown ¶
MongoDB ¶

• #1810 Investigate different structures which could be used to store different types of information (4 days)

o ROI
o Measurement
o Annotation
o Links

• Create a test dataset from the XLS incell file and upload to MongoDB (1 day)

o Insert speed
o Query speed
o Index requirements

Omero ¶

• #1809 Create a test framework, modifying Populateroi to upload plate data to MongoDB. (3 days)

o Upload a number of plates to the service
o We have already a MongoDB version of OMEROPojos, and a method for storing table data in MongoDB

• Create a new version of the Tables API service that talks to MongoDB (3 days)
• Add annotations to ROI, compare performance for different tag objects. (1 day)
• Test performance of the System, measurements, tags, roi, shape retrieval. (1 day)

Omero.insight ¶

• insight#1173 Have the measurement tool working against MongoDB (4 days)

o Save ROI to the Database and MongoDB in tandem.