Nullable (RType) columns

[jamoore]

Currently all omero.column types are not nullable. A signpost value (-1, -INF, etc) must be used to mark a null value. Instead, a secondary column per nullable column (or a single "Nulls" matrix for all nullable columns) should be added to the OmeroTables to mark nulls. The values in the columns themselves will be most likely be Ice's default values (0, 0.0) but should be considered UNDEFINED. In reading the data, null RType instances will be inserted into the RType lists.

[jamoore]

Added this to #1638 after closing #4650

[spli]

I implemented something similar on the client-side in ​https://github.com/ome/omero-wndcharm/blob/8c7eecf318c3efee6e77c4941139bd601a43abbc/OmeroWndcharm/TableConnection.py#L379
though it's not actually used.

Doing this server side implies changing the API (or adding new NullableXXXColumns) so the values field of each column returns lists of rtypes instead of primitives e.g. LongColumn?.values will be list(rlong) instead of list(long).

The bigger question is the balance between the Tables API vs the Tables storage format, i.e. Do we actively encourage people to use Tables because it (currently) uses HDF5 so they can download and open it outside OMERO, or is it there as an option but the main purpose is that people access it via OMERO?

[jamoore]

I would say the latter. I.e. we want to provide an API which provides us flexibility. If users want to then get an HDF5 file, hopefully we can offer it, but most likely they could do it themselves pretty easily.