Improve Ricerca handling of negative examples

[spli]

The handling of negative results in Ricerca doesn't take into account the dissimilarity score, only the ranking:
​https://github.com/icaoberg/ricerca/blob/17e0252b49d1f197553cd6d1319e143631ebf9f3/ricerca/content.py#L151

This means when the negative set of samples is very spread out, and for many positive and negative samples d([-],x) >> d([+],x), a negative image can still come out ahead of what you'd intuitively expect to be a positive sample- see the following 1D example below where x1- is ranked above x2+ and x3+. This is probably an even bigger problem with high-dimensional data.

[-] = negative reference
[+] = positive reference
x1, x4: Negative samples
x2, x3: Positive samples


0        9   13 16           29                        55

-        +  [+] +           [-]                        -
|        |      |                                      |
x1       x2     x3                                     x4

                              rev.   avg
    d[+]  d[-]  rank+  rank-  rank-  rank
x1   13    29    3      4      1     2
x2    4    20    2      2      3     2.5
x3    3    13    1      1      4     2.5
x4   42    26    4      3      2     3

---

Reported by pwalczysko (#11200)

When selecting negative images in OMERO.Searcher to refine the search, then the negative images selected can easily appear in the next round of refinement in the "found" set. The workflow which was particularly prone to this was

• be at "100" in Retrieved Images pop-down menu
• refine your search
• works fine
• change the Retrieved Images number to "20" not changing anything else
• click "Refine.." button
• the negative images are in the found results, and also the whole result seems way off anyway - when looking for cats, suddenly plenty of hedgehogs came up, although they were already filtered out
• the top 20 images are not the same as in the search results before the Retrieved images number was changed
• one person, which was clearly marked as negative in the left pane, was included now in the search results

[jamoore]

Unlikely that "icao-berg-x" is actually Ivan's user name.

[icao-berg-x]

is there a way that can i recreate these results? can you send us the images along with their features or give us access to the test server?

[icao-berg-x]

yes, it is :)

Replying to jamoore:

Unlikely that "icao-berg-x" is actually Ivan's user name.

[jamoore]

I know, I know, I'm quiet now :)

[spli]

Replying to icao-berg-x:

is there a way that can i recreate these results? can you send us the images along with their features or give us access to the test server?

I can't seem to recreate this right now. I can give you access to the server and/or images.

[jamoore]

Cleaning up searcher backlog.