Bug: No thumbnails for new Pyramid

[wmoore]

When import completes, thumbnails are automatically created but the same does not happen for Pyramid generation.

In the web, this PR looks for latest thumbnail version,
​https://github.com/openmicroscopy/openmicroscopy/pull/2979 but fails to update when Pyramids are complete because no new thumbs are created.

[jamoore]

Despite any changes required due to ​https://www.openmicroscopy.org/community/viewtopic.php?f=4&t=7593&start=10, this likely only requires an extra step in the PixelDataHandler (roughly around ​line 146) to also initiate thumbnail generation. A caveat will be that this will add time to the overall process which will need to be evaluated. A more complicated solution would be to trigger an asynchronous thumbnail generation.

If no one picks this up for m1, this should be pushed to m3.

[jburel]

Unlikely to happen for m1 due to the list of remaining tasks, Pushing to m3

[jamoore]

Reproduced the behavior that Will saw with the following:

• bin/omero admin ice server disable PixelData-0
• bin/omero admin ice server stop PixelData-0
• touch 'big&sizeX=20000&sizeY=20000.fake'
• bin/omero import 'big&sizeX=20000&sizeY=20000.fake'
• Open up web
• List orphaned folder
• See clock SVG holder
• bin/omero admin ice server enable PixelData-0
• watch var/log/PixelData-0.log until pyramid generation is done
• Go to web and click on in-app refresh
• Thumbnail is still the clock.
• Click on browser refresh solves the issue and the pyramid thumbnail appears.

Talking to Chris, this issue is at least partly in the browser caching layer. The server does not store the the clock thumbnails and so there's nothing that can be deleted.

Following the same workflow in insight, using the refresh icon in app does show the new thumbnail.

It's not clear what should be changed server-side at this point. Passing back to Will for discussion.

[wmoore]

OK - I'll see if I can work a solution in Javascript or Django to always try refreshing thumbs if version == 0.

[jamoore]

Will: I looked further, and the background processes simply aren't made for performing this kind of transaction. If you're unsuccessful, I can try a workaround in which I create a Message which gets handled elsewhere. More ideal would be having the proper MQ in place, at which point we can just request it to be performed in the background.

[wmoore]

I looked at the thumbnail version numbers when Pyramids are generated. Even before the pyramid is complete, the thumbnail table has Version=0. This makes it impossible to tell whether a Pyramid is currently being generated, since Version=0 is also true for non-tiled small images with thumbnails generated on import. Forcing the web NOT to cache any thumbnail with version==0 means that no regular images will have their thumbnails cached until their rendering settings are changed & saved (and a new thumbnail is generated). So we need some other way to prevent caching of thumbnails for big images during pyramid generation. Is it possible to have the thumbnail version set to None or -1 until a first thumbnail is generated for pyramids? Seems that having version=0 when no thumbnail has been created is really a bug.

[jamoore]

The version= setting happens in the main server so I can definitely look into setting that to null. Alternatively, I can look into nulling it when pyramid generation either begins or ends if that would be better.

It's the generating of them that I'd like to keep out of the PixelData service for now.

[wmoore]

Actually, nulling the Thumbnail version when pyramid generation ends would be best I think, since then we wouldn't needlessly try to refresh thumbnail until it's done. However, if this meant that the version=0 during pyramid generation (and also version=0 after first thumbnail is created) then we wouldn't be able to tell from the version whether pyramid generation is in progress. That's OK for the current use-case, but if it was possible to have version= -1 during pyramid generation, then set to null when done, then goes to 0 when the first thumbnail is created, that would be nice in case you wanted to make use of that info for some future functionality.

[wmoore]

Another request for this fix: ​https://www.openmicroscopy.org/community/viewtopic.php?f=5&t=7779&p=15580

[jamoore]

​https://github.com/openmicroscopy/openmicroscopy/pull/3713

[jamoore]

Assuming the latest on my PR has this fixed.

[jmoore <josh@…>]

(In [365b04292eef2e865e2864a195690960aae73caf/ome.git] on branch develop) Set pyramid thumb versions to -1 (Fix #12538)

Thumbnails that are generated while pyramids are being
created (i.e. the clock symbol) should have a version of
-1 so that clients know to refresh.

There is currently no case where version == null is
used or even possible.

[jmoore <josh@…>]

(In [0ac0f535d51fda5ec00a7fb45d28b8f48918f1de/ome.git] on branch develop) Handle dupe thumbnails (Fix #12538)

The retrieveThumbnail method was internally resetting
the cached metadata object when inProgress was true.
Now, we temporarily copy this value so that following
saves don't create new objects.

[Sébastien Besson <seb.besson@…>]

(In [67fe98f195f6f1496f3e26251eebf59aedeaad22/ome.git] on branch develop) Merge pull request #3713 from joshmoore/12538-tb-versions

Set pyramid thumb versions to -1 (Fix #12538)