Notice: In order to edit this ticket you need to be either: a Product Owner, The owner or the reporter of the ticket, or, in case of a Task not yet assigned, a team_member"

Task #10325 (accepted)

Opened 7 years ago

Last modified 4 years ago

Introduce ImportSet to API

Reported by: jamoore Owned by: jamoore
Priority: critical Milestone: Unscheduled
Component: OmeroFs Version: 5.0.3
Keywords: fs Cc: fs@…, wmoore, cxallan
Resources: n.a. Referenced By: n.a.
References: n.a. Remaining Time: 1.5d
Sprint: n.a.

Description (last modified by jamoore)

Currently the hierarchy is:

  /Fileset
     -> /OriginalFile collection
     -> /Image collection

The issue is that on import clients may not know what all is contained in the fileset, and therefore there will need to be a "pulling apart" (or the use of a DirectoryReader in Bio-Formats which reads everything a la ImportCandidates).

If the import action of the clients creates a ImportSet rather than a Fileset, there will be fewer overall top-level containers, and the server can adjust the Filesets as necessary. (NB: It would be important the clients could delete a single fileset from that import set)

The new hierarchy might look something like:

  /ImportSet
     -> /OriginalFile collection
     -> /Fileset collection
           -> /Image collection
           -> /OriginalFile collection (reused pointers?)

Jean-Marie has raised the issue that we will need to also examine how we deal with multi-image filesets, possibly linking Fileset under Dataset, though this cannot be done for the demo. In which case, we may want to omit ImportSet (i.e. this ticket) as well.

Note: while introducing this API, importFileset or similar should also be made into an async request, since for Plates it can take *significant* time.

Change History (27)

comment:1 Changed 7 years ago by jmoore

  • Sprint set to FS Demo 2

comment:2 Changed 7 years ago by jmoore

  • Sprint FS Demo 2 deleted

Not included in fs demo 2 tag. Removing from sprint

comment:3 Changed 7 years ago by jmoore

  • Sprint set to FS Demo 3

comment:4 Changed 7 years ago by jburel

Need to create a new ticket to examine the concept of possibly linking Fileset under Dataset, before closing that task.

comment:5 Changed 7 years ago by mtbcarroll

ManagedRepositoryI.replacementMap's importSetId term requires correction

comment:6 Changed 7 years ago by jamoore

mtbcarroll: agreed. My thinking was to use a nicely formatted timestamp, since "import-set" will become just a API concept (as opposed to a model concept).

comment:7 Changed 7 years ago by mtbcarroll

Okay, addressed the timestamp issue under https://github.com/openmicroscopy/openmicroscopy/pull/998

comment:8 Changed 7 years ago by jamoore

  • Owner set to jamoore
  • Remaining Time set to 1.5

in-progress

comment:9 Changed 7 years ago by jamoore

  • Sprint changed from FS Demo 3 to FS Demo 4

Pushing to "cleanup" bit of next demo.

comment:10 Changed 7 years ago by jamoore

  • Sprint changed from FS Demo 4.x to FS demo 4.1

This is my remaining cleanup tasks. It needs to be in-place for the UI/workflow work, and will continue to become less and less rebase-able.

comment:11 Changed 7 years ago by jburel

I understood the idea of importSet was no longer in the pipeline. So we will have ImportSet and FileSet.

comment:12 Changed 7 years ago by jamoore

  • Summary changed from Introduce ImportSet to Introduce ImportSet to API

ImportSet is only being introduced at the API level. I.e. you can upload a bunch of files and they will turn into multiple filesets. Otherwise, these filesets are unrelated (i.e. no import set object in the DB)

comment:13 Changed 7 years ago by jamoore

  • Sprint changed from FS demo 4.1 to FS demo 4.2

Didn't have time to work on this. Will do so while the import UI is being re-organized.

comment:14 Changed 7 years ago by jamoore

  • Status changed from new to accepted

comment:15 Changed 6 years ago by jburel

  • Sprint changed from FS demo 4.2 to FS Demo 4.3

Moved from sprint FS demo 4.2

comment:16 Changed 6 years ago by jamoore

  • Priority changed from major to critical
  • Sprint changed from FS Demo 4.3 to FS demo 4.x

This will require a lot of coordination and will likely de-stabilize initially. Pushing.

comment:17 Changed 6 years ago by jamoore

  • Description modified (diff)

comment:18 Changed 6 years ago by jamoore

  • Milestone changed from 5.0.0-beta1 to 5.0.0-beta2
  • Sprint FS demo 4.x deleted
  • Version set to 4.4.8

comment:19 Changed 6 years ago by jamoore

  • Sprint set to OMERO 5 Beta 2 (1)
  • Version changed from 4.4.8 to 5.0.0-beta1

comment:20 Changed 6 years ago by jamoore

Pushed branch with origin/develop merged in to https://github.com/joshmoore/openmicroscopy/tree/10325-import-set-api

Next steps:

comment:21 Changed 6 years ago by jamoore

  • Milestone changed from 5.0.0-beta2 to 5.0.0-beta3
  • Sprint OMERO 5 Beta 2 (1) deleted

comment:22 Changed 5 years ago by jamoore

Referencing ticket #10310 has changed sprint.

comment:23 Changed 5 years ago by jamoore

  • Milestone changed from 5.1.0 to 5.1.0-m1

API / DB changes should be out asap.

comment:24 Changed 5 years ago by jamoore

  • Milestone changed from 5.1.0-m1 to 5.1.0
  • Version changed from 5.0.0-beta1 to 5.0.3

Would definitely be a nice to have for 5.1 but I'm not optimistic that it will happen.

comment:25 Changed 5 years ago by jamoore

  • Milestone changed from 5.1.0 to 5.x

comment:26 Changed 4 years ago by mtbcarroll

"Priority: critical": aim for 5.2 or perhaps push further? Happy to try to help with this if I can.

comment:27 Changed 4 years ago by jamoore

  • Milestone changed from 5.x to Unscheduled
Note: See TracTickets for help on using tickets. You may also have a look at Agilo extensions to the ticket.

1.3.13-PRO © 2008-2011 Agilo Software all rights reserved (this page was served in: 0.80809 sec.)

We're Hiring!