Task #10807 (closed)
Opened 11 years ago
Closed 8 years ago
Final DB modifications
Reported by: | jamoore | Owned by: | |
---|---|---|---|
Priority: | critical | Milestone: | Unscheduled |
Component: | Model | Version: | 5.0.0 |
Keywords: | fs | Cc: | fs@… |
Resources: | n.a. | Referenced By: | n.a. |
References: | n.a. | Remaining Time: | n.a. |
Sprint: | n.a. |
Description (last modified by jamoore)
If we have time to perform further DB changes, the following would simplify much of the OriginalFile? and binary logic put in place for FS:
Mimetype
- register() should check for mimetypes
- originalfile super class so there's no mimetype??
- OriginalFile? extends BaseFile? to remove sha1, mimetype, etc.: Dropping several of the OriginalFile? fields would significantly simplify synchronization between the FS and the DB.
- Move DIRECTORY_MIMETYPE to constants.
- Mimetypes (including "Directory") should be compared to standards: There are various mimetypes relating to directories. Perhaps we could either use of those, or use an "application/vnn.openmicroscopy.dir" style
filesets
- add javaVersion to FilesetVersionInfo?. (And Python?) (see 888)
- Review all the version info set by python/Java/C++
- where to put tags during upload
- check for other definitions of "Reader" (with formatString.replaceAll)
- include FileInfo (e.g. companion=true, series=2 in filesetEntry (See #10979)
Misc
- PixelsType?.MIN|MAX in database
Change History (31)
comment:1 Changed 11 years ago by mtbcarroll
comment:2 Changed 11 years ago by jamoore
- Description modified (diff)
comment:3 Changed 11 years ago by jamoore
- Keywords fs added
comment:4 Changed 11 years ago by jamoore
- Description modified (diff)
- Priority changed from major to critical
- Sprint changed from FS demo 4.x to FS demo 4.2
- Summary changed from Final OriginalFile modifications to Final DB modifications
comment:5 Changed 11 years ago by jburel
- Sprint changed from FS demo 4.2 to FS Demo 4.3
Moved from sprint FS demo 4.2
comment:6 Changed 11 years ago by jamoore
- Description modified (diff)
comment:7 Changed 11 years ago by mtbcarroll
The java version information could be stored in a hstore column; will that need extra code in Property.java and a new ParameterizedType implementation used by PostgresSqlAction?
comment:9 Changed 11 years ago by mtbcarroll
If we use a late enough JDBC driver: http://postgresql.1045698.n5.nabble.com/Patch-Add-support-for-hstore-extension-and-map-it-to-from-java-util-Map-td5699633.html
comment:10 Changed 11 years ago by mtbcarroll
also note #7069
comment:11 Changed 10 years ago by mtbcarroll
Is there any further work for beta2 here that must be accounted for by #11516?
comment:12 Changed 10 years ago by jamoore
Some of these it would certainly be good to address for 5.0.0, but whether or not we get that done for beta2 is unclear.
comment:13 Changed 10 years ago by jamoore
- Milestone changed from OMERO-5 to 5.0.0-beta2
- Sprint FS demo 4.x deleted
- Version set to 4.4.9
Moving to beta2 for evaluation.
comment:14 Changed 10 years ago by jamoore
- Description modified (diff)
comment:15 Changed 10 years ago by jburel
Model PR considered for the coming release
- https://github.com/openmicroscopy/bioformats/pull/761 (doc)
- https://github.com/openmicroscopy/bioformats/pull/762 (annotatable objects)
- https://github.com/openmicroscopy/bioformats/pull/763 (new Map annotation)
comment:16 Changed 10 years ago by jburel
Model changes will happen in Beta3.
Pushing for code freeze for Beta2. Moving to Beta3
comment:17 Changed 10 years ago by jamoore
- Milestone changed from 5.0.0-beta2 to 5.0.0-beta3
comment:18 Changed 10 years ago by mtbcarroll
So, should we get rid of filesetversioninfo and turn it instead into a map property of fileset?
comment:19 Changed 10 years ago by jamoore
- Milestone changed from 5.1.0 to 5.1.0-m1
- Version changed from 4.4.9 to 5.0.0
Quite likely. Moving this. Let's discuss ASAP.
comment:20 Changed 10 years ago by jamoore
In conversation with J-m and Mark, brain-storming other uses of maps / map-annotations:
- experimenter.config (e.g. "userPreferences")
- script provenance
- perhaps something for open-with (per-user-per-host)
- mounted drives for in-place
- location of files/data/images (url/cloud/etc)
- external links (connection to other DB) (Josh: NS and map annotation?)
Need a discussion with the wider team on any of other similar uses. Mark to open an initial PR that others can discuss and possibly open secondary PRs against his.
comment:21 Changed 10 years ago by mtbcarroll
Once MapAnnotation is merged I'll rebase and force-push https://github.com/openmicroscopy/openmicroscopy/pull/2301 and will then be happy to change its title / description as other PRs come in against it.
comment:22 Changed 10 years ago by mtbcarroll
This whole story and worthwhile comments could be migrated to Trello.
comment:23 Changed 10 years ago by mtbcarroll
Maps with model enumerations as keys might be useful. Clients can reference their string values and they can be stored efficiently in the database with foreign keys into the enumeration table.
comment:24 Changed 10 years ago by jamoore
and filesets should like have a link to their repository objects.
comment:25 Changed 10 years ago by jburel
- Milestone changed from 5.1.0-m1 to 5.1.0-m2
few more changes to come. so pushing to m2.
comment:26 Changed 10 years ago by jamoore
Whatever bits of this are going to be done for 5.1.0 should definitely be done for m2 and the rest pushed to post 5.1.0.
comment:27 Changed 9 years ago by jamoore
- Milestone changed from 5.1.0-m2 to 5.x
Per https://trello.com/c/cur5hd3e/44-5-2-db-changes (and in discussion with Mark), moving all of these changes to 5.x. Intended for 5.2.0-m1 (or similar)
comment:28 Changed 9 years ago by mtbcarroll
Regarding FileInfo, it seems that at present files like test_images_good/bruker/wis/RA_060110.051/, with many images in the fileset, are annotating every image with every companion file: in that case, causing 6,006 file annotations, so there might be a significant win there for certain kinds of MIF.
comment:29 Changed 9 years ago by mtbcarroll
"Intended for 5.2.0-m1" and "Priority: critical": should we start chatting? The OriginalFile reworking also bears on https://trello.com/c/dPfFR1H4/15-duplicate-request-in-cmd.
comment:30 Changed 8 years ago by jamoore
- Milestone changed from 5.x to Unscheduled
comment:31 Changed 8 years ago by jburel
- Resolution set to fixed
- Status changed from new to closed
Things like repositories and directories and whatnot should perhaps each have separate classes in the hierarchy.