Task #4872 (closed)
Bug: Import History slows large import queues
Reported by: | jamoore | Owned by: | bwzloranger |
---|---|---|---|
Priority: | critical | Milestone: | OMERO-Beta4.3 |
Component: | Import | Version: | n.a. |
Keywords: | n.a. | Cc: | cxallan, jburel |
Resources: | n.a. | Referenced By: | n.a. |
References: | n.a. | Remaining Time: | 0.0d |
Sprint: | n.a. |
Description
After IMPORT_DONE there's an up-to 12 second pause for BK (see forum URL) before any other real activity:
2011-04-04 11:02:50,688 138544 [2-thread-1] INFO ormats.importer.gui.LoggingImportMonitor - DATA_STORED 2011-04-04 11:02:51,094 138950 [2-thread-1] INFO ormats.importer.gui.LoggingImportMonitor - IMPORT_OVERLAYS 2011-04-04 11:02:51,094 138950 [2-thread-1] INFO ormats.importer.gui.LoggingImportMonitor - IMPORT_THUMBNAILING 2011-04-04 11:02:51,259 139115 [2-thread-1] INFO ormats.importer.gui.LoggingImportMonitor - IMPORT_DONE 2011-04-04 11:02:51,269 139125 [2-thread-1] DEBUG ome.formats.OMEROMetadataStoreClient - Creating root! 2011-04-04 11:02:51,276 139132 [2-thread-1] DEBUG ome.formats.OMEROMetadataStoreClient - Creating root! 2011-04-04 11:03:02,671 150527 [2-thread-1] INFO ome.formats.importer.ImportLibrary - Metadata only import? false
Looking in Tables-0.log, there's an average of 6.6 seconds per call to table.update()
2011-04-04 16:19:18,867 INFO [ omero.remote] (Dummy-11 ) Meth: TableI.update 2011-04-04 16:19:26,182 INFO [ omero.tables.TableI] (Dummy-11 ) Updated 5861 row(s) of data to Table-...
Actions:
- table history should be configurable from the GUI and importer.ini
- table history methods should be logged and displayed in the gui (possibly before IMPORT_ONE)
- (alternatively) table history methods should run in the background and be cancelable
- depending on the required changes, this work should be ported to 4.2
See: http://openmicroscopy.org/community/viewtopic.php?f=4&t=645&sid=7a4931be888a091fb3903dd7a0895d26
Change History (5)
comment:1 Changed 13 years ago by bwzloranger
- Remaining Time set to 2
comment:2 Changed 13 years ago by bwzloranger <bwzloranger@…>
- Remaining Time changed from 2 to 0
- Resolution set to fixed
- Status changed from new to closed
comment:3 Changed 13 years ago by bwzloranger
Some discussion about deprecating the history feature entirely (with import more reliable now and insight including its own version). There is now an option for users to to disable the history system in options, and admins can also alter the import.config file to override the option entirely.
comment:4 Changed 13 years ago by jburel
In that case, it might be better to have if off by default.
comment:5 Changed 13 years ago by bwzloranger <bwzloranger@…>
(In [7a6101ef8a55a888b5c4b8d5761345766cc59469/ome.git] on branch develop) Fixes #4872. Added options to allow user and admin history disable. Now disabled by default.
(In [37f4baf3d24de7bfb06a4f26f519dbf98fb555cb/ome.git] on branch develop) Fixes #4872. Added options to allow user and admin history disable