Task #11184 (closed)
Opened 11 years ago
Closed 10 years ago
Unexplained import overhead
| Reported by: | jamoore | Owned by: | jamoore |
|---|---|---|---|
| Priority: | critical | Milestone: | 5.0.0-rc1 |
| Component: | OmeroFs | Version: | 5.0.0-beta1 |
| Keywords: | n.a. | Cc: | fs@… |
| Resources: | n.a. | Referenced By: | n.a. |
| References: | n.a. | Remaining Time: | n.a. |
| Sprint: | OMERO 5 Beta 2 (1) |
Description (last modified by jamoore)
While preparing the FS workshop for Paris, there were a few surprisingly slow imports. An LEI dataset for example took roughly 100 seconds for some 140 files, while the pixel upload was much slower. Watching the stack traces via jstack in at least one instance, pointed at some of the directory creation logic to be taking too long. Perhaps due to one transaction per file?
This code path needs to be optimized. Though it's usable, there's no reason any of the OME5 imports should be significantly slower than the OME4 imports, unless there's significant non-binary data contained in the files.
Change History (4)
comment:1 Changed 11 years ago by jamoore
- Description modified (diff)
- Version set to 4.4.8
comment:2 Changed 11 years ago by jamoore
- Version changed from 4.4.8 to 5.0.0-beta1
comment:3 Changed 10 years ago by jburel
- Milestone changed from OMERO-5 to 5.0.0-beta2
- Sprint changed from FS demo 4.x to OMERO 5 Beta 2 (1)
comment:4 Changed 10 years ago by jamoore
- Resolution set to duplicate
- Status changed from new to closed
Marking as a duplicate of #11250 since Roger added performance numbers. Thanks.
Note: See
TracTickets for help on using
tickets.
You may also have a look at Agilo extensions to the ticket.
Moving to Beta2 for review