Task #10892 (closed)
Opened 11 years ago
Closed 11 years ago
Bug: chgrp/delete performance
| Reported by: | jamoore | Owned by: | jamoore |
|---|---|---|---|
| Priority: | blocker | Milestone: | 5.0.0-beta1 |
| Component: | Services | Version: | n.a. |
| Keywords: | fs | Cc: | mtbcarroll, fs@… |
| Resources: | n.a. | Referenced By: | n.a. |
| References: | n.a. | Remaining Time: | n.a. |
| Sprint: | FS demo 4.4 |
Description
There have been numerous reports of slow graph operations. Moving 56 lei for example taking minutes. Minimally some investigation should be done into where this is happening.
A previous report about passing many Delete("/OriginalFile") instances in a single DoAll? seemed to be waiting completely at the JDBC level, i.e. the size of the transaction with the savepoints was simply too large.
Attachments (1)
Change History (9)
comment:1 Changed 11 years ago by jamoore
comment:2 Changed 11 years ago by jburel
- Sprint changed from FS demo 4.2 to FS Demo 4.3
Moved from sprint FS demo 4.2
comment:3 Changed 11 years ago by jamoore
I'm happy to look at this for demo 4.3, but I'm not very confident that a solution can be found in that time scale. The most likely solution that occurs to me is to drop the use of PG savepoints/rollbacks completely and rely on our own pre-/post-check logic. But replacing a DB feature like that requires (like our permission checks) requires extensive testing. Happy to be proved wrong though.
comment:4 Changed 11 years ago by wmoore
I just tried moving 3 images from Leica/joel/three_channel_seqen.lei in web, which always fails (locally) with the web hanging and a server restart needed. The log file from the chgrp submission to "hang" is attached http://trac.openmicroscopy.org.uk/ome/attachment/ticket/10892/chgrp-lei.log.zip
comment:5 Changed 11 years ago by jamoore
Will, by "locally" you mean on your own server? If so, what's the the memory setting? Do you have any OutOfMemory? lines in the log?
comment:6 Changed 11 years ago by jburel
- Sprint changed from FS Demo 4.3 to FS demo 4.4
Moved from sprint FS Demo 4.3
comment:7 Changed 11 years ago by jamoore
- Owner set to jamoore
comment:8 Changed 11 years ago by jamoore
- Resolution set to fixed
- Status changed from new to closed
The BTM fix introduced in https://github.com/openmicroscopy/openmicroscopy/pull/1243 for #11033 looks to have largely solved the performance issues. If problems arise again, please re-open.
As mentioned under PR 1142, chgrping the lei can succeed, but is running into O(n^2) issues since for each /Image, all the /FilesetEntry links are also attempted, which is unnecessary.