Changes between Initial Version and Version 9 of Ticket #3163

Timestamp:

10/27/10 12:59:14 (14 years ago)

Author:

jmoore

Comment:

omero.delete.ids phase is now quite short:

2010-10-27 10:22:09,378 INFO  [                 org.perf4j.TimingLogger] (3-thread-1) start[1288171269918] time[59460] tag[omero.delete.ids.274319]

or one minute for 275K items.

On average, 10K delete steps take about a minute. A dataset with 15K+ images consists of about 270K delete steps each, or roughly 30 minutes delete time with the latest server:

omero@howe:~/OMERO-CURRENT$ bin/omero -s mike@localhost hql -q "select d.id, count(dil) from Dataset d join d.imageLinks dil group by d.id"
Previously logged in to localhost as mike
Password:
Created session 84332076-88a6-4398-9261-9827ca7ca547 (mike@localhost). Idle timeout: 10.0 min. Current group: Swedlow Lab
#  | Col1 | Col2
----+------+-------
...
20 | 600  | 15360
21 | 618  | 77
22 | 626  | 15360
23 | 648  | 1
24 | 655  | 10
(25 rows)
omero@howe:~/OMERO-CURRENT$ time bin/omero delete /Dataset:600
Using session 84332076-88a6-4398-9261-9827ca7ca547 (mike@localhost). Idle timeout: 10.0 min. Current group: Swedlow Lab
Deleting /Dataset 600... ok

real    32m29.207s
user    0m2.180s
sys     0m0.770s

omero@howe:~/OMERO-CURRENT$ tail -n 500000 var/log/Blitz-0.log | perl -ne 'print if /omero.delete.step.\d+0000\]/'
2010-10-27 10:41:47,382 INFO  [                 org.perf4j.TimingLogger] (3-thread-1) start[1288172507381] time[1] tag[omero.delete.step.180000]
2010-10-27 10:43:01,948 INFO  [                 org.perf4j.TimingLogger] (3-thread-1) start[1288172581947] time[1] tag[omero.delete.step.190000]
2010-10-27 10:44:19,389 INFO  [                 org.perf4j.TimingLogger] (3-thread-1) start[1288172659388] time[1] tag[omero.delete.step.200000]
2010-10-27 10:45:34,224 INFO  [                 org.perf4j.TimingLogger] (3-thread-1) start[1288172734223] time[1] tag[omero.delete.step.210000]
2010-10-27 10:46:48,890 INFO  [                 org.perf4j.TimingLogger] (3-thread-1) start[1288172808889] time[1] tag[omero.delete.step.220000]
2010-10-27 10:48:04,126 INFO  [                 org.perf4j.TimingLogger] (3-thread-1) start[1288172884125] time[1] tag[omero.delete.step.230000]
2010-10-27 10:49:19,016 INFO  [                 org.perf4j.TimingLogger] (3-thread-1) start[1288172959015] time[1] tag[omero.delete.step.240000]
2010-10-27 10:50:34,640 INFO  [                 org.perf4j.TimingLogger] (3-thread-1) start[1288173034639] time[1] tag[omero.delete.step.250000]
2010-10-27 10:51:49,835 INFO  [                 org.perf4j.TimingLogger] (3-thread-1) start[1288173109833] time[2] tag[omero.delete.step.260000]
2010-10-27 10:53:05,468 INFO  [                 org.perf4j.TimingLogger] (3-thread-1) start[1288173185467] time[1] tag[omero.delete.step.270000]

Closing this. Other optimizations may be necessary later.