Warning: Can't synchronize with repository "(default)" (/home/git/ome.git does not appear to be a Git repository.). Look in the Trac log for more information.
Notice: In order to edit this ticket you need to be either: a Product Owner, The owner or the reporter of the ticket, or, in case of a Task not yet assigned, a team_member"

Task #13225 (closed)

Opened 6 years ago

Closed 4 years ago

Last modified 4 years ago

RFE: paging for cleansing large repositories

Reported by: mtbcarroll Owned by: mtbcarroll
Priority: major Milestone: Unscheduled
Component: Client Version: OMERO-5.2.0
Keywords: fs Cc: server@…, olivier.miquel@…
Resources: n.a. Referenced By: n.a.
References: n.a. Remaining Time: n.a.
Sprint: n.a.

Description

In [ome-users] excessive time to upgrade database / cleanse it is reported that bin/omero admin cleanse dies in PublicRepositoryI.listFiles (RepositoryDaoImpl.getOriginalFiles) on a directory described at http://lists.openmicroscopy.org.uk/pipermail/ome-users/2016-May/005992.html as,

It's a 25G directory containing 54 436 files.

root at kansas:/home/omero/apps/OMERO/OMERO.data/ManagedRepository# du -hs 
bellisj_454/2015-10/01
25G    bellisj_454/2015-10/01
root at kansas:/home/omero/apps/OMERO/OMERO.data/ManagedRepository# find 
./bellisj_454/2015-10/01 -type f | wc -l
54436

We need to reproduce this problem in our own testing then introduce paging as we should be able to cope with large repositories.

Change History (6)

comment:1 Changed 6 years ago by mtbcarroll

  • Cc olivier.miquel@… added

comment:2 Changed 4 years ago by jamoore

comment:3 Changed 4 years ago by mtbcarroll

  • Owner changed from jamoore to mtbcarroll

I can batch that query for 5.4.0.

comment:5 Changed 4 years ago by mtbcarroll

  • Resolution set to fixed
  • Status changed from new to closed

comment:6 Changed 4 years ago by mtbcarroll

Fix should be in OMERO 5.4.0.

Note: See TracTickets for help on using tickets. You may also have a look at Agilo extensions to the ticket.

1.3.13-PRO © 2008-2011 Agilo Software all rights reserved (this page was served in: 0.95740 sec.)

We're Hiring!