Warning: Can't synchronize with repository "(default)" (/home/git/ome.git does not appear to be a Git repository.). Look in the Trac log for more information.
Notice: In order to edit this ticket you need to be either: a Product Owner, The owner or the reporter of the ticket, or, in case of a Task not yet assigned, a team_member"

Task #2764 (closed)

Opened 14 years ago

Closed 14 years ago

Last modified 14 years ago

BUG: OME-TIFF export fails with "Too many files"

Reported by: jburel Owned by: cxallan
Priority: critical Milestone: OMERO-Beta4.2.2
Component: General Version: n.a.
Keywords: n.a. Cc: jamoore, cxallan, mlinkert, cneves
Resources: n.a. Referenced By: n.a.
References: n.a. Remaining Time: 0.0d
Sprint: n.a.

Description (last modified by jmoore)

Possibly due to a leak in OMETiffWriter on close() the server runs out of file descriptors (See ​loci:#543)

Workaround:
Export should throw an exception to the user if close() fails, so that the issue is noticed more quickly.

Long-term a fix for this descriptor leak and any others in the OMERO code base need to be fixed.

Original exception from QA

java.lang.Exception: org.openmicroscopy.shoola.env.data.DSAccessException: Cannot access data.
Cannot export the image as an OME-TIFF


at org.openmicroscopy.shoola.env.data.OMEROGateway.handleException(OMEROGateway.java:732)


at org.openmicroscopy.shoola.env.data.OMEROGateway.exportImageAsOMETiff(OMEROGateway.java:6357)


at org.openmicroscopy.shoola.env.data.OmeroImageServiceImpl.exportImageAsOMETiff(OmeroImageServiceImpl.java:830)


at org.openmicroscopy.shoola.env.data.views.calls.ExportLoader$1.doCall(ExportLoader.java:76)


at org.openmicroscopy.shoola.env.data.views.BatchCall.doStep(BatchCall.java:145)


at org.openmicroscopy.shoola.util.concur.tasks.CompositeTask.doStep(CompositeTask.java:226)


at org.openmicroscopy.shoola.env.data.views.CompositeBatchCall.doStep(CompositeBatchCall.java:126)


at org.openmicroscopy.shoola.util.concur.tasks.ExecCommand.exec(ExecCommand.java:165)


at org.openmicroscopy.shoola.util.concur.tasks.ExecCommand.run(ExecCommand.java:274)


at org.openmicroscopy.shoola.util.concur.tasks.AsyncProcessor$Runner.run(AsyncProcessor.java:91)


at java.lang.Thread.run(Unknown Source)


Caused by: omero.InternalException


serverStackTrace = "java.io.FileNotFoundException: /home/omero/omero/tmp/omero_omero/2813@localhost/__omero_export__8090264881308579116.ome.tiff (Too many open files)

Change History (21)

comment:1 Changed 14 years ago by jmoore

  • Description modified (diff)
  • Summary changed from BUG: OME-TIFF export to BUG: OME-TIFF export fails with "Too many files"

comment:2 Changed 14 years ago by jmoore

  • Description modified (diff)

comment:3 Changed 14 years ago by jmoore

  • Owner set to jmoore
  • Status changed from new to assigned

comment:4 Changed 14 years ago by jmoore

  • Remaining Time set to 0
  • Resolution set to fixed
  • Status changed from assigned to closed

(In [7973]) Calling writer.close before Exporter return (Fix #2764)

This will cause any exceptions to be thrown to the client
as InternalException so that we can more quickly detect
issues like this.

comment:5 Changed 14 years ago by cxallan

  • Resolution fixed deleted
  • Status changed from closed to reopened

comment:6 Changed 14 years ago by cxallan

Re-opened to evaluate the extent of the issue with 4.1.x as well.

comment:7 Changed 14 years ago by cxallan

  • Owner changed from jmoore to cxallan
  • Status changed from reopened to new

comment:8 Changed 14 years ago by cxallan

  • Status changed from new to assigned

comment:9 Changed 14 years ago by cxallan

  • Sprint set to 2010-09-09 (16)

comment:10 Changed 14 years ago by cxallan

(In [8044]) Updating to ​bio-formats:r6921; a solution for leaking file handles on export in Beta 4.1.x. (See #2764)

comment:11 Changed 14 years ago by cxallan

(In [8064]) Updating to ​bio-formats:r6947; fixing leaking filehandles during plane writing. (See #2764)

comment:12 Changed 14 years ago by cxallan

  • Resolution set to fixed
  • Status changed from assigned to closed

comment:13 Changed 14 years ago by cxallan

  • Cc mlinkert-x added
  • Resolution fixed deleted
  • Status changed from closed to reopened

Re-opening with the same issue happening in the 4.2.1 release version.

comment:14 Changed 14 years ago by cxallan

Looks like the culprit is ​bio-formats:r7155. Specifically the opening of a TiffParser with currentId and not closing it. As a result we end up leaking an FD per plane write.

comment:15 Changed 14 years ago by jmoore

  • Cc cneves-x added
  • Milestone changed from OMERO-Beta4.2.1 to OMERO-Beta4.2.2
  • Sprint 2010-09-09 (16) deleted

comment:16 Changed 14 years ago by cxallan

  • Resolution set to fixed
  • Status changed from reopened to closed

Fixed upstream in ​bio-formats:r7226 and merged into Trunk with r8645.

comment:17 Changed 14 years ago by cxallan

Merged into the Bio-Formats Beta 4.2 branch in ​bio-formats:r7263.

comment:18 Changed 14 years ago by cxallan <cxallan@…>

(In [a3eaee4afb4dc4e360a620d09d035729e4e39bde/ome.git]on branches 4_1, dev_4_1, dev_4_1_custom) Updating to ​bio-formats:r6921; a solution for leaking file handles on export in Beta 4.1.x. (See #2764)

git-svn-id: ​file:///home/svn/omero/branches/Beta4.1@8044 05709c45-44f0-0310-885b-81a1db45b4a6

comment:19 Changed 14 years ago by jmoore <josh@…>

(In [0ed10cb9b92a9fd02ad7c28a2267f1026d3a36d2/ome.git] on branch dev_4_1_custom) Merge remote branch 'origin/dev_4_1' into dev_4_1_custom

Conflicts:

.classpath
components/tools/OmeroImporter/src/ome/formats/importer/ImportLibrary.java
components/tools/OmeroImporter/src/ome/formats/importer/cli/CommandLineImporter.java
etc/omero.properties
lib/repository/bio-formats-9071a01.jar
lib/repository/jai_imageio-9071a01.jar
lib/repository/loci-common-9071a01.jar
lib/repository/mdbtools-java-9071a01.jar
lib/repository/ome-xml-9071a01.jar
lib/repository/poi-loci-9071a01.jar

comment:20 Changed 14 years ago by cxallan <cxallan@…>

(In [a3eaee4afb4dc4e360a620d09d035729e4e39bde/ome.git]on branches 4_1, 4_1_custom, dev_4_1, dev_4_1_custom, josh/4.1-web-backport) Updating to ​bio-formats:r6921; a solution for leaking file handles on export in Beta 4.1.x. (See #2764)

git-svn-id: ​file:///home/svn/omero/branches/Beta4.1@8044 05709c45-44f0-0310-885b-81a1db45b4a6

comment:21 Changed 14 years ago by jmoore <josh@…>

(In [0ed10cb9b92a9fd02ad7c28a2267f1026d3a36d2/ome.git]on branches 4_1_custom, dev_4_1_custom, josh/4.1-web-backport) Merge remote branch 'origin/dev_4_1' into dev_4_1_custom

Conflicts:

.classpath
components/tools/OmeroImporter/src/ome/formats/importer/ImportLibrary.java
components/tools/OmeroImporter/src/ome/formats/importer/cli/CommandLineImporter.java
etc/omero.properties
lib/repository/bio-formats-9071a01.jar
lib/repository/jai_imageio-9071a01.jar
lib/repository/loci-common-9071a01.jar
lib/repository/mdbtools-java-9071a01.jar
lib/repository/ome-xml-9071a01.jar
lib/repository/poi-loci-9071a01.jar

Note: See TracTickets for help on using tickets. You may also have a look at Agilo extensions to the ticket.

1.3.13-PRO © 2008-2011 Agilo Software all rights reserved (this page was served in: 0.71164 sec.)

© 2000-2014 University of Dundee & Open Microscopy Environment. Creative Commons Attribution 3.0 Unported License
OME source code is available under the GNU General public license or through commercial license from Glencoe Software

We're Hiring!