Too Many Open Files

[cxallan]

The binary services currently open many files from disk without explicitly closing the file handle. The garbage collector is not able to keep up over time which eventually results in the operating system's "process max file handle count" being exhausted.

[cxallan]

Some of this will have been fixed by the thumbnail service changes in r457 but r1069 should stamp out any further occurrences.

[cxallan]

Further squashing done in r1081. For some reason the PixelBuffer?'s don't seem to be garbage collected correctly and in order to insure that we don't end up with file handle exhaustion I've moved the "close()" closer to the mapped byte buffer retrieval.

[cxallan]

This is still happening intermitantly. It seems to regularly happen on Valewalker:

cxallan@valewalker ~ $ java -version
java version "1.6.0"
Java(TM) SE Runtime Environment (build 1.6.0-b105)
Java HotSpot?(TM) Server VM (build 1.6.0-b105, mixed mode)

[jburel]

Problem scenario:

For testing, I had to browse michael's datasets (70 thumbnails in total) each time
I repeated the information 4 / 5 times (logged in, logged out)
I browsed the same datasets each time.

The 5th time, some of the thumbnails were not loaded.
Follow the error message

Error message
Caused by: javax.naming.CommunicationException: Failed to retrieve stub from server   valewalker.openmicroscopy.org.uk:1099 [Root exception is java.io.EOFException]
at org.jnp.interfaces.NamingContext.getServer(NamingContext.java:263)
at org.jnp.interfaces.NamingContext.checkRef(NamingContext.java:1385)...
Caused by: java.io.EOFException 
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2228) at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2694)
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:761)
at java.io.ObjectInputStream.<init>(ObjectInputStream.java:277)
at org.jnp.interfaces.NamingContext.getServer(NamingContext.java:250)

[cxallan]

Changes in r1557 should resolve this.

[cxallan]

r1558 plugs another edge case where we can leak file handles.

[cxallan]

r1559 plugs another edge case where we can leak file handles in the ImageIO logic in the thumbnail bean, this should also increase performance. It appears as though we are in fact leaking memory here as a result of close() not being called on a ByteArrayInputStream.

[cxallan]

r1560 deals with the ImageOutputStream that the ImageIO logic uses internally. This should fix the presence of temporary files and also increases performance significantly. Thumbnail retrieval is now approximately 10x faster than before.

[cxallan]

This is now evident in milestone:3.0-Beta3.1 projection code.

[cxallan]

Fixed in r2875 in Trunk and r2876 in the milestone:3.0-Beta3.1 branch.