Warning: Can't synchronize with repository "(default)" (/home/git/ome.git does not appear to be a Git repository.). Look in the Trac log for more information.
Notice: In order to edit this ticket you need to be either: a Product Owner, The owner or the reporter of the ticket, or, in case of a Task not yet assigned, a team_member"

Task #11011 (closed)

Opened 11 years ago

Closed 10 years ago

Bug: OOM PermGen space during Kryo

Reported by: jamoore Owned by:
Priority: critical Milestone: 5.0.2
Component: Performance Version: 5.0.1
Keywords: fs Cc: fs@…, cxallan, sbesson, khgillen
Resources: n.a. Referenced By: n.a.
References: n.a. Remaining Time: n.a.
Sprint: n.a.

Description

During testing, gretzky OOM'd with "PermGen? space" which is a seldom occurrence these days. The log file shows:

2013-05-30 13:03:16,367 DEBUG [                   loci.formats.Memoizer] (l.Server-5) loaded memo file: /OMERO/BioFormatsCache/OMERO/ManagedRepository/user-4_5/2013-05/30/10-44-32.064/.20110629-Spat-1_1.tif.bfmemo (5091492 bytes)
2013-05-30 13:03:16,367 DEBUG [                   loci.formats.Memoizer] (l.Server-5) start[1369915395579] time[788] tag[loci.formats.Memoizer.loadMemo]
2013-05-30 13:03:16,367 DEBUG [                   loci.formats.Memoizer] (l.Server-5) start[1369915395579] time[788] tag[loci.formats.Memoizer.setId]
2013-05-30 13:03:16,371 INFO  [                 org.perf4j.TimingLogger] (l.Server-5) start[1369915395548] time[822] tag[omero.call.exception]
2013-05-30 13:03:16,373 ERROR [        ome.services.util.ServiceHandler] (l.Server-5) java.lang.Error:  Wrapped Exception: (java.lang.OutOfMemoryError):
PermGen space
java.lang.OutOfMemoryError: PermGen space
        at java.lang.Class.getDeclaredConstructors0(Native Method) ~[na:1.5.0_22]
        at java.lang.Class.privateGetDeclaredConstructors(Class.java:2357) ~[na:1.5.0_22]
        at java.lang.Class.getConstructor0(Class.java:2671) ~[na:1.5.0_22]

Though this may be coincidental, the fact is that Kryo does do class loading magic which would likely use more of the perm gen space. If so, the default perm gen value may have to be bumped. If this continues to happen (i.e. if kryo is not built for being used in a long-running application), then something more fundamental will have to be done (i.e. dropping kryo).

Change History (3)

comment:1 Changed 11 years ago by jamoore

  • Cc sbesson khgillen added

Note: likely gretzky was running a quite old JDK5 like howe is currently:

hudson@howe:~$ ls -al  /proc/19078/exe 
lrwxrwxrwx 1 hudson hudson 0 2013-05-30 14:44 /proc/19078/exe -> /home/hudson/hudson/tools/hudson.model.JDK/5_0_22/bin/java

All the jobs which use a JDK5 for compilation should likely be carefully checked to deploy with JDK6.

comment:2 Changed 11 years ago by jamoore

Another full hang today but without an OOM. See logs in omero@gretzky:~/Josh. This time PG was locked, most likely by various deletes.

comment:3 Changed 10 years ago by jamoore

  • Milestone changed from 5.x to 5.0.2
  • Resolution set to worksforme
  • Sprint FS demo 4.x deleted
  • Status changed from new to closed
  • Version set to 5.0.1

Closing as we've not seen any more of this. Minimum JDK version is now 6+ on these branches, so this hopefully won't be seen again.

Note: See TracTickets for help on using tickets. You may also have a look at Agilo extensions to the ticket.

1.3.13-PRO © 2008-2011 Agilo Software all rights reserved (this page was served in: 0.61742 sec.)

We're Hiring!