Notice: In order to edit this ticket you need to be either: a Product Owner, The owner or the reporter of the ticket, or, in case of a Task not yet assigned, a team_member"

Task #12315 (new)

Opened 5 years ago

Last modified 3 years ago

Bug: DropBox can't remove temporary files on Windows

Reported by: bpindelski Owned by: spli
Priority: major Milestone: Unscheduled
Component: Client Version: 5.0.1
Keywords: n.a. Cc: cblackburn, jamoore, jburel, spli
Resources: n.a. Referenced By: https://trello.com/c/EOL8YL14/54-bug-import
References: n.a. Remaining Time: n.a.
Sprint: n.a.

Description

When testing DropBox? with Ice 3.5.1 during the 5.0.2 release cycle, it has been discovered that a DropBox? import leaves a lot of error messages in the DropBox.log file:

DropBox.log:2014-05-22 16:11:32,821 ERROR [              omero.util.TempFileManager] (Thread-3  ) Failed to remove file C:\Windows\system32\config\systemprofile\AppData\Roaming\omero\tmp\omero_WIN7$\2684\candidateskhu_bverr
DropBox.log:2014-05-22 16:57:11,243 ERROR [              omero.util.TempFileManager] (Thread-222) Failed to remove file C:\Windows\system32\config\systemprofile\AppData\Roaming\omero\tmp\omero_WIN7$\2684\dropboxouuizmerr
DropBox.log:2014-05-22 16:57:11,243 ERROR [              omero.util.TempFileManager] (Thread-223) Failed to remove file C:\Windows\system32\config\systemprofile\AppData\Roaming\omero\tmp\omero_WIN7$\2684\dropboxk_smdterr
DropBox.log:2014-05-22 16:57:11,243 ERROR [              omero.util.TempFileManager] (Thread-149) Failed to remove file C:\Windows\system32\config\systemprofile\AppData\Roaming\omero\tmp\omero_WIN7$\2684\dropbox2xw9nlerr
DropBox.log:2014-05-22 16:57:11,336 ERROR [              omero.util.TempFileManager] (Thread-219) Failed to remove file C:\Windows\system32\config\systemprofile\AppData\Roaming\omero\tmp\omero_WIN7$\2684\dropboxnipjyberr
DropBox.log:2014-05-22 16:57:11,414 ERROR [              omero.util.TempFileManager] (Thread-222) Failed to remove file C:\Windows\system32\config\systemprofile\AppData\Roaming\omero\tmp\omero_WIN7$\2684\dropboxiz49vbout
DropBox.log:2014-05-22 16:57:11,414 ERROR [              omero.util.TempFileManager] (Thread-223) Failed to remove file C:\Windows\system32\config\systemprofile\AppData\Roaming\omero\tmp\omero_WIN7$\2684\dropboxxiju1aout
DropBox.log:2014-05-22 16:57:11,414 ERROR [              omero.util.TempFileManager] (Thread-149) Failed to remove file C:\Windows\system32\config\systemprofile\AppData\Roaming\omero\tmp\omero_WIN7$\2684\dropboxsb6468out
DropBox.log:2014-05-22 16:57:11,414 ERROR [              omero.util.TempFileManager] (Thread-219) Failed to remove file C:\Windows\system32\config\systemprofile\AppData\Roaming\omero\tmp\omero_WIN7$\2684\dropboxeqlmstout

Josh's suspicion is that "(...) it's probably leaking file handles if the file can't be cleaned up."

Change History (8)

comment:1 Changed 5 years ago by jburel

  • Cc jamoore jburel added
  • Referenced By set to https://trello.com/c/EOL8YL14/54-bug-import

comment:2 Changed 4 years ago by jamoore

  • Cc spli added

Anything we're going to try for 5.1.0 or push?

comment:3 Changed 4 years ago by spli

  • Owner set to spli

comment:4 Changed 4 years ago by spli

Verified wirh OMERO.server-5.1.0-m3-928-390d925-ice34-b554

Those temporary files look like they're used by Python to record the output from the Java importer. See the create_path() and remove_path() calls in

​https://github.com/openmicroscopy/openmicroscopy/blob/v5.1.0-m3/components/tools/OmeroFS/src/fsDropBoxMonitorClient.py#L629
​https://github.com/openmicroscopy/openmicroscopy/blob/v5.1.0-m3/components/tools/OmeroFS/src/fsDropBoxMonitorClient.py#L675

and

​https://github.com/openmicroscopy/openmicroscopy/blob/v5.1.0-m3/components/tools/OmeroPy/src/omero/util/import_candidates.py#L62
​https://github.com/openmicroscopy/openmicroscopy/blob/v5.1.0-m3/components/tools/OmeroPy/src/omero/util/import_candidates.py#L72

However running the import_candidates method from a shell doesn't show any problems:

H:\>set DEBUG=true

H:\>echo %DEBUG%
true

H:\>c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\bin\omero shel
l --login
Deprecated warning: use the 'bin/omero --debug=x [args]' to debug
Running omero with debugging == 1
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\admin.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\basics.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\cecog.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\chgrp.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\db.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\delete.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\download.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\export.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\fs.py
DEBUG:omero.util.TempFileManager:Added file C:\Users\spli\AppData\Roaming\.lock_
testaohkl2.tmp
2015-02-10 16:40:50,854 DEBUG [              omero.util.TempFileManager] (MainTh
read) Added file C:\Users\spli\AppData\Roaming\.lock_testaohkl2.tmp
DEBUG:omero.util.TempFileManager:Chose gloabl tmpdir: C:\Users\spli\AppData\Roam
ing
2015-02-10 16:40:50,855 DEBUG [              omero.util.TempFileManager] (MainTh
read) Chose gloabl tmpdir: C:\Users\spli\AppData\Roaming
DEBUG:omero.util.TempFileManager:Using temp dir: C:\Users\spli\AppData\Roaming\o
mero\tmp\omero_spli\4048
2015-02-10 16:40:50,865 DEBUG [              omero.util.TempFileManager] (MainTh
read) Using temp dir: C:\Users\spli\AppData\Roaming\omero\tmp\omero_spli\4048
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\group.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\hql.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\import.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\ldap.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\node.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\obj.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\perf.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\prefs.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\script.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\search.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\server.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\sessions.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\submit.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\tag.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\upload.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\user.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\web.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\__init__.py
DEBUG:omero.util.UpgradeCheck:Attempting to connect to http://upgrade.openmicros
copy.org.uk/?os.version=Windows-2008ServerR2-6.1.7601-SP1&os.arch=AMD64&version=
5.1.0-m3-960-3b028da-ice34-b557&python.compiler=MSC+v.1500+64+bit+%28AMD64%29&py
thon.build=%28%27r266%3A84297%27%2C+%27Aug+24+2010+18%3A13%3A38%27%29&os.name=Wi
ndows&python.version=2.6.6
2015-02-10 16:40:51,272 DEBUG [                 omero.util.UpgradeCheck] (MainTh
read) Attempting to connect to http://upgrade.openmicroscopy.org.uk/?os.version=
Windows-2008ServerR2-6.1.7601-SP1&os.arch=AMD64&version=5.1.0-m3-960-3b028da-ice
34-b557&python.compiler=MSC+v.1500+64+bit+%28AMD64%29&python.build=%28%27r266%3A
84297%27%2C+%27Aug+24+2010+18%3A13%3A38%27%29&os.name=Windows&python.version=2.6
.6
INFO:omero.util.UpgradeCheck:no update needed
2015-02-10 16:40:51,401 INFO  [                 omero.util.UpgradeCheck] (MainTh
read) no update needed
INFO:omero.util.Resources:Starting
2015-02-10 16:40:52,591 INFO  [                    omero.util.Resources] (Thread
-1  ) Starting
DEBUG:omero.util.Resources:Adding object <omero.clients.Entry instance at 0x0000
000005D4B9C8>
2015-02-10 16:40:52,592 DEBUG [                    omero.util.Resources] (MainTh
read) Adding object <omero.clients.Entry instance at 0x0000000005D4B9C8>
DEBUG:omero.util.Resources:Executing
2015-02-10 16:40:52,592 DEBUG [                    omero.util.Resources] (Thread
-1  ) Executing
DEBUG:omero.util.Resources:Checking <omero.clients.Entry instance at 0x000000000
5D4B9C8>
2015-02-10 16:40:52,595 DEBUG [                    omero.util.Resources] (Thread
-1  ) Checking <omero.clients.Entry instance at 0x0000000005D4B9C8>
DEBUG:omero.util.Resources:Sleeping 300
2015-02-10 16:40:52,598 DEBUG [                    omero.util.Resources] (Thread
-1  ) Sleeping 300
Using session 06056e5b-98d8-4bcc-9f6b-8da10d90911c (user-1@trout.openmicroscopy.
org:4064). Idle timeout: 10.0 min. Current group: private-1
WARNING: Readline services not available on this platform.
WARNING: Proper color support under MS Windows requires the pyreadline library.
You can find it at:
http://ipython.scipy.org/moin/PyReadline/Intro
Gary's readline needs the ctypes module, from:
http://starship.python.net/crew/theller/ctypes
(Note that ctypes is already part of Python versions 2.5 and newer).

Defaulting color scheme to 'NoColor'


In [1]: import omero.util.import_candidates

In [2]: omero.util.import_candidates.as_dictionary(['test.jpg'])
DEBUG:omero.util.TempFileManager:Added file C:\Users\spli\AppData\Roaming\omero\
tmp\omero_spli\4048\candidatescnf7feerr
2015-02-10 16:41:05,243 DEBUG [              omero.util.TempFileManager] (MainTh
read) Added file C:\Users\spli\AppData\Roaming\omero\tmp\omero_spli\4048\candida
tescnf7feerr
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\admin.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\basics.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\cecog.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\chgrp.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\db.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\delete.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\download.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\export.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\fs.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\group.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\hql.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\import.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\ldap.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\node.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\obj.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\perf.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\prefs.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\script.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\search.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\server.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\sessions.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\submit.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\tag.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\upload.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\user.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\web.py
Loading c:\jenkins\workspace\OMERO-5.1-latest-deploy-win\OMERO.server\lib\python
\omero\plugins\__init__.py
2015-02-10 16:41:07,422 623        [      main] INFO          ome.formats.import
er.ImportConfig - OMERO Version: 5.1.0-m3-960-3b028da-ice34-b557
2015-02-10 16:41:07,470 671        [      main] INFO          ome.formats.import
er.ImportConfig - Bioformats version: 5.1.0-m3-960-3b028da-ice34-b557 revision:
b14596b date: 10 February 2015
2015-02-10 16:41:07,498 699        [      main] INFO   formats.importer.cli.Comm
andLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2015-02-10 16:41:08,390 1591       [      main] INFO      ome.formats.importer.I
mportCandidates - Depth: 4 Metadata Level: MINIMUM
2015-02-10 16:41:08,672 1873       [      main] INFO      ome.formats.importer.I
mportCandidates - 1 file(s) parsed into 1 group(s) with 1 call(s) to setId in 26
5ms. (281ms total) [0 unknowns]
DEBUG:omero.util.TempFileManager:Removed file C:\Users\spli\AppData\Roaming\omer
o\tmp\omero_spli\4048\candidatescnf7feerr
2015-02-10 16:41:08,716 DEBUG [              omero.util.TempFileManager] (MainTh
read) Removed file C:\Users\spli\AppData\Roaming\omero\tmp\omero_spli\4048\candi
datescnf7feerr
Out[2]: {'H:\\\\test.jpg': ['H:\\\\test.jpg']}

In [3]: quit()
Do you really want to exit ([y]/n)?
Closing client: <omero.clients.BaseClient object at 0x0000000005CC1470>
DEBUG:omero.util.Resources:Cleaning <omero.clients.Entry instance at 0x000000000
5D4B9C8>
2015-02-10 16:41:16,424 DEBUG [                    omero.util.Resources] (MainTh
read) Cleaning <omero.clients.Entry instance at 0x0000000005D4B9C8>
DEBUG:omero.util.Resources:Cleanup done
2015-02-10 16:41:16,427 DEBUG [                    omero.util.Resources] (MainTh
read) Cleanup done
INFO:omero.util.Resources:Halted
2015-02-10 16:41:16,460 INFO  [                    omero.util.Resources] (Thread
-1  ) Halted
DEBUG:omero.util.TempFileManager:Removing tree: C:\Users\spli\AppData\Roaming\om
ero\tmp\omero_spli\4048
2015-02-10 16:41:16,463 DEBUG [              omero.util.TempFileManager] (Dummy-
4   ) Removing tree: C:\Users\spli\AppData\Roaming\omero\tmp\omero_spli\4048

H:\>

comment:5 Changed 4 years ago by jburel

Simon: any update?
Push to 5.1.3 or later

comment:6 Changed 4 years ago by spli

  • Milestone changed from 5.1.2 to 5.1.4

comment:7 Changed 4 years ago by jamoore

  • Milestone changed from 5.1.4 to OMERO-5.1.4

Splitting 5.1.4 due to milestone decoupling

comment:8 Changed 3 years ago by jamoore

  • Milestone changed from 5.x to Unscheduled
Note: See TracTickets for help on using tickets. You may also have a look at Agilo extensions to the ticket.

1.3.13-PRO © 2008-2011 Agilo Software all rights reserved (this page was served in: 0.78971 sec.)

© 2000-2014 University of Dundee & Open Microscopy Environment. Creative Commons Attribution 3.0 Unported License
OME source code is available under the GNU General public license or through commercial license from Glencoe Software

We're Hiring!