Review Failling Python tests

[wmoore]

Turn off each failing test by adding a X to the name. E.g.

def XtestThumbnial()

Create BUG ticket for each with a link to the X commit.

For info how to run all Python tests locally, look in /docs/hudson/OMERO-py.sh

set -e
set -u
set -x

ulimit -n 8192
ulimit -a

#
# Run tests
#
./build.py clean
./build.py
./build.py -f components/tools/OmeroPy/build.xml -Dtest.with.fail=true test
./build.py -f components/tools/OmeroPy/build.xml -Dtest.with.fail=true integration

cd examples
python ../target/scons/scons.py run_py=1

[wmoore]

First attempt at running $ sh docs/hudson/OMERO-py.sh

...
BUILD SUCCESSFUL
Total time: 10 minutes 44 seconds
+ ./build.py -f components/tools/OmeroPy/build.xml -Dtest.with.fail=true test
Buildfile: /Users/will/Desktop/OMERO/components/tools/OmeroPy/build.xml
Entering /Users/will/Desktop/OMERO/components/tools/OmeroPy...

python-test:
Entering /Users/will/Desktop/OMERO/components/tools/OmeroPy...
Entering /Users/will/Desktop/OMERO/components/tools/OmeroPy...

python-test-suite:
Password:
Sorry, try again.
Password:
Sorry, try again.
Password:
Sorry, try again.
sudo: 3 incorrect password attempts
Running tests...
----------------------------------------------------------------------
  test421Upgrade (t_config.TestConfig) ... OK (0.243s)
  testAsDict (t_config.TestConfig) ... OK (0.005s)
...
  testUsingThePath2 (t_tempfiles.TestTemps) ... OK (0.001s)
  testUsingThePathAndAFile (t_tempfiles.TestTemps) ... OK (0.001s)
----------------------------------------------------------------------
Ran 77 tests in 0.469s
OK
Generating XML reports...
Error on kill process
NO SUDO

BUILD SUCCESSFUL
Total time: 11 seconds
+ ./build.py -f components/tools/OmeroPy/build.xml -Dtest.with.fail=true integration
Buildfile: /Users/will/Desktop/OMERO/components/tools/OmeroPy/build.xml
Entering /Users/will/Desktop/OMERO/components/tools/OmeroPy...

python-integration:
Traceback (most recent call last):
  File "/Users/will/Desktop/OMERO/components/tools/OmeroPy/test/integration/integration_suite.py", line 62, in <module>
    xmlrunner.XMLTestRunner(verbose=True, output='target/reports').run(additional_tests())
  File "/Users/will/Desktop/OMERO/components/tools/OmeroPy/test/integration/integration_suite.py", line 56, in additional_tests
    suite.addTest(load("tablestest.suite._additional_tests"))
  File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/unittest.py", line 553, in loadTestsFromName
    test = obj()
  File "/Users/will/Desktop/OMERO/components/tools/OmeroPy/test/tablestest/suite.py", line 22, in _additional_tests
    suite.addTest(load("tablestest.hdfstorage"))
  File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/unittest.py", line 541, in loadTestsFromName
    parent, obj = obj, getattr(obj, part)
AttributeError: 'module' object has no attribute 'hdfstorage'

BUILD FAILED
/Users/will/Desktop/OMERO/components/tools/python.xml:78: The following error occurred while executing this line:
/Users/will/Desktop/OMERO/components/tools/python.xml:35: exec returned: 1

Total time: 3 seconds
wjm:OMERO will$

[Will Moore <will@…>]

(In [468159a97c7bbede6ac694eb3738282c973b25a1/ome.git] on branch develop) Adding imports to fix failing OmeroPy tests. See #4334

[Will Moore <will@…>]

(In [b4b34710f3c8b35352f9156fa1540f7f26b52224/ome.git] on branch develop) Turing off test testTwoDifferentHosts() in client_ctors.py. See #4334

[Will Moore <will@…>]

(In [016f026961d8fd6b68352fc967c8911b2535711a/ome.git] on branch develop) Turning off testTicket2196SetSecurityContext() in isession.py. See #4334

[Will Moore <will@…>]

(In [28ed4bb0d18774ee5915b306ed0c16374d016a44/ome.git] on branch develop) Turning off testEditScript() testScriptValidation() and testSpeedOfThumbnailFigure(). See #4334

[Will Moore <will@…>]

(In [546384c4a681b06203bd68a14aa26782a55d04ac/ome.git] on branch develop) Turned off test1() and testReplaceOfficialScript(). See #4334

[Will Moore <will@…>]

(In [deeaf18b040fd7f3c2ec4fefa9328c24698874a5/ome.git] on branch develop) Turning off testtestGetScriptWithDetails() testUploadAndScript() and testUserCantUploadOfficalScript(). See #4334

[Will Moore <will@…>]

(In [5c8114a0ecfc48fc8263595568db40d145c270bd/ome.git] on branch develop) Turning off testParseAllOfficialScripts() which fails parsing flim-omero.py
See #4334

[Will Moore <will@…>]

(In [d6e5388fd743306c9f89066577452eac277459dd/ome.git] on branch develop) Turned off test3138() in tickets4000.py. See #4334

[jmoore]

Will, I'm re-opening this now that OMERO-trunk-components is running properly (#4404). You can see the results under ​http://hudson.openmicroscopy.org.uk/job/OMERO-trunk-components/297/testReport/?.

Test Result
43 failures
942 tests

Configuration Name   	                Duration   	All   	Failed   	Skipped   
component=start,label=x86-windows	0.1 sec	        1	0	        0
component=start,label=linux	        0.1 sec	        1	0	        0
component=fs,label=x86-windows		                0	0	        0
component=java,label=linux	        5 min 21 sec	346	0	        0
component=matlab,label=matlab		                0	0	        0
component=py,label=linux	        49 sec	        287	7	        0
component=java,label=x86-windows		        0	0	        0
component=web,label=x86-windows		                0	0	        0
component=py,label=x86-windows	        1 hr 1 min	287	36	        0
component=cpp,label=macosx		                0	0	        0
component=fs,label=linux	        4.6 sec	        20	0	        0
component=web,label=linux		                0	0	        0
component=cpp,label=x86-windows		                0	0	        0
component=cpp,label=linux		                0	0	        0

Failed Tests
component=py,label=linux
Test Name   	Duration   	Age   
clitest.script.TestScript.testFullSession 	0.238	25
clitest.script.TestScript.testReplace 	0.238	25
scriptstest.makeMovieTest.TestMakeMovie.testNoParams 	0.238	25
clitest.script.TestScript.test1 	0.238	148
clitest.script.TestScript.testReplaceOfficial 	0.238	148
integration.scripts.TestScripts.testSpeedOfThumbnailFigure 	0.238	148
tablestest.service.TestTables.test2910 	0.238	246
component=py,label=x86-windows
Test Name   	Duration   	Age   
clitest.script.TestScript.test1 	17.421	8
clitest.script.TestScript.testFullSession 	17.421	8
clitest.script.TestScript.testReplace 	17.421	8
clitest.script.TestScript.testReplaceOfficial 	17.421	8
integration.clientusage.TestClientUsage.testClientClosedAutomatically 	17.421	8
integration.clientusage.TestClientUsage.testClientClosedManually 	17.421	8
integration.clientusage.TestClientUsage.testCreateInsecureClientTicket2099 	17.421	8
integration.clientusage.TestClientUsage.testUseSharedMemory 	17.421	8
integration.figureExportScripts.TestFigureExportScripts.testMovieFigure 	17.421	8
integration.figureExportScripts.TestFigureExportScripts.testRoiFigure 	17.421	8
integration.figureExportScripts.TestFigureExportScripts.testSplitViewFigure 	17.421	8
integration.figureExportScripts.TestFigureExportScripts.testThumbnailFigure 	17.421	8
integration.isession.TestISession.testCreationDestructionClosing 	17.421	8
integration.isession.TestISession.testSimpleDestruction 	17.421	8
integration.scripts.TestScripts.testScriptValidation 	17.421	8
integration.scripts.TestScripts.testUpload2562 	17.421	8
integration.scripts.TestScripts.testEditScript 	17.421	8
integration.scripts.TestScripts.testParamLoadingPerformanceTicket2285 	17.421	8
integration.scripts.TestScripts.testRunScript 	17.421	8
integration.scripts.TestScripts.testSpeedOfThumbnailFigure 	17.421	8
integration.scripts.TestScripts.testUploadOfficialScript 	17.421	8
integration.tickets2000.TestTickets2000.test1018CreationDestructionClosing 	17.421	8
scriptstest.makeMovieTest.TestMakeMovie.testNoParams 	17.421	8
scriptstest.ping.TestPing.testPingParametersViaISCript 	17.421	8
scriptstest.ping.TestPing.testPingStdout 	17.421	8
scriptstest.ping.TestPing.testPingViaISCript 	17.421	8
scriptstest.ping.TestPing.testProcessCallback 	17.421	8
scriptstest.ping.TestPing.testProcessShutdown 	17.421	8
scriptstest.ping.TestPing.testProcessShutdownOneway 	17.421	8
scriptstest.ping.TestPing.testProcessorDetach 	17.421	8
scriptstest.ping.TestPing.testProcessorGetResultsBeforeFinished 	17.421	8
scriptstest.ping.TestPing.testProcessorStop 	17.421	8
scriptstest.rand.TestRand.testRand 	17.421	8
tablestest.hdfstorage.TestHdfStorage.testInitializationOnInitializedFileFails 	17.421	8
tablestest.hdfstorage.TestHdfStorage.testLocking 	17.421	8
tablestest.service.TestTables.test2910 	17.421	8

Page generated: Feb 21, 2011 6:47:44 AMJenkins ver. 1.397

[jmoore <josh@…>]

(In [b916d16af531b7decc8dea6d7e07023e652c6dc6/ome.git] on branch dev_4_2) Turning off testEditScript() testScriptValidation() and testSpeedOfThumbnailFigure(). See #4334

[wmoore]

Josh - looks like the current report contains no failures. ​http://hudson.openmicroscopy.org.uk/job/OMERO-trunk-components/313/testReport/?

OK to close this ticket now? Do I need to make tickets for fixing testEditScript() testScriptValidation() and testSpeedOfThumbnailFigure() ?

[jmoore]

Will, build 313 doesn't have any test failures because tests didn't run. If the 2 "start" jobs fail, then nothing else progresses (I'll look into why that's happening). The last build to have tests was: ​http://hudson.openmicroscopy.org.uk/job/OMERO-trunk-components/309/testReport/?

[jburel]

Moved from sprint 2011-02-24 (6)

[jburel]

Moved from sprint 2011-03-10 (7)

[jburel]

Moved from sprint 2011-03-24 (8)

[wmoore]

Josh - should I be doing anything with this ticket? Looks like current build is failing to run tests ​http://hudson.openmicroscopy.org.uk/job/OMERO-trunk-components/430/

[cxallan]

Moved from sprint 2011-04-07 (9)

[jburel]

Moved from sprint 2011-04-21 (10)

[jburel]

Moved from sprint 2011-05-05 (11)

[jmoore]

Will, I'm not sure. How many tests are currently failing for you? Do we need to mark any more disabled and create individual tickets for them? Or should we just add another "review failing python tickets" in 4.3.1? (Really, it should be a constant thing, but for the moment, ticketing it may be helpful)

[jburel]

Moved from sprint 2011-05-19 (12)

[wmoore]

Having commented out hdf tests...

$ ./build.py -f components/tools/OmeroPy/build.xml -Dtest.with.fail=true integration
...
----------------------------------------------------------------------
Ran 193 tests in 2939.157s
FAILED (failures=5, errors=30)
Generating XML reports...

BUILD SUCCESSFUL
Total time: 49 minutes 12 seconds

[jmoore]

So, on the one hand, "test.with.fail" isn't appropriately failing and on the other, we still have a load of failing tests? Do you have the list of the failures?

[Will Moore <will@…>]

(In [cec267b752bdc788366e0090026b3509b84942ea/ome.git] on branch develop) Fix failing testValidateRoiMovieCall. See #4334

[wmoore]

Created tickets or fixed all the errors / failures in comment 26 above.

[wmoore]

All but one (and hdf table-tests) passing now:

$ cd OmeroPy
$ ulimit -n 8000
$ . source
$ python test/integration/integration_suite.py

...

======================================================================
FAIL [16.238s]: testRomioToPyramid (integration.rawpixelsstore.TestRPS)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/will/Desktop/OMERO/components/tools/OmeroPy/test/integration/rawpixelsstore.py", line 108, in testRomioToPyramid
    self.assert_(success)
AssertionError

----------------------------------------------------------------------
Ran 199 tests in 1879.778s

FAILED (failures=1)

[Will Moore <will@…>]

(In [497cefeae07abfee790f0aaa487bfab371189296/ome.git] on branch develop) Turning on gatewaytest in intergration_suite.py. See #5935, #4334