Investigate bfconvert performance on Windows vs. Linux

[mlinkert]

From Rubén Muñoz:

Bioformats is much slower on Windows than on Linux. Could that be? I am having this for long time.
 
Same (local) files, same loci_tools.jar and loci.formats.tools.ImageConverter give 4x slower rates on windows.
Does it have to do with java interpreter. Does it have to do with the filesystem?
 
-Xmx512m was the only argumet used (no compression).

[mlinkert]

Given the age of this ticket, it's probably worth verifying the extent to which this is still a problem. The original use case was converting ScanR plates to OME-TIFF, so a first step here would be concrete numbers for running bfconvert on Windows and Linux with the ScanR plate in test_images_good - probably local disk only to start with, and best/worst out of 3-5 runs to see the effects of OS/FS-level caching.

[mlinkert]

With the v5.6.0-m2 tag and 5 runs of bfconvert test_images_good/scanr/Maik_001/experiment_descriptor.xml plate.ome.tiff :

Linux, local disk:

72.198s elapsed (137.48611+330.22223ms per plane, 3161ms overhead)
38.78s elapsed (55.131943+190.13889ms per plane, 2535ms overhead)
41.291s elapsed (59.65278+199.27083ms per plane, 3098ms overhead)
41.69s elapsed (59.9375+203.98611ms per plane, 2733ms overhead)
39.863s elapsed (56.208332+188.95139ms per plane, 3340ms overhead)

Linux, network disk:

62.417s elapsed (168.92361+236.08333ms per plane, 3317ms overhead)
60.811s elapsed (156.84027+236.14583ms per plane, 3239ms overhead)
68.398s elapsed (163.76389+277.25696ms per plane, 4188ms overhead)
68.976s elapsed (180.16667+255.86806ms per plane, 5076ms overhead)
84.561s elapsed (209.77777+331.38196ms per plane, 5016ms overhead)

Windows, local disk:

105.801s elapsed (66.59028+607.7778ms per plane, 7485ms overhead)
95.472s elapsed (63.86111+571.7986ms per plane, 2839ms overhead)
91.754s elapsed (62.118057+546.1458ms per plane, 2832ms overhead)
93.178s elapsed (63.40972+555.74304ms per plane, 2834ms overhead)
94.7s elapsed (64.25694+564.3542ms, 2972ms overhead)

Windows, network disk:

103.376s elapsed (145.66667+528.2778ms per plane, 5196ms overhead)
100.821s elapsed (144.0625+525.25ms per plane, 3350ms overhead)
107.713s elapsed (147.05556+559.43054ms per plane, 4673ms overhead)
108.523s elapsed (149.92361+569.2361ms per plane, 3784ms overhead)
108.114s elapsed (148.46527+563.0208ms per plane, 4588ms overhead)

This suggests that there is still a noticeable difference between Windows and Linux, with the plane write time alone being the real difference. I don't have a good idea at the moment of why that is the case, but hopefully it narrows the focus for further investigation.

[mlinkert]

Proposed IO-level fix: ​https://github.com/ome/ome-common-java/pull/15

Closing in favor of open PR and follow-up card: ​https://trello.com/c/OimiHAQY/175-profile-tiff-writing-on-windows