Arrange good user environment coverage across CI jobs

[mtbcarroll]

Users may build and run OMERO components under various circumstances: compilers (Oracle/OpenJDK 6/7, gcc, Visual Studio, etc.), libraries (e.g., Ice), operating systems, web browsers, etc. With reference to our own installation guide and what is actually observed in our community (perhaps we need to expand data gathering?), we should generate a list of what we should be including in routine testing.

Indeed, especially with hopefully getting more PowerEdge C6100 CI nodes, expanding automated testing coverage, @sbesson's recent work, etc., it is probably approaching time to generate a matrix of Jenkins jobs against operating systems, software, versions, to ensure that CI nodes cover a wide range of environments that include the likelier.

(Also, using the current LSC infrastructure, the VMs' slowness might be useful in catching bugs that only present when CPU or IO are slow: that's another environmental variable.)

Given the combinatorial aspects of this problem, some jobs could run weekly instead of daily or more, so long as the chosen frequent jobs are highly likely to catch regressions promptly.

To the extent that CI node setup can become a scripted checklist, that lowers longer-term sysadmin burden and eases disaster recovery: it doesn't much matter if nodes die if we can easily recreate them.

[czmacleod]

The installation and maintenance burden here can be greatly mitigated through the use of some configuration management to setup the CI nodes. Puppet / Ansible or the like.

[khgillen]

Replying to czmacleod:

The installation and maintenance burden here can be greatly mitigated through the use of some configuration management to setup the CI nodes. Puppet / Ansible or the like.

At the moment the CI nodes are indeed under configuration management, the base OS is installed with Kickstart from a spacewalk server ready for OME staff to manage. We can create kickstart profiles specifically for the CI nodes once we have a solid set of standard requirements - this is at least one viable option. If we are deviating away entirely from LSC supported platforms then other options like Puppet can be explored.

[jburel]

Passing it to seb

[mtbcarroll]

This may be superseded by ​https://trello.com/c/1caNiqss/206-prerequisite-versions-tested.

[spli]

@mtbcarroll If you're happy with the Trello card then close this ticket?

[mtbcarroll]

Happy for Seb to do so if he thinks it's covered. Partly depends on the aspects regarding being aware of what's actually extant in our user community and if we need to adjust stats gathering accordingly.

[sbesson]

Moving this to Trello completely since this is internal.