4.4.9 release setup

[jamoore]

General TODOs

• Add a sync job which pushes tags from gh:ome/* to gh:snoopy/* (UNNEEDED)
• Setup proper directory permissions for the hudson user (#11523)

---

Ideal job

The ideal job for performing a release would be runnable via: %build OME-release now version=4.4.8. It would perform the following steps:

• Checkout the latest ome/ome code (Don)
• Update all submodule pointers similar to OMERO-submods-{stable,develop} (Done)
• Push that branch to snoopy (Done: pushes a tag)
• Open a PR (Note: when this commit is pushed back to ome/ome, the PR will be autoclosed) (Not needed)
• Kick off the ice33 and ice34 builds. (Done; as matrix job)
• ...
• Once all other activities are finished, there are 2 options:
  • Either delete the job executions, in which case there's no evidence of the tag that snoopy created; or
  • Sign the snoopy commit and push it to ome/ome and promote the builds. The release was a success!

---

Larger tasks (ticketable)

• Remove need for version PRs in bioformats (Mostly done; not maven - see #11523)
• Move release notes into source (See ​https://github.com/PyTables/PyTables/pull/240/files. From Roger: "Being able to update the release notes with the PR to implement a feature/fix a bug would be very nice, then it's always in a "ready to release" state ahead of the release. " (#11523)

[sbesson]

PRs have been opened against Bio-Formats to get rid of the version number (​https://github.com/openmicroscopy/bioformats/pull/636)

The release process is tested on the develop branch for the Bio-Formats component with the following job: ​http://hudson.openmicroscopy.org.uk/job/BIOFORMATS-trunk-release/.
The current workflow of the job is the following:

• checkout the latest version of the code
• run scc tag-release locally
• build the artifacts (using ant) including the documentation PDF
• copy the artifacts to ​http://downloads.openmicroscopy.org/bio-formats/$RELEASE
• generate the main Bio-Formats landing page pointing at this artefacts and run the linkchecker on it
• copy the landing page to ​http://downloads.openmicroscopy.org/bio-formats/$RELEASE/index.html
• also copy an .htaccess and an .htpasswd files to ​http://downloads.openmicroscopy.org/bio-formats/$RELEASE to hide these pages publicly

Next steps for this task are the following

• move the OMERO landing pages under version code under ​https://github.com/openmicroscopy/openmicroscopy
• set up OMERO-trunk-release, OMERO-trunk-release-ice34 and OMERO-trunk-virtualbox-release jobs
• generate OMERO artifacts and its downloads pages under ​http://downloads.openmicroscopy.org/omero/$RELEASE
• add a top level OME-trunk-release job to trigger OMERO and Bio-Formats releases at once
• reintegrate the logic to check the artifacts MD5 (see ​https://github.com/openmicroscopy/ome-release)
• reuse this release infrastructure on the dev_4_4 line (i.e. job setup, PR rebasing...)

[sbesson]

​http://hudson.openmicroscopy.org.uk/view/Mgmt/job/OME-sync-tags/ job weekly scheduled to sync the tags of ​https://github/openmicroscopy/openmicroscopy.git and ​https://github/openmicroscopy/bioformats.git to their corresponding snoopycrimecop forks.

[sbesson]

Recent progress on this blocker can be checked under the ​Release tab of the CI server:

• ​BIOFORMATS-stable-release checks out the head of dev_4_4, runs scc tag-relase RELEASE, builds the artefacts and copies them under ​http://downloads.openmicroscopy.org/bio-formats/RELEASE with an .htaccess and a .htpasswd file. The Bio-Formats Sphinx landing page (hosted under bioformats.git_ is also generated and copied over to ​http://downloads.openmicroscopy.org/bio-formats/RELEASE/index..html

• ​OMERO-stable-release is a multi-configuration jobs which checks out the head of dev_4_4, runs scc tag-relase RELEASE, builds the OMERO artifacts and copies them under ​http://downloads.openmicroscopy.org/omero/RELEASE with the same .htaccess and a .htpasswd file as above. Three configurations are currently defined for this job corresponding to the supported Ice versions i.e. 3.3, 3.4 and 3.5. The Ice environment is installed from the ​ICE-setup under hudson's $HOME and the desired environment is activated for each matrix job.

• ​OME-stable-release currently triggers these two jobs and passes its RELEASE variable

Current blockers of the 4.4.9 release process are the following:

• Virtualbox artifacts: a new job ​OMERO-stable-release-virtualbox has been copied from ​OMERO-stable-virtualbox to build an ova from any of the above OMERO-stable-release jobs. First run of this job failed. Current caveats are:
  • the ​setup_omero.sh explicitely uses ?xpath=/freeStyleBuild/url to retrieve the server which may not work with the matrix job format above
  • a free style job may not be set as the downstream job of a multi-configuration job
  Alternately, we could reuse/implement the machinery proposed in ​https://github.com/manics/openmicroscopy/commit/65cb4f98ba8314a5f42fb845a4835742056ee823 and build/archive a virtualbox as part of ​OMERO-stable-release for each Ice configuration using the local zip containing the server

• Downloads page generation: these downloads pages were generated using ​https://github.com/openmicroscopy/ome-release repository and the ​OME-release-stable job. Assuming we still want to expose 4.4.9 on these pages,
  • the OMERO page needs to be amended to include the 3.5 server: (Done)
  • the Bio-Formats page needs to be amended to include the Matlab zip bundle (Done)
  • the generation logic needs to be revisited: the prep.sh step can be skipped since all artifacts are copied under a well-defined location, SNAPSHOT_PATH default value needs to be modified (Deleted)
  • the OME-release-stable job should be renamed to be more meaningful and potentially set as a downstream of OMERO-stable-release and BIOFORMATS-stable-release e.g. using the Joint plugin (Deleted; further renaming in #11523)

• Landing pages: the following pages will need to be generated:
  • ​http://downloads.openmicroscopy.org/bio-formats/index.html with links to both stable and develop latest versions (TBD)
  • ​http://downloads.openmicroscopy.org/bio-formats/5.0.0-beta1/index.html i.e. the current ​http://downloads.openmicroscopy.org/bio-formats/index.html (TBD)
  • ​http://downloads.openmicroscopy.org/bio-formats/4.4.9/index.html (Done;4.4.9-RC1.html)
  • ​http://downloads.openmicroscopy.org/omero/index.html with links to both stable and develop latest versions
  • ​http://downloads.openmicroscopy.org/omero/5.0.0-beta1/index.html i.e. the current ​http://downloads.openmicroscopy.org/omero/index.html
  • ​http://downloads.openmicroscopy.org/omero/4.4.9/index.html

Re-assigning this ticket to Josh while away at meeting in Edinburgh.

[jamoore]

Follow-on #11523 opened.