Release Process

The page provides the steps needed to make and publish a release of OpenDDS.

Notes About OpenDDS Releases

Temporary Notes

List of current problems with the release process and their workarounds. If there’s nothing to note then the list should just consist of - N/A.

Attention

  • N/A

File Updates

This guide will document what should be updated manually. Almost all manual updates should be done before the release and are documented in Update Files in the Repo as Needed.

All authoritative references to OpenDDS versions in the source code (like VERSION.txt or dds/Version.h) will be automatically updated by the release script, so normally there’s no need to do it manually. Other files such as NEWS.md and README.md shouldn’t be updated manually specifically for a release.

References to ACE/TAO versions are handled using acetao.ini, which is automatically updated by .github/workflows/update-ace-tao.yml.

Versioning

Micro release (a.k.a. patch release)

Releases with X.Y.Z versions. They should only contain fixes. They should avoid additional functionality and breaking changes if possible. Some parts of the release process are different for micro releases. See Micro Releases for more information.

Minor release

Releases with X.Y.0 versions. They should only contain fixes and additional functionality. They should avoid breaking changes if possible.

Major release

Releases with X.0.0 versions. They are generally the only time breaking changes can be made.

Version metadata

Text that follows the version in most circumstances. Currently there’s only two uses for this:

  • The default state of the repo is for the metadata to be dev (e.g. 4.0.0-dev). The release script will remove the metadata before a release and restore it afterwards.

  • A preview release/prerelease/release candidate can be indicated by pre.N where N starts at 1 (e.g. 4.0.0-pre.2). Currently this isn’t supported by the release script, so it would have to be manually created.

Latest release

The release that has the highest version number. Some parts of the release process are only done for the latest release.

Prior to the Release

Address Automated Code Analysis Results

Update Files in the Repo as Needed

These are files or the parts of the files that the release script won’t be able to automatically update or should be checked beforehand to make sure they are correct.

  • Check Release Notes to make sure the release notes for the next release look correct. If they need changes, then the news fragments should be changed before release.

    See also

    News documents how news fragments works.

  • The AUTHORS file should also be checked. This file is automatically generated from git history by the release script during the release. It can also be generated ahead of time to get a preview:

    • Fetch the latest master branch and run tools/scripts/gitrelease.pl --update-authors to see what the file will look like after release. This doesn’t require any other arguments like the release script normally does.

    • If the file changed, but something is wrong, then it might be good to correct it before the release. For example a contributor might have used a Git client that is inserting an name/email that might not be what they want in the file or including a name/email might be in addition to their existing name/email combo. Corrections will have to be inserted into .mailmap using the format described in the git documentation listed in the file. In addition to that there are general rules in the release script such as ignoring bots, preferring objectcomputing.com email addresses to ociweb.com ones, and dealing with GitHub-specific issues.

    • Run tools/scripts/gitrelease.pl --update-authors again to make sure the changes worked.

  • Update README.md and Dependencies for any platform or dependency changes.

  • Document changes to building OpenDDS, at least in Building and Installing, but possibly also in java/README and java/INSTALL.

Update the Modeling SDK version numbers and release notes

Our convention recently has been to only update these if changes have been made to the Modeling SDK plugins in the current release cycle. Notes are in tools/modeling/plugins/org.opendds.modeling.help/html/gettingstarted/maintopic.html (View as HTML). Version numbers are updated by running tools/modeling/update_version.pl.

Generate the Modeling SDK Eclipse update site

Our convention recently has been to only update these if changes have been made to the Modeling SDK plugins in the current release cycle.

Follow all the steps in tools/modeling/features/org.opendds.modeling.site/README.txt The step dealing with version numbers is already taken care of by the above section “Update the Modeling SDK version numbers and release notes”. The result of this process is adding the update site contents to the repository for http://www.opendds.org (which will be synced to the live site in the steps below).

Update the opendds.org Website

The www.opendds.org website is hosted by GitHub as special branch named gh-pages in the OpenDDS repository. The website is updated when changes are pushed to that branch. To do this follow these steps:

  • Clone the OpenDDS repository and checkout the branch named website-next-release to make changes to website source files This branch is used instead of gh-pages to hold changes that shouldn’t be public until the release is made. When the release script runs it has a step for merging website-next-release into gh-pages.

    git clone -b gh-pages git@github.com:OpenDDS/OpenDDS.git website
cd website
git checkout -B website-next-release

  • To have Jekyll generate and serve gh-pages website locally in order to validate website changes:

    • You will need to have Ruby and bundler installed.

    • Run the commands:

      bundle install
bundle exec jekyll serve

    • This will generate the website from the Jekyll source files and serve the generated website on localhost port TCP/4000

    See also

    The gh-pages README

    Detailed instructions

    Testing your GitHub Pages site locally with Jekyll

    Official GitHub tutorial

  • Update the website source files and commit local changes to the website-next-release branch.

  • Push local website-next-release branch changes to the central repository or your forked repository

  • If using a forked repo, generate a pull request for the website-next-release branch to get the changes in to the central OpenDDS/OpenDDS repository.

The release script will merge website-next-release into gh-pages on the OpenDDS/OpenDDS repository during the release process.

Check if GHA Workflows need Updating

Note

This should only be done for the latest release.

For non-micro releases, if the release workflows haven’t been updated in a while, manually trigger them to make sure they work. If any don’t, then make changes as necessary.

For micro releases check to see if the workflows have been updated on master and backport those changes to the release series branch. Then manually trigger them to make sure they work.

The workflows are:

These will be triggered by the release script, then must be uploaded after release.

Making a Release

The release script (tools/scripts/gitrelease.pl) performs or validates the release steps. All the steps can be listed using the --list-all option. The steps that would be ran with the full set arguments can be listed with the --list options. By default it will try to run all the steps it can or you can run an arbitrary subset of the steps using the --steps option. Some manual steps are required. It will make modifications to the repository of the current working directory while using a directory of your choosing for intermediate and release files.

Before Running the Release Script

  • Release Script Prerequisites:

  • Choose a directory for the WORKSPACE argument. It doesn’t have to exist but the release script must be able to create it if it doesn’t. It should not contain files created by previous release (mocked or otherwise).

  • You should start a new clone of the OpenDDS repository for just for this release. That clone shouldn’t be inside the directory being passed as WORKSPACE.

    git clone git@github.com:OpenDDS/OpenDDS.git

    For micro releases, check out the relevant branch that the release will come from and pass --branch=BRANCH along with the --micro argument.

  • Set tokens the release script needs to interact with web services by exporting them as environment variables. These are GITHUB_TOKEN for your GitHub Personal Access Token and READ_THE_DOCS_TOKEN for your Read the Docs API token as shown below:

    export GITHUB_TOKEN=ff00ff00ff00ff00ff00ff00ff00ff00ff00ff00
export READ_THE_DOCS_TOKEN=ff00ff00ff00ff00ff00ff00ff00ff00ff00ff00

Releasing from a Container

Performing the release from a container is recommended as it provides a standardized and repeatable environment.

The following script demonstrates how to prepare a container to perform a release:

podman run --rm -ti ubuntu /bin/bash
apt update
apt install zip git g++ make libssl-dev zlib1g-dev rustc cargo python3 python3-venv vim
cpan -T -i Pithub Net::SFTP::Foreign Time::Piece LWP::UserAgent LWP::Protocol::https Net::SSLeay
git config --global user.email "YOUR EMAIL"
git config --global user.name "YOUR NAME"
export GITHUB_TOKEN="YOUR GITHUB TOKEN"
export READ_THE_DOCS_TOKEN="YOUR READ THE DOCS TOKEN"
mkdir ~/.ssh
# Copy in your SSH private key for GitHub
vim ~/.ssh/id_rsa
# Copy in your SSH public key for GitHub
vim ~/.ssh/id_rsa.pub
chmod 400 ~/.ssh/id_rsa
# Create a workspace
mkdir workspace
# Clone OpenDDS
git clone git@github.com:OpenDDS/OpenDDS.git
cd OpenDDS

Then, you can execute the release script or, for a micro release, check out the appropriate branch and then execute the release script.

Running the Release Script

The release script is located at tools/scripts/gitrelease.pl and should be ran from the root of the repo. (See above note in mock releases for the exception) There are two required arguments, the WORKSPACE and VERSION arguments:

  • WORKSPACE is the directory where the script will place all intermediate files. If it doesn’t exist the script will try to create it for you. This should be different for different releases of OpenDDS.

  • VERSION is the version to release.

Run the script with just the required arguments to validate each step of the process. It will stop at the first error and give you instructions of what to do. In most cases --remedy should be used to continue.

When the script wants to commit something, it will show you the git diff. Press q and it will ask you for confirmation that it’s okay to commit it.

The most important options are:

  • --list, which lists the steps with their number and description

  • --remedy, which tells the release script to attempt to resolve issues with the release

  • --steps, which will specify the steps to run If one of the steps isn’t verifying correctly, but you already manually fixed it, you can skip the step by passing --step ^STEP where STEP is the step you want to skip. You can also skip whole ranges of the steps. See --help for the notation it accepts.

  • --micro, which excludes the steps that probably are not desired when doing a micro release and requires --branch.

Run perl tools/scripts/gitrelease.pl --help to see the full help.

Here is an example of what to run for a version 1.0.0 release command assuming that the release script can take care of everything for us:

tools/scripts/gitrelease.pl ../1.0.0-release-workspace 1.0.0 --remedy

Micro Releases

As stated earlier, a micro release only contains fixes. If a bug is found in a release version, the fix should first be applied to the development branch for the corresponding major version and the master branch if they are not the same. To start a micro release, cherry pick one or more fixes from the development branch to the release branch. See the --cherry-pick-prs option of the release script for help with this task. Release branches have the form branch-DDS-X.Y. The cherry-picked commits should include the relevant news files.

The release script has a --micro option which skips steps that probably are not relevant to micro releases. You must pass the --branch argument as you should be on the release branch for the minor release. As of writing these steps skipped are:

  • Merging website-next-release with gh-pages

Some other notes about using --micro:

  • The notation of the version argument has no effect on if the script is doing a micro release.

  • Steps are skipped if they are one of the ones listed above, even if that step number is the only one explicitly passed in.

Otherwise the script should behave the same way.

Here is an example of what to run for a version 1.0.1 release assuming that the release script can take care of everything for us:

git checkout branch-DDS-1.0
tools/scripts/gitrelease.pl ../1.0.1-release-workspace 1.0.1 --micro --branch=branch-DDS-1.0 --remedy

Doing Mock Releases with the Release Script

It is possible to do a mock release where basically everything is tested, but the script will make sure it’s not making any real changes to the real thing. To set this up, you must do the following:

  • Fork OpenDDS on GitHub.

    • To avoid conflicts with regular work on a fork you might already have, it’s recommended to create a new organization for this purpose and create a token for the repository just like for an actual release. Pass the organization name using --github-user.

    • This can be skipped if code involving GitHub doesn’t need to be tested and --skip-github is passed.

  • Pass --mock. This actually isn’t absolutely necessary, but it is useful as it does some basic checks to make sure the mock release won’t interfere with the actual releases.

It’s possible to use and edit gitrelease.pl without having to commit changes to it for a mock release if you use two repos. One repo, lets call it $MOCK_ROOT, is the one cloned from the mock organization mentioned in the previous instructions and is where the release is going to happen. The other, $WORKING_ROOT, is a normal repo where you can edit gitrelease.pl and other files and push changes to your normal GitHub fork. Running $WORKING_ROOT/tools/scripts/gitrelease.pl from $MOCK_ROOT will work because gitrelease.pl does everything relative to the current working directory. This also might be possible with git worktree instead of fully separate repos but this hasn’t been tested.

After Running the Release Script

Test the release package

A simple test of Messenger will do. The git tag is already cloned for you as part of the release process.

Update News on Master

Note

This should only be done for the latest release that’s also a micro release.

The news on the master branch has to be updated to account for the micro release. Updating the news consists of:

Upload Artifacts from Release Workflows

Note

This should only be done for the latest release.

The release script has steps that trigger release workflows on GitHub Actions and print out links to the runs. After they have finished successfully, run the release script with the version and workspace arguments and the --upload-artifacts option. If the workflows are still in progress it will say so and give the links again. If the workflows were successful, it will download the artifacts, package them, and upload them to GitHub.

Remove Files Used for Release

Once everything is been finished, the repo and workspace directory used for release can usually be safely deleted. Erring on the side of caution though, they could be kept around for at least a few days after the release to help rerun steps if necessary or inspecting contents of the workspace directory for debugging purposes.