OpenDDS Release Process#

The page provides the steps needed to make and publish a release of OpenDDS.

Notes About OpenDDS Releases#

A micro (aka patch) (X.Y.Z) release is somewhat different from a minor (X.Y.0) release. The steps where this is the case will be marked as such.
All authoritative references to versions in the source code (like VERSION.txt or dds/Version.h) will be automatically updated by the release script, so there’s no need to do it manually. This also applies to NEWS.md where the release script expects the start of the current entry to be exactly the same as when it inserted it and it will change it to the release version. So the only thing that needs to manually edited is the list of changes.

Temporary Notes#

List of current problems with the release process and their workarounds. If there’s nothing to note then the list should just consist of - N/A.

The readthedocs.org version needs to be activated manually after the tag has been pushed. There’s a part of the script to do this automatically, but it’s only partially working.

Prior to the Release#

Address Automated Code Analysis Results#

Address any serious compiler warnings seen on the CI
Address any serious diagnostic messages generated by Coverity at https://scan.coverity.com/projects/opendds?tab=overview

Update Files in the Repo as Needed#

These are files or the parts of the files that the release script won’t be able to automatically update.

Update the NEWS.md to include the changes made to OpenDDS that impact the capabilities of OpenDDS (features added, issues resolved, optimizations made, etc.).
- Use the scripts in tools/scripts/release_notes to prepare a spreadsheet. See tools/scripts/release_notes/README.md for details.
- The team then fills out the spreadsheet.
- Fill in the details for the current release.
The release script will add a new section for next release.
The AUTHORS file should also be checked and updating the news file is a good time to do it. This file is automatically generated from git history by the release script during the release. It can also be generated ahead of time to get a preview:
- Fetch the latest master branch and run ./tools/scripts/gitrelease.pl --update-authors to see what the file will look like after release. This doesn’t require any other arguments like the release script normally does.
- If the file changed, but something is wrong, then it might be good to correct it before the release. For example a contributor might have used a Git client that is inserting an name/email that might not be what they want in the file or including a name/email might be in addition to their existing name/email combo. Corrections will have to be inserted into .mailmap using the format described in the git documentation listed in the file. In addition to that there are general rules in the release script such as ignoring bots, preferring objectcomputing.com email addresses to ociweb.com ones, and dealing with GitHub-specific issues.
- Run ./tools/scripts/gitrelease.pl --update-authors again to make sure the changes worked.
Update README.md and docs/dependencies.md for any platform or dependency changes, such as updates to the ACE/TAO version being used. Specifically, double check that the ACE and TAO versions listed in README.md match the versions in the configure script.
Document changes to building OpenDDS, at least in INSTALL.md, but possibly also in java/README and java/INSTALL.

Update the Modeling SDK version numbers and release notes#

Our convention recently has been to only update these if changes have been made to the Modeling SDK plugins in the current release cycle. Notes are in tools/modeling/plugins/org.opendds.modeling.help/html/gettingstarted/maintopic.html (View as HTML). Version numbers are updated by running tools/modeling/update_version.pl.

Generate the Modeling SDK Eclipse update site#

Our convention recently has been to only update these if changes have been made to the Modeling SDK plugins in the current release cycle.

Follow all the steps in tools/modeling/features/org.opendds.modeling.site/README.txt The step dealing with version numbers is already taken care of by the above section “Update the Modeling SDK version numbers and release notes”. The result of this process is adding the update site contents to the repository for http://www.opendds.org (which will be synced to the live site in the steps below).

Update the opendds.org Website#

The www.opendds.org website is hosted by GitHub as special branch named gh-pages in the OpenDDS repository. The website is updated when changes are pushed to that branch. To do this follow these steps:

Clone the OpenDDS repository and checkout the branch named website-next-release to make changes to website source files This branch is used instead of gh-pages to hold changes that shouldn’t be public until the release is made. When the release script runs it has a step for merging website-next-release into gh-pages.
```
git clone -b gh-pages git@github.com:OpenDDS/OpenDDS.git website
cd website
git checkout -B website-next-release
```
To have Jekyll generate and serve gh-pages website locally in order to validate website changes:
- You will need to have Ruby and bundler installed.
- Run the commands:
```
bundle install
bundle exec jekyll serve
```
- This will generate the website from the Jekyll source files and serve the generated website on localhost port TCP/4000
See also

The gh-pages README
Detailed instructions

Testing your GitHub Pages site locally with Jekyll
Official GitHub tutorial
Update the website source files and commit local changes to the website-next-release branch.
Push local website-next-release branch changes to the central repository or your forked repository
If using a forked repo, generate a pull request for the website-next-release branch to get the changes in to the central OpenDDS/OpenDDS repository.

The release script will merge website-next-release into gh-pages on the OpenDDS/OpenDDS repository during the release process.

Check if Shapes Demo GHA Workflow needs Updating#

Note

This should only be done for the latest release. This is any release that’s not a micro release of the non-latest minor release.

For non-micro releases if the shapes demo workflow hasn’t been updated in a while, manually trigger one to make sure it works. If it doesn’t then make changes as necessary.

For micro releases check to see if the workflow has been updated on master and backport those changes to the release series branch. Then manually trigger it to make sure it works.

Making a Release#

The release script (tools/scripts/gitrelease.pl) performs or validates the release steps. All the steps can be listed using the --list-all option. The steps that would be ran with the full set arguments can be listed with the --list options. By default it will try to run all the steps it can or you can run an arbitrary subset of the steps using the --steps option. Some manual steps are required. It will make modifications to the repository of the current working directory while using a directory of your choosing for intermediate and release files.

Before Running the Release Script#

Release Script Prerequisites:
- Commands available:
  - md5sum and sha256sum
  - zip, unzip, and zipinfo
  - Git version 2.5 or later
- Your GitHub account has been added as a member of the OpenDDS organization with the appropriate permissions.
  - You have uploaded your SSH public key to your GitHub account
  - You have created a Personal Access Token for your GitHub account
- The following Perl CPAN modules are required (Perl core modules should not be listed here):
  To install them run:
```
cpan -i Pithub Net::SFTP::Foreign Time::Piece LWP::UserAgent LWP::Protocol::https
```
Choose a directory for the WORKSPACE argument. It doesn’t have to exist but the release script must be able to create it if it doesn’t. It should not contain files created by previous release (mocked or otherwise).
You should start a new clone of the OpenDDS repository for just for this release. That clone shouldn’t be inside the directory being passed as WORKSPACE.
```
git clone git@github.com:OpenDDS/OpenDDS.git
```
For micro releases, check out the relevant branch that the release will come from and pass --branch=BRANCH along with the --micro argument.
Export a GITHUB_TOKEN Bash shell variable with your GitHub Personal Access Token as shown below:
```
export GITHUB_TOKEN=ff00ff00ff00ff00ff00ff00ff00ff00ff00ff00
```

Running the Release Script#

The release script is located at tools/scripts/gitrelease.pl and should be ran from the root of the repo. (See above note in mock releases for the exception) There are two required arguments, the WORKSPACE and VERSION arguments:

WORKSPACE is the directory where the script will place all intermediate files. If it doesn’t exist the script will try to create it for you. This should be different for different releases of OpenDDS.
VERSION is the version to release.

Run the script with just the required arguments to validate each step of the process. It will stop at the first error and give you instructions of what to do. In most cases --remedy should be used to continue.

When the script wants to commit something, it will show you the git diff. Press q and it will ask you for confirmation that it’s okay to commit it.

The most important options are:

--list, which lists the steps with their number and description
--remedy, which tells the release script to attempt to resolve issues with the release
--steps, which will specify the steps to run If one of the steps isn’t verifying correctly, but you already manually fixed it, you can skip the step by passing --step ^STEP where STEP is the step you want to skip. You can also skip whole ranges of the steps. See --help for the notation it accepts.
--micro, which excludes the steps that probably are not desired when doing a micro release and requires --branch.

Run perl tools/scripts/gitrelease.pl --help to see the full help.

Here is an example of what to run for a version 1.0.0 release command assuming that the release script can take care of everything for us:

perl tools/scripts/gitrelease.pl ../1.0.0-release-workspace 1.0.0 --remedy

Micro Releases#

The release script has a --micro option which skips steps that probably are not relevant to micro releases. You must pass the --branch argument as you should be on the release branch for the minor release. As of writing these steps skipped are:

Merging website-next-release with gh-pages

Some other notes about using --micro:

The notation of the version argument has no effect on if the script is doing a micro release.
Steps are skipped if they are one of the ones listed above, even if that step number is the only one explicitly passed in.

Otherwise the script should behave the same way.

Here is an example of what to run for a version 1.0.1 release assuming that the release script can take care of everything for us:

git checkout branch-DDS-1.0
perl tools/scripts/gitrelease.pl ../1.0.1-release-workspace 1.0.1 --micro --branch=branch-DDS-1.0 --remedy

Doing Mock Releases with the Release Script#

It is possible to do a mock release where basically everything is tested, but the script will make sure it’s not making any real changes to the real thing. To set this up, you must do the following:

Fork OpenDDS on GitHub.
- To avoid conflicts with regular work on a fork you might already have, it’s recommended to create a new organization for this purpose and create a token for the repository just like for an actual release. Pass the organization name using --github-user.
- This can be skipped if code involving GitHub doesn’t need to be tested and --skip-github is passed.
Pass --mock. This actually isn’t absolutely necessary, but it is useful as it does some basic checks to make sure the mock release won’t interfere with the actual releases.

It’s possible to use and edit gitrelease.pl without having to commit changes to it for a mock release if you use two repos. One repo, lets call it $MOCK_ROOT, is the one cloned from the mock organization mentioned in the previous instructions and is where the release is going to happen. The other, $WORKING_ROOT, is a normal repo where you can edit gitrelease.pl and other files and push changes to your normal GitHub fork. Running $WORKING_ROOT/tools/scripts/gitrelease.pl from $MOCK_ROOT will work because gitrelease.pl does everything relative to the current working directory. This also might be possible with git worktree instead of fully separate repos but this hasn’t been tested.

After Running the Release Script#

Test the release package#

A simple test of Messenger will do. The git tag is already cloned for you as part of the release process.

Upload the Shapes Demo Binaries#

Note

This should only be done for the latest release. This is any release that’s not a micro release of the non-latest minor release.

During the release script there’s a step called “Trigger Shapes Demo Build” that triggers a workflow on GitHub to build the shapes demo for the new release. If it was successful it will print out the link to the run so it can be monitored. After it’s done run the release script with the version and workspace arguments and the --upload-shapes-demo option. If the workflow is still in progress it will say so and give the link again. If the workflow is successful it will download the shapes demo binaries, package them, and upload them to GitHub.

Remove Files Used for Release#

Once everything is been finished, the repo and workspace directory used for release can usually be safely deleted. Erring on the side of caution though, they could be kept around for at least a few days after the release to help rerun steps if necessary or inspecting contents of the workspace directory for debugging purposes.