Release Process#
The page provides the steps needed to make and publish a release of OpenDDS.
Notes About OpenDDS Releases#
Temporary Notes#
List of current problems with the release process and their workarounds.
If there’s nothing to note then the list should just consist of - N/A
.
Attention
N/A
File Updates#
This guide will document what should be updated manually. Almost all manual updates should be done before the release and are documented in Update Files in the Repo as Needed.
All authoritative references to OpenDDS versions in the source code (like VERSION.txt
or dds/Version.h
) will be automatically updated by the release script, so normally there’s no need to do it manually.
Other files such as NEWS.md
and README.md
shouldn’t be updated manually specifically for a release.
References to ACE/TAO versions are handled using acetao.ini
, which is automatically updated by .github/workflows/update-ace-tao.yml
.
Versioning#
- Micro release (a.k.a. patch release)
Releases with
X.Y.Z
versions. They should only contain fixes. They should avoid additional functionality and breaking changes if possible. Some parts of the release process are different for micro releases. See Micro Releases for more information.- Minor release
Releases with
X.Y.0
versions. They should only contain fixes and additional functionality. They should avoid breaking changes if possible.- Major release
Releases with
X.0.0
versions. They are generally the only time breaking changes can be made.- Version metadata
Text that follows the version in most circumstances. Currently there’s only two uses for this:
The default state of the repo is for the metadata to be
dev
(e.g.4.0.0-dev
). The release script will remove the metadata before a release and restore it afterwards.A preview release/prerelease/release candidate can be indicated by
pre.N
whereN
starts at 1 (e.g.4.0.0-pre.2
). Currently this isn’t supported by the release script, so it would have to be manually created.
- Latest release
The release that has the highest version number. Some parts of the release process are only done for the latest release.
Prior to the Release#
Address Automated Code Analysis Results#
Address any serious compiler warnings seen on the CI
Address any serious diagnostic messages generated by Coverity at https://scan.coverity.com/projects/opendds?tab=overview
Update Files in the Repo as Needed#
These are files or the parts of the files that the release script won’t be able to automatically update or should be checked beforehand to make sure they are correct.
Check Release Notes to make sure the release notes for the next release look correct. If they need changes, then the news fragments should be changed before release.
See also
News documents how news fragments works.
The
AUTHORS
file should also be checked. This file is automatically generated from git history by the release script during the release. It can also be generated ahead of time to get a preview:Fetch the latest
master
branch and runtools/scripts/gitrelease.pl --update-authors
to see what the file will look like after release. This doesn’t require any other arguments like the release script normally does.If the file changed, but something is wrong, then it might be good to correct it before the release. For example a contributor might have used a Git client that is inserting an name/email that might not be what they want in the file or including a name/email might be in addition to their existing name/email combo. Corrections will have to be inserted into
.mailmap
using the format described in the git documentation listed in the file. In addition to that there are general rules in the release script such as ignoring bots, preferringobjectcomputing.com
email addresses toociweb.com
ones, and dealing with GitHub-specific issues.Run
tools/scripts/gitrelease.pl --update-authors
again to make sure the changes worked.
Update
README.md
and Dependencies for any platform or dependency changes.Document changes to building OpenDDS, at least in Building and Installing, but possibly also in
java/README
andjava/INSTALL
.
Update the Modeling SDK version numbers and release notes#
Our convention recently has been to only update these if changes have been made to the Modeling SDK plugins in the current release cycle.
Notes are in tools/modeling/plugins/org.opendds.modeling.help/html/gettingstarted/maintopic.html
(View as HTML).
Version numbers are updated by running tools/modeling/update_version.pl
.
Generate the Modeling SDK Eclipse update site#
Our convention recently has been to only update these if changes have been made to the Modeling SDK plugins in the current release cycle.
Follow all the steps in tools/modeling/features/org.opendds.modeling.site/README.txt
The step dealing with version numbers is already taken care of by the above section “Update the Modeling SDK version numbers and release notes”.
The result of this process is adding the update site contents to the repository for http://www.opendds.org (which will be synced to the live site in the steps below).
Update the opendds.org Website#
The www.opendds.org website is hosted by GitHub as special branch named gh-pages
in the OpenDDS repository.
The website is updated when changes are pushed to that branch.
To do this follow these steps:
Clone the OpenDDS repository and checkout the branch named
website-next-release
to make changes to website source files This branch is used instead ofgh-pages
to hold changes that shouldn’t be public until the release is made. When the release script runs it has a step for mergingwebsite-next-release
intogh-pages
.git clone -b gh-pages git@github.com:OpenDDS/OpenDDS.git website cd website git checkout -B website-next-release
To have Jekyll generate and serve
gh-pages
website locally in order to validate website changes:Run the commands:
bundle install bundle exec jekyll serve
This will generate the website from the Jekyll source files and serve the generated website on localhost port TCP/4000
See also
- The gh-pages README
Detailed instructions
- Testing your GitHub Pages site locally with Jekyll
Official GitHub tutorial
Update the website source files and commit local changes to the
website-next-release
branch.Push local
website-next-release
branch changes to the central repository or your forked repositoryIf using a forked repo, generate a pull request for the
website-next-release
branch to get the changes in to the central OpenDDS/OpenDDS repository.
The release script will merge website-next-release
into gh-pages
on the OpenDDS/OpenDDS
repository during the release process.
Check if GHA Workflows need Updating#
Note
This should only be done for the latest release.
For non-micro releases, if the release workflows haven’t been updated in a while, manually trigger them to make sure they work. If any don’t, then make changes as necessary.
For micro releases check to see if the workflows have been updated on master and backport those changes to the release series branch. Then manually trigger them to make sure they work.
The workflows are:
These will be triggered by the release script, then must be uploaded after release.
Making a Release#
The release script (tools/scripts/gitrelease.pl
) performs or validates the release steps.
All the steps can be listed using the --list-all
option.
The steps that would be ran with the full set arguments can be listed with the --list
options.
By default it will try to run all the steps it can or you can run an arbitrary subset of the steps using the --steps
option.
Some manual steps are required.
It will make modifications to the repository of the current working directory while using a directory of your choosing for intermediate and release files.
Before Running the Release Script#
Release Script Prerequisites:
Commands available:
md5sum
andsha256sum
zip
,unzip
, andzipinfo
Git version 2.5 or later
Your GitHub account meets the following requirements:
It has been added as a member of the OpenDDS organization with the appropriate permissions.
It has permissions to update the release artifacts for the OMG RTPS Interop repo for Upload Artifacts from Release Workflows.
You have uploaded your SSH public key to your GitHub account
You have created a Personal Access Token for your GitHub account
You are a maintainer on the OpenDDS Read the Docs project.
The following Perl CPAN modules are required (Perl core modules should not be listed here):
To install them run:
cpan -T -i Pithub Net::SFTP::Foreign Time::Piece LWP::UserAgent LWP::Protocol::https
Choose a directory for the
WORKSPACE
argument. It doesn’t have to exist but the release script must be able to create it if it doesn’t. It should not contain files created by previous release (mocked or otherwise).You should start a new clone of the OpenDDS repository for just for this release. That clone shouldn’t be inside the directory being passed as
WORKSPACE
.git clone git@github.com:OpenDDS/OpenDDS.git
For micro releases, check out the relevant branch that the release will come from and pass
--branch=BRANCH
along with the--micro
argument.Set tokens the release script needs to interact with web services by exporting them as environment variables. These are
GITHUB_TOKEN
for your GitHub Personal Access Token andREAD_THE_DOCS_TOKEN
for your Read the Docs API token as shown below:export GITHUB_TOKEN=ff00ff00ff00ff00ff00ff00ff00ff00ff00ff00 export READ_THE_DOCS_TOKEN=ff00ff00ff00ff00ff00ff00ff00ff00ff00ff00
Running the Release Script#
The release script is located at tools/scripts/gitrelease.pl
and should be ran from the root of the repo. (See above note in mock releases for the exception)
There are two required arguments, the WORKSPACE
and VERSION
arguments:
WORKSPACE
is the directory where the script will place all intermediate files. If it doesn’t exist the script will try to create it for you. This should be different for different releases of OpenDDS.VERSION
is the version to release.
Run the script with just the required arguments to validate each step of the process.
It will stop at the first error and give you instructions of what to do.
In most cases --remedy
should be used to continue.
When the script wants to commit something, it will show you the git diff
.
Press q
and it will ask you for confirmation that it’s okay to commit it.
The most important options are:
--list
, which lists the steps with their number and description--remedy
, which tells the release script to attempt to resolve issues with the release--steps
, which will specify the steps to run If one of the steps isn’t verifying correctly, but you already manually fixed it, you can skip the step by passing--step ^STEP
whereSTEP
is the step you want to skip. You can also skip whole ranges of the steps. See--help
for the notation it accepts.--micro
, which excludes the steps that probably are not desired when doing a micro release and requires--branch
.
Run perl tools/scripts/gitrelease.pl --help
to see the full help.
Here is an example of what to run for a version 1.0.0 release command assuming that the release script can take care of everything for us:
tools/scripts/gitrelease.pl ../1.0.0-release-workspace 1.0.0 --remedy
Micro Releases#
The release script has a --micro
option which skips steps that probably are not relevant to micro releases.
You must pass the --branch
argument as you should be on the release branch for the minor release.
As of writing these steps skipped are:
Merging
website-next-release
withgh-pages
Some other notes about using --micro
:
The notation of the version argument has no effect on if the script is doing a micro release.
Steps are skipped if they are one of the ones listed above, even if that step number is the only one explicitly passed in.
Otherwise the script should behave the same way.
Here is an example of what to run for a version 1.0.1 release assuming that the release script can take care of everything for us:
git checkout branch-DDS-1.0
tools/scripts/gitrelease.pl ../1.0.1-release-workspace 1.0.1 --micro --branch=branch-DDS-1.0 --remedy
Doing Mock Releases with the Release Script#
It is possible to do a mock release where basically everything is tested, but the script will make sure it’s not making any real changes to the real thing. To set this up, you must do the following:
Fork OpenDDS on GitHub.
To avoid conflicts with regular work on a fork you might already have, it’s recommended to create a new organization for this purpose and create a token for the repository just like for an actual release. Pass the organization name using
--github-user
.This can be skipped if code involving GitHub doesn’t need to be tested and
--skip-github
is passed.
Pass
--mock
. This actually isn’t absolutely necessary, but it is useful as it does some basic checks to make sure the mock release won’t interfere with the actual releases.
It’s possible to use and edit gitrelease.pl
without having to commit changes to it for a mock release if you use two repos.
One repo, lets call it $MOCK_ROOT
, is the one cloned from the mock organization mentioned in the previous instructions and is where the release is going to happen.
The other, $WORKING_ROOT
, is a normal repo where you can edit gitrelease.pl
and other files and push changes to your normal GitHub fork.
Running $WORKING_ROOT/tools/scripts/gitrelease.pl
from $MOCK_ROOT
will work because gitrelease.pl
does everything relative to the current working directory.
This also might be possible with git worktree
instead of fully separate repos but this hasn’t been tested.
After Running the Release Script#
Test the release package#
A simple test of Messenger will do. The git tag is already cloned for you as part of the release process.
Update News on Master#
Note
This should only be done for the latest release that’s also a :ref:`micro release <release-micro-release>.
The news on the master branch has to be updated to account for the micro release. Updating the news consists of:
Copy the micro release entry in
NEWS.md
.Copy the micro release file in
docs/news.d/_releases
.Remove the news fragments in
docs/news.d
for the PRs that were backported.
Upload Artifacts from Release Workflows#
Note
This should only be done for the latest release.
The release script has steps that trigger release workflows on GitHub Actions and print out links to the runs.
After they have finished successfully, run the release script with the version and workspace arguments and the --upload-artifacts
option.
If the workflows are still in progress it will say so and give the links again.
If the workflows were successful, it will download the artifacts, package them, and upload them to GitHub.
Remove Files Used for Release#
Once everything is been finished, the repo and workspace directory used for release can usually be safely deleted. Erring on the side of caution though, they could be kept around for at least a few days after the release to help rerun steps if necessary or inspecting contents of the workspace directory for debugging purposes.