GitHub Actions Summary and FAQ#
GitHub Actions is the continuous integration solution currently being used to evaluate the readiness of pull requests. It builds OpenDDS and runs the test suite across a wide variety of operation systems and build configurations.
Legend for GitHub Actions Build Names#
u20/u22 - Ubuntu 20.04/22.04
w19/w22 - Windows Server 2019/Windows Server 2022
m11/m12 - macOS 11/12
GitHub Runner Images <https://github.com/actions/runner-images>
x86 - Windows 32 bit. If not specified, x64 is implied.
re - Release build. If not specified, Debug is implied.
clangX/gccY - compiler used to build OpenDDS. If not specified, the default system compiler is used. Windows Server 2019 uses Visual Studio 2019 Windows Server 2022 uses Visual Studio 2022
stat - Static build
bsafe/esafe - Base Safety/Extended Safety build
sec - Security build
asan/tsan - Address/Thread Sanitizer build
o1 - enables
d0 - enables
i0 - enables
p1 - enables
w1 - enables wide characters
v1 - enables versioned namespace
j/j<N> - Java version default/N
ace7 - uses ace7tao3 rather than ace6tao2
xer0 - disables xerces
qt - enables
ws - enables
js0 - enables
a1 - enables TAO’s Anys using
This is a mask in an attempt to keep names shorter.
Triggering the Build And Test Workflow#
There are a couple ways in which a run of build and test workflow can be started.
Any pull request targeting master will automatically run the OpenDDS workflows. This form of workflow run will simulate a merge between the branch and master.
Push events on branches prefixed
gh_wf_ will trigger workflow runs on the fork in which the branch resides.
These fork runs of GitHub Actions can be viewed in the “Actions” tab.
Runs of the workflow on forks will not simulate a merge between the branch and master.
There are a number of job types that are contained in the file build_and_test.yml. Where possible, a configuration will contain 3 jobs. The first job that is run is ACE_TAO_. This will create an artifact which is used later by the OpenDDS build. The second job is build_, which uses the previous ACE_TAO_ job to configure and build OpenDDS. This job will then export an artifact to be used in the third step. The third step is the test_ job, which runs the appropriate tests for the associated OpenDDS configuration.
Certain builds do not follow this 3 step model. Static and Release builds have a large footprint and therefore cannot fit the entire test suite onto a GitHub Actions runner. As a result, they only build and run a subset of the tests in their final jobs, but then have multiple final jobs to increase test coverage. These jobs are prefixed by:
compiler_ (and for some build configurations, compiler2_) which runs the
unit_ which runs the unit tests located in
To shorten the runtime of the continuous integration, some other builds will not run the test suite.
All builds with safety profile disabled and ownership profile enabled, will run the
Test runs which only contain CMake tests are prefixed by
.lst files contain a list of tests with configuration options that will turn tests on or off.
The test_ jobs pass in
MacOS, Windows 22, Static, and Release builds instead use
The Thread Sanatizer build uses
This separation of .lst files is due to how excluding all but a few tests in the
dcps_tests.lst would require adding a new config option to every test we didn’t want to run.
There is a separate security test list,
tests/security/security_tests.lst, which governs the security tests which are run when
--security is passed to
The last list file used by
tools/modeling/tests/modeling_tests.lst, which is included by passing
To disable a test in GitHub Actions,
!GH_ACTIONS must be added next to the test in the .lst file.
There are similar test blockers which only block for specific GitHub Actions configurations from running marked tests:
!GH_ACTIONS_OPENDDS_SAFETY_PROFILEblocks Safety Profile builds
!GH_ACTIONS_M10blocks the MacOS10 runners
This option currently does nothing because GitHub sees MacOS runners as unresponsive when they attempt to run some of the more intensive tests in dcps_tests.lst.
!GH_ACTIONS_ASANblocks the Address Sanitizer builds
!GH_ACTIONS_W22blocks the Windows Server 2022 runner
These blocks are necessary because certain tests cannot properly run on GitHub Actions due to how the runners are configured.
-Config GH_ACTIONS is assumed by
auto_run_tests.pl when running on GitHub Actions, but the other test configurations must be passed using
- Running Tests
auto_run_tests.pland the lst files work in general.
.github/workflows/lint.yml workflow runs
.github/workflows/lint_build_and_test.pl, which checks that the
.github/workflows/build_and_test.yml workflow has gcc-problem-matcher and msvc-problem-matcher in the correct places.
Running this script requires the YAML CPAN module. As a safety measure, it has some picky rules about how steps are named and ordered. In simplified terms, these rules include:
If used, the problem matcher must be appropriate for the platform the job is running on.
The problem matcher must not be declared before steps that are named “setup gtest” or named like “build ACE/TAO”. This should reduce any warnings from Google Test or ACE/TAO.
A problem matcher should be declared before steps that start with “build” or contain “make”. These steps should also contain
Certain tests are blocked from GitHub actions because their failures are either unfixable, or are not represented on the scoreboard. If this is the case, we have to assume that the failure is due to some sort of limitation caused by the GitHub Actions runners.
Only Failing on CI#
Multicast times out waiting for remote peer. Fails on
tests/DCPS/Thrasher/run_test.pl high/aggressive/medium XXXX XXXX
The more intense thrasher tests cause consistent failures due to the increased load from ASAN. GitHub Actions fails these tests very consistently compared to the scoreboard which is more intermittent. Fails on
Failing Both CI and scoreboard#
These tests fail on the CI as well as the scoreboard, but will remain blocked on the CI until fixed. Each test has a list of the builds it was failing on before being blocked.
The tests are run using autobuild which creates a number of output files that are turned into a GitHub artifact.
This artifact is processed by the “check results” step which uses the script
tools/scripts/autobuild_brief_html_to_text.pl to catch failures and print them in an organized manner.
Due to this being a part of the “test” jobs, the results of each run will appear as soon as the job is finished.
Artifacts from the continuous integration run can be downloaded by clicking details on one of the Build & Test runs. Once all jobs are completed, a dropdown will appear on the bar next to “Re-run jobs”, called “Artifacts” which lists each artifact that can be downloaded.
Alternatively, clicking the “Summary” button at the top of the list of jobs will list all the available artifacts at the bottom of the page.
Using Artifacts to Replicate Builds#
You can download the
build_ artifacts then use them for a local build, so long as your operating system is the same as the one on the runner.
git clonethe ACE_TAO branch which is targeted by the build. This is usually going to be
git clone --recursivethe OpenDDS branch on which the CI was run.
Merge OpenDDS master into your cloned branch.
tar xvfJfrom inside the cloned ACE_TAO, targeting the
tar xvfJfrom inside the cloned OpenDDS, targeting the
Adjust the setenv.sh located inside OpenDDS to match the new locations for your ACE_TAO, and OpenDDS. The word “runner” should not appear within the setenv.sh once you are finished.
You should now have a working duplicate of the build that was run on GitHub Actions. This can be used for debugging as a way to quickly set up a problematic build.
Using Artifacts to View More Test Information#
Tests failures which are recorded on GitHub only contain a brief capture of output surrounding a failure.
This is useful for some tests, but it can often be helpful to view more of a test run.
This can be done by downloading the artifact for a test step you are viewing.
This test step artifact contains a number of files including
This is the full log of all output from all test runs done for the corresponding job.
It should be opened in either a text editor or Firefox, as Chrome will have issues due to the length of the file.
The OpenDDS workflows create .tar.xz archives of certain build artifacts which can then be up uploaded and shared between jobs (and the user) as part of GitHub Actions’ “artifact” API. A cache key comparison made using the relevant git commit SHA will determine whether to rebuild the artifact, or to use the cached artifact.