Documentation Guidelines#

This Sphinx-based documentation is hosted on Read the Docs and can be located here. It can also be built locally. To do this follow the steps in the following section.

Building#

Run docs/build.py, passing the kinds of documentation desired. Multiple kinds can be passed, and they are documented in the following sections.

Requirements#

The script requires Python 3.10 or later and an internet connection if the script needs to download dependencies or check the validity of external links. It will also try to download extra information for things like omgspec, but it shouldn’t be a fatal error if there’s not an internet connection.

You might receive a message like this when running for the first time:

build.py: Creating venv...
The virtual environment was not created successfully because ensurepip is not
available.  On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.

    apt install python3.9-venv

If you do, then follow the directions it gives, remove the docs/.venv directory, and try again.

HTML#

HTML documentation can be built and viewed using docs/build.py html -o. If it was built successfully, then the front page will be at docs/_build/html/index.html.

A single page variant is also available using docs/build.py singlehtml -o If it was built successfully, then the page will be at docs/_build/singlehtml/index.html.

Dash#

Documentation can be built for Dash, Zeal, and other Dash-compatible applications using doc2dash. The command for this is docs/build.py dash. This will create a docs/_build/OpenDDS.docset directory that must be manually moved to where other docsets are stored.

PDF#

Note

The PDF output is currently much less optimized than the HTML-based outputs.

Note

The Sphinx PDF builder has additional dependencies on LaTeX that are documented here.

PDF documentation can be built and viewed using docs/build.py pdf -o. If it was built successfully, then the PDF file will be at docs/_build/latex/opendds.pdf.

Strict Checks#

docs/build.py strict will promote Sphinx warnings to errors and check to see if links resolve to a valid web page.

Note

The documentation includes dynamic links to files in the GitHub repo created by ghfile. These links will be invalid until the git commit they were built under is pushed to a Github fork of OpenDDS. This also means running will cause those links to marked as broken. A workaround for this is to pass -c master or another commit, branch, or tag that is desired.

Building Manually#

It is recommended to use build.py to build the documentation as it will handle dependencies automatically. If necessary though, Sphinx can be ran directly.

To build the documentation the dependencies need to be installed first. Run this from the docs directory to do this:

pip3 install -r requirements.txt

Then sphinx-build can be ran. For example to build the HTML documentation:

sphinx-build -M html . _build

RST/Sphinx Usage#

  • See Sphinx reStructuredText Primer for basic RST usage.

  • Inline code such as class names like DataReader and other symbolic text such as commands like ls should use double backticks: ``TEXT``. This distinguishes it as code, makes it easier to distinguish characters, and reduces the chance of needing to escape characters if they happen to be special for RST.

  • One sentence per line should be perfered. This makes it easier to see what changed in a git diff or GitHub PR and easier to move sentences around in editors like Vim. It also avoids inconsistencies involving what the maximum line length is. This might make it more annoying to read the documentation raw, but that’s not the intended way to do so anyway.

Custom Domains#

Sphinx domains are a way to document collections of hierarchical definitions such as APIs. Sphinx has a number of built-in domains such as Python and C++, but it helps to have custom ones. Custom domains in OpenDDS should use classes derived from the ones in docs/sphinx_extensions/custom_domain.py.

All of the custom domain directives can and should have RST content nested in them. They all support the :no-index:, :no-index-entry:, and :no-contents-entry: directive options. See Basic Markup for more information.

CMake Domain#

For Using OpenDDS in a CMake Project there’s a custom CMake Sphinx domain in docs/sphinx_extensions/cmake_domain.py. There is an official CMake domain used by CMake for their own documentation, but it would be impractical for us to use because it requires a separate RST file for every property and variable.

.. cmake:func:: <name>#

Use to document public CMake functions.

.. cmake:func:: opendds_cmake_function

  ::

    opendds_cmake_function(<target> [ARG <value>])

  This is a function.

  .. cmake:func:arg:: target

    This is a positional argument

  .. cmake:func:arg:: ARG <value>

    This is a keyword argument

Turns into:

opendds_cmake_function#
opendds_cmake_function(<target> [ARG <value>])

This is a function.

target#

This is a positional argument

ARG <value>#

This is a keyword argument

.. cmake:func:arg:: <name> [<subargument>...]#

Use to document arguments and options for public CMake functions. Should be nested in cmake:func of the function the argument belongs to.

:cmake:func:#

Use to reference a cmake:func by name or reference a cmake:func:arg by function name followed by argument name in parentheses. For example:

:cmake:func:`opendds_cmake_function`

:cmake:func:`opendds_cmake_function(target)`

:cmake:func:`opendds_cmake_function(ARG)`

Turns into:

.. cmake:var:: <name>#

Use to document public variables.

.. cmake:var:: OPENDDS_CMAKE_VARIABLE

  This is a variable

Turns into:

OPENDDS_CMAKE_VARIABLE#

This is a variable

:cmake:var:#

Use to reference a cmake:var by name.

:cmake:var:`OPENDDS_CMAKE_VARIABLE`

Turns into:

.. cmake:prop:: <name>#

Use to document properties on CMake targets, or possibly other kinds of properties, that we’re looking for or creating for the user.

.. cmake:prop:: OPENDDS_CMAKE_PROPERTY

  This is a property

Turns into:

OPENDDS_CMAKE_PROPERTY#

This is a property

:cmake:prop:#

Use to reference a cmake:prop by name.

:cmake:prop:`OPENDDS_CMAKE_PROPERTY`

Turns into:

.. cmake:tgt:: <name>#

Use to document a library or executable CMake target meant to users that can be imported or exported.

.. cmake:tgt:: OpenDDS::MessengerPigeonTransport

  Transport for IP over messenger pigeon (:rfc:`1149`)

Turns into:

OpenDDS::MessengerPigeonTransport#

Transport for IP over messenger pigeon (RFC 1149)

:cmake:tgt:#

Use to reference a cmake:tgt by name.

:cmake:tgt:`OpenDDS::MessengerPigeonTransport`

Turns into:

Config Domain#

For Run-time Configuration there’s a custom configuration Sphinx domain in docs/sphinx_extensions/config_domain.py.

.. cfg:sec:: <name>[@<discriminator>][/<argument>]#

Use to document a configuration section that can contain cfg:prop and most other RST content. <discriminator> is an optional extension of the name to document cases where the available properties depend on something. Using discriminators requires separate cfg:sec entires. <arguments> is just for display and has no restrictions on the contents.

:cfg:sec:#

Use to reference a cfg:sec by name and optional discriminator. If the section has a discriminator, it must be separated by a @ symbol. Do not include arguments if it has arguments in the directive. The possible formats are <sect_name> and <sect_name>@<disc_name>.

.. cfg:prop:: <name>=<values>#

Use to document a configuration property that can contain cfg:val and most other RST content. Must be in a cfg:sec. <values> describe what sort of text is accepted. It is just for display and has no restrictions on the contents, but should follow the following conventions to describe the accepted values:

  • | indicates an OR

  • [] indicates an optional part of the value

  • ... indicates the previous part can be repeated

  • Words surrounded angle brackets (ex: <prop_name>) indicate placeholders.

  • Everything else should be considered literal.

For example: log_level=none|error|warn|debug, memory_limit=<uint64>, addresses=<ip>[:<port>],....

:required:#

Indicates the property is required for the section

:default:#

The default value of the property if omitted

:cfg:prop:#

Use to reference a cfg:prop by name. Do not include values if it has values in the directive. The possible formats are:

  • <prop_name>

    Inside of a cfg:sec, it refers to a property in that section. Outside of a cfg:sec, the property is assumed to be common.

  • [<sect_name>]<prop_name>

  • [<sect_name>@<disc_name>]<prop_name>

.. cfg:val:: [<]<name>[>]#

Use to document a part of what a configuration property accepts. Must be in a cfg:prop. The optional angle brackets (<>) are just for display and are meant to help distinguish between the value being a literal and a placeholder.

:cfg:val:#

Use to reference a cfg:val by name. Do not include brackets if it has brackets in the directive. The possible formats are:

  • <val_name>

    This must be inside a cfg:prop.

  • <prop_name>=<val_name>

    Inside of a cfg:sec, it refers to a value of a property in that section. Outside of a cfg:sec, the property is assumed to be common.

  • [<sect_name>]<prop_name>=<val_name>

  • [<sect_name>@<disc_name>]<prop_name>=<val_name>

Example#

This is a example made up for the following INI file:

[server/Alpha]
os=windows

[server/Beta]
os=linux
distro=Debian

Outside their sections, references to properties and values must be complete: :cfg:val:`[server]os=linux`, :cfg:prop:`[server@linux]distro`

Otherwise the ``common`` section will be assumed.

.. cfg:sec:: server/<name>

  A property or value's section can be omitted from references within their sections: :cfg:prop:`os`, :cfg:val:`os=windows`

  .. cfg:prop:: os=windows|linux
    :required:

    A value's property can be omitted from references within their properties: :cfg:val:`linux`

    .. cfg:val:: windows

      Implied titles will be shortened within their scopes: :cfg:prop:`[server]os`, :cfg:val:`[server]os=windows`

    .. cfg:val:: linux

      Sections with discriminators require them in the reference targets: :cfg:sec:`server@linux`, :cfg:prop:`[server@linux]distro`

.. cfg:sec:: server@linux/<name>

  .. cfg:prop:: distro=<name>
    :default: ``Ubuntu``

Turns into:

Outside their sections, references to properties and values must be complete: [server] os=linux, [server] distro (linux)

Otherwise the common section will be assumed.

[server/<name>]#

A property or value’s section can be omitted from references within their sections: os, os=windows

os=windows|linux#

Config store key: SERVER_<name>_OS
Required

A value’s property can be omitted from references within their properties: linux

windows#

Implied titles will be shortened within their scopes: os, windows

linux#

Sections with discriminators require them in the reference targets: [server] (linux), [server] distro (linux)

[server/<name>] (linux)#
distro=<name>#

Config store key: SERVER_<name>_DISTRO
Default: Ubuntu

News#

The news for a release (aka release notes) is created from separate reStructuredText files called fragments. This makes it possible to change the news without everyone editing one file. Managing the news fragments is handled as part of the Sphinx documentation to make it easy to link to other relevant documents and files in the source code. It also makes it possible to easily preview the news before a release.

News Fragments#

To add content to the news for a release, a fragment file with the content must be created in docs/news.d. It can be named anything as long as it’s reasonably unique, doesn’t start with an underscore, and has an .rst file extension. The branch name of the PR for this change might be a good name, for example: fix_rtps_segfault.rst.

The following is a simple news fragment example:

docs/news.d/_example.rst#
# This is a news fragment example. Copy this file in this directory and change
# it to have content added to the release notes for the next release.
# See https://opendds.readthedocs.io/en/master/internal/docs.html#news for details

# This will have to be replaced with the PR number after the PR is created:
.. news-prs: 0

.. news-start-section: Additions
- This is an addition we made.

  - This is detailed information about the addition.
    This is some more.

.. news-end-section

Fragments contain RST content for the news along with some RST directive-like metadata. Lines starting with # are ignored as comments. Content for the news must be formatted as a list. It will usually be one list item, but can be multiple items if they are separate changes from a user perspective.

Note

If using nested lists, reStructuredText requires these to have empty lines before and after as shown in the example above.

Content must be contained somewhere within a section, indicated by news-start-section and news-end-section. Having text outside sections that isn’t a comment or a special news directive is an error.

Content should provide a reasonable amount of context to users so they can figure out if and how a change will impact or benefit them. Some of this can be achieved through subsections. Content should link to the rest of the Sphinx documentation and to source files using :ghfile: if relevant.

PRs will usually have just have one fragment file, but can have more then one. For example, say there is a PR that is making changes associated with an existing fragment file, but also has changes that are separate from the PRs in that fragment. This might indicate a new PR should be made for those changes, but if that’s impractical, a new separate fragment file for those changes can be made.

Directives#

news-prs#

news-prs should declare all the PRs that make up the changes to OpenDDS described in the fragment. It’s an error to omit news-prs from a fragment or to have more than one news-prs directive. Do not add # at the start of the PR numbers. Please add follow-on PRs separated by spaces as they are created.

.. news-prs: 1002 1003 1008

If the changes don’t have a PR associated with them, such as a policy change, then put none instead:

.. news-prs: none
news-start-section and news-end-section#

news-start-section and news-end-section directives contain the news content and define how they’re grouped. All sections with the same name across all the fragments are merged together in the final result. The top-level sections must be one of the following:

Additions

New user-relevant features

Platform Support and Dependencies

Additional support or changes to support for OS, C++ standards, compilers, and dependencies.

Deprecations

User-relevant features that are being planned on being removed, but removing now would break a significant number of user’s use cases. This marks them for removal in the next major version.

Removals

User-relevant features removed that were either deprecated, “experimental”, or “internal” that users might be relying on anyways.

Security

Fixes for issues that might compromise the security of a system using OpenDDS. This doesn’t need to be related to DDS Security.

Fixes

User-relevant fixes not related to security or documentation.

Documentation

Significant changes to documentation, either in the primary Sphinx documentation or in secondary documentation. This can be documenting an existing feature or major changes to existing documentation. Documentation for a new feature should be linked in the news item for it, not here. Minor corrections like spelling and changes to internal documentation probably don’t need to be mentioned in the news.

Notes

Changes that might be relevant to users, but might not fit in any of the other sections. This could be a change in policy or change outside the OpenDDS source code that doesn’t have a PR associated with it.

Sections can be nested as subsections. There is no requirement for subsections or restrictions on their names, but it’s suggested to group otherwise separate changes if they are all impact the same part of OpenDDS. For example:

.. news-start-section: Fixes
- Non-RTPS Fix
.. news-start-section: RTPS
- RTPS Fix
.. news-start-section: RTPS Relay
- RTPS Relay Fix
- Another RTPS Relay Fix
.. news-end-section
- Another RTPS Fix
.. news-end-section
- Another Non-RTPS Fix
.. news-end-section

Will result in:

This will allow other fragments to add to “RTPS” or “RTPS Relay”.

news-rank#

Rank is an integer used to help sort content and sections. Setting the rank higher using news-rank can headline a change within a section. This is optional and if omitted or if rank is the same, then items are ranked by the oldest PR number.

Once used, news-rank applies to both subsections and content directly in the section between the news-start-section and the news-end-section. Content in subsections will need a separate news-rank to change the order, as sorting of subsections is separate. When sections have different ranks across fragments (or even in the same fragment), then the largest of ranks is used.

For an example of all that, let us say we had the following fragments:

These are the older changes#
.. news-prs: 1111
.. news-start-section: Additions
# Rank 0 because it's a new section
- Improved logging in some cases.
# Lets give XTypes as a section some priority
.. news-rank: 20
.. news-start-section: XTypes
# Rank 0 because it's a new section
- Added ability to randomize ``DynamicData``
.. news-rank: 10
- Added lossless casting from any type to any other type in ``DynamicData``.
.. news-end-section
# This gets the same rank 20 as the XTypes section
- Made logging worse in some cases.
.. news-end-section
These are the newer changes#
.. news-prs: 2222
.. news-start-section: Additions
# Rank 0, the rank 0 older item in the other fragment will get priority
- Unoptimized all code that's not related to pigeons
# Let's say we want to give this highest priority
.. news-rank: 50
- Added a new transport for IP over messenger pigeon (:rfc:`1149`)
# This will be overruled by the rank 20 XTypes in the other fragment
.. news-rank: 10
.. news-start-section: XTypes
.. news-rank: 100
- New pigeon-based ``DynamicData``
.. news-end-section
.. news-end-section

These will result in:

  • Added a new transport for IP over messenger pigeon (RFC 1149) (PR #2222) [Rank 50]

  • Made logging worse in some cases. (PR #1111) [Rank 20]

  • XTypes [Rank 20]

    • New pigeon-based DynamicData (PR #2222) [Rank 100]

    • Added lossless casting from any type to any other type in DynamicData. (PR #1111) [Rank 10]

    • Added ability to randomize DynamicData (PR #1111) [Rank 0]

  • Improved logging in some cases. (PR #1111) [Rank 0]

  • Unoptimized all code that’s not related to pigeons (PR #2222) [Rank 0]

The ranks are included in previews of the news as an aid to deciding what the rank of new news content should be. The preview can be viewed in Release Notes or alternatively by running docs/news.py preview. The ranks are not included in the final news for a release.

One final thing to note is that top-level sections, like “Additions”, have a fixed rank that can’t be changed so they always appear in the same order.

Generating the News#

Before a release, a preview of the whole news for the next release will always be available in Release Notes. It’s also possible to see the source of that preview by running docs/news.py preview or docs/news.py preview-all. During a release the fragments are permanently committed to docs/news.d/_releases and NEWS.md and the fragment files in docs/news.d are removed.

See also

Release Process