Building dds
from Source¶
Note
This page assumes that you have ready the Setting Up a Build/Development Environment page, and that you are running all commands from within the Poetry-generated virtual environment.
The main entrypoint for the dds
CI process is the dds-ci
command, which
will build and test the dds
from the repository sources. dds-ci
accepts
several optional command-line arguments to tweak its behavior.
Running a Build Only¶
If you only wish to obtain a built dds
executable, the --no-test
parameter can be given:
$ dds-ci --no-test
This will skip the audit-build and testing phases of CI and build only the final
dds
executable.
Rapid Iterations for Development¶
If you are making frequent changes to dds
’s source code and want a fast
development process, use --rapid
:
$ dds-ci --rapid
This will build the build step only, and builds an executable with maximum debug and audit information, including AddressSanitizer and UndefinedBehaviorSanitizer. This will also execute the unit tests, which should run completely in under two seconds (if they are slower, then it may be a bug).
Toolchain Control¶
dds-ci
will automatically select and build with an appropriate
toolchain based on what it detects of the host
platform, but you may want to tweak those options.
The dds-ci
script accepts two toolchain options:
--main-toolchain
This is the toolchain that is used to create a final release-built executable. If you build with
--no-test
, this toolchain will be used.
--test-toolchain
This is the toolchain that is used to create an auditing
and debuggable executable of dds
. This is the toolchain that is used if you
build with --rapid
.
If you build with neither --rapid
nor --no-test
, then dds-ci
will
build two dds
executables: One with the --test-toolchain
that is
passed through the test suite, and another for --main-toolchain
that is
built for distribution.
The default toolchains are files contained within the tools/
directory of
the repository. When dds-ci
builds dds
, it will print the path to the
toolchain file that is selected for that build.
While these provided toolchains will work perfectly well in CI, you may need to
tweak these for your build setup. For example: gcc-9-*.jsonc
toolchains
assume that the GCC 9 executables are named gcc-9
and g++-9
, which is
incorrect on some Unix and Linux distributions.
It is recommended to tweak these files as necessary to get the build working on your system. However, do not include those tweaks in a commit unless they are necessary to get the build running in CI.
What’s Happening?¶
The dds-ci
script performs the following actions, in order:
If given
--clean
, remove any prior build output and downloaded dependencies.Prepare the prior version of
dds
that will build the current version (usually, just download it). This is placed in_prebuilt/
.Import the
old-catalog.json
into a catalog database stored within_prebuilt/
. This will be used to resolve the third-party packages thatdds
itself uses.Invoke the build of
dds
using the prebuiltdds
obtained from the prior bootstrap phase. If--no-test
or--rapid
was specified, the CI script stops here.Launch
pytest
with the generateddds
executable and start the final release build simultaneously, and wait for both to finish.
Unit Tests¶
Various pieces of dds
contain unit tests. These are stored within the
src/
directory itself in *.test.cpp
files. They are built and executed
as part of the iteration cycle unconditionally. These tests execute in
milliseconds so as not to burden the development iteration cycle. The more
rigorous tests are executed separately by PyTest.
Speeding Up the Build¶
dds
’s build is unfortunately demanding, but can be sped up by additional
tools:
Use the LLVM lld
Linker¶
Installing the LLVM lld
linker will significantly improve the time it
takes for dds
and its unit test executables to link. dds-ci
will
automatically recognize the presence of lld
if it has been installed
properly.
Note
dds-ci
(and GCC) look for an executable called ld.ldd
on the
executable PATH (no version suffix!). You may need to symlink the
version-suffixed executable with ld.ldd
in another location on PATH so
that dds-ci
(and GCC) can find it.
Use ccache
¶
dds-ci
will also recognize ccache
and add it as a compiler-launcher if
it is installed on your PATH. This won’t improve initial compilation times, but
can make subsequent compilations significantly faster when files are unchanged.