dds Alpha Release 6 is Available!
This is the sixth alpha release of
dds, and the final “alpha” release before
it enters beta.
- Get it from the downloads page!
- Learn how to get and use
- Ask questions, send feedback, and more in the
Changes in Alpha 6
It’s been a few months, but there’s been some significant improvements.
Feature: Remote Package Repositories!
The largest, most significant, and most fundamental feature of any package
managing tool has arrived. This release finally introduces package
dds. The prior methods of package procurement were clunky,
only designed to allow
dds development to be dogfed, and slow to update and
deploy. This package repositories feature required a huge amount of yak-shaving
and took the majority of development time for the past year.
The new package repositories work similar to how you would see them in other
package managing software: A remote server maintains a listing of packages that
it provides, and services clients wishing to download packages. Here’s a rundown
of how they work in
dds (as of alpha.6):
- Package repositories are static content. Any HTTP server that can serve
from a filesystem can serve a repository. There is no dynamic data or API on
ddsrepositories. This makes
ddsrepositories trivial to manage as a system administrator.
ddsdownloads the package listings from a repository as a database of all listings in a single file.
ddsimports the listings and associates them with the remote repository. It can then be locally browsed, searched, and resolved without communicating with the remote.
ddsships with a new subcommand,
repoman, designed to manage repository directories that are then served on HTTP.
- Right now there is only one repository by default, at
repo-1.dds.pizza. (The name is qualified with a
-1suffix to allow room for the stable
repo.dds.pizzain the future.) This repository is registered when
ddsperforms its local database migration.
- New repositories can be added or removed as needed with the
dds pkg reposubcommand. Package listings are updated with
dds pkg repo update.
At the moment,
repo-1.dds.pizza does not have a huge catalog, but over the
coming weeks you can anticipate it growing as time is allocated to fleshing it
The packages available for use can be obtained using the
dds pkg search
Note that although
dds can communicate via HTTPS, it does not yet perform
TLS certificate validation!
Refer to the documentation on remote packages for more information.
Feature: Cached Compiler Output
When a compiler emits output to stdout/stderr,
dds will display that output as
a compiler warning. If
dds build is executed again without any changes to the
source files, compilation of those files is skipped. This is referred to as
incremental compilation, and is essential for a rapid build development cycle.
Any major build tool today features incremental compilation, and
dds has had
it since the beginning.
But what happens the output of the compiler when we skip compiling a file due to
incremental compilation? For most tools, and in
dds alpha.5, the answer was
simple: We didn’t run the compiler, so there was no output to display.
dds alpha.6 changes that, and now, when
dds sees a compiler emit some output
after a compilation, that output is stored for later display. On subsequent
builds, if the compilation of a source file is skipped but the previous
compilation emitted compiler warnings, those compiler warnings are displayed
again. This prevents warnings from being forgotten because an incremental
rebuild skipped over the compilation of a file that was up-to-date. Compiler
output that appears from a prior compilation is annotated as “cached compiler
output” to distinguish it.
Feature: New CLI
The command-line interface of
dds has been completely rewritten from scratch.
This improves what was previously a confusing set of subcommands and reduces
them by reorganizing them into a more useful hierarchy. Additionally,
now issue “did-you-mean” messages for every command-line argument, improving
are a new approach to compile-time library configuration customization.
supports these natively via the
--tweaks-dir command-line argument to
dds build-deps, and
--tweaks-dir should be provided as an argument to the build and point to the
directory from which tweak-header
#include directives should resolve. The
recursive file listing of a
--tweaks-dir is used as a cache key during
incremental compilation and
dds will insert a cache-busting compiler
option to prevent tools like
ccache from missing tweak-header
additions (a known issue when the result of
__has_include() can change).
Before remote repositories,
dds used a “calalog” JSON format. This has been
entirely removed, as well as “fs-transforms.” If a package needs to be
rearranged or patched before import, it should be patched separately, archived
as a source distribution, and imported into an HTTP repository for
dds had a featured called the “local repository”, which was just a directory
containing source distributions of external packages. This terminology has been
dropped in favor of the “local package cache.” Also, the “catalog database” is
now just the “local database.” These changes are simply to relieve undue
confusion, especially now that
dds has remote repositories which have
remote catalog databases.
ddsoutput is now much more colorful!
- More documentation on installation, and docs on basic dependency management.
ddscan “install itself” with the
dds install-yourselfsubcommand (refer to the documentation on download and installation).
- Updated the embedded Catch2 header.
- Fix garbled output on build cancellation.
- Reduced static binary size on Linux.
- Fix behavior on Windows with non-ASCII paths.
A Year’s Retrospective
It’s been nearly a year since
It’s been a loooong time, and we’re still in alpha??
Yep, we’re still only in testing phase. What’s taking so long?
If you read the prior release post, you’ll probably remember that there’s been
a lot of yak-shaving, and that’s the main reason that it has taken so long.
Almost every component in
dds’s stack is “home-grown,” including
“fundamentals” such as HTTP, networking, gzip, and command-line parsing (of
I’ve heard often complaints that “software development anymore is just gluing
libraries and frameworks together and no one ever makes anything,” and I can now
say with some confidence that I’m glad that I don’t have to constantly
re-invent everything when I want to just download a file. Over this last year
I’ve experienced it. It took me months just to get
dds to download a single
webpage over HTTP. It’s been a great learning experience, but its taken quite a
bit of time, and only now do I have something to show for it.
Going forward, iteration between dependencies and components will be significantly faster now that there is a separate package repository that can be updated when I need to post a bugfix to an external component.
A Year of Eating Dog Food
dds and every library that I’ve built along the way are built and tested by
dds itself. Times that I have used CMake or Visual Studio in the past year
could be counted on one hand, and I can say that I’m very confident and
comfortable with using
dds as my main driver, even if it is still in an early
stage and has not had a package repository all this time.
Of course I will be biased, but I offer this perspective in hopes that others
may take the time to try it out with their own personal projects. I have been
dds with intent that it can be anyone’s main development iteration
tool, from simple prototyping, to library development, to tooling, and
eventually to full desktop applications.
The Final Stretch: What’s Next?
As noted, this is the last alpha-release of
dds, after which it will be
released as beta, and the focus will be on bugfixes, performance,
documentation, and code cleanup. Following beta will be the first stable release
1.0.0, of course!
Until the beta, though, there are still some features to add:
- TLS certificate validation:
ddsdoes not yet authenticate and validate TLS certificates when it connects to a TLS server! Certificate validation will need to be handled properly, and will also entail new
ddsconfiguration options on certificate authority trust and validation bypass options.
- Package signing: The previous post mentioned that packages are not yet
signed. While usage of
TLSand certificate trust somewhat alleviates some concerns, we will want package authors to be able to sign packages themselves before those packages are posted to an HTTP repository.
- A new command-line UX: The
ddscommand-line is fairly rudimentary and consists of basic logging, but a more concise and informative UI is preferrable.
- An interprocess API for external tooling and IDE integration: Currently,
dds’s only integration is by emitting a
compile_commands.jsonfile. While this offers some amount of functionality to some tools, it is insufficient for many.
Try It Out!
I’m very happy with how
dds is going, and I am excited to continue work. In
the coming weeks I’ll be updating the alpha package repository with more (and
updated) packages, and I will be listening for feedback.
If your brand new, or just curious:
Check the Getting/Installing
dds page of the documentation.
Send Feedback, Ask Questions, Post Ideas, and More
GitHub has released a beta feature of the platform, known as Discussions. I
have opted to enable them on the
dds repository to answer user questions and
solicit feedback rather than use GitHub issues for that purpose.
If you have any questions or want to provide feedback, please feel free to open a new discussion on the Discussions section. I’ll especially be looking out for “How do I…?” questions, since those may warrant documentation changes.