Remote Packages and Repositories¶
dds stores a local database of available packages, along with their
dependency statements and information about how a source distribution thereof
may be obtained.
Inside the database are package repositories, which are remote servers that
contain their own database of packages, and may also contain the packages
themselves. An arbitrary number of package repositories may be added to the
local database. When
dds updates its package information, it will download
the package database from each registered remote and import the listings into
its own local database, making them available for download.
Viewing Available Packages¶
The default catalog database is stored in a user-local location, and the
available packages can be listed with
dds pkg search:
$ dds pkg search Name: abseil Versions: 2018.6.0, 2019.8.8, 2020.2.25 From: repo-1.dds.pizza Name: asio Versions: 1.12.0, 1.12.1, 1.12.2, 1.13.0, 1.14.0, 1.14.1, 1.16.0, 1.16.1 From: repo-1.dds.pizza Name: boost.leaf Versions: 0.1.0, 0.2.0, 0.2.1, 0.2.2, 0.2.3, 0.2.4, 0.2.5, 0.3.0 From: repo-1.dds.pizza Name: boost.mp11 Versions: 1.70.0, 1.71.0, 1.72.0, 1.73.0 From: repo-1.dds.pizza
Optionally, one can search with a glob/fnmatch-style pattern:
$ dds pkg search 'neo-*' Name: neo-buffer Versions: 0.2.1, 0.3.0, 0.4.0, 0.4.1, 0.4.2 From: repo-1.dds.pizza Name: neo-compress Versions: 0.1.0, 0.1.1, 0.2.0 From: repo-1.dds.pizza Name: neo-concepts Versions: 0.2.2, 0.3.0, 0.3.1, 0.3.2, 0.4.0 From: repo-1.dds.pizza
A remote package repository consists of an HTTP(S) server serving the following:
An accessible directory containing source distributions of various packages, and
An accessible database file that contains a listing of packages and some repository metadata.
The exact details of the directory layout and database are not covered here, and are not necessary to make use of a repository.
dds uses a repository, it pulls down the database file and imports its
contents into its own local database, associating the imported package listings
with the remote repository which provides them. Pulling the entire database at
dds to perform much faster dependency resolution and reduces
the round-trips associated with using a dynamic package repository.
Adding a Repository¶
Adding a remote repository to the local database is a simple single command:
$ dds pkg repo add "https://repo-1.dds.pizza" [info ] Pulling repository contents for repo-1.dds.pizza [https://repo-1.dds.pizza/]
This will tell
dds to add
https://repo-1.dds.pizza as a remote
repository and immediately pull its package listings for later lookup. This
initial update can be suppressed with the
After the initial
pkg repo add, the repository is not identified by its
URL, but by its name, which is provided by the repository itself. The name
is printed the first time it is added, and can be seen using
pkg repo ls.
A list of package repositories can be seen with the
pkg repo ls subcommand:
$ dds pkg repo ls
A repository can be removed by the
pkg repo remove subcommand:
$ dds pkg repo remove <name>
<name> is given as the name (not URL!) of the repository.
Note that removing a repository will make all of its corresponding remote packages unavailable, while packages that have been pulled into the local cache will remain available even after removing a repository.
Updating Repository Data¶
Repository data and package listings can be updated with the
pkg repo update
$ dds pkg repo update
This will pull down the databases of all registered remote repositories. If
dds can detect that a repository’s database is unchanged since a prior
update, that update will be skipped.
The Default Repository¶
dds first initializes its local package database, it will add a single
https://repo-1.dds.pizza/, which has the name
repo-1.dds.pizza. At the time of writing, this is the only official
repository, and is populated sparsely with hand-curated and prepared packages.
In the future, the catalog of packages will grow and be partially automated.
There is nothing intrinsically special about this repository other than it being
the default when
dds first creates its package database. It can be removed
as any other, should one want tighter control over package availability.
Managing a Repository¶
dds repository is simply a directory of static files, so any HTTP server
that can serve from a filesystem can be used as a repository.
ships with a subcommand,
repoman, that can be used to manage a repository
Initializing a Repository¶
Before anything can be done, a directory should be converted to a repository by
$ dds repoman init ./my-repo-dir --name=my-experimental-repo
This will add the basic metadata into
./my-repo-dir such that
be able to pull package data from it.
--name argument should be used to give the repository a unique name. The
name should be globally unique to avoid collisions: When
dds pulls a
repository that declares a given name, it will replace the package listings
associated with any repository of that name. As such, generic names like
packages shouldn’t be used in production.
The packages in a repository can be listed using
dds repoman ls <repo-dir>.
This will simply print each package identifier that is present in the
Importing Source Distributions¶
If you have a source distribution archive, it can be imported with the
dds repoman import command:
$ dds repoman import ./my-repo email@example.com
Multiple archive paths may be provided to import them all at once.
Adding a Package by URL¶
A repository can also list packages that it does not host itself. Such a package
listing can be added “by URL,” where the URL tells
dds how to pull the
source distribution of the package. Beyond basic HTTP(S) URLs,
dds can also
clone packages via
$ dds repoman add ./my-repo git+https://github.com/vector-of-bool/neo-fun.git#0.5.2
The above URL tells
dds that it can use
git clone against
https://github.com/vector-of-bool/neo-fun.git and ask for tag
get a source distribution directory that can be imported. Note the fragment on
git URLs! The fragment is required to specify the branch or tag to clone.
If the package is available on GitHub,
dds has a shorthand URL for that:
$ dds repoman add ./my-repo github:vector-of-bool/neo-fun/0.6.0
github: URL scheme tells
dds to clone from GitHub. A
must have exactly three path elements to determine what to download:
github: URL lacks an authority element, and as such does not use
repoman add will immediately attempt to pull a source distribution from
the given URL so that it may import the package’s metadata into its listing
database. You cannot add a URL that is not already accessible.
A package can be removed from a repository with
dds repoman remove <repo-dir> <pkg-id>, where
<pkg-id> is the
<name>@<version> of the package to remove.