Remote Packages and Repositories¶

dds stores a local database of available packages, along with their dependency statements and information about how a source distribution thereof may be obtained.

Inside the database are package repositories, which are remote servers that contain their own database of packages, and may also contain the packages themselves. An arbitrary number of package repositories may be added to the local database. When dds updates its package information, it will download the package database from each registered remote and import the listings into its own local database, making them available for download.

Viewing Available Packages¶

The default catalog database is stored in a user-local location, and the available packages can be listed with dds pkg search:

$ dds pkg search
    Name: abseil
Versions: 2018.6.0, 2019.8.8, 2020.2.25
    From: repo-1.dds.pizza

    Name: asio
Versions: 1.12.0, 1.12.1, 1.12.2, 1.13.0, 1.14.0, 1.14.1, 1.16.0, 1.16.1
    From: repo-1.dds.pizza

    Name: boost.leaf
Versions: 0.1.0, 0.2.0, 0.2.1, 0.2.2, 0.2.3, 0.2.4, 0.2.5, 0.3.0
    From: repo-1.dds.pizza

    Name: boost.mp11
Versions: 1.70.0, 1.71.0, 1.72.0, 1.73.0
    From: repo-1.dds.pizza

Optionally, one can search with a glob/fnmatch-style pattern:

$ dds pkg search 'neo-*'
    Name: neo-buffer
Versions: 0.2.1, 0.3.0, 0.4.0, 0.4.1, 0.4.2
    From: repo-1.dds.pizza

    Name: neo-compress
Versions: 0.1.0, 0.1.1, 0.2.0
    From: repo-1.dds.pizza

    Name: neo-concepts
Versions: 0.2.2, 0.3.0, 0.3.1, 0.3.2, 0.4.0
    From: repo-1.dds.pizza

Remote Repositories¶

A remote package repository consists of an HTTP(S) server serving the following:

An accessible directory containing source distributions of various packages, and
An accessible database file that contains a listing of packages and some repository metadata.

The exact details of the directory layout and database are not covered here, and are not necessary to make use of a repository.

When dds uses a repository, it pulls down the database file and imports its contents into its own local database, associating the imported package listings with the remote repository which provides them. Pulling the entire database at once allows dds to perform much faster dependency resolution and reduces the round-trips associated with using a dynamic package repository.

Adding a Repository¶

Adding a remote repository to the local database is a simple single command:

$ dds pkg repo add "https://repo-1.dds.pizza"
[info ] Pulling repository contents for repo-1.dds.pizza [https://repo-1.dds.pizza/]

This will tell dds to add https://repo-1.dds.pizza as a remote repository and immediately pull its package listings for later lookup. This initial update can be suppressed with the --no-update flag.

Note

After the initial pkg repo add, the repository is not identified by its URL, but by its name, which is provided by the repository itself. The name is printed the first time it is added, and can be seen using pkg repo ls.

Listing Repositories¶

A list of package repositories can be seen with the pkg repo ls subcommand:

$ dds pkg repo ls

Removing Repositories¶

A repository can be removed by the pkg repo remove subcommand:

$ dds pkg repo remove <name>

Where <name> is given as the name (not URL!) of the repository.

Note that removing a repository will make all of its corresponding remote packages unavailable, while packages that have been pulled into the local cache will remain available even after removing a repository.

Updating Repository Data¶

Repository data and package listings can be updated with the pkg repo update subcommand:

$ dds pkg repo update

This will pull down the databases of all registered remote repositories. If dds can detect that a repository’s database is unchanged since a prior update, that update will be skipped.

The Default Repository¶

When dds first initializes its local package database, it will add a single remote repository: https://repo-1.dds.pizza/, which has the name repo-1.dds.pizza. At the time of writing, this is the only official dds repository, and is populated sparsely with hand-curated and prepared packages. In the future, the catalog of packages will grow and be partially automated.

There is nothing intrinsically special about this repository other than it being the default when dds first creates its package database. It can be removed as any other, should one want tighter control over package availability.

Managing a Repository¶

A dds repository is simply a directory of static files, so any HTTP server that can serve from a filesystem can be used as a repository. dds also ships with a subcommand, repoman, that can be used to manage a repository directory.

Initializing a Repository¶

Before anything can be done, a directory should be converted to a repository by using repoman init:

$ dds repoman init ./my-repo-dir --name=my-experimental-repo

This will add the basic metadata into ./my-repo-dir such that dds will be able to pull package data from it.

The --name argument should be used to give the repository a unique name. The name should be globally unique to avoid collisions: When dds pulls a repository that declares a given name, it will replace the package listings associated with any repository of that name. As such, generic names like main or packages shouldn’t be used in production.

Listing Contents¶

The packages in a repository can be listed using dds repoman ls <repo-dir>. This will simply print each package identifier that is present in the repository.

Importing Source Distributions¶

If you have a source distribution archive, it can be imported with the appropriately named dds repoman import command:

$ dds repoman import ./my-repo some-pkg@1.2.3.tar.gz

Multiple archive paths may be provided to import them all at once.

Adding a Package by URL¶

A repository can also list packages that it does not host itself. Such a package listing can be added “by URL,” where the URL tells dds how to pull the source distribution of the package. Beyond basic HTTP(S) URLs, dds can also clone packages via git:

$ dds repoman add ./my-repo git+https://github.com/vector-of-bool/neo-fun.git#0.5.2

The above URL tells dds that it can use git clone against https://github.com/vector-of-bool/neo-fun.git and ask for tag 0.5.2 to get a source distribution directory that can be imported. Note the fragment on git URLs! The fragment is required to specify the branch or tag to clone.

If the package is available on GitHub, dds has a shorthand URL for that:

$ dds repoman add ./my-repo github:vector-of-bool/neo-fun/0.6.0

The github: URL scheme tells dds to clone from GitHub. A github: URL must have exactly three path elements to determine what to download: github:{owner}/{repository}/{branch-or-tag}.

Note

The github: URL lacks an authority element, and as such does not use the double-slash.

Note

repoman add will immediately attempt to pull a source distribution from the given URL so that it may import the package’s metadata into its listing database. You cannot add a URL that is not already accessible.

Removing Packages¶

A package can be removed from a repository with dds repoman remove <repo-dir> <pkg-id>, where <pkg-id> is the <name>@<version> of the package to remove.

Next: Library and Package Dependencies