NIX-PUSH(1)

NAME
nix-push - generate a binary cache

SYNOPSIS

nix-push --dest dest-dir [--bzip2] [--none] [--force] [--link] [--manifest] [--manifest-path filename] [--url-prefix url] [--key-file path] paths...

DESCRIPTION

The command nix-push produces a binary cache, a directory containing compressed Nix archives (NARs) plus some metadata of the closure of the specified store paths. This directory can then be made available through a web server to other Nix installations, allowing them to skip building from source and instead download binaries from the cache automatically.

nix-push performs the following actions.

    1. Each path in paths is built (using nix-store --realise).

    2. All paths in the closure of paths are determined (using nix-store --query --requisites --include-outputs). Note that since the --include-outputs flag is used, if paths includes a store derivation, you get a combined source/binary distribution (e.g., source tarballs will be included).

    3. All store paths determined in the previous step are packaged into a NAR (using nix-store --dump) and compressed using xz or bzip2. The resulting files have the extension .nar.xz or .nar.bz2. Also for each store path, Nix generates a file with extension .narinfo containing metadata such as the references, cryptographic hash and size of each path.

    4. Optionally, a single manifest file is created that contains the same metadata as the .narinfo files. This is for compatibility with Nix versions prior to 1.2 (see nix-pull for details).

    5. A file named nix-cache-info is placed in the destination directory. The existence of this file marks the directory as a binary cache.

OPTIONS

--dest dest-dir
    Set the destination directory to dir, which is created if it does not exist. This flag is required.

--bzip2
    Compress NARs using bzip2 instead of xz. The latter compresses about 30% better on typical archives, decompresses about twice as fast, but compresses a lot slower and is not supported by Nix prior to version 1.2.

--none
    Do not compress NARs.

--force
    Overwrite .narinfo files if they already exist.

--link
    By default, NARs are generated in the Nix store and then copied to dest-dir. If this option is given, hard links are used instead. This only works if dest-dir is on the same filesystem as the Nix store.

--manifest
    Force the generation of a manifest suitable for use by nix-pull. The manifest is stored as dest-dir/MANIFEST.

--manifest-path filename
    Like --manifest, but store the manifest in filename.

--url-prefix url
    Manifests are expected to contain the absolute URLs of NARs. For generating these URLs, the prefix url is used. It defaults to file://dest-dir.

--key-file path
    Sign the binary cache using the secret key stored in path. This secret key must have been created using nix-store --generate-binary-cache-key. Users of this binary cache should add the corresponding public key to the option binary-cache-public-keys in nix.conf.

EXAMPLES

To add the closure of Thunderbird to a binary cache:

    $ nix-push --dest /tmp/cache $(nix-build -A thunderbird)

Assuming that /tmp/cache is exported by a web server as http://example.org/cache, you can then use this cache on another machine to speed up the installation of Thunderbird:

    $ nix-build -A thunderbird --option binary-caches http://example.org/cache

Alternatively, you could add binary-caches = http://example.org/cache to nix.conf.

To also include build-time dependencies (such as source tarballs):

    $ nix-push --dest /tmp/cache $(nix-instantiate -A thunderbird)

To generate a manifest suitable for nix-pull:

    $ nix-push --dest /tmp/cache $(nix-build -A thunderbird) --manifest

On another machine you can then do:

    $ nix-pull http://example.org/cache

to cause the binaries to be used by subsequent Nix operations.

To generate a signed binary cache, you must first generate a key pair, in this example called cache.example.org-1, storing the secret key in ./sk and the public key in ./pk:

    $ nix-store --generate-binary-cache-key cache.example.org-1 sk pk
    $ cat sk
    cache.example.org-1:jcMRQYFo8pQKzTtimpQLIPeHkMYZjfhB24hGfwF+u9PuX8H8FO7q564+X3G/JDlqqIqGar3OXRRwS9N3Wh3vbw==
    $ cat pk
    cache.example.org-1:7l/B/BTu6ueuPl9xvyQ5aqiKhmq9zl0UcEvTd1od728=

You can then generate a binary cache signed with the secret key:

    $ nix-push --dest /tmp/cache --key-file ./sk $(type -p firefox)

Users who wish to verify the integrity of binaries downloaded from your cache would add the following to their nix.conf:

    binary-caches = http://cache.example.org
    signed-binary-caches = *
    binary-cache-public-keys = cache.example.org-1:7l/B/BTu6ueuPl9xvyQ5aqiKhmq9zl0UcEvTd1od728=

Nix will then ignore any binary that has a missing, incorrect or unrecognised signature.

BINARY CACHE FORMAT AND OPERATION

A binary cache with URL url only denotes a valid binary cache if the file url/nix-cache-info exists. If this file does not exist (or cannot be downloaded), the cache is ignored. If it does exist, it must be a text file containing cache properties. Here’s an example:

    StoreDir: /nix/store
    WantMassQuery: 1
    Priority: 10

The properties that are currently supported are:

StoreDir
    The path of the Nix store to which this binary cache applies. Binaries are not relocatable — a binary built for /nix/store won’t generally work in /home/alice/store — so to prevent binaries from being used in a wrong store, a binary cache is only used if its StoreDir matches the local Nix configuration. The default is /nix/store.

WantMassQuery
    Query operations such as nix-env -qas can cause thousands of cache queries, and thus thousands of HTTP requests, to determine which packages are available in binary form. While these requests are small, not every server may appreciate a potential onslaught of queries. If WantMassQuery is set to 0 (default), “mass queries” such as nix-env -qas will skip this cache. Thus a package may appear not to have a binary substitute. However, the binary will still be used when you actually install the package. If WantMassQuery is set to 1, mass queries will use this cache.

Priority
    Each binary cache has a priority (defaulting to 50). Binary caches are checked for binaries in order of ascending priority; thus a higher number denotes a lower priority. The binary cache https://cache.nixos.org has priority 40.

Every time Nix needs to build some store path p, it will check each configured binary cache to see if it has a NAR file for p, until it finds one. If no cache has a NAR, Nix will fall back to building the path from source (if applicable). To see if a cache with URL url has a binary for p, Nix fetches url/h, where h is the hash part of p. Thus, if we have a cache https://cache.nixos.org and we want to obtain the store path

    /nix/store/a8922c0h87iilxzzvwn2hmv8x210aqb9-glibc-2.7

then Nix will attempt to fetch

    https://cache.nixos.org/a8922c0h87iilxzzvwn2hmv8x210aqb9.narinfo

(Commands such as nix-env -qas will issue an HTTP HEAD request, since it only needs to know if the .narinfo file exists.) The .narinfo file is a simple text file that looks like this:

    StorePath: /nix/store/a8922c0h87iilxzzvwn2hmv8x210aqb9-glibc-2.7
    URL: nar/0zzjpdz46mdn74v09m053yczlz4am038g8r74iy8w43gx8801h70.nar.bz2
    Compression: bzip2
    FileHash: sha256:0zzjpdz46mdn74v09m053yczlz4am038g8r74iy8w43gx8801h70
    FileSize: 24473768
    NarHash: sha256:0s491y1h9hxj5ghiizlxk7ax6jwbha00zwn7lpyd5xg5bhf60vzg
    NarSize: 109521136
    References: 2ma2k0ys8knh4an48n28vigcmc2z8773-linux-headers-2.6.23.16 ...
    Deriver: 7akyyc87ka32xwmqza9dvyg5pwx3j212-glibc-2.7.drv
    Sig: cache.example.org-1:WepnSp2UT0odDpR3NRjPVhJBHmdBgSBSTbHpdh4SCz92nGXwFY82bkPEmISoC0hGqBXDXEmB6y3Ohgna3mMgDg==

The fields are as follows:

StorePath
    The full store path, including the name part (e.g., glibc-2.7). It must match the requested store path.

URL
    The URL of the NAR, relative to the binary cache URL.

Compression
    The compression method; either xz or bzip2.

FileHash
    The SHA-256 hash of the compressed NAR.

FileSize
    The size of the compressed NAR.

NarHash
    The SHA-256 hash of the uncompressed NAR. This is equal to the hash of the store path as returned by nix-store -q --hash p.

NarSize
    The size of the uncompressed NAR.

References
    The references of the store path, without the Nix store prefix.

Deriver
    The deriver of the store path, without the Nix store prefix. This field is optional.

System
    The Nix platform type of this binary, if known. This field is optional.

Sig
    A signature of the the form key-name:sig, where key-name is the symbolic name of the key pair used to sign and verify the cache (e.g. cache.example.org-1), and sig is the actual signature, computed over the StorePath, NarHash, NarSize and References fields using the Ed25519 public-key signature system[1].

Thus, in our example, after recursively ensuring that the references exist (e.g., /nix/store/2ma2k0ys8knh4an48n28vigcmc2z8773-linux-headers-2.6.23.16), Nix will fetch

    https://cache.nixos.org/nar/0zzjpdz46mdn74v09m053yczlz4am038g8r74iy8w43gx8801h70.nar.bz2

and decompress and unpack it to /nix/store/a8922c0h87iilxzzvwn2hmv8x210aqb9-glibc-2.7.

AUTHOR

Eelco Dolstra
LogicBlox
    Author

COPYRIGHT
Copyright © 2004-2014 Eelco Dolstra

NOTES

1.  Ed25519 public-key signature system
    http://ed25519.cr.yp.to/