This is guix.info, produced by makeinfo version 6.7 from guix.texi.

Copyright © 2012-2022 Ludovic Courtès
Copyright © 2013, 2014, 2016 Andreas Enge
Copyright © 2013 Nikita Karetnikov
Copyright © 2014, 2015, 2016 Alex Kost
Copyright © 2015, 2016 Mathieu Lirzin
Copyright © 2014 Pierre-Antoine Rault
Copyright © 2015 Taylan Ulrich Bayırlı/Kammer
Copyright © 2015, 2016, 2017, 2019, 2020, 2021 Leo Famulari
Copyright © 2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022 Ricardo
Wurmus
Copyright © 2016 Ben Woodcroft
Copyright © 2016, 2017, 2018, 2021 Chris Marusich
Copyright © 2016, 2017, 2018, 2019, 2020, 2021, 2022 Efraim Flashner
Copyright © 2016 John Darrington
Copyright © 2016, 2017 Nikita Gillmann
Copyright © 2016, 2017, 2018, 2019, 2020 Jan Nieuwenhuizen
Copyright © 2016, 2017, 2018, 2019, 2020, 2021 Julien Lepiller
Copyright © 2016 Alex ter Weele
Copyright © 2016, 2017, 2018, 2019, 2020, 2021 Christopher Baines
Copyright © 2017, 2018, 2019 Clément Lassieur
Copyright © 2017, 2018, 2020, 2021, 2022 Mathieu Othacehe
Copyright © 2017 Federico Beffa
Copyright © 2017, 2018 Carlo Zancanaro
Copyright © 2017 Thomas Danckaert
Copyright © 2017 humanitiesNerd
Copyright © 2017, 2021 Christine Lemmer-Webber
Copyright © 2017, 2018, 2019, 2020, 2021, 2022 Marius Bakke
Copyright © 2017, 2019, 2020, 2022 Hartmut Goebel
Copyright © 2017, 2019, 2020, 2021, 2022 Maxim Cournoyer
Copyright © 2017–2022 Tobias Geerinckx-Rice
Copyright © 2017 George Clemmer
Copyright © 2017 Andy Wingo
Copyright © 2017, 2018, 2019, 2020 Arun Isaac
Copyright © 2017 nee
Copyright © 2018 Rutger Helling
Copyright © 2018, 2021 Oleg Pykhalov
Copyright © 2018 Mike Gerwitz
Copyright © 2018 Pierre-Antoine Rouby
Copyright © 2018, 2019 Gábor Boskovits
Copyright © 2018, 2019, 2020, 2022 Florian Pelz
Copyright © 2018 Laura Lazzati
Copyright © 2018 Alex Vong
Copyright © 2019 Josh Holland
Copyright © 2019, 2020 Diego Nicola Barbato
Copyright © 2019 Ivan Petkov
Copyright © 2019 Jakob L. Kreuze
Copyright © 2019 Kyle Andrews
Copyright © 2019 Alex Griffin
Copyright © 2019, 2020, 2021, 2022 Guillaume Le Vaillant
Copyright © 2020 Liliana Marie Prikler
Copyright © 2019, 2020, 2021, 2022 Simon Tournier
Copyright © 2020 Wiktor Żelazny
Copyright © 2020 Damien Cassou
Copyright © 2020 Jakub Kądziołka
Copyright © 2020 Jack Hill
Copyright © 2020 Naga Malleswari
Copyright © 2020, 2021 Brice Waegeneire
Copyright © 2020 R Veera Kumar
Copyright © 2020, 2021 Pierre Langlois
Copyright © 2020 pinoaffe
Copyright © 2020 André Batista
Copyright © 2020, 2021 Alexandru-Sergiu Marton
Copyright © 2020 raingloom
Copyright © 2020 Daniel Brooks
Copyright © 2020 John Soo
Copyright © 2020 Jonathan Brielmaier
Copyright © 2020 Edgar Vincent
Copyright © 2021, 2022 Maxime Devos
Copyright © 2021 B. Wilson
Copyright © 2021 Xinglu Chen
Copyright © 2021 Raghav Gururajan
Copyright © 2021 Domagoj Stolfa
Copyright © 2021 Hui Lu
Copyright © 2021 pukkamustard
Copyright © 2021 Alice Brenon
Copyright © 2021, 2022 Josselin Poiret
Copyright © 2021 muradm
Copyright © 2021, 2022 Andrew Tropin
Copyright © 2021 Sarah Morgensen
Copyright © 2022 Remco van ’t Veer
Copyright © 2022 Aleksandr Vityazev
Copyright © 2022 Philip M^{c}Grath
Copyright © 2022 Karl Hallsby
Copyright © 2022 Justin Veilleux
Copyright © 2022 Reily Siegel
Copyright © 2022 Simon Streit
Copyright © 2022 (
Copyright © 2022 John Kehayias

   Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3 or
any later version published by the Free Software Foundation; with no
Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.  A
copy of the license is included in the section entitled “GNU Free
Documentation License”.
INFO-DIR-SECTION System administration
START-INFO-DIR-ENTRY
* Guix: (guix).       Manage installed software and system configuration.
* guix package: (guix)Invoking guix package.  Installing, removing, and upgrading packages.
* guix gc: (guix)Invoking guix gc.            Reclaiming unused disk space.
* guix pull: (guix)Invoking guix pull.        Update the list of available packages.
* guix system: (guix)Invoking guix system.    Manage the operating system configuration.
* guix deploy: (guix)Invoking guix deploy.    Manage operating system configurations for remote hosts.
END-INFO-DIR-ENTRY

INFO-DIR-SECTION Software development
START-INFO-DIR-ENTRY
* guix shell: (guix)Invoking guix shell.      Creating software environments.
* guix environment: (guix)Invoking guix environment.  Building development environments with Guix.
* guix build: (guix)Invoking guix build.      Building packages.
* guix pack: (guix)Invoking guix pack.        Creating binary bundles.
END-INFO-DIR-ENTRY


File: guix.info,  Node: origin Reference,  Prev: package Reference,  Up: Defining Packages

9.2.2 ‘origin’ Reference
------------------------

This section documents “origins”.  An ‘origin’ declaration specifies
data that must be “produced”—downloaded, usually—and whose content hash
is known in advance.  Origins are primarily used to represent the source
code of packages (*note Defining Packages::).  For that reason, the
‘origin’ form allows you to declare patches to apply to the original
source code as well as code snippets to modify it.

 -- Data Type: origin
     This is the data type representing a source code origin.

     ‘uri’
          An object containing the URI of the source.  The object type
          depends on the ‘method’ (see below).  For example, when using
          the URL-FETCH method of ‘(guix download)’, the valid ‘uri’
          values are: a URL represented as a string, or a list thereof.

     ‘method’
          A monadic procedure that handles the given URI.  The procedure
          must accept at least three arguments: the value of the ‘uri’
          field and the hash algorithm and hash value specified by the
          ‘hash’ field.  It must return a store item or a derivation in
          the store monad (*note The Store Monad::); most methods return
          a fixed-output derivation (*note Derivations::).

          Commonly used methods include ‘url-fetch’, which fetches data
          from a URL, and ‘git-fetch’, which fetches data from a Git
          repository (see below).

     ‘sha256’
          A bytevector containing the SHA-256 hash of the source.  This
          is equivalent to providing a ‘content-hash’ SHA256 object in
          the ‘hash’ field described below.

     ‘hash’
          The ‘content-hash’ object of the source—see below for how to
          use ‘content-hash’.

          You can obtain this information using ‘guix download’ (*note
          Invoking guix download::) or ‘guix hash’ (*note Invoking guix
          hash::).

     ‘file-name’ (default: ‘#f’)
          The file name under which the source code should be saved.
          When this is ‘#f’, a sensible default value will be used in
          most cases.  In case the source is fetched from a URL, the
          file name from the URL will be used.  For version control
          checkouts, it is recommended to provide the file name
          explicitly because the default is not very descriptive.

     ‘patches’ (default: ‘'()’)
          A list of file names, origins, or file-like objects (*note
          file-like objects: G-Expressions.) pointing to patches to be
          applied to the source.

          This list of patches must be unconditional.  In particular, it
          cannot depend on the value of ‘%current-system’ or
          ‘%current-target-system’.

     ‘snippet’ (default: ‘#f’)
          A G-expression (*note G-Expressions::) or S-expression that
          will be run in the source directory.  This is a convenient way
          to modify the source, sometimes more convenient than a patch.

     ‘patch-flags’ (default: ‘'("-p1")’)
          A list of command-line flags that should be passed to the
          ‘patch’ command.

     ‘patch-inputs’ (default: ‘#f’)
          Input packages or derivations to the patching process.  When
          this is ‘#f’, the usual set of inputs necessary for patching
          are provided, such as GNU Patch.

     ‘modules’ (default: ‘'()’)
          A list of Guile modules that should be loaded during the
          patching process and while running the code in the ‘snippet’
          field.

     ‘patch-guile’ (default: ‘#f’)
          The Guile package that should be used in the patching process.
          When this is ‘#f’, a sensible default is used.

 -- Data Type: content-hash VALUE [ALGORITHM]
     Construct a content hash object for the given ALGORITHM, and with
     VALUE as its hash value.  When ALGORITHM is omitted, assume it is
     ‘sha256’.

     VALUE can be a literal string, in which case it is base32-decoded,
     or it can be a bytevector.

     The following forms are all equivalent:

          (content-hash "05zxkyz9bv3j9h0xyid1rhvh3klhsmrpkf3bcs6frvlgyr2gwilj")
          (content-hash "05zxkyz9bv3j9h0xyid1rhvh3klhsmrpkf3bcs6frvlgyr2gwilj"
                        sha256)
          (content-hash (base32
                         "05zxkyz9bv3j9h0xyid1rhvh3klhsmrpkf3bcs6frvlgyr2gwilj"))
          (content-hash (base64 "kkb+RPaP7uyMZmu4eXPVkM4BN8yhRd8BTHLslb6f/Rc=")
                        sha256)

     Technically, ‘content-hash’ is currently implemented as a macro.
     It performs sanity checks at macro-expansion time, when possible,
     such as ensuring that VALUE has the right size for ALGORITHM.

   As we have seen above, how exactly the data an origin refers to is
retrieved is determined by its ‘method’ field.  The ‘(guix download)’
module provides the most common method, ‘url-fetch’, described below.

 -- Scheme Procedure: url-fetch URL HASH-ALGO HASH [name] [#:executable?
          #f]
     Return a fixed-output derivation that fetches data from URL (a
     string, or a list of strings denoting alternate URLs), which is
     expected to have hash HASH of type HASH-ALGO (a symbol).  By
     default, the file name is the base name of URL; optionally, NAME
     can specify a different file name.  When EXECUTABLE? is true, make
     the downloaded file executable.

     When one of the URL starts with ‘mirror://’, then its host part is
     interpreted as the name of a mirror scheme, taken from
     ‘%mirror-file’.

     Alternatively, when URL starts with ‘file://’, return the
     corresponding file name in the store.

   Likewise, the ‘(guix git-download)’ module defines the ‘git-fetch’
origin method, which fetches data from a Git version control repository,
and the ‘git-reference’ data type to describe the repository and
revision to fetch.

 -- Scheme Procedure: git-fetch REF HASH-ALGO HASH
     Return a fixed-output derivation that fetches REF, a
     ‘<git-reference>’ object.  The output is expected to have recursive
     hash HASH of type HASH-ALGO (a symbol).  Use NAME as the file name,
     or a generic name if ‘#f’.

 -- Data Type: git-reference
     This data type represents a Git reference for ‘git-fetch’ to
     retrieve.

     ‘url’
          The URL of the Git repository to clone.

     ‘commit’
          This string denotes either the commit to fetch (a hexadecimal
          string), or the tag to fetch.  You can also use a “short”
          commit ID or a ‘git describe’ style identifier such as
          ‘v1.0.1-10-g58d7909c97’.

     ‘recursive?’ (default: ‘#f’)
          This Boolean indicates whether to recursively fetch Git
          sub-modules.

     The example below denotes the ‘v2.10’ tag of the GNU Hello
     repository:

          (git-reference
            (url "https://git.savannah.gnu.org/git/hello.git")
            (commit "v2.10"))

     This is equivalent to the reference below, which explicitly names
     the commit:

          (git-reference
            (url "https://git.savannah.gnu.org/git/hello.git")
            (commit "dc7dc56a00e48fe6f231a58f6537139fe2908fb9"))

   For Mercurial repositories, the module ‘(guix hg-download)’ defines
the ‘hg-fetch’ origin method and ‘hg-reference’ data type for support of
the Mercurial version control system.

 -- Scheme Procedure: hg-fetch REF HASH-ALGO HASH [name]
     Return a fixed-output derivation that fetches REF, a
     ‘<hg-reference>’ object.  The output is expected to have recursive
     hash HASH of type HASH-ALGO (a symbol).  Use NAME as the file name,
     or a generic name if ‘#false’.


File: guix.info,  Node: Defining Package Variants,  Next: Writing Manifests,  Prev: Defining Packages,  Up: Programming Interface

9.3 Defining Package Variants
=============================

One of the nice things with Guix is that, given a package definition,
you can easily _derive_ variants of that package—for a different
upstream version, with different dependencies, different compilation
options, and so on.  Some of these custom packages can be defined
straight from the command line (*note Package Transformation Options::).
This section describes how to define package variants in code.  This can
be useful in “manifests” (*note Writing Manifests::) and in your own
package collection (*note Creating a Channel::), among others!

   As discussed earlier, packages are first-class objects in the Scheme
language.  The ‘(guix packages)’ module provides the ‘package’ construct
to define new package objects (*note package Reference::).  The easiest
way to define a package variant is using the ‘inherit’ keyword together
with ‘package’.  This allows you to inherit from a package definition
while overriding the fields you want.

   For example, given the ‘hello’ variable, which contains a definition
for the current version of GNU Hello, here’s how you would define a
variant for version 2.2 (released in 2006, it’s vintage!):

     (use-modules (gnu packages base))    ;for 'hello'

     (define hello-2.2
       (package
         (inherit hello)
         (version "2.2")
         (source (origin
                   (method url-fetch)
                   (uri (string-append "mirror://gnu/hello/hello-" version
                                       ".tar.gz"))
                   (sha256
                    (base32
                     "0lappv4slgb5spyqbh6yl5r013zv72yqg2pcl30mginf3wdqd8k9"))))))

   The example above corresponds to what the ‘--with-source’ package
transformation option does.  Essentially ‘hello-2.2’ preserves all the
fields of ‘hello’, except ‘version’ and ‘source’, which it overrides.
Note that the original ‘hello’ variable is still there, in the ‘(gnu
packages base)’ module, unchanged.  When you define a custom package
like this, you are really _adding_ a new package definition; the
original one remains available.

   You can just as well define variants with a different set of
dependencies than the original package.  For example, the default ‘gdb’
package depends on ‘guile’, but since that is an optional dependency,
you can define a variant that removes that dependency like so:

     (use-modules (gnu packages gdb))   ;for 'gdb'

     (define gdb-sans-guile
       (package
         (inherit gdb)
         (inputs (modify-inputs (package-inputs gdb)
                   (delete "guile")))))

   The ‘modify-inputs’ form above removes the ‘"guile"’ package from the
‘inputs’ field of ‘gdb’.  The ‘modify-inputs’ macro is a helper that can
prove useful anytime you want to remove, add, or replace package inputs.

 -- Scheme Syntax: modify-inputs INPUTS CLAUSES
     Modify the given package inputs, as returned by ‘package-inputs’ &
     co., according to the given clauses.  Each clause must have one of
     the following forms:

     ‘(delete NAME...)’
          Delete from the inputs packages with the given NAMEs
          (strings).

     ‘(prepend PACKAGE...)’
          Add PACKAGEs to the front of the input list.

     ‘(append PACKAGE...)’
          Add PACKAGEs to the end of the input list.

     The example below removes the GMP and ACL inputs of Coreutils and
     adds libcap to the front of the input list:

          (modify-inputs (package-inputs coreutils)
            (delete "gmp" "acl")
            (prepend libcap))

     The example below replaces the ‘guile’ package from the inputs of
     ‘guile-redis’ with ‘guile-2.2’:

          (modify-inputs (package-inputs guile-redis)
            (replace "guile" guile-2.2))

     The last type of clause is ‘append’, to add inputs at the back of
     the list.

   In some cases, you may find it useful to write functions
(“procedures”, in Scheme parlance) that return a package based on some
parameters.  For example, consider the ‘luasocket’ library for the Lua
programming language.  We want to create ‘luasocket’ packages for major
versions of Lua.  One way to do that is to define a procedure that takes
a Lua package and returns a ‘luasocket’ package that depends on it:

     (define (make-lua-socket name lua)
       ;; Return a luasocket package built with LUA.
       (package
         (name name)
         (version "3.0")
         ;; several fields omitted
         (inputs (list lua))
         (synopsis "Socket library for Lua")))

     (define-public lua5.1-socket
       (make-lua-socket "lua5.1-socket" lua-5.1))

     (define-public lua5.2-socket
       (make-lua-socket "lua5.2-socket" lua-5.2))

   Here we have defined packages ‘lua5.1-socket’ and ‘lua5.2-socket’ by
calling ‘make-lua-socket’ with different arguments.  *Note
(guile)Procedures::, for more info on procedures.  Having top-level
public definitions for these two packages means that they can be
referred to from the command line (*note Package Modules::).

   These are pretty simple package variants.  As a convenience, the
‘(guix transformations)’ module provides a high-level interface that
directly maps to the more sophisticated package transformation options
(*note Package Transformation Options::):

 -- Scheme Procedure: options->transformation OPTS
     Return a procedure that, when passed an object to build (package,
     derivation, etc.), applies the transformations specified by OPTS
     and returns the resulting objects.  OPTS must be a list of
     symbol/string pairs such as:

          ((with-branch . "guile-gcrypt=master")
           (without-tests . "libgcrypt"))

     Each symbol names a transformation and the corresponding string is
     an argument to that transformation.

   For instance, a manifest equivalent to this command:

     guix build guix \
       --with-branch=guile-gcrypt=master \
       --with-debug-info=zlib

...  would look like this:

     (use-modules (guix transformations))

     (define transform
       ;; The package transformation procedure.
       (options->transformation
        '((with-branch . "guile-gcrypt=master")
          (with-debug-info . "zlib"))))

     (packages->manifest
      (list (transform (specification->package "guix"))))

   The ‘options->transformation’ procedure is convenient, but it’s
perhaps also not as flexible as you may like.  How is it implemented?
The astute reader probably noticed that most package transformation
options go beyond the superficial changes shown in the first examples of
this section: they involve “input rewriting”, whereby the dependency
graph of a package is rewritten by replacing specific inputs by others.

   Dependency graph rewriting, for the purposes of swapping packages in
the graph, is what the ‘package-input-rewriting’ procedure in ‘(guix
packages)’ implements.

 -- Scheme Procedure: package-input-rewriting REPLACEMENTS
          [REWRITE-NAME] [#:deep? #t]
     Return a procedure that, when passed a package, replaces its direct
     and indirect dependencies, including implicit inputs when DEEP? is
     true, according to REPLACEMENTS.  REPLACEMENTS is a list of package
     pairs; the first element of each pair is the package to replace,
     and the second one is the replacement.

     Optionally, REWRITE-NAME is a one-argument procedure that takes the
     name of a package and returns its new name after rewrite.

Consider this example:

     (define libressl-instead-of-openssl
       ;; This is a procedure to replace OPENSSL by LIBRESSL,
       ;; recursively.
       (package-input-rewriting `((,openssl . ,libressl))))

     (define git-with-libressl
       (libressl-instead-of-openssl git))

Here we first define a rewriting procedure that replaces OPENSSL with
LIBRESSL.  Then we use it to define a “variant” of the GIT package that
uses LIBRESSL instead of OPENSSL.  This is exactly what the
‘--with-input’ command-line option does (*note ‘--with-input’: Package
Transformation Options.).

   The following variant of ‘package-input-rewriting’ can match packages
to be replaced by name rather than by identity.

 -- Scheme Procedure: package-input-rewriting/spec REPLACEMENTS [#:deep?
          #t]
     Return a procedure that, given a package, applies the given
     REPLACEMENTS to all the package graph, including implicit inputs
     unless DEEP? is false.  REPLACEMENTS is a list of spec/procedures
     pair; each spec is a package specification such as ‘"gcc"’ or
     ‘"guile@2"’, and each procedure takes a matching package and
     returns a replacement for that package.

   The example above could be rewritten this way:

     (define libressl-instead-of-openssl
       ;; Replace all the packages called "openssl" with LibreSSL.
       (package-input-rewriting/spec `(("openssl" . ,(const libressl)))))

   The key difference here is that, this time, packages are matched by
spec and not by identity.  In other words, any package in the graph that
is called ‘openssl’ will be replaced.

   A more generic procedure to rewrite a package dependency graph is
‘package-mapping’: it supports arbitrary changes to nodes in the graph.

 -- Scheme Procedure: package-mapping PROC [CUT?] [#:deep? #f]
     Return a procedure that, given a package, applies PROC to all the
     packages depended on and returns the resulting package.  The
     procedure stops recursion when CUT? returns true for a given
     package.  When DEEP? is true, PROC is applied to implicit inputs as
     well.


File: guix.info,  Node: Writing Manifests,  Next: Build Systems,  Prev: Defining Package Variants,  Up: Programming Interface

9.4 Writing Manifests
=====================

‘guix’ commands let you specify package lists on the command line.  This
is convenient, but as the command line becomes longer and less trivial,
it quickly becomes more convenient to have that package list in what we
call a “manifest”.  A manifest is some sort of a “bill of materials”
that defines a package set.  You would typically come up with a code
snippet that builds the manifest, store it in a file, say
‘manifest.scm’, and then pass that file to the ‘-m’ (or ‘--manifest’)
option that many ‘guix’ commands support.  For example, here’s what a
manifest for a simple package set might look like:

     ;; Manifest for three packages.
     (specifications->manifest '("gcc-toolchain" "make" "git"))

   Once you have that manifest, you can pass it, for example, to ‘guix
package’ to install just those three packages to your profile (*note
‘-m’ option of ‘guix package’: profile-manifest.):

     guix package -m manifest.scm

...  or you can pass it to ‘guix shell’ (*note ‘-m’ option of ‘guix
shell’: shell-manifest.) to spawn an ephemeral environment:

     guix shell -m manifest.scm

...  or you can pass it to ‘guix pack’ in pretty much the same way
(*note ‘-m’ option of ‘guix pack’: pack-manifest.).  You can store the
manifest under version control, share it with others so they can easily
get set up, etc.

   But how do you write your first manifest?  To get started, maybe
you’ll want to write a manifest that mirrors what you already have in a
profile.  Rather than start from a blank page, ‘guix package’ can
generate a manifest for you (*note ‘guix package --export-manifest’:
export-manifest.):

     # Write to 'manifest.scm' a manifest corresponding to the
     # default profile, ~/.guix-profile.
     guix package --export-manifest > manifest.scm

   Or maybe you’ll want to “translate” command-line arguments into a
manifest.  In that case, ‘guix shell’ can help (*note ‘guix shell
--export-manifest’: shell-export-manifest.):

     # Write a manifest for the packages specified on the command line.
     guix shell --export-manifest gcc-toolchain make git > manifest.scm

   In both cases, the ‘--export-manifest’ option tries hard to generate
a faithful manifest; in particular, it takes package transformation
options into account (*note Package Transformation Options::).

     Note: Manifests are _symbolic_: they refer to packages of the
     channels _currently in use_ (*note Channels::).  In the example
     above, ‘gcc-toolchain’ might refer to version 11 today, but it
     might refer to version 13 two years from now.

     If you want to “pin” your software environment to specific package
     versions and variants, you need an additional piece of information:
     the list of channel revisions in use, as returned by ‘guix
     describe’.  *Note Replicating Guix::, for more information.

   Once you’ve obtained your first manifest, perhaps you’ll want to
customize it.  Since your manifest is code, you now have access to all
the Guix programming interfaces!

   Let’s assume you want a manifest to deploy a custom variant of GDB,
the GNU Debugger, that does not depend on Guile, together with another
package.  Building on the example seen in the previous section (*note
Defining Package Variants::), you can write a manifest along these
lines:

     (use-modules (guix packages)
                  (gnu packages gdb)               ;for 'gdb'
                  (gnu packages version-control))  ;for 'git'

     ;; Define a variant of GDB without a dependency on Guile.
     (define gdb-sans-guile
       (package
         (inherit gdb)
         (inputs (modify-inputs (package-inputs gdb)
                   (delete "guile")))))

     ;; Return a manifest containing that one package plus Git.
     (packages->manifest (list gdb-sans-guile git))

   Note that in this example, the manifest directly refers to the ‘gdb’
and ‘git’ variables, which are bound to a ‘package’ object (*note
package Reference::), instead of calling ‘specifications->manifest’ to
look up packages by name as we did before.  The ‘use-modules’ form at
the top lets us access the core package interface (*note Defining
Packages::) and the modules that define ‘gdb’ and ‘git’ (*note Package
Modules::).  Seamlessly, we’re weaving all this together—the
possibilities are endless, unleash your creativity!

   The data type for manifests as well as supporting procedures are
defined in the ‘(guix profiles)’ module, which is automatically
available to code passed to ‘-m’.  The reference follows.

 -- Data Type: manifest
     Data type representing a manifest.

     It currently has one field:

     ‘entries’
          This must be a list of ‘manifest-entry’ records—see below.

 -- Data Type: manifest-entry
     Data type representing a manifest entry.  A manifest entry contains
     essential metadata: a name and version string, the object (usually
     a package) for that entry, the desired output (*note Packages with
     Multiple Outputs::), and a number of optional pieces of information
     detailed below.

     Most of the time, you won’t build a manifest entry directly;
     instead, you will pass a package to ‘package->manifest-entry’,
     described below.  In some unusual cases though, you might want to
     create manifest entries for things that are _not_ packages, as in
     this example:

          ;; Manually build a single manifest entry for a non-package object.
          (let ((hello (program-file "hello" #~(display "Hi!"))))
            (manifest-entry
              (name "foo")
              (version "42")
              (item
               (computed-file "hello-directory"
                               #~(let ((bin (string-append #$output "/bin")))
                                   (mkdir #$output) (mkdir bin)
                                    (symlink #$hello
                                             (string-append bin "/hello")))))))

     The available fields are the following:

     ‘name’
     ‘version’
          Name and version string for this entry.

     ‘item’
          A package or other file-like object (*note file-like objects:
          G-Expressions.).

     ‘output’ (default: ‘"out"’)
          Output of ‘item’ to use, in case ‘item’ has multiple outputs
          (*note Packages with Multiple Outputs::).

     ‘dependencies’ (default: ‘'()’)
          List of manifest entries this entry depends on.  When building
          a profile, dependencies are added to the profile.

          Typically, the propagated inputs of a package (*note
          ‘propagated-inputs’: package Reference.) end up having a
          corresponding manifest entry in among the dependencies of the
          package’s own manifest entry.

     ‘search-paths’ (default: ‘'()’)
          The list of search path specifications honored by this entry
          (*note Search Paths::).

     ‘properties’ (default: ‘'()’)
          List of symbol/value pairs.  When building a profile, those
          properties get serialized.

          This can be used to piggyback additional metadata—e.g., the
          transformations applied to a package (*note Package
          Transformation Options::).

     ‘parent’ (default: ‘(delay #f)’)
          A promise pointing to the “parent” manifest entry.

          This is used as a hint to provide context when reporting an
          error related to a manifest entry coming from a ‘dependencies’
          field.

 -- Scheme Procedure: concatenate-manifests LST
     Concatenate the manifests listed in LST and return the resulting
     manifest.

 -- Scheme Procedure: package->manifest-entry PACKAGE [OUTPUT]
          [#:properties]
     Return a manifest entry for the OUTPUT of package PACKAGE, where
     OUTPUT defaults to ‘"out"’, and with the given PROPERTIES.  By
     default PROPERTIES is the empty list or, if one or more package
     transformations were applied to PACKAGE, it is an association list
     representing those transformations, suitable as an argument to
     ‘options->transformation’ (*note ‘options->transformation’:
     Defining Package Variants.).

     The code snippet below builds a manifest with an entry for the
     default output and the ‘send-email’ output of the ‘git’ package:

          (use-modules (gnu packages version-control))

          (manifest (list (package->manifest-entry git)
                          (package->manifest-entry git "send-email")))

 -- Scheme Procedure: packages->manifest PACKAGES
     Return a list of manifest entries, one for each item listed in
     PACKAGES.  Elements of PACKAGES can be either package objects or
     package/string tuples denoting a specific output of a package.

     Using this procedure, the manifest above may be rewritten more
     concisely:

          (use-modules (gnu packages version-control))

          (packages->manifest (list git `(,git "send-email")))

 -- Scheme Procedure: package->development-manifest PACKAGE [SYSTEM]
          [#:target]
     Return a manifest for the “development inputs” of PACKAGE for
     SYSTEM, optionally when cross-compiling to TARGET.  Development
     inputs include both explicit and implicit inputs of PACKAGE.

     Like the ‘-D’ option of ‘guix shell’ (*note ‘guix shell -D’:
     shell-development-option.), the resulting manifest describes the
     environment in which one can develop PACKAGE.  For example, suppose
     you’re willing to set up a development environment for Inkscape,
     with the addition of Git for version control; you can describe that
     “bill of materials” with the following manifest:

          (use-modules (gnu packages inkscape)          ;for 'inkscape'
                       (gnu packages version-control))  ;for 'git'

          (concatenate-manifests
           (list (package->development-manifest inkscape)
                 (packages->manifest (list git))))

     In this example, the development manifest that
     ‘package->development-manifest’ returns includes the compiler
     (GCC), the many supporting libraries (Boost, GLib, GTK, etc.), and
     a couple of additional development tools—these are the dependencies
     ‘guix show inkscape’ lists.

   Last, the ‘(gnu packages)’ module provides higher-level facilities to
build manifests.  In particular, it lets you look up packages by
name—see below.

 -- Scheme Procedure: specifications->manifest SPECS
     Given SPECS, a list of specifications such as ‘"emacs@25.2"’ or
     ‘"guile:debug"’, return a manifest.  Specs have the format that
     command-line tools such as ‘guix install’ and ‘guix package’
     understand (*note Invoking guix package::).

     As an example, it lets you rewrite the Git manifest that we saw
     earlier like this:

          (specifications->manifest '("git" "git:send-email"))

     Notice that we do not need to worry about ‘use-modules’, importing
     the right set of modules, and referring to the right variables.
     Instead, we directly refer to packages in the same way as on the
     command line, which can often be more convenient.


File: guix.info,  Node: Build Systems,  Next: Build Phases,  Prev: Writing Manifests,  Up: Programming Interface

9.5 Build Systems
=================

Each package definition specifies a “build system” and arguments for
that build system (*note Defining Packages::).  This ‘build-system’
field represents the build procedure of the package, as well as implicit
dependencies of that build procedure.

   Build systems are ‘<build-system>’ objects.  The interface to create
and manipulate them is provided by the ‘(guix build-system)’ module, and
actual build systems are exported by specific modules.

   Under the hood, build systems first compile package objects to
“bags”.  A “bag” is like a package, but with less ornamentation—in other
words, a bag is a lower-level representation of a package, which
includes all the inputs of that package, including some that were
implicitly added by the build system.  This intermediate representation
is then compiled to a derivation (*note Derivations::).  The
‘package-with-c-toolchain’ is an example of a way to change the implicit
inputs that a package’s build system pulls in (*note
‘package-with-c-toolchain’: package Reference.).

   Build systems accept an optional list of “arguments”.  In package
definitions, these are passed via the ‘arguments’ field (*note Defining
Packages::).  They are typically keyword arguments (*note keyword
arguments in Guile: (guile)Optional Arguments.).  The value of these
arguments is usually evaluated in the “build stratum”—i.e., by a Guile
process launched by the daemon (*note Derivations::).

   The main build system is ‘gnu-build-system’, which implements the
standard build procedure for GNU and many other packages.  It is
provided by the ‘(guix build-system gnu)’ module.

 -- Scheme Variable: gnu-build-system
     ‘gnu-build-system’ represents the GNU Build System, and variants
     thereof (*note configuration and makefile conventions:
     (standards)Configuration.).

     In a nutshell, packages using it are configured, built, and
     installed with the usual ‘./configure && make && make check && make
     install’ command sequence.  In practice, a few additional steps are
     often needed.  All these steps are split up in separate “phases”.
     *Note Build Phases::, for more info on build phases and ways to
     customize them.

     In addition, this build system ensures that the “standard”
     environment for GNU packages is available.  This includes tools
     such as GCC, libc, Coreutils, Bash, Make, Diffutils, grep, and sed
     (see the ‘(guix build-system gnu)’ module for a complete list).  We
     call these the “implicit inputs” of a package, because package
     definitions do not have to mention them.

     This build system supports a number of keyword arguments, which can
     be passed via the ‘arguments’ field of a package.  Here are some of
     the main parameters:

     ‘#:phases’
          This argument specifies build-side code that evaluates to an
          alist of build phases.  *Note Build Phases::, for more
          information.

     ‘#:configure-flags’
          This is a list of flags (strings) passed to the ‘configure’
          script.  *Note Defining Packages::, for an example.

     ‘#:make-flags’
          This list of strings contains flags passed as arguments to
          ‘make’ invocations in the ‘build’, ‘check’, and ‘install’
          phases.

     ‘#:out-of-source?’
          This Boolean, ‘#f’ by default, indicates whether to run builds
          in a build directory separate from the source tree.

          When it is true, the ‘configure’ phase creates a separate
          build directory, changes to that directory, and runs the
          ‘configure’ script from there.  This is useful for packages
          that require it, such as ‘glibc’.

     ‘#:tests?’
          This Boolean, ‘#t’ by default, indicates whether the ‘check’
          phase should run the package’s test suite.

     ‘#:test-target’
          This string, ‘"check"’ by default, gives the name of the
          makefile target used by the ‘check’ phase.

     ‘#:parallel-build?’
     ‘#:parallel-tests?’
          These Boolean values specify whether to build, respectively
          run the test suite, in parallel, with the ‘-j’ flag of ‘make’.
          When they are true, ‘make’ is passed ‘-jN’, where N is the
          number specified as the ‘--cores’ option of ‘guix-daemon’ or
          that of the ‘guix’ client command (*note ‘--cores’: Common
          Build Options.).

     ‘#:validate-runpath?’
          This Boolean, ‘#t’ by default, determines whether to
          “validate” the ‘RUNPATH’ of ELF binaries (‘.so’ shared
          libraries as well as executables) previously installed by the
          ‘install’ phase.  *Note the ‘validate-runpath’ phase:
          phase-validate-runpath, for details.

     ‘#:substitutable?’
          This Boolean, ‘#t’ by default, tells whether the package
          outputs should be substitutable—i.e., whether users should be
          able to obtain substitutes for them instead of building
          locally (*note Substitutes::).

     ‘#:allowed-references’
     ‘#:disallowed-references’
          When true, these arguments must be a list of dependencies that
          must not appear among the references of the build results.
          If, upon build completion, some of these references are
          retained, the build process fails.

          This is useful to ensure that a package does not erroneously
          keep a reference to some of it build-time inputs, in cases
          where doing so would, for example, unnecessarily increase its
          size (*note Invoking guix size::).

     Most other build systems support these keyword arguments.

   Other ‘<build-system>’ objects are defined to support other
conventions and tools used by free software packages.  They inherit most
of ‘gnu-build-system’, and differ mainly in the set of inputs implicitly
added to the build process, and in the list of phases executed.  Some of
these build systems are listed below.

 -- Scheme Variable: ant-build-system
     This variable is exported by ‘(guix build-system ant)’.  It
     implements the build procedure for Java packages that can be built
     with Ant build tool (https://ant.apache.org/).

     It adds both ‘ant’ and the “Java Development Kit” (JDK) as provided
     by the ‘icedtea’ package to the set of inputs.  Different packages
     can be specified with the ‘#:ant’ and ‘#:jdk’ parameters,
     respectively.

     When the original package does not provide a suitable Ant build
     file, the parameter ‘#:jar-name’ can be used to generate a minimal
     Ant build file ‘build.xml’ with tasks to build the specified jar
     archive.  In this case the parameter ‘#:source-dir’ can be used to
     specify the source sub-directory, defaulting to “src”.

     The ‘#:main-class’ parameter can be used with the minimal ant
     buildfile to specify the main class of the resulting jar.  This
     makes the jar file executable.  The ‘#:test-include’ parameter can
     be used to specify the list of junit tests to run.  It defaults to
     ‘(list "**/*Test.java")’.  The ‘#:test-exclude’ can be used to
     disable some tests.  It defaults to ‘(list "**/Abstract*.java")’,
     because abstract classes cannot be run as tests.

     The parameter ‘#:build-target’ can be used to specify the Ant task
     that should be run during the ‘build’ phase.  By default the “jar”
     task will be run.

 -- Scheme Variable: android-ndk-build-system
     This variable is exported by ‘(guix build-system android-ndk)’.  It
     implements a build procedure for Android NDK (native development
     kit) packages using a Guix-specific build process.

     The build system assumes that packages install their public
     interface (header) files to the subdirectory ‘include’ of the ‘out’
     output and their libraries to the subdirectory ‘lib’ the ‘out’
     output.

     It’s also assumed that the union of all the dependencies of a
     package has no conflicting files.

     For the time being, cross-compilation is not supported - so right
     now the libraries and header files are assumed to be host tools.

 -- Scheme Variable: asdf-build-system/source
 -- Scheme Variable: asdf-build-system/sbcl
 -- Scheme Variable: asdf-build-system/ecl

     These variables, exported by ‘(guix build-system asdf)’, implement
     build procedures for Common Lisp packages using “ASDF”
     (https://common-lisp.net/project/asdf/).  ASDF is a system
     definition facility for Common Lisp programs and libraries.

     The ‘asdf-build-system/source’ system installs the packages in
     source form, and can be loaded using any common lisp
     implementation, via ASDF.  The others, such as
     ‘asdf-build-system/sbcl’, install binary systems in the format
     which a particular implementation understands.  These build systems
     can also be used to produce executable programs, or lisp images
     which contain a set of packages pre-loaded.

     The build system uses naming conventions.  For binary packages, the
     package name should be prefixed with the lisp implementation, such
     as ‘sbcl-’ for ‘asdf-build-system/sbcl’.

     Additionally, the corresponding source package should be labeled
     using the same convention as python packages (see *note Python
     Modules::), using the ‘cl-’ prefix.

     In order to create executable programs and images, the build-side
     procedures ‘build-program’ and ‘build-image’ can be used.  They
     should be called in a build phase after the
     ‘create-asdf-configuration’ phase, so that the system which was
     just built can be used within the resulting image.  ‘build-program’
     requires a list of Common Lisp expressions to be passed as the
     ‘#:entry-program’ argument.

     By default, all the ‘.asd’ files present in the sources are read to
     find system definitions.  The ‘#:asd-files’ parameter can be used
     to specify the list of ‘.asd’ files to read.  Furthermore, if the
     package defines a system for its tests in a separate file, it will
     be loaded before the tests are run if it is specified by the
     ‘#:test-asd-file’ parameter.  If it is not set, the files
     ‘<system>-tests.asd’, ‘<system>-test.asd’, ‘tests.asd’, and
     ‘test.asd’ will be tried if they exist.

     If for some reason the package must be named in a different way
     than the naming conventions suggest, or if several systems must be
     compiled, the ‘#:asd-systems’ parameter can be used to specify the
     list of system names.

 -- Scheme Variable: cargo-build-system
     This variable is exported by ‘(guix build-system cargo)’.  It
     supports builds of packages using Cargo, the build tool of the Rust
     programming language (https://www.rust-lang.org).

     It adds ‘rustc’ and ‘cargo’ to the set of inputs.  A different Rust
     package can be specified with the ‘#:rust’ parameter.

     Regular cargo dependencies should be added to the package
     definition similarly to other packages; those needed only at build
     time to native-inputs, others to inputs.  If you need to add
     source-only crates then you should add them to via the
     ‘#:cargo-inputs’ parameter as a list of name and spec pairs, where
     the spec can be a package or a source definition.  Note that the
     spec must evaluate to a path to a gzipped tarball which includes a
     ‘Cargo.toml’ file at its root, or it will be ignored.  Similarly,
     cargo dev-dependencies should be added to the package definition
     via the ‘#:cargo-development-inputs’ parameter.

     In its ‘configure’ phase, this build system will make any source
     inputs specified in the ‘#:cargo-inputs’ and
     ‘#:cargo-development-inputs’ parameters available to cargo.  It
     will also remove an included ‘Cargo.lock’ file to be recreated by
     ‘cargo’ during the ‘build’ phase.  The ‘package’ phase will run
     ‘cargo package’ to create a source crate for future use.  The
     ‘install’ phase installs the binaries defined by the crate.  Unless
     ‘install-source? #f’ is defined it will also install a source crate
     repository of itself and unpacked sources, to ease in future
     hacking on rust packages.

 -- Scheme Variable: chicken-build-system
     This variable is exported by ‘(guix build-system chicken)’.  It
     builds CHICKEN Scheme (https://call-cc.org/) modules, also called
     “eggs” or “extensions”.  CHICKEN generates C source code, which
     then gets compiled by a C compiler, in this case GCC.

     This build system adds ‘chicken’ to the package inputs, as well as
     the packages of ‘gnu-build-system’.

     The build system can’t (yet) deduce the egg’s name automatically,
     so just like with ‘go-build-system’ and its ‘#:import-path’, you
     should define ‘#:egg-name’ in the package’s ‘arguments’ field.

     For example, if you are packaging the ‘srfi-1’ egg:

          (arguments '(#:egg-name "srfi-1"))

     Egg dependencies must be defined in ‘propagated-inputs’, not
     ‘inputs’ because CHICKEN doesn’t embed absolute references in
     compiled eggs.  Test dependencies should go to ‘native-inputs’, as
     usual.

 -- Scheme Variable: copy-build-system
     This variable is exported by ‘(guix build-system copy)’.  It
     supports builds of simple packages that don’t require much
     compiling, mostly just moving files around.

     It adds much of the ‘gnu-build-system’ packages to the set of
     inputs.  Because of this, the ‘copy-build-system’ does not require
     all the boilerplate code often needed for the
     ‘trivial-build-system’.

     To further simplify the file installation process, an
     ‘#:install-plan’ argument is exposed to let the packager specify
     which files go where.  The install plan is a list of ‘(SOURCE
     TARGET [FILTERS])’.  FILTERS are optional.

        • When SOURCE matches a file or directory without trailing
          slash, install it to TARGET.
             • If TARGET has a trailing slash, install SOURCE basename
               beneath TARGET.
             • Otherwise install SOURCE as TARGET.

        • When SOURCE is a directory with a trailing slash, or when
          FILTERS are used, the trailing slash of TARGET is implied with
          the same meaning as above.
             • Without FILTERS, install the full SOURCE _content_ to
               TARGET.
             • With FILTERS among ‘#:include’, ‘#:include-regexp’,
               ‘#:exclude’, ‘#:exclude-regexp’, only select files are
               installed depending on the filters.  Each filters is
               specified by a list of strings.
                  • With ‘#:include’, install all the files which the
                    path suffix matches at least one of the elements in
                    the given list.
                  • With ‘#:include-regexp’, install all the files which
                    the subpaths match at least one of the regular
                    expressions in the given list.
                  • The ‘#:exclude’ and ‘#:exclude-regexp’ filters are
                    the complement of their inclusion counterpart.
                    Without ‘#:include’ flags, install all files but
                    those matching the exclusion filters.  If both
                    inclusions and exclusions are specified, the
                    exclusions are done on top of the inclusions.
          In all cases, the paths relative to SOURCE are preserved
          within TARGET.

     Examples:

        • ‘("foo/bar" "share/my-app/")’: Install ‘bar’ to
          ‘share/my-app/bar’.
        • ‘("foo/bar" "share/my-app/baz")’: Install ‘bar’ to
          ‘share/my-app/baz’.
        • ‘("foo/" "share/my-app")’: Install the content of ‘foo’ inside
          ‘share/my-app’, e.g., install ‘foo/sub/file’ to
          ‘share/my-app/sub/file’.
        • ‘("foo/" "share/my-app" #:include ("sub/file"))’: Install only
          ‘foo/sub/file’ to ‘share/my-app/sub/file’.
        • ‘("foo/sub" "share/my-app" #:include ("file"))’: Install
          ‘foo/sub/file’ to ‘share/my-app/file’.

 -- Scheme Variable: clojure-build-system
     This variable is exported by ‘(guix build-system clojure)’.  It
     implements a simple build procedure for Clojure
     (https://clojure.org/) packages using plain old ‘compile’ in
     Clojure.  Cross-compilation is not supported yet.

     It adds ‘clojure’, ‘icedtea’ and ‘zip’ to the set of inputs.
     Different packages can be specified with the ‘#:clojure’, ‘#:jdk’
     and ‘#:zip’ parameters, respectively.

     A list of source directories, test directories and jar names can be
     specified with the ‘#:source-dirs’, ‘#:test-dirs’ and ‘#:jar-names’
     parameters, respectively.  Compile directory and main class can be
     specified with the ‘#:compile-dir’ and ‘#:main-class’ parameters,
     respectively.  Other parameters are documented below.

     This build system is an extension of ‘ant-build-system’, but with
     the following phases changed:

     ‘build’
          This phase calls ‘compile’ in Clojure to compile source files
          and runs ‘jar’ to create jars from both source files and
          compiled files according to the include list and exclude list
          specified in ‘#:aot-include’ and ‘#:aot-exclude’,
          respectively.  The exclude list has priority over the include
          list.  These lists consist of symbols representing Clojure
          libraries or the special keyword ‘#:all’ representing all
          Clojure libraries found under the source directories.  The
          parameter ‘#:omit-source?’ decides if source should be
          included into the jars.

     ‘check’
          This phase runs tests according to the include list and
          exclude list specified in ‘#:test-include’ and
          ‘#:test-exclude’, respectively.  Their meanings are analogous
          to that of ‘#:aot-include’ and ‘#:aot-exclude’, except that
          the special keyword ‘#:all’ now stands for all Clojure
          libraries found under the test directories.  The parameter
          ‘#:tests?’ decides if tests should be run.

     ‘install’
          This phase installs all jars built previously.

     Apart from the above, this build system also contains an additional
     phase:

     ‘install-doc’
          This phase installs all top-level files with base name
          matching ‘%doc-regex’.  A different regex can be specified
          with the ‘#:doc-regex’ parameter.  All files (recursively)
          inside the documentation directories specified in ‘#:doc-dirs’
          are installed as well.

 -- Scheme Variable: cmake-build-system
     This variable is exported by ‘(guix build-system cmake)’.  It
     implements the build procedure for packages using the CMake build
     tool (https://www.cmake.org).

     It automatically adds the ‘cmake’ package to the set of inputs.
     Which package is used can be specified with the ‘#:cmake’
     parameter.

     The ‘#:configure-flags’ parameter is taken as a list of flags
     passed to the ‘cmake’ command.  The ‘#:build-type’ parameter
     specifies in abstract terms the flags passed to the compiler; it
     defaults to ‘"RelWithDebInfo"’ (short for “release mode with
     debugging information”), which roughly means that code is compiled
     with ‘-O2 -g’, as is the case for Autoconf-based packages by
     default.

 -- Scheme Variable: dune-build-system
     This variable is exported by ‘(guix build-system dune)’.  It
     supports builds of packages using Dune (https://dune.build/), a
     build tool for the OCaml programming language.  It is implemented
     as an extension of the ‘ocaml-build-system’ which is described
     below.  As such, the ‘#:ocaml’ and ‘#:findlib’ parameters can be
     passed to this build system.

     It automatically adds the ‘dune’ package to the set of inputs.
     Which package is used can be specified with the ‘#:dune’ parameter.

     There is no ‘configure’ phase because dune packages typically don’t
     need to be configured.  The ‘#:build-flags’ parameter is taken as a
     list of flags passed to the ‘dune’ command during the build.

     The ‘#:jbuild?’ parameter can be passed to use the ‘jbuild’ command
     instead of the more recent ‘dune’ command while building a package.
     Its default value is ‘#f’.

     The ‘#:package’ parameter can be passed to specify a package name,
     which is useful when a package contains multiple packages and you
     want to build only one of them.  This is equivalent to passing the
     ‘-p’ argument to ‘dune’.

 -- Scheme variable: elm-build-system
     This variable is exported by ‘(guix build-system elm)’.  It
     implements a build procedure for Elm (https://elm-lang.org)
     packages similar to ‘elm install’.

     The build system adds an Elm compiler package to the set of inputs.
     The default compiler package (currently ‘elm-sans-reactor’) can be
     overridden using the ‘#:elm’ argument.  Additionally, Elm packages
     needed by the build system itself are added as implicit inputs if
     they are not already present: to suppress this behavior, use the
     ‘#:implicit-elm-package-inputs?’ argument, which is primarily
     useful for bootstrapping.

     The ‘"dependencies"’ and ‘"test-dependencies"’ in an Elm package’s
     ‘elm.json’ file correspond to ‘propagated-inputs’ and ‘inputs’,
     respectively.

     Elm requires a particular structure for package names: *note Elm
     Packages:: for more details, including utilities provided by ‘(guix
     build-system elm)’.

     There are currently a few noteworthy limitations to
     ‘elm-build-system’:

        • The build system is focused on “packages” in the Elm sense of
          the word: Elm “projects” which declare ‘{ "type": "package" }’
          in their ‘elm.json’ files.  Using ‘elm-build-system’ to build
          Elm “applications” (which declare ‘{ "type": "application" }’)
          is possible, but requires ad-hoc modifications to the build
          phases.  For examples, see the definitions of the
          ‘elm-todomvc’ example application and the ‘elm’ package itself
          (because the front-end for the ‘elm reactor’ command is an Elm
          application).

        • Elm supports multiple versions of a package coexisting
          simultaneously under ‘ELM_HOME’, but this does not yet work
          well with ‘elm-build-system’.  This limitation primarily
          affects Elm applications, because they specify exact versions
          for their dependencies, whereas Elm packages specify supported
          version ranges.  As a workaround, the example applications
          mentioned above use the ‘patch-application-dependencies’
          procedure provided by ‘(guix build elm-build-system)’ to
          rewrite their ‘elm.json’ files to refer to the package
          versions actually present in the build environment.
          Alternatively, Guix package transformations (*note Defining
          Package Variants::) could be used to rewrite an application’s
          entire dependency graph.

        • We are not yet able to run tests for Elm projects because
          neither ‘elm-test-rs’
          (https://github.com/mpizenberg/elm-test-rs) nor the
          Node.js-based ‘elm-test’
          (https://github.com/rtfeldman/node-test-runner) runner has
          been packaged for Guix yet.

 -- Scheme Variable: go-build-system
     This variable is exported by ‘(guix build-system go)’.  It
     implements a build procedure for Go packages using the standard Go
     build mechanisms
     (https://golang.org/cmd/go/#hdr-Compile_packages_and_dependencies).

     The user is expected to provide a value for the key ‘#:import-path’
     and, in some cases, ‘#:unpack-path’.  The import path
     (https://golang.org/doc/code.html#ImportPaths) corresponds to the
     file system path expected by the package’s build scripts and any
     referring packages, and provides a unique way to refer to a Go
     package.  It is typically based on a combination of the package
     source code’s remote URI and file system hierarchy structure.  In
     some cases, you will need to unpack the package’s source code to a
     different directory structure than the one indicated by the import
     path, and ‘#:unpack-path’ should be used in such cases.

     Packages that provide Go libraries should install their source code
     into the built output.  The key ‘#:install-source?’, which defaults
     to ‘#t’, controls whether or not the source code is installed.  It
     can be set to ‘#f’ for packages that only provide executable files.

     Packages can be cross-built, and if a specific architecture or
     operating system is desired then the keywords ‘#:goarch’ and
     ‘#:goos’ can be used to force the package to be built for that
     architecture and operating system.  The combinations known to Go
     can be found in their documentation
     ("https://golang.org/doc/install/source#environment").

 -- Scheme Variable: glib-or-gtk-build-system
     This variable is exported by ‘(guix build-system glib-or-gtk)’.  It
     is intended for use with packages making use of GLib or GTK+.

     This build system adds the following two phases to the ones defined
     by ‘gnu-build-system’:

     ‘glib-or-gtk-wrap’
          The phase ‘glib-or-gtk-wrap’ ensures that programs in ‘bin/’
          are able to find GLib “schemas” and GTK+ modules
          (https://developer.gnome.org/gtk3/stable/gtk-running.html).
          This is achieved by wrapping the programs in launch scripts
          that appropriately set the ‘XDG_DATA_DIRS’ and ‘GTK_PATH’
          environment variables.

          It is possible to exclude specific package outputs from that
          wrapping process by listing their names in the
          ‘#:glib-or-gtk-wrap-excluded-outputs’ parameter.  This is
          useful when an output is known not to contain any GLib or GTK+
          binaries, and where wrapping would gratuitously add a
          dependency of that output on GLib and GTK+.

     ‘glib-or-gtk-compile-schemas’
          The phase ‘glib-or-gtk-compile-schemas’ makes sure that all
          GSettings schemas
          (https://developer.gnome.org/gio/stable/glib-compile-schemas.html)
          of GLib are compiled.  Compilation is performed by the
          ‘glib-compile-schemas’ program.  It is provided by the package
          ‘glib:bin’ which is automatically imported by the build
          system.  The ‘glib’ package providing ‘glib-compile-schemas’
          can be specified with the ‘#:glib’ parameter.

     Both phases are executed after the ‘install’ phase.

 -- Scheme Variable: guile-build-system
     This build system is for Guile packages that consist exclusively of
     Scheme code and that are so lean that they don’t even have a
     makefile, let alone a ‘configure’ script.  It compiles Scheme code
     using ‘guild compile’ (*note (guile)Compilation::) and installs the
     ‘.scm’ and ‘.go’ files in the right place.  It also installs
     documentation.

     This build system supports cross-compilation by using the
     ‘--target’ option of ‘guild compile’.

     Packages built with ‘guile-build-system’ must provide a Guile
     package in their ‘native-inputs’ field.

 -- Scheme Variable: julia-build-system
     This variable is exported by ‘(guix build-system julia)’.  It
     implements the build procedure used by julia
     (https://julialang.org/) packages, which essentially is similar to
     running ‘julia -e 'using Pkg; Pkg.add(package)'’ in an environment
     where ‘JULIA_LOAD_PATH’ contains the paths to all Julia package
     inputs.  Tests are run by calling ‘/test/runtests.jl’.

     The Julia package name and uuid is read from the file
     ‘Project.toml’.  These values can be overridden by passing the
     argument ‘#:julia-package-name’ (which must be correctly
     capitalized) or ‘#:julia-package-uuid’.

     Julia packages usually manage their binary dependencies via
     ‘JLLWrappers.jl’, a Julia package that creates a module (named
     after the wrapped library followed by ‘_jll.jl’.

     To add the binary path ‘_jll.jl’ packages, you need to patch the
     files under ‘src/wrappers/’, replacing the call to the macro
     ‘JLLWrappers.@generate_wrapper_header’, adding as a second argument
     containing the store path the binary.

     As an example, in the MbedTLS Julia package, we add a build phase
     (*note Build Phases::) to insert the absolute file name of the
     wrapped MbedTLS package:

          (add-after 'unpack 'override-binary-path
            (lambda* (#:key inputs #:allow-other-keys)
              (for-each (lambda (wrapper)
                          (substitute* wrapper
                            (("generate_wrapper_header.*")
                             (string-append
                              "generate_wrapper_header(\"MbedTLS\", \""
                              (assoc-ref inputs "mbedtls-apache") "\")\n"))))
                        ;; There's a Julia file for each platform, override them all.
                        (find-files "src/wrappers/" "\\.jl$"))))

     Some older packages that aren’t using ‘Project.toml’ yet, will
     require this file to be created, too.  It is internally done if the
     arguments ‘#:julia-package-name’ and ‘#:julia-package-uuid’ are
     provided.

 -- Scheme Variable: maven-build-system
     This variable is exported by ‘(guix build-system maven)’.  It
     implements a build procedure for Maven (https://maven.apache.org)
     packages.  Maven is a dependency and lifecycle management tool for
     Java.  A user of Maven specifies dependencies and plugins in a
     ‘pom.xml’ file that Maven reads.  When Maven does not have one of
     the dependencies or plugins in its repository, it will download
     them and use them to build the package.

     The maven build system ensures that maven will not try to download
     any dependency by running in offline mode.  Maven will fail if a
     dependency is missing.  Before running Maven, the ‘pom.xml’ (and
     subprojects) are modified to specify the version of dependencies
     and plugins that match the versions available in the guix build
     environment.  Dependencies and plugins must be installed in the
     fake maven repository at ‘lib/m2’, and are symlinked into a proper
     repository before maven is run.  Maven is instructed to use that
     repository for the build and installs built artifacts there.
     Changed files are copied to the ‘lib/m2’ directory of the package
     output.

     You can specify a ‘pom.xml’ file with the ‘#:pom-file’ argument, or
     let the build system use the default ‘pom.xml’ file in the sources.

     In case you need to specify a dependency’s version manually, you
     can use the ‘#:local-packages’ argument.  It takes an association
     list where the key is the groupId of the package and its value is
     an association list where the key is the artifactId of the package
     and its value is the version you want to override in the ‘pom.xml’.

     Some packages use dependencies or plugins that are not useful at
     runtime nor at build time in Guix.  You can alter the ‘pom.xml’
     file to remove them using the ‘#:exclude’ argument.  Its value is
     an association list where the key is the groupId of the plugin or
     dependency you want to remove, and the value is a list of
     artifactId you want to remove.

     You can override the default ‘jdk’ and ‘maven’ packages with the
     corresponding argument, ‘#:jdk’ and ‘#:maven’.

     The ‘#:maven-plugins’ argument is a list of maven plugins used
     during the build, with the same format as the ‘inputs’ fields of
     the package declaration.  Its default value is
     ‘(default-maven-plugins)’ which is also exported.

 -- Scheme Variable: minetest-mod-build-system
     This variable is exported by ‘(guix build-system minetest)’.  It
     implements a build procedure for Minetest
     (https://www.minetest.net) mods, which consists of copying Lua
     code, images and other resources to the location Minetest searches
     for mods.  The build system also minimises PNG images and verifies
     that Minetest can load the mod without errors.

 -- Scheme Variable: minify-build-system
     This variable is exported by ‘(guix build-system minify)’.  It
     implements a minification procedure for simple JavaScript packages.

     It adds ‘uglify-js’ to the set of inputs and uses it to compress
     all JavaScript files in the ‘src’ directory.  A different minifier
     package can be specified with the ‘#:uglify-js’ parameter, but it
     is expected that the package writes the minified code to the
     standard output.

     When the input JavaScript files are not all located in the ‘src’
     directory, the parameter ‘#:javascript-files’ can be used to
     specify a list of file names to feed to the minifier.

 -- Scheme Variable: ocaml-build-system
     This variable is exported by ‘(guix build-system ocaml)’.  It
     implements a build procedure for OCaml (https://ocaml.org)
     packages, which consists of choosing the correct set of commands to
     run for each package.  OCaml packages can expect many different
     commands to be run.  This build system will try some of them.

     When the package has a ‘setup.ml’ file present at the top-level, it
     will run ‘ocaml setup.ml -configure’, ‘ocaml setup.ml -build’ and
     ‘ocaml setup.ml -install’.  The build system will assume that this
     file was generated by OASIS (http://oasis.forge.ocamlcore.org/) and
     will take care of setting the prefix and enabling tests if they are
     not disabled.  You can pass configure and build flags with the
     ‘#:configure-flags’ and ‘#:build-flags’.  The ‘#:test-flags’ key
     can be passed to change the set of flags used to enable tests.  The
     ‘#:use-make?’ key can be used to bypass this system in the build
     and install phases.

     When the package has a ‘configure’ file, it is assumed that it is a
     hand-made configure script that requires a different argument
     format than in the ‘gnu-build-system’.  You can add more flags with
     the ‘#:configure-flags’ key.

     When the package has a ‘Makefile’ file (or ‘#:use-make?’ is ‘#t’),
     it will be used and more flags can be passed to the build and
     install phases with the ‘#:make-flags’ key.

     Finally, some packages do not have these files and use a somewhat
     standard location for its build system.  In that case, the build
     system will run ‘ocaml pkg/pkg.ml’ or ‘ocaml pkg/build.ml’ and take
     care of providing the path to the required findlib module.
     Additional flags can be passed via the ‘#:build-flags’ key.
     Install is taken care of by ‘opam-installer’.  In this case, the
     ‘opam’ package must be added to the ‘native-inputs’ field of the
     package definition.

     Note that most OCaml packages assume they will be installed in the
     same directory as OCaml, which is not what we want in guix.  In
     particular, they will install ‘.so’ files in their module’s
     directory, which is usually fine because it is in the OCaml
     compiler directory.  In guix though, these libraries cannot be
     found and we use ‘CAML_LD_LIBRARY_PATH’.  This variable points to
     ‘lib/ocaml/site-lib/stubslibs’ and this is where ‘.so’ libraries
     should be installed.

 -- Scheme Variable: python-build-system
     This variable is exported by ‘(guix build-system python)’.  It
     implements the more or less standard build procedure used by Python
     packages, which consists in running ‘python setup.py build’ and
     then ‘python setup.py install --prefix=/gnu/store/...’.

     For packages that install stand-alone Python programs under ‘bin/’,
     it takes care of wrapping these programs so that their
     ‘GUIX_PYTHONPATH’ environment variable points to all the Python
     libraries they depend on.

     Which Python package is used to perform the build can be specified
     with the ‘#:python’ parameter.  This is a useful way to force a
     package to be built for a specific version of the Python
     interpreter, which might be necessary if the package is only
     compatible with a single interpreter version.

     By default guix calls ‘setup.py’ under control of ‘setuptools’,
     much like ‘pip’ does.  Some packages are not compatible with
     setuptools (and pip), thus you can disable this by setting the
     ‘#:use-setuptools?’ parameter to ‘#f’.

     If a ‘"python"’ output is available, the package is installed into
     it instead of the default ‘"out"’ output.  This is useful for
     packages that include a Python package as only a part of the
     software, and thus want to combine the phases of
     ‘python-build-system’ with another build system.  Python bindings
     are a common usecase.

 -- Scheme Variable: pyproject-build-system
     This is a variable exported by ‘guix build-system pyproject’.  It
     is based on PYTHON-BUILD-SYSTEM, and adds support for
     ‘pyproject.toml’ and PEP 517 (https://peps.python.org/pep-0517/).
     It also supports a variety of build backends and test frameworks.

     The API is slightly different from PYTHON-BUILD-SYSTEM:
        • ‘#:use-setuptools?’ and ‘#:test-target’ is removed.
        • ‘#:build-backend’ is added.  It defaults to ‘#false’ and will
          try to guess the appropriate backend based on
          ‘pyproject.toml’.
        • ‘#:test-backend’ is added.  It defaults to ‘#false’ and will
          guess an appropriate test backend based on what is available
          in package inputs.
        • ‘#:test-flags’ is added.  The default is ‘'()’.  These flags
          are passed as arguments to the test command.  Note that flags
          for verbose output is always enabled on supported backends.

     It is considered “experimental” in that the implementation details
     are not set in stone yet, however users are encouraged to try it
     for new Python projects (even those using ‘setup.py’).  The API is
     subject to change, but any breaking changes in the Guix channel
     will be dealt with.

     Eventually this build system will be deprecated and merged back
     into PYTHON-BUILD-SYSTEM, probably some time in 2024.

 -- Scheme Variable: perl-build-system
     This variable is exported by ‘(guix build-system perl)’.  It
     implements the standard build procedure for Perl packages, which
     either consists in running ‘perl Build.PL --prefix=/gnu/store/...’,
     followed by ‘Build’ and ‘Build install’; or in running ‘perl
     Makefile.PL PREFIX=/gnu/store/...’, followed by ‘make’ and ‘make
     install’, depending on which of ‘Build.PL’ or ‘Makefile.PL’ is
     present in the package distribution.  Preference is given to the
     former if both ‘Build.PL’ and ‘Makefile.PL’ exist in the package
     distribution.  This preference can be reversed by specifying ‘#t’
     for the ‘#:make-maker?’ parameter.

     The initial ‘perl Makefile.PL’ or ‘perl Build.PL’ invocation passes
     flags specified by the ‘#:make-maker-flags’ or
     ‘#:module-build-flags’ parameter, respectively.

     Which Perl package is used can be specified with ‘#:perl’.

 -- Scheme Variable: renpy-build-system
     This variable is exported by ‘(guix build-system renpy)’.  It
     implements the more or less standard build procedure used by Ren’py
     games, which consists of loading ‘#:game’ once, thereby creating
     bytecode for it.

     It further creates a wrapper script in ‘bin/’ and a desktop entry
     in ‘share/applications’, both of which can be used to launch the
     game.

     Which Ren’py package is used can be specified with ‘#:renpy’.
     Games can also be installed in outputs other than “out” by using
     ‘#:output’.

 -- Scheme Variable: qt-build-system
     This variable is exported by ‘(guix build-system qt)’.  It is
     intended for use with applications using Qt or KDE.

     This build system adds the following two phases to the ones defined
     by ‘cmake-build-system’:

     ‘check-setup’
          The phase ‘check-setup’ prepares the environment for running
          the checks as commonly used by Qt test programs.  For now this
          only sets some environment variables:
          ‘QT_QPA_PLATFORM=offscreen’, ‘DBUS_FATAL_WARNINGS=0’ and
          ‘CTEST_OUTPUT_ON_FAILURE=1’.

          This phase is added before the ‘check’ phase.  It’s a separate
          phase to ease adjusting if necessary.

     ‘qt-wrap’
          The phase ‘qt-wrap’ searches for Qt5 plugin paths, QML paths
          and some XDG in the inputs and output.  In case some path is
          found, all programs in the output’s ‘bin/’, ‘sbin/’,
          ‘libexec/’ and ‘lib/libexec/’ directories are wrapped in
          scripts defining the necessary environment variables.

          It is possible to exclude specific package outputs from that
          wrapping process by listing their names in the
          ‘#:qt-wrap-excluded-outputs’ parameter.  This is useful when
          an output is known not to contain any Qt binaries, and where
          wrapping would gratuitously add a dependency of that output on
          Qt, KDE, or such.

          This phase is added after the ‘install’ phase.

 -- Scheme Variable: r-build-system
     This variable is exported by ‘(guix build-system r)’.  It
     implements the build procedure used by R (https://r-project.org)
     packages, which essentially is little more than running ‘R CMD
     INSTALL --library=/gnu/store/...’ in an environment where
     ‘R_LIBS_SITE’ contains the paths to all R package inputs.  Tests
     are run after installation using the R function
     ‘tools::testInstalledPackage’.

 -- Scheme Variable: rakudo-build-system
     This variable is exported by ‘(guix build-system rakudo)’.  It
     implements the build procedure used by Rakudo (https://rakudo.org/)
     for Perl6 (https://perl6.org/) packages.  It installs the package
     to ‘/gnu/store/.../NAME-VERSION/share/perl6’ and installs the
     binaries, library files and the resources, as well as wrap the
     files under the ‘bin/’ directory.  Tests can be skipped by passing
     ‘#f’ to the ‘tests?’ parameter.

     Which rakudo package is used can be specified with ‘rakudo’.  Which
     perl6-tap-harness package used for the tests can be specified with
     ‘#:prove6’ or removed by passing ‘#f’ to the ‘with-prove6?’
     parameter.  Which perl6-zef package used for tests and installing
     can be specified with ‘#:zef’ or removed by passing ‘#f’ to the
     ‘with-zef?’ parameter.

 -- Scheme Variable: rebar-build-system
     This variable is exported by ‘(guix build-system rebar)’.  It
     implements a build procedure around rebar3 (https://rebar3.org), a
     build system for programs written in the Erlang language.

     It adds both ‘rebar3’ and the ‘erlang’ to the set of inputs.
     Different packages can be specified with the ‘#:rebar’ and
     ‘#:erlang’ parameters, respectively.

     This build system is based on ‘gnu-build-system’, but with the
     following phases changed:

     ‘unpack’
          This phase, after unpacking the source like the
          ‘gnu-build-system’ does, checks for a file ‘contents.tar.gz’
          at the top-level of the source.  If this file exists, it will
          be unpacked, too.  This eases handling of package hosted at
          <https://hex.pm/>, the Erlang and Elixir package repository.

     ‘bootstrap’
     ‘configure’
          There are no ‘bootstrap’ and ‘configure’ phase because erlang
          packages typically don’t need to be configured.

     ‘build’
          This phase runs ‘rebar3 compile’ with the flags listed in
          ‘#:rebar-flags’.

     ‘check’
          Unless ‘#:tests? #f’ is passed, this phase runs ‘rebar3
          eunit’, or some other target specified with ‘#:test-target’,
          with the flags listed in ‘#:rebar-flags’,

     ‘install’
          This installs the files created in the default profile, or
          some other profile specified with ‘#:install-profile’.

 -- Scheme Variable: texlive-build-system
     This variable is exported by ‘(guix build-system texlive)’.  It is
     used to build TeX packages in batch mode with a specified engine.
     The build system sets the ‘TEXINPUTS’ variable to find all TeX
     source files in the inputs.

     By default it runs ‘luatex’ on all files ending on ‘ins’.  A
     different engine and format can be specified with the
     ‘#:tex-format’ argument.  Different build targets can be specified
     with the ‘#:build-targets’ argument, which expects a list of file
     names.  The build system adds only ‘texlive-bin’ and
     ‘texlive-latex-base’ (both from ‘(gnu packages tex’) to the inputs.
     Both can be overridden with the arguments ‘#:texlive-bin’ and
     ‘#:texlive-latex-base’, respectively.

     The ‘#:tex-directory’ parameter tells the build system where to
     install the built files under the texmf tree.

 -- Scheme Variable: ruby-build-system
     This variable is exported by ‘(guix build-system ruby)’.  It
     implements the RubyGems build procedure used by Ruby packages,
     which involves running ‘gem build’ followed by ‘gem install’.

     The ‘source’ field of a package that uses this build system
     typically references a gem archive, since this is the format that
     Ruby developers use when releasing their software.  The build
     system unpacks the gem archive, potentially patches the source,
     runs the test suite, repackages the gem, and installs it.
     Additionally, directories and tarballs may be referenced to allow
     building unreleased gems from Git or a traditional source release
     tarball.

     Which Ruby package is used can be specified with the ‘#:ruby’
     parameter.  A list of additional flags to be passed to the ‘gem’
     command can be specified with the ‘#:gem-flags’ parameter.

 -- Scheme Variable: waf-build-system
     This variable is exported by ‘(guix build-system waf)’.  It
     implements a build procedure around the ‘waf’ script.  The common
     phases—‘configure’, ‘build’, and ‘install’—are implemented by
     passing their names as arguments to the ‘waf’ script.

     The ‘waf’ script is executed by the Python interpreter.  Which
     Python package is used to run the script can be specified with the
     ‘#:python’ parameter.

 -- Scheme Variable: scons-build-system
     This variable is exported by ‘(guix build-system scons)’.  It
     implements the build procedure used by the SCons software
     construction tool.  This build system runs ‘scons’ to build the
     package, ‘scons test’ to run tests, and then ‘scons install’ to
     install the package.

     Additional flags to be passed to ‘scons’ can be specified with the
     ‘#:scons-flags’ parameter.  The default build and install targets
     can be overridden with ‘#:build-targets’ and ‘#:install-targets’
     respectively.  The version of Python used to run SCons can be
     specified by selecting the appropriate SCons package with the
     ‘#:scons’ parameter.

 -- Scheme Variable: haskell-build-system
     This variable is exported by ‘(guix build-system haskell)’.  It
     implements the Cabal build procedure used by Haskell packages,
     which involves running ‘runhaskell Setup.hs configure
     --prefix=/gnu/store/...’ and ‘runhaskell Setup.hs build’.  Instead
     of installing the package by running ‘runhaskell Setup.hs install’,
     to avoid trying to register libraries in the read-only compiler
     store directory, the build system uses ‘runhaskell Setup.hs copy’,
     followed by ‘runhaskell Setup.hs register’.  In addition, the build
     system generates the package documentation by running ‘runhaskell
     Setup.hs haddock’, unless ‘#:haddock? #f’ is passed.  Optional
     Haddock parameters can be passed with the help of the
     ‘#:haddock-flags’ parameter.  If the file ‘Setup.hs’ is not found,
     the build system looks for ‘Setup.lhs’ instead.

     Which Haskell compiler is used can be specified with the
     ‘#:haskell’ parameter which defaults to ‘ghc’.

 -- Scheme Variable: dub-build-system
     This variable is exported by ‘(guix build-system dub)’.  It
     implements the Dub build procedure used by D packages, which
     involves running ‘dub build’ and ‘dub run’.  Installation is done
     by copying the files manually.

     Which D compiler is used can be specified with the ‘#:ldc’
     parameter which defaults to ‘ldc’.

 -- Scheme Variable: emacs-build-system
     This variable is exported by ‘(guix build-system emacs)’.  It
     implements an installation procedure similar to the packaging
     system of Emacs itself (*note (emacs)Packages::).

     It first creates the ‘package-autoloads.el’ file, then it byte
     compiles all Emacs Lisp files.  Differently from the Emacs
     packaging system, the Info documentation files are moved to the
     standard documentation directory and the ‘dir’ file is deleted.
     The Elisp package files are installed directly under
     ‘share/emacs/site-lisp’.

 -- Scheme Variable: font-build-system
     This variable is exported by ‘(guix build-system font)’.  It
     implements an installation procedure for font packages where
     upstream provides pre-compiled TrueType, OpenType, etc. font files
     that merely need to be copied into place.  It copies font files to
     standard locations in the output directory.

 -- Scheme Variable: meson-build-system
     This variable is exported by ‘(guix build-system meson)’.  It
     implements the build procedure for packages that use Meson
     (https://mesonbuild.com) as their build system.

     It adds both Meson and Ninja (https://ninja-build.org/) to the set
     of inputs, and they can be changed with the parameters ‘#:meson’
     and ‘#:ninja’ if needed.

     This build system is an extension of ‘gnu-build-system’, but with
     the following phases changed to some specific for Meson:

     ‘configure’
          The phase runs ‘meson’ with the flags specified in
          ‘#:configure-flags’.  The flag ‘--buildtype’ is always set to
          ‘debugoptimized’ unless something else is specified in
          ‘#:build-type’.

     ‘build’
          The phase runs ‘ninja’ to build the package in parallel by
          default, but this can be changed with ‘#:parallel-build?’.

     ‘check’
          The phase runs ‘meson test’ with a base set of options that
          cannot be overridden.  This base set of options can be
          extended via the ‘#:test-options’ argument, for example to
          select or skip a specific test suite.

     ‘install’
          The phase runs ‘ninja install’ and can not be changed.

     Apart from that, the build system also adds the following phases:

     ‘fix-runpath’
          This phase ensures that all binaries can find the libraries
          they need.  It searches for required libraries in
          subdirectories of the package being built, and adds those to
          ‘RUNPATH’ where needed.  It also removes references to
          libraries left over from the build phase by ‘meson’, such as
          test dependencies, that aren’t actually required for the
          program to run.

     ‘glib-or-gtk-wrap’
          This phase is the phase provided by
          ‘glib-or-gtk-build-system’, and it is not enabled by default.
          It can be enabled with ‘#:glib-or-gtk?’.

     ‘glib-or-gtk-compile-schemas’
          This phase is the phase provided by
          ‘glib-or-gtk-build-system’, and it is not enabled by default.
          It can be enabled with ‘#:glib-or-gtk?’.

 -- Scheme Variable: linux-module-build-system
     ‘linux-module-build-system’ allows building Linux kernel modules.

     This build system is an extension of ‘gnu-build-system’, but with
     the following phases changed:

     ‘configure’
          This phase configures the environment so that the Linux
          kernel’s Makefile can be used to build the external kernel
          module.

     ‘build’
          This phase uses the Linux kernel’s Makefile in order to build
          the external kernel module.

     ‘install’
          This phase uses the Linux kernel’s Makefile in order to
          install the external kernel module.

     It is possible and useful to specify the Linux kernel to use for
     building the module (in the ‘arguments’ form of a package using the
     ‘linux-module-build-system’, use the key ‘#:linux’ to specify it).

 -- Scheme Variable: node-build-system
     This variable is exported by ‘(guix build-system node)’.  It
     implements the build procedure used by Node.js
     (https://nodejs.org), which implements an approximation of the ‘npm
     install’ command, followed by an ‘npm test’ command.

     Which Node.js package is used to interpret the ‘npm’ commands can
     be specified with the ‘#:node’ parameter which defaults to ‘node’.

   Lastly, for packages that do not need anything as sophisticated, a
“trivial” build system is provided.  It is trivial in the sense that it
provides basically no support: it does not pull any implicit inputs, and
does not have a notion of build phases.

 -- Scheme Variable: trivial-build-system
     This variable is exported by ‘(guix build-system trivial)’.

     This build system requires a ‘#:builder’ argument.  This argument
     must be a Scheme expression that builds the package output(s)—as
     with ‘build-expression->derivation’ (*note
     ‘build-expression->derivation’: Derivations.).

 -- Scheme Variable: channel-build-system
     This variable is exported by ‘(guix build-system channel)’.

     This build system is meant primarily for internal use.  A package
     using this build system must have a channel specification as its
     ‘source’ field (*note Channels::); alternatively, its source can be
     a directory name, in which case an additional ‘#:commit’ argument
     must be supplied to specify the commit being built (a hexadecimal
     string).

     The resulting package is a Guix instance of the given channel,
     similar to how ‘guix time-machine’ would build it.


File: guix.info,  Node: Build Phases,  Next: Build Utilities,  Prev: Build Systems,  Up: Programming Interface

9.6 Build Phases
================

Almost all package build systems implement a notion “build phases”: a
sequence of actions that the build system executes, when you build the
package, leading to the installed byproducts in the store.  A notable
exception is the “bare-bones” ‘trivial-build-system’ (*note Build
Systems::).

   As discussed in the previous section, those build systems provide a
standard list of phases.  For ‘gnu-build-system’, the main build phases
are the following:

‘set-paths’
     Define search path environment variables for all the input
     packages, including ‘PATH’ (*note Search Paths::).

‘unpack’
     Unpack the source tarball, and change the current directory to the
     extracted source tree.  If the source is actually a directory, copy
     it to the build tree, and enter that directory.

‘patch-source-shebangs’
     Patch shebangs encountered in source files so they refer to the
     right store file names.  For instance, this changes ‘#!/bin/sh’ to
     ‘#!/gnu/store/...-bash-4.3/bin/sh’.

‘configure’
     Run the ‘configure’ script with a number of default options, such
     as ‘--prefix=/gnu/store/...’, as well as the options specified by
     the ‘#:configure-flags’ argument.

‘build’
     Run ‘make’ with the list of flags specified with ‘#:make-flags’.
     If the ‘#:parallel-build?’ argument is true (the default), build
     with ‘make -j’.

‘check’
     Run ‘make check’, or some other target specified with
     ‘#:test-target’, unless ‘#:tests? #f’ is passed.  If the
     ‘#:parallel-tests?’ argument is true (the default), run ‘make check
     -j’.

‘install’
     Run ‘make install’ with the flags listed in ‘#:make-flags’.

‘patch-shebangs’
     Patch shebangs on the installed executable files.

‘strip’
     Strip debugging symbols from ELF files (unless ‘#:strip-binaries?’
     is false), copying them to the ‘debug’ output when available (*note
     Installing Debugging Files::).

‘validate-runpath’
     Validate the ‘RUNPATH’ of ELF binaries, unless
     ‘#:validate-runpath?’ is false (*note Build Systems::).

     This validation step consists in making sure that all the shared
     libraries needed by an ELF binary, which are listed as ‘DT_NEEDED’
     entries in its ‘PT_DYNAMIC’ segment, appear in the ‘DT_RUNPATH’
     entry of that binary.  In other words, it ensures that running or
     using those binaries will not result in a “file not found” error at
     run time.  *Note ‘-rpath’: (ld)Options, for more information on
     ‘RUNPATH’.

   Other build systems have similar phases, with some variations.  For
example, ‘cmake-build-system’ has same-named phases but its ‘configure’
phases runs ‘cmake’ instead of ‘./configure’.  Others, such as
‘python-build-system’, have a wholly different list of standard phases.
All this code runs on the “build side”: it is evaluated when you
actually build the package, in a dedicated build process spawned by the
build daemon (*note Invoking guix-daemon::).

   Build phases are represented as association lists or “alists” (*note
(guile)Association Lists::) where each key is a symbol for the name of
the phase and the associated value is a procedure that accepts an
arbitrary number of arguments.  By convention, those procedures receive
information about the build in the form of “keyword parameters”, which
they can use or ignore.

   For example, here is how ‘(guix build gnu-build-system)’ defines
‘%standard-phases’, the variable holding its alist of build phases(1):

     ;; The build phases of 'gnu-build-system'.

     (define* (unpack #:key source #:allow-other-keys)
       ;; Extract the source tarball.
       (invoke "tar" "xvf" source))

     (define* (configure #:key outputs #:allow-other-keys)
       ;; Run the 'configure' script.  Install to output "out".
       (let ((out (assoc-ref outputs "out")))
         (invoke "./configure"
                 (string-append "--prefix=" out))))

     (define* (build #:allow-other-keys)
       ;; Compile.
       (invoke "make"))

     (define* (check #:key (test-target "check") (tests? #true)
                     #:allow-other-keys)
       ;; Run the test suite.
       (if tests?
           (invoke "make" test-target)
           (display "test suite not run\n")))

     (define* (install #:allow-other-keys)
       ;; Install files to the prefix 'configure' specified.
       (invoke "make" "install"))

     (define %standard-phases
       ;; The list of standard phases (quite a few are omitted
       ;; for brevity).  Each element is a symbol/procedure pair.
       (list (cons 'unpack unpack)
             (cons 'configure configure)
             (cons 'build build)
             (cons 'check check)
             (cons 'install install)))

   This shows how ‘%standard-phases’ is defined as a list of
symbol/procedure pairs (*note (guile)Pairs::).  The first pair
associates the ‘unpack’ procedure with the ‘unpack’ symbol—a name; the
second pair defines the ‘configure’ phase similarly, and so on.  When
building a package that uses ‘gnu-build-system’ with its default list of
phases, those phases are executed sequentially.  You can see the name of
each phase started and completed in the build log of packages that you
build.

   Let’s now look at the procedures themselves.  Each one is defined
with ‘define*’: ‘#:key’ lists keyword parameters the procedure accepts,
possibly with a default value, and ‘#:allow-other-keys’ specifies that
other keyword parameters are ignored (*note (guile)Optional
Arguments::).

   The ‘unpack’ procedure honors the ‘source’ parameter, which the build
system uses to pass the file name of the source tarball (or version
control checkout), and it ignores other parameters.  The ‘configure’
phase only cares about the ‘outputs’ parameter, an alist mapping package
output names to their store file name (*note Packages with Multiple
Outputs::).  It extracts the file name of for ‘out’, the default output,
and passes it to ‘./configure’ as the installation prefix, meaning that
‘make install’ will eventually copy all the files in that directory
(*note configuration and makefile conventions:
(standards)Configuration.).  ‘build’ and ‘install’ ignore all their
arguments.  ‘check’ honors the ‘test-target’ argument, which specifies
the name of the Makefile target to run tests; it prints a message and
skips tests when ‘tests?’ is false.

   The list of phases used for a particular package can be changed with
the ‘#:phases’ parameter of the build system.  Changing the set of build
phases boils down to building a new alist of phases based on the
‘%standard-phases’ alist described above.  This can be done with
standard alist procedures such as ‘alist-delete’ (*note (guile)SRFI-1
Association Lists::); however, it is more convenient to do so with
‘modify-phases’ (*note ‘modify-phases’: Build Utilities.).

   Here is an example of a package definition that removes the
‘configure’ phase of ‘%standard-phases’ and inserts a new phase before
the ‘build’ phase, called ‘set-prefix-in-makefile’:

     (define-public example
       (package
         (name "example")
         ;; other fields omitted
         (build-system gnu-build-system)
         (arguments
          '(#:phases (modify-phases %standard-phases
                       (delete 'configure)
                       (add-before 'build 'set-prefix-in-makefile
                         (lambda* (#:key outputs #:allow-other-keys)
                           ;; Modify the makefile so that its
                           ;; 'PREFIX' variable points to "out".
                           (let ((out (assoc-ref outputs "out")))
                             (substitute* "Makefile"
                               (("PREFIX =.*")
                                (string-append "PREFIX = "
                                               out "\n")))))))))))

   The new phase that is inserted is written as an anonymous procedure,
introduced with ‘lambda*’; it honors the ‘outputs’ parameter we have
seen before.  *Note Build Utilities::, for more about the helpers used
by this phase, and for more examples of ‘modify-phases’.

   Keep in mind that build phases are code evaluated at the time the
package is actually built.  This explains why the whole ‘modify-phases’
expression above is quoted (it comes after the ‘'’ or apostrophe): it is
“staged” for later execution.  *Note G-Expressions::, for an explanation
of code staging and the “code strata” involved.

   ---------- Footnotes ----------

   (1) We present a simplified view of those build phases, but do take a
look at ‘(guix build gnu-build-system)’ to see all the details!


File: guix.info,  Node: Build Utilities,  Next: Search Paths,  Prev: Build Phases,  Up: Programming Interface

9.7 Build Utilities
===================

As soon as you start writing non-trivial package definitions (*note
Defining Packages::) or other build actions (*note G-Expressions::), you
will likely start looking for helpers for “shell-like” actions—creating
directories, copying and deleting files recursively, manipulating build
phases, and so on.  The ‘(guix build utils)’ module provides such
utility procedures.

   Most build systems load ‘(guix build utils)’ (*note Build Systems::).
Thus, when writing custom build phases for your package definitions, you
can usually assume those procedures are in scope.

   When writing G-expressions, you can import ‘(guix build utils)’ on
the “build side” using ‘with-imported-modules’ and then put it in scope
with the ‘use-modules’ form (*note (guile)Using Guile Modules::):

     (with-imported-modules '((guix build utils))  ;import it
       (computed-file "empty-tree"
                      #~(begin
                          ;; Put it in scope.
                          (use-modules (guix build utils))

                          ;; Happily use its 'mkdir-p' procedure.
                          (mkdir-p (string-append #$output "/a/b/c")))))

   The remainder of this section is the reference for most of the
utility procedures provided by ‘(guix build utils)’.

9.7.1 Dealing with Store File Names
-----------------------------------

This section documents procedures that deal with store file names.

 -- Scheme Procedure: %store-directory
     Return the directory name of the store.

 -- Scheme Procedure: store-file-name? FILE
     Return true if FILE is in the store.

 -- Scheme Procedure: strip-store-file-name FILE
     Strip the ‘/gnu/store’ and hash from FILE, a store file name.  The
     result is typically a ‘"PACKAGE-VERSION"’ string.

 -- Scheme Procedure: package-name->name+version NAME
     Given NAME, a package name like ‘"foo-0.9.1b"’, return two values:
     ‘"foo"’ and ‘"0.9.1b"’.  When the version part is unavailable, NAME
     and ‘#f’ are returned.  The first hyphen followed by a digit is
     considered to introduce the version part.

9.7.2 File Types
----------------

The procedures below deal with files and file types.

 -- Scheme Procedure: directory-exists? DIR
     Return ‘#t’ if DIR exists and is a directory.

 -- Scheme Procedure: executable-file? FILE
     Return ‘#t’ if FILE exists and is executable.

 -- Scheme Procedure: symbolic-link? FILE
     Return ‘#t’ if FILE is a symbolic link (aka.  a “symlink”).

 -- Scheme Procedure: elf-file? FILE
 -- Scheme Procedure: ar-file? FILE
 -- Scheme Procedure: gzip-file? FILE
     Return ‘#t’ if FILE is, respectively, an ELF file, an ‘ar’ archive
     (such as a ‘.a’ static library), or a gzip file.

 -- Scheme Procedure: reset-gzip-timestamp FILE [#:keep-mtime? #t]
     If FILE is a gzip file, reset its embedded timestamp (as with ‘gzip
     --no-name’) and return true.  Otherwise return ‘#f’.  When
     KEEP-MTIME? is true, preserve FILE’s modification time.

9.7.3 File Manipulation
-----------------------

The following procedures and macros help create, modify, and delete
files.  They provide functionality comparable to common shell utilities
such as ‘mkdir -p’, ‘cp -r’, ‘rm -r’, and ‘sed’.  They complement
Guile’s extensive, but low-level, file system interface (*note
(guile)POSIX::).

 -- Scheme Syntax: with-directory-excursion DIRECTORY BODY...
     Run BODY with DIRECTORY as the process’s current directory.

     Essentially, this macro changes the current directory to DIRECTORY
     before evaluating BODY, using ‘chdir’ (*note (guile)Processes::).
     It changes back to the initial directory when the dynamic extent of
     BODY is left, be it via normal procedure return or via a non-local
     exit such as an exception.

 -- Scheme Procedure: mkdir-p DIR
     Create directory DIR and all its ancestors.

 -- Scheme Procedure: install-file FILE DIRECTORY
     Create DIRECTORY if it does not exist and copy FILE in there under
     the same name.

 -- Scheme Procedure: make-file-writable FILE
     Make FILE writable for its owner.

 -- Scheme Procedure: copy-recursively SOURCE DESTINATION [#:log
          (current-output-port)] [#:follow-symlinks? #f] [#:copy-file
          copy-file] [#:keep-mtime? #f] [#:keep-permissions? #t]
     Copy SOURCE directory to DESTINATION.  Follow symlinks if
     FOLLOW-SYMLINKS? is true; otherwise, just preserve them.  Call
     COPY-FILE to copy regular files.  When KEEP-MTIME? is true, keep
     the modification time of the files in SOURCE on those of
     DESTINATION.  When KEEP-PERMISSIONS? is true, preserve file
     permissions.  Write verbose output to the LOG port.

 -- Scheme Procedure: delete-file-recursively DIR [#:follow-mounts? #f]
     Delete DIR recursively, like ‘rm -rf’, without following symlinks.
     Don’t follow mount points either, unless FOLLOW-MOUNTS? is true.
     Report but ignore errors.

 -- Scheme Syntax: substitute* FILE ((REGEXP MATCH-VAR...) BODY...) ...
     Substitute REGEXP in FILE by the string returned by BODY.  BODY is
     evaluated with each MATCH-VAR bound to the corresponding positional
     regexp sub-expression.  For example:

          (substitute* file
            (("hello")
             "good morning\n")
            (("foo([a-z]+)bar(.*)$" all letters end)
             (string-append "baz" letters end)))

     Here, anytime a line of FILE contains ‘hello’, it is replaced by
     ‘good morning’.  Anytime a line of FILE matches the second regexp,
     ‘all’ is bound to the complete match, ‘letters’ is bound to the
     first sub-expression, and ‘end’ is bound to the last one.

     When one of the MATCH-VAR is ‘_’, no variable is bound to the
     corresponding match substring.

     Alternatively, FILE may be a list of file names, in which case they
     are all subject to the substitutions.

     Be careful about using ‘$’ to match the end of a line; by itself it
     won’t match the terminating newline of a line.

9.7.4 File Search
-----------------

This section documents procedures to search and filter files.

 -- Scheme Procedure: file-name-predicate REGEXP
     Return a predicate that returns true when passed a file name whose
     base name matches REGEXP.

 -- Scheme Procedure: find-files DIR [PRED] [#:stat lstat]
          [#:directories? #f] [#:fail-on-error? #f]
     Return the lexicographically sorted list of files under DIR for
     which PRED returns true.  PRED is passed two arguments: the
     absolute file name, and its stat buffer; the default predicate
     always returns true.  PRED can also be a regular expression, in
     which case it is equivalent to ‘(file-name-predicate PRED)’.  STAT
     is used to obtain file information; using ‘lstat’ means that
     symlinks are not followed.  If DIRECTORIES? is true, then
     directories will also be included.  If FAIL-ON-ERROR? is true,
     raise an exception upon error.

   Here are a few examples where we assume that the current directory is
the root of the Guix source tree:

     ;; List all the regular files in the current directory.
     (find-files ".")
     ⇒ ("./.dir-locals.el" "./.gitignore" ...)

     ;; List all the .scm files under gnu/services.
     (find-files "gnu/services" "\\.scm$")
     ⇒ ("gnu/services/admin.scm" "gnu/services/audio.scm" ...)

     ;; List ar files in the current directory.
     (find-files "." (lambda (file stat) (ar-file? file)))
     ⇒ ("./libformat.a" "./libstore.a" ...)

 -- Scheme Procedure: which PROGRAM
     Return the complete file name for PROGRAM as found in ‘$PATH’, or
     ‘#f’ if PROGRAM could not be found.

 -- Scheme Procedure: search-input-file INPUTS NAME
 -- Scheme Procedure: search-input-directory INPUTS NAME
     Return the complete file name for NAME as found in INPUTS;
     ‘search-input-file’ searches for a regular file and
     ‘search-input-directory’ searches for a directory.  If NAME could
     not be found, an exception is raised.

     Here, INPUTS must be an association list like ‘inputs’ and
     ‘native-inputs’ as available to build phases (*note Build
     Phases::).

   Here is a (simplified) example of how ‘search-input-file’ is used in
a build phase of the ‘wireguard-tools’ package:

     (add-after 'install 'wrap-wg-quick
       (lambda* (#:key inputs outputs #:allow-other-keys)
         (let ((coreutils (string-append (assoc-ref inputs "coreutils")
                                         "/bin")))
           (wrap-program (search-input-file outputs "bin/wg-quick")
             #:sh (search-input-file inputs "bin/bash")
             `("PATH" ":" prefix ,(list coreutils))))))

9.7.5 Program Invocation
------------------------

You’ll find handy procedures to spawn processes in this module,
essentially convenient wrappers around Guile’s ‘system*’ (*note
‘system*’: (guile)Processes.).

 -- Scheme Procedure: invoke PROGRAM ARGS...
     Invoke PROGRAM with the given ARGS.  Raise an ‘&invoke-error’
     exception if the exit code is non-zero; otherwise return ‘#t’.

     The advantage compared to ‘system*’ is that you do not need to
     check the return value.  This reduces boilerplate in
     shell-script-like snippets for instance in package build phases.

 -- Scheme Procedure: invoke-error? C
     Return true if C is an ‘&invoke-error’ condition.

 -- Scheme Procedure: invoke-error-program C
 -- Scheme Procedure: invoke-error-arguments C
 -- Scheme Procedure: invoke-error-exit-status C
 -- Scheme Procedure: invoke-error-term-signal C
 -- Scheme Procedure: invoke-error-stop-signal C
     Access specific fields of C, an ‘&invoke-error’ condition.

 -- Scheme Procedure: report-invoke-error C [PORT]
     Report to PORT (by default the current error port) about C, an
     ‘&invoke-error’ condition, in a human-friendly way.

     Typical usage would look like this:

          (use-modules (srfi srfi-34) ;for 'guard'
                       (guix build utils))

          (guard (c ((invoke-error? c)
                     (report-invoke-error c)))
            (invoke "date" "--imaginary-option"))

          ⊣ command "date" "--imaginary-option" failed with status 1

 -- Scheme Procedure: invoke/quiet PROGRAM ARGS...
     Invoke PROGRAM with ARGS and capture PROGRAM’s standard output and
     standard error.  If PROGRAM succeeds, print nothing and return the
     unspecified value; otherwise, raise a ‘&message’ error condition
     that includes the status code and the output of PROGRAM.

     Here’s an example:

          (use-modules (srfi srfi-34) ;for 'guard'
                       (srfi srfi-35) ;for 'message-condition?'
                       (guix build utils))

          (guard (c ((message-condition? c)
                     (display (condition-message c))))
            (invoke/quiet "date")  ;all is fine
            (invoke/quiet "date" "--imaginary-option"))

          ⊣ 'date --imaginary-option' exited with status 1; output follows:

              date: unrecognized option '--imaginary-option'
              Try 'date --help' for more information.

9.7.6 Build Phases
------------------

The ‘(guix build utils)’ also contains tools to manipulate build phases
as used by build systems (*note Build Systems::).  Build phases are
represented as association lists or “alists” (*note (guile)Association
Lists::) where each key is a symbol naming the phase and the associated
value is a procedure (*note Build Phases::).

   Guile core and the ‘(srfi srfi-1)’ module both provide tools to
manipulate alists.  The ‘(guix build utils)’ module complements those
with tools written with build phases in mind.

 -- Scheme Syntax: modify-phases PHASES CLAUSE...
     Modify PHASES sequentially as per each CLAUSE, which may have one
     of the following forms:

          (delete OLD-PHASE-NAME)
          (replace OLD-PHASE-NAME NEW-PHASE)
          (add-before OLD-PHASE-NAME NEW-PHASE-NAME NEW-PHASE)
          (add-after OLD-PHASE-NAME NEW-PHASE-NAME NEW-PHASE)

     Where every PHASE-NAME above is an expression evaluating to a
     symbol, and NEW-PHASE an expression evaluating to a procedure.

   The example below is taken from the definition of the ‘grep’ package.
It adds a phase to run after the ‘install’ phase, called
‘fix-egrep-and-fgrep’.  That phase is a procedure (‘lambda*’ is for
anonymous procedures) that takes a ‘#:outputs’ keyword argument and
ignores extra keyword arguments (*note (guile)Optional Arguments::, for
more on ‘lambda*’ and optional and keyword arguments.)  The phase uses
‘substitute*’ to modify the installed ‘egrep’ and ‘fgrep’ scripts so
that they refer to ‘grep’ by its absolute file name:

     (modify-phases %standard-phases
       (add-after 'install 'fix-egrep-and-fgrep
         ;; Patch 'egrep' and 'fgrep' to execute 'grep' via its
         ;; absolute file name instead of searching for it in $PATH.
         (lambda* (#:key outputs #:allow-other-keys)
           (let* ((out (assoc-ref outputs "out"))
                  (bin (string-append out "/bin")))
             (substitute* (list (string-append bin "/egrep")
                                (string-append bin "/fgrep"))
               (("^exec grep")
                (string-append "exec " bin "/grep")))))))

   In the example below, phases are modified in two ways: the standard
‘configure’ phase is deleted, presumably because the package does not
have a ‘configure’ script or anything similar, and the default ‘install’
phase is replaced by one that manually copies the executable files to be
installed:

     (modify-phases %standard-phases
       (delete 'configure)      ;no 'configure' script
       (replace 'install
         (lambda* (#:key outputs #:allow-other-keys)
           ;; The package's Makefile doesn't provide an "install"
           ;; rule so do it by ourselves.
           (let ((bin (string-append (assoc-ref outputs "out")
                                     "/bin")))
             (install-file "footswitch" bin)
             (install-file "scythe" bin)))))

9.7.7 Wrappers
--------------

It is not unusual for a command to require certain environment variables
to be set for proper functioning, typically search paths (*note Search
Paths::).  Failing to do that, the command might fail to find files or
other commands it relies on, or it might pick the “wrong” ones—depending
on the environment in which it runs.  Examples include:

   • a shell script that assumes all the commands it uses are in ‘PATH’;

   • a Guile program that assumes all its modules are in
     ‘GUILE_LOAD_PATH’ and ‘GUILE_LOAD_COMPILED_PATH’;

   • a Qt application that expects to find certain plugins in
     ‘QT_PLUGIN_PATH’.

   For a package writer, the goal is to make sure commands always work
the same rather than depend on some external settings.  One way to
achieve that is to “wrap” commands in a thin script that sets those
environment variables, thereby ensuring that those run-time dependencies
are always found.  The wrapper would be used to set ‘PATH’,
‘GUILE_LOAD_PATH’, or ‘QT_PLUGIN_PATH’ in the examples above.

   To ease that task, the ‘(guix build utils)’ module provides a couple
of helpers to wrap commands.

 -- Scheme Procedure: wrap-program PROGRAM [#:sh SH] [#:rest VARIABLES]
     Make a wrapper for PROGRAM.  VARIABLES should look like this:

          '(VARIABLE DELIMITER POSITION LIST-OF-DIRECTORIES)

     where DELIMITER is optional.  ‘:’ will be used if DELIMITER is not
     given.

     For example, this call:

          (wrap-program "foo"
                        '("PATH" ":" = ("/gnu/.../bar/bin"))
                        '("CERT_PATH" suffix ("/gnu/.../baz/certs"
                                              "/qux/certs")))

     will copy ‘foo’ to ‘.foo-real’ and create the file ‘foo’ with the
     following contents:

          #!location/of/bin/bash
          export PATH="/gnu/.../bar/bin"
          export CERT_PATH="$CERT_PATH${CERT_PATH:+:}/gnu/.../baz/certs:/qux/certs"
          exec -a $0 location/of/.foo-real "$@"

     If PROGRAM has previously been wrapped by ‘wrap-program’, the
     wrapper is extended with definitions for VARIABLES.  If it is not,
     SH will be used as the interpreter.

 -- Scheme Procedure: wrap-script PROGRAM [#:guile GUILE] [#:rest
          VARIABLES]
     Wrap the script PROGRAM such that VARIABLES are set first.  The
     format of VARIABLES is the same as in the ‘wrap-program’ procedure.
     This procedure differs from ‘wrap-program’ in that it does not
     create a separate shell script.  Instead, PROGRAM is modified
     directly by prepending a Guile script, which is interpreted as a
     comment in the script’s language.

     Special encoding comments as supported by Python are recreated on
     the second line.

     Note that this procedure can only be used once per file as Guile
     scripts are not supported.


File: guix.info,  Node: Search Paths,  Next: The Store,  Prev: Build Utilities,  Up: Programming Interface

9.8 Search Paths
================

Many programs and libraries look for input data in a “search path”, a
list of directories: shells like Bash look for executables in the
command search path, a C compiler looks for ‘.h’ files in its header
search path, the Python interpreter looks for ‘.py’ files in its search
path, the spell checker has a search path for dictionaries, and so on.

   Search paths can usually be defined or overridden via environment
variables (*note (libc)Environment Variables::).  For example, the
search paths mentioned above can be changed by defining the ‘PATH’,
‘C_INCLUDE_PATH’, ‘PYTHONPATH’ (or ‘GUIX_PYTHONPATH’), and ‘DICPATH’
environment variables—you know, all these something-PATH variables that
you need to get right or things “won’t be found”.

   You may have noticed from the command line that Guix “knows” which
search path environment variables should be defined, and how.  When you
install packages in your default profile, the file
‘~/.guix-profile/etc/profile’ is created, which you can “source” from
the shell to set those variables.  Likewise, if you ask ‘guix shell’ to
create an environment containing Python and NumPy, a Python library, and
if you pass it the ‘--search-paths’ option, it will tell you about
‘PATH’ and ‘GUIX_PYTHONPATH’ (*note Invoking guix shell::):

     $ guix shell python python-numpy --pure --search-paths
     export PATH="/gnu/store/...-profile/bin"
     export GUIX_PYTHONPATH="/gnu/store/...-profile/lib/python3.9/site-packages"

   When you omit ‘--search-paths’, it defines these environment
variables right away, such that Python can readily find NumPy:

     $ guix shell python python-numpy -- python3
     Python 3.9.6 (default, Jan  1 1970, 00:00:01)
     [GCC 10.3.0] on linux
     Type "help", "copyright", "credits" or "license" for more information.
     >>> import numpy
     >>> numpy.version.version
     '1.20.3'

   For this to work, the definition of the ‘python’ package _declares_
the search path it cares about and its associated environment variable,
‘GUIX_PYTHONPATH’.  It looks like this:

     (package
       (name "python")
       (version "3.9.9")
       ;; some fields omitted...
       (native-search-paths
        (list (search-path-specification
               (variable "GUIX_PYTHONPATH")
               (files (list "lib/python/3.9/site-packages"))))))

   What this ‘native-search-paths’ field says is that, when the ‘python’
package is used, the ‘GUIX_PYTHONPATH’ environment variable must be
defined to include all the ‘lib/python/3.9/site-packages’
sub-directories encountered in its environment.  (The ‘native-’ bit
means that, if we are in a cross-compilation environment, only native
inputs may be added to the search path; *note ‘search-paths’: package
Reference.)  In the NumPy example above, the profile where ‘python’
appears contains exactly one such sub-directory, and ‘GUIX_PYTHONPATH’
is set to that.  When there are several
‘lib/python/3.9/site-packages’—this is the case in package build
environments—they are all added to ‘GUIX_PYTHONPATH’, separated by
colons (‘:’).

     Note: Notice that ‘GUIX_PYTHONPATH’ is specified as part of the
     definition of the ‘python’ package, and _not_ as part of that of
     ‘python-numpy’.  This is because this environment variable
     “belongs” to Python, not NumPy: Python actually reads the value of
     that variable and honors it.

     Corollary: if you create a profile that does not contain ‘python’,
     ‘GUIX_PYTHONPATH’ will _not_ be defined, even if it contains
     packages that provide ‘.py’ files:

          $ guix shell python-numpy --search-paths --pure
          export PATH="/gnu/store/...-profile/bin"

     This makes a lot of sense if we look at this profile in isolation:
     no software in this profile would read ‘GUIX_PYTHONPATH’.

   Of course, there are many variations on that theme: some packages
honor more than one search path, some use separators other than colon,
some accumulate several directories in their search path, and so on.  A
more complex example is the search path of libxml2: the value of the
‘XML_CATALOG_FILES’ environment variable is space-separated, it must
contain a list of ‘catalog.xml’ files (not directories), which are to be
found in ‘xml’ sub-directories—nothing less.  The search path
specification looks like this:

     (package
       (name "libxml2")
       ;; some fields omitted
       (native-search-paths
        (list (search-path-specification
               (variable "XML_CATALOG_FILES")
               (separator " ")
               (files '("xml"))
               (file-pattern "^catalog\\.xml$")
               (file-type 'regular)))))

   Worry not, search path specifications are usually not this tricky.

   The ‘(guix search-paths)’ module defines the data type of search path
specifications and a number of helper procedures.  Below is the
reference of search path specifications.

 -- Data Type: search-path-specification
     The data type for search path specifications.

     ‘variable’
          The name of the environment variable for this search path (a
          string).

     ‘files’
          The list of sub-directories (strings) that should be added to
          the search path.

     ‘separator’ (default: ‘":"’)
          The string used to separate search path components.

          As a special case, a ‘separator’ value of ‘#f’ specifies a
          “single-component search path”—in other words, a search path
          that cannot contain more than one element.  This is useful in
          some cases, such as the ‘SSL_CERT_DIR’ variable (honored by
          OpenSSL, cURL, and a few other packages) or the
          ‘ASPELL_DICT_DIR’ variable (honored by the GNU Aspell spell
          checker), both of which must point to a single directory.

     ‘file-type’ (default: ‘'directory’)
          The type of file being matched—‘'directory’ or ‘'regular’,
          though it can be any symbol returned by ‘stat:type’ (*note
          ‘stat’: (guile)File System.).

          In the libxml2 example above, we would match regular files; in
          the Python example, we would match directories.

     ‘file-pattern’ (default: ‘#f’)
          This must be either ‘#f’ or a regular expression specifying
          files to be matched _within_ the sub-directories specified by
          the ‘files’ field.

          Again, the libxml2 example shows a situation where this is
          needed.

   Some search paths are not tied by a single package but to many
packages.  To reduce duplications, some of them are pre-defined in
‘(guix search-paths)’.

 -- Scheme Variable: $SSL_CERT_DIR
 -- Scheme Variable: $SSL_CERT_FILE
     These two search paths indicate where X.509 certificates can be
     found (*note X.509 Certificates::).

   These pre-defined search paths can be used as in the following
example:

     (package
       (name "curl")
       ;; some fields omitted ...
       (native-search-paths (list $SSL_CERT_DIR $SSL_CERT_FILE)))

   How do you turn search path specifications on one hand and a bunch of
directories on the other hand in a set of environment variable
definitions?  That’s the job of ‘evaluate-search-paths’.

 -- Scheme Procedure: evaluate-search-paths SEARCH-PATHS DIRECTORIES
          [GETENV]
     Evaluate SEARCH-PATHS, a list of search-path specifications, for
     DIRECTORIES, a list of directory names, and return a list of
     specification/value pairs.  Use GETENV to determine the current
     settings and report only settings not already effective.

   The ‘(guix profiles)’ provides a higher-level helper procedure,
‘load-profile’, that sets the environment variables of a profile.


File: guix.info,  Node: The Store,  Next: Derivations,  Prev: Search Paths,  Up: Programming Interface

9.9 The Store
=============

Conceptually, the “store” is the place where derivations that have been
built successfully are stored—by default, ‘/gnu/store’.  Sub-directories
in the store are referred to as “store items” or sometimes “store
paths”.  The store has an associated database that contains information
such as the store paths referred to by each store path, and the list of
_valid_ store items—results of successful builds.  This database resides
in ‘LOCALSTATEDIR/guix/db’, where LOCALSTATEDIR is the state directory
specified via ‘--localstatedir’ at configure time, usually ‘/var’.

   The store is _always_ accessed by the daemon on behalf of its clients
(*note Invoking guix-daemon::).  To manipulate the store, clients
connect to the daemon over a Unix-domain socket, send requests to it,
and read the result—these are remote procedure calls, or RPCs.

     Note: Users must _never_ modify files under ‘/gnu/store’ directly.
     This would lead to inconsistencies and break the immutability
     assumptions of Guix’s functional model (*note Introduction::).

     *Note ‘guix gc --verify’: Invoking guix gc, for information on how
     to check the integrity of the store and attempt recovery from
     accidental modifications.

   The ‘(guix store)’ module provides procedures to connect to the
daemon, and to perform RPCs.  These are described below.  By default,
‘open-connection’, and thus all the ‘guix’ commands, connect to the
local daemon or to the URI specified by the ‘GUIX_DAEMON_SOCKET’
environment variable.

 -- Environment Variable: GUIX_DAEMON_SOCKET
     When set, the value of this variable should be a file name or a URI
     designating the daemon endpoint.  When it is a file name, it
     denotes a Unix-domain socket to connect to.  In addition to file
     names, the supported URI schemes are:

     ‘file’
     ‘unix’
          These are for Unix-domain sockets.
          ‘file:///var/guix/daemon-socket/socket’ is equivalent to
          ‘/var/guix/daemon-socket/socket’.

     ‘guix’
          These URIs denote connections over TCP/IP, without encryption
          nor authentication of the remote host.  The URI must specify
          the host name and optionally a port number (by default port
          44146 is used):

               guix://master.guix.example.org:1234

          This setup is suitable on local networks, such as clusters,
          where only trusted nodes may connect to the build daemon at
          ‘master.guix.example.org’.

          The ‘--listen’ option of ‘guix-daemon’ can be used to instruct
          it to listen for TCP connections (*note ‘--listen’: Invoking
          guix-daemon.).

     ‘ssh’
          These URIs allow you to connect to a remote daemon over SSH.
          This feature requires Guile-SSH (*note Requirements::) and a
          working ‘guile’ binary in ‘PATH’ on the destination machine.
          It supports public key and GSSAPI authentication.  A typical
          URL might look like this:

               ssh://charlie@guix.example.org:22

          As for ‘guix copy’, the usual OpenSSH client configuration
          files are honored (*note Invoking guix copy::).

     Additional URI schemes may be supported in the future.

          Note: The ability to connect to remote build daemons is
          considered experimental as of 1.4.0.  Please get in touch with
          us to share any problems or suggestions you may have (*note
          Contributing::).

 -- Scheme Procedure: open-connection [URI] [#:reserve-space? #t]
     Connect to the daemon over the Unix-domain socket at URI (a
     string).  When RESERVE-SPACE? is true, instruct it to reserve a
     little bit of extra space on the file system so that the garbage
     collector can still operate should the disk become full.  Return a
     server object.

     FILE defaults to ‘%default-socket-path’, which is the normal
     location given the options that were passed to ‘configure’.

 -- Scheme Procedure: close-connection SERVER
     Close the connection to SERVER.

 -- Scheme Variable: current-build-output-port
     This variable is bound to a SRFI-39 parameter, which refers to the
     port where build and error logs sent by the daemon should be
     written.

   Procedures that make RPCs all take a server object as their first
argument.

 -- Scheme Procedure: valid-path? SERVER PATH
     Return ‘#t’ when PATH designates a valid store item and ‘#f’
     otherwise (an invalid item may exist on disk but still be invalid,
     for instance because it is the result of an aborted or failed
     build).

     A ‘&store-protocol-error’ condition is raised if PATH is not
     prefixed by the store directory (‘/gnu/store’).

 -- Scheme Procedure: add-text-to-store SERVER NAME TEXT [REFERENCES]
     Add TEXT under file NAME in the store, and return its store path.
     REFERENCES is the list of store paths referred to by the resulting
     store path.

 -- Scheme Procedure: build-derivations STORE DERIVATIONS [MODE]
     Build DERIVATIONS, a list of ‘<derivation>’ objects, ‘.drv’ file
     names, or derivation/output pairs, using the specified
     MODE—‘(build-mode normal)’ by default.

   Note that the ‘(guix monads)’ module provides a monad as well as
monadic versions of the above procedures, with the goal of making it
more convenient to work with code that accesses the store (*note The
Store Monad::).

   This section is currently incomplete.


File: guix.info,  Node: Derivations,  Next: The Store Monad,  Prev: The Store,  Up: Programming Interface

9.10 Derivations
================

Low-level build actions and the environment in which they are performed
are represented by “derivations”.  A derivation contains the following
pieces of information:

   • The outputs of the derivation—derivations produce at least one file
     or directory in the store, but may produce more.

   • The inputs of the derivations—i.e., its build-time
     dependencies—which may be other derivations or plain files in the
     store (patches, build scripts, etc.).

   • The system type targeted by the derivation—e.g., ‘x86_64-linux’.

   • The file name of a build script in the store, along with the
     arguments to be passed.

   • A list of environment variables to be defined.

   Derivations allow clients of the daemon to communicate build actions
to the store.  They exist in two forms: as an in-memory representation,
both on the client- and daemon-side, and as files in the store whose
name end in ‘.drv’—these files are referred to as “derivation paths”.
Derivations paths can be passed to the ‘build-derivations’ procedure to
perform the build actions they prescribe (*note The Store::).

   Operations such as file downloads and version-control checkouts for
which the expected content hash is known in advance are modeled as
“fixed-output derivations”.  Unlike regular derivations, the outputs of
a fixed-output derivation are independent of its inputs—e.g., a source
code download produces the same result regardless of the download method
and tools being used.

   The outputs of derivations—i.e., the build results—have a set of
“references”, as reported by the ‘references’ RPC or the ‘guix gc
--references’ command (*note Invoking guix gc::).  References are the
set of run-time dependencies of the build results.  References are a
subset of the inputs of the derivation; this subset is automatically
computed by the build daemon by scanning all the files in the outputs.

   The ‘(guix derivations)’ module provides a representation of
derivations as Scheme objects, along with procedures to create and
otherwise manipulate derivations.  The lowest-level primitive to create
a derivation is the ‘derivation’ procedure:

 -- Scheme Procedure: derivation STORE NAME BUILDER ARGS [#:outputs
          '("out")] [#:hash #f] [#:hash-algo #f] [#:recursive? #f]
          [#:inputs '()] [#:env-vars '()] [#:system (%current-system)]
          [#:references-graphs #f] [#:allowed-references #f]
          [#:disallowed-references #f] [#:leaked-env-vars #f]
          [#:local-build? #f] [#:substitutable? #t] [#:properties '()]
     Build a derivation with the given arguments, and return the
     resulting ‘<derivation>’ object.

     When HASH and HASH-ALGO are given, a “fixed-output derivation” is
     created—i.e., one whose result is known in advance, such as a file
     download.  If, in addition, RECURSIVE? is true, then that fixed
     output may be an executable file or a directory and HASH must be
     the hash of an archive containing this output.

     When REFERENCES-GRAPHS is true, it must be a list of file
     name/store path pairs.  In that case, the reference graph of each
     store path is exported in the build environment in the
     corresponding file, in a simple text format.

     When ALLOWED-REFERENCES is true, it must be a list of store items
     or outputs that the derivation’s output may refer to.  Likewise,
     DISALLOWED-REFERENCES, if true, must be a list of things the
     outputs may _not_ refer to.

     When LEAKED-ENV-VARS is true, it must be a list of strings denoting
     environment variables that are allowed to “leak” from the daemon’s
     environment to the build environment.  This is only applicable to
     fixed-output derivations—i.e., when HASH is true.  The main use is
     to allow variables such as ‘http_proxy’ to be passed to derivations
     that download files.

     When LOCAL-BUILD? is true, declare that the derivation is not a
     good candidate for offloading and should rather be built locally
     (*note Daemon Offload Setup::).  This is the case for small
     derivations where the costs of data transfers would outweigh the
     benefits.

     When SUBSTITUTABLE? is false, declare that substitutes of the
     derivation’s output should not be used (*note Substitutes::).  This
     is useful, for instance, when building packages that capture
     details of the host CPU instruction set.

     PROPERTIES must be an association list describing “properties” of
     the derivation.  It is kept as-is, uninterpreted, in the
     derivation.

Here’s an example with a shell script as its builder, assuming STORE is
an open connection to the daemon, and BASH points to a Bash executable
in the store:

     (use-modules (guix utils)
                  (guix store)
                  (guix derivations))

     (let ((builder   ; add the Bash script to the store
             (add-text-to-store store "my-builder.sh"
                                "echo hello world > $out\n" '())))
       (derivation store "foo"
                   bash `("-e" ,builder)
                   #:inputs `((,bash) (,builder))
                   #:env-vars '(("HOME" . "/homeless"))))
     ⇒ #<derivation /gnu/store/...-foo.drv => /gnu/store/...-foo>

   As can be guessed, this primitive is cumbersome to use directly.  A
better approach is to write build scripts in Scheme, of course!  The
best course of action for that is to write the build code as a
“G-expression”, and to pass it to ‘gexp->derivation’.  For more
information, *note G-Expressions::.

   Once upon a time, ‘gexp->derivation’ did not exist and constructing
derivations with build code written in Scheme was achieved with
‘build-expression->derivation’, documented below.  This procedure is now
deprecated in favor of the much nicer ‘gexp->derivation’.

 -- Scheme Procedure: build-expression->derivation STORE NAME EXP
          [#:system (%current-system)] [#:inputs '()] [#:outputs
          '("out")] [#:hash #f] [#:hash-algo #f] [#:recursive? #f]
          [#:env-vars '()] [#:modules '()] [#:references-graphs #f]
          [#:allowed-references #f] [#:disallowed-references #f]
          [#:local-build? #f] [#:substitutable? #t] [#:guile-for-build
          #f]
     Return a derivation that executes Scheme expression EXP as a
     builder for derivation NAME.  INPUTS must be a list of ‘(name
     drv-path sub-drv)’ tuples; when SUB-DRV is omitted, ‘"out"’ is
     assumed.  MODULES is a list of names of Guile modules from the
     current search path to be copied in the store, compiled, and made
     available in the load path during the execution of EXP—e.g.,
     ‘((guix build utils) (guix build gnu-build-system))’.

     EXP is evaluated in an environment where ‘%outputs’ is bound to a
     list of output/path pairs, and where ‘%build-inputs’ is bound to a
     list of string/output-path pairs made from INPUTS.  Optionally,
     ENV-VARS is a list of string pairs specifying the name and value of
     environment variables visible to the builder.  The builder
     terminates by passing the result of EXP to ‘exit’; thus, when EXP
     returns ‘#f’, the build is considered to have failed.

     EXP is built using GUILE-FOR-BUILD (a derivation).  When
     GUILE-FOR-BUILD is omitted or is ‘#f’, the value of the
     ‘%guile-for-build’ fluid is used instead.

     See the ‘derivation’ procedure for the meaning of
     REFERENCES-GRAPHS, ALLOWED-REFERENCES, DISALLOWED-REFERENCES,
     LOCAL-BUILD?, and SUBSTITUTABLE?.

Here’s an example of a single-output derivation that creates a directory
containing one file:

     (let ((builder '(let ((out (assoc-ref %outputs "out")))
                       (mkdir out)    ; create /gnu/store/...-goo
                       (call-with-output-file (string-append out "/test")
                         (lambda (p)
                           (display '(hello guix) p))))))
       (build-expression->derivation store "goo" builder))

     ⇒ #<derivation /gnu/store/...-goo.drv => ...>


File: guix.info,  Node: The Store Monad,  Next: G-Expressions,  Prev: Derivations,  Up: Programming Interface

9.11 The Store Monad
====================

The procedures that operate on the store described in the previous
sections all take an open connection to the build daemon as their first
argument.  Although the underlying model is functional, they either have
side effects or depend on the current state of the store.

   The former is inconvenient: the connection to the build daemon has to
be carried around in all those functions, making it impossible to
compose functions that do not take that parameter with functions that
do.  The latter can be problematic: since store operations have side
effects and/or depend on external state, they have to be properly
sequenced.

   This is where the ‘(guix monads)’ module comes in.  This module
provides a framework for working with “monads”, and a particularly
useful monad for our uses, the “store monad”.  Monads are a construct
that allows two things: associating “context” with values (in our case,
the context is the store), and building sequences of computations (here
computations include accesses to the store).  Values in a monad—values
that carry this additional context—are called “monadic values”;
procedures that return such values are called “monadic procedures”.

   Consider this “normal” procedure:

     (define (sh-symlink store)
       ;; Return a derivation that symlinks the 'bash' executable.
       (let* ((drv (package-derivation store bash))
              (out (derivation->output-path drv))
              (sh  (string-append out "/bin/bash")))
         (build-expression->derivation store "sh"
                                       `(symlink ,sh %output))))

   Using ‘(guix monads)’ and ‘(guix gexp)’, it may be rewritten as a
monadic function:

     (define (sh-symlink)
       ;; Same, but return a monadic value.
       (mlet %store-monad ((drv (package->derivation bash)))
         (gexp->derivation "sh"
                           #~(symlink (string-append #$drv "/bin/bash")
                                      #$output))))

   There are several things to note in the second version: the ‘store’
parameter is now implicit and is “threaded” in the calls to the
‘package->derivation’ and ‘gexp->derivation’ monadic procedures, and the
monadic value returned by ‘package->derivation’ is “bound” using ‘mlet’
instead of plain ‘let’.

   As it turns out, the call to ‘package->derivation’ can even be
omitted since it will take place implicitly, as we will see later (*note
G-Expressions::):

     (define (sh-symlink)
       (gexp->derivation "sh"
                         #~(symlink (string-append #$bash "/bin/bash")
                                    #$output)))

   Calling the monadic ‘sh-symlink’ has no effect.  As someone once
said, “you exit a monad like you exit a building on fire: by running”.
So, to exit the monad and get the desired effect, one must use
‘run-with-store’:

     (run-with-store (open-connection) (sh-symlink))
     ⇒ /gnu/store/...-sh-symlink

   Note that the ‘(guix monad-repl)’ module extends the Guile REPL with
new “commands” to make it easier to deal with monadic procedures:
‘run-in-store’, and ‘enter-store-monad’ (*note Using Guix
Interactively::).  The former is used to “run” a single monadic value
through the store:

     scheme@(guile-user)> ,run-in-store (package->derivation hello)
     $1 = #<derivation /gnu/store/...-hello-2.9.drv => ...>

   The latter enters a recursive REPL, where all the return values are
automatically run through the store:

     scheme@(guile-user)> ,enter-store-monad
     store-monad@(guile-user) [1]> (package->derivation hello)
     $2 = #<derivation /gnu/store/...-hello-2.9.drv => ...>
     store-monad@(guile-user) [1]> (text-file "foo" "Hello!")
     $3 = "/gnu/store/...-foo"
     store-monad@(guile-user) [1]> ,q
     scheme@(guile-user)>

Note that non-monadic values cannot be returned in the ‘store-monad’
REPL.

   Other meta-commands are available at the REPL, such as ‘,build’ to
build a file-like object (*note Using Guix Interactively::).

   The main syntactic forms to deal with monads in general are provided
by the ‘(guix monads)’ module and are described below.

 -- Scheme Syntax: with-monad MONAD BODY ...
     Evaluate any ‘>>=’ or ‘return’ forms in BODY as being in MONAD.

 -- Scheme Syntax: return VAL
     Return a monadic value that encapsulates VAL.

 -- Scheme Syntax: >>= MVAL MPROC ...
     “Bind” monadic value MVAL, passing its “contents” to monadic
     procedures MPROC...(1).  There can be one MPROC or several of them,
     as in this example:

          (run-with-state
              (with-monad %state-monad
                (>>= (return 1)
                     (lambda (x) (return (+ 1 x)))
                     (lambda (x) (return (* 2 x)))))
            'some-state)

          ⇒ 4
          ⇒ some-state

 -- Scheme Syntax: mlet MONAD ((VAR MVAL) ...) BODY ...
 -- Scheme Syntax: mlet* MONAD ((VAR MVAL) ...) BODY ...
     Bind the variables VAR to the monadic values MVAL in BODY, which is
     a sequence of expressions.  As with the bind operator, this can be
     thought of as “unpacking” the raw, non-monadic value “contained” in
     MVAL and making VAR refer to that raw, non-monadic value within the
     scope of the BODY.  The form (VAR -> VAL) binds VAR to the “normal”
     value VAL, as per ‘let’.  The binding operations occur in sequence
     from left to right.  The last expression of BODY must be a monadic
     expression, and its result will become the result of the ‘mlet’ or
     ‘mlet*’ when run in the MONAD.

     ‘mlet*’ is to ‘mlet’ what ‘let*’ is to ‘let’ (*note (guile)Local
     Bindings::).

 -- Scheme System: mbegin MONAD MEXP ...
     Bind MEXP and the following monadic expressions in sequence,
     returning the result of the last expression.  Every expression in
     the sequence must be a monadic expression.

     This is akin to ‘mlet’, except that the return values of the
     monadic expressions are ignored.  In that sense, it is analogous to
     ‘begin’, but applied to monadic expressions.

 -- Scheme System: mwhen CONDITION MEXP0 MEXP* ...
     When CONDITION is true, evaluate the sequence of monadic
     expressions MEXP0..MEXP* as in an ‘mbegin’.  When CONDITION is
     false, return ‘*unspecified*’ in the current monad.  Every
     expression in the sequence must be a monadic expression.

 -- Scheme System: munless CONDITION MEXP0 MEXP* ...
     When CONDITION is false, evaluate the sequence of monadic
     expressions MEXP0..MEXP* as in an ‘mbegin’.  When CONDITION is
     true, return ‘*unspecified*’ in the current monad.  Every
     expression in the sequence must be a monadic expression.

   The ‘(guix monads)’ module provides the “state monad”, which allows
an additional value—the state—to be _threaded_ through monadic procedure
calls.

 -- Scheme Variable: %state-monad
     The state monad.  Procedures in the state monad can access and
     change the state that is threaded.

     Consider the example below.  The ‘square’ procedure returns a value
     in the state monad.  It returns the square of its argument, but
     also increments the current state value:

          (define (square x)
            (mlet %state-monad ((count (current-state)))
              (mbegin %state-monad
                (set-current-state (+ 1 count))
                (return (* x x)))))

          (run-with-state (sequence %state-monad (map square (iota 3))) 0)
          ⇒ (0 1 4)
          ⇒ 3

     When “run” through ‘%state-monad’, we obtain that additional state
     value, which is the number of ‘square’ calls.

 -- Monadic Procedure: current-state
     Return the current state as a monadic value.

 -- Monadic Procedure: set-current-state VALUE
     Set the current state to VALUE and return the previous state as a
     monadic value.

 -- Monadic Procedure: state-push VALUE
     Push VALUE to the current state, which is assumed to be a list, and
     return the previous state as a monadic value.

 -- Monadic Procedure: state-pop
     Pop a value from the current state and return it as a monadic
     value.  The state is assumed to be a list.

 -- Scheme Procedure: run-with-state MVAL [STATE]
     Run monadic value MVAL starting with STATE as the initial state.
     Return two values: the resulting value, and the resulting state.

   The main interface to the store monad, provided by the ‘(guix store)’
module, is as follows.

 -- Scheme Variable: %store-monad
     The store monad—an alias for ‘%state-monad’.

     Values in the store monad encapsulate accesses to the store.  When
     its effect is needed, a value of the store monad must be
     “evaluated” by passing it to the ‘run-with-store’ procedure (see
     below).

 -- Scheme Procedure: run-with-store STORE MVAL [#:guile-for-build]
          [#:system (%current-system)]
     Run MVAL, a monadic value in the store monad, in STORE, an open
     store connection.

 -- Monadic Procedure: text-file NAME TEXT [REFERENCES]
     Return as a monadic value the absolute file name in the store of
     the file containing TEXT, a string.  REFERENCES is a list of store
     items that the resulting text file refers to; it defaults to the
     empty list.

 -- Monadic Procedure: binary-file NAME DATA [REFERENCES]
     Return as a monadic value the absolute file name in the store of
     the file containing DATA, a bytevector.  REFERENCES is a list of
     store items that the resulting binary file refers to; it defaults
     to the empty list.

 -- Monadic Procedure: interned-file FILE [NAME] [#:recursive? #t]
          [#:select? (const #t)]
     Return the name of FILE once interned in the store.  Use NAME as
     its store name, or the basename of FILE if NAME is omitted.

     When RECURSIVE? is true, the contents of FILE are added
     recursively; if FILE designates a flat file and RECURSIVE? is true,
     its contents are added, and its permission bits are kept.

     When RECURSIVE? is true, call ‘(SELECT? FILE STAT)’ for each
     directory entry, where FILE is the entry’s absolute file name and
     STAT is the result of ‘lstat’; exclude entries for which SELECT?
     does not return true.

     The example below adds a file to the store, under two different
     names:

          (run-with-store (open-connection)
            (mlet %store-monad ((a (interned-file "README"))
                                (b (interned-file "README" "LEGU-MIN")))
              (return (list a b))))

          ⇒ ("/gnu/store/rwm...-README" "/gnu/store/44i...-LEGU-MIN")

   The ‘(guix packages)’ module exports the following package-related
monadic procedures:

 -- Monadic Procedure: package-file PACKAGE [FILE] [#:system
          (%current-system)] [#:target #f] [#:output "out"]
     Return as a monadic value in the absolute file name of FILE within
     the OUTPUT directory of PACKAGE.  When FILE is omitted, return the
     name of the OUTPUT directory of PACKAGE.  When TARGET is true, use
     it as a cross-compilation target triplet.

     Note that this procedure does _not_ build PACKAGE.  Thus, the
     result might or might not designate an existing file.  We recommend
     not using this procedure unless you know what you are doing.

 -- Monadic Procedure: package->derivation PACKAGE [SYSTEM]
 -- Monadic Procedure: package->cross-derivation PACKAGE TARGET [SYSTEM]
     Monadic version of ‘package-derivation’ and
     ‘package-cross-derivation’ (*note Defining Packages::).

   ---------- Footnotes ----------

   (1) This operation is commonly referred to as “bind”, but that name
denotes an unrelated procedure in Guile.  Thus we use this somewhat
cryptic symbol inherited from the Haskell language.


File: guix.info,  Node: G-Expressions,  Next: Invoking guix repl,  Prev: The Store Monad,  Up: Programming Interface

9.12 G-Expressions
==================

So we have “derivations”, which represent a sequence of build actions to
be performed to produce an item in the store (*note Derivations::).
These build actions are performed when asking the daemon to actually
build the derivations; they are run by the daemon in a container (*note
Invoking guix-daemon::).

   It should come as no surprise that we like to write these build
actions in Scheme.  When we do that, we end up with two “strata” of
Scheme code(1): the “host code”—code that defines packages, talks to the
daemon, etc.—and the “build code”—code that actually performs build
actions, such as making directories, invoking ‘make’, and so on (*note
Build Phases::).

   To describe a derivation and its build actions, one typically needs
to embed build code inside host code.  It boils down to manipulating
build code as data, and the homoiconicity of Scheme—code has a direct
representation as data—comes in handy for that.  But we need more than
the normal ‘quasiquote’ mechanism in Scheme to construct build
expressions.

   The ‘(guix gexp)’ module implements “G-expressions”, a form of
S-expressions adapted to build expressions.  G-expressions, or “gexps”,
consist essentially of three syntactic forms: ‘gexp’, ‘ungexp’, and
‘ungexp-splicing’ (or simply: ‘#~’, ‘#$’, and ‘#$@’), which are
comparable to ‘quasiquote’, ‘unquote’, and ‘unquote-splicing’,
respectively (*note ‘quasiquote’: (guile)Expression Syntax.).  However,
there are major differences:

   • Gexps are meant to be written to a file and run or manipulated by
     other processes.

   • When a high-level object such as a package or derivation is
     unquoted inside a gexp, the result is as if its output file name
     had been introduced.

   • Gexps carry information about the packages or derivations they
     refer to, and these dependencies are automatically added as inputs
     to the build processes that use them.

   This mechanism is not limited to package and derivation objects:
“compilers” able to “lower” other high-level objects to derivations or
files in the store can be defined, such that these objects can also be
inserted into gexps.  For example, a useful type of high-level objects
that can be inserted in a gexp is “file-like objects”, which make it
easy to add files to the store and to refer to them in derivations and
such (see ‘local-file’ and ‘plain-file’ below).

   To illustrate the idea, here is an example of a gexp:

     (define build-exp
       #~(begin
           (mkdir #$output)
           (chdir #$output)
           (symlink (string-append #$coreutils "/bin/ls")
                    "list-files")))

   This gexp can be passed to ‘gexp->derivation’; we obtain a derivation
that builds a directory containing exactly one symlink to
‘/gnu/store/...-coreutils-8.22/bin/ls’:

     (gexp->derivation "the-thing" build-exp)

   As one would expect, the ‘"/gnu/store/...-coreutils-8.22"’ string is
substituted to the reference to the COREUTILS package in the actual
build code, and COREUTILS is automatically made an input to the
derivation.  Likewise, ‘#$output’ (equivalent to ‘(ungexp output)’) is
replaced by a string containing the directory name of the output of the
derivation.

   In a cross-compilation context, it is useful to distinguish between
references to the _native_ build of a package—that can run on the
host—versus references to cross builds of a package.  To that end, the
‘#+’ plays the same role as ‘#$’, but is a reference to a native package
build:

     (gexp->derivation "vi"
        #~(begin
            (mkdir #$output)
            (mkdir (string-append #$output "/bin"))
            (system* (string-append #+coreutils "/bin/ln")
                     "-s"
                     (string-append #$emacs "/bin/emacs")
                     (string-append #$output "/bin/vi")))
        #:target "aarch64-linux-gnu")

In the example above, the native build of COREUTILS is used, so that
‘ln’ can actually run on the host; but then the cross-compiled build of
EMACS is referenced.

   Another gexp feature is “imported modules”: sometimes you want to be
able to use certain Guile modules from the “host environment” in the
gexp, so those modules should be imported in the “build environment”.
The ‘with-imported-modules’ form allows you to express that:

     (let ((build (with-imported-modules '((guix build utils))
                    #~(begin
                        (use-modules (guix build utils))
                        (mkdir-p (string-append #$output "/bin"))))))
       (gexp->derivation "empty-dir"
                         #~(begin
                             #$build
                             (display "success!\n")
                             #t)))

In this example, the ‘(guix build utils)’ module is automatically pulled
into the isolated build environment of our gexp, such that ‘(use-modules
(guix build utils))’ works as expected.

   Usually you want the _closure_ of the module to be imported—i.e., the
module itself and all the modules it depends on—rather than just the
module; failing to do that, attempts to use the module will fail because
of missing dependent modules.  The ‘source-module-closure’ procedure
computes the closure of a module by looking at its source file headers,
which comes in handy in this case:

     (use-modules (guix modules))   ;for 'source-module-closure'

     (with-imported-modules (source-module-closure
                              '((guix build utils)
                                (gnu build image)))
       (gexp->derivation "something-with-vms"
                         #~(begin
                             (use-modules (guix build utils)
                                          (gnu build image))
                             ...)))

   In the same vein, sometimes you want to import not just pure-Scheme
modules, but also “extensions” such as Guile bindings to C libraries or
other “full-blown” packages.  Say you need the ‘guile-json’ package
available on the build side, here’s how you would do it:

     (use-modules (gnu packages guile))  ;for 'guile-json'

     (with-extensions (list guile-json)
       (gexp->derivation "something-with-json"
                         #~(begin
                             (use-modules (json))
                             ...)))

   The syntactic form to construct gexps is summarized below.

 -- Scheme Syntax: #~EXP
 -- Scheme Syntax: (gexp EXP)
     Return a G-expression containing EXP.  EXP may contain one or more
     of the following forms:

     ‘#$OBJ’
     ‘(ungexp OBJ)’
          Introduce a reference to OBJ.  OBJ may have one of the
          supported types, for example a package or a derivation, in
          which case the ‘ungexp’ form is replaced by its output file
          name—e.g., ‘"/gnu/store/...-coreutils-8.22’.

          If OBJ is a list, it is traversed and references to supported
          objects are substituted similarly.

          If OBJ is another gexp, its contents are inserted and its
          dependencies are added to those of the containing gexp.

          If OBJ is another kind of object, it is inserted as is.

     ‘#$OBJ:OUTPUT’
     ‘(ungexp OBJ OUTPUT)’
          This is like the form above, but referring explicitly to the
          OUTPUT of OBJ—this is useful when OBJ produces multiple
          outputs (*note Packages with Multiple Outputs::).

     ‘#+OBJ’
     ‘#+OBJ:output’
     ‘(ungexp-native OBJ)’
     ‘(ungexp-native OBJ OUTPUT)’
          Same as ‘ungexp’, but produces a reference to the _native_
          build of OBJ when used in a cross compilation context.

     ‘#$output[:OUTPUT]’
     ‘(ungexp output [OUTPUT])’
          Insert a reference to derivation output OUTPUT, or to the main
          output when OUTPUT is omitted.

          This only makes sense for gexps passed to ‘gexp->derivation’.

     ‘#$@LST’
     ‘(ungexp-splicing LST)’
          Like the above, but splices the contents of LST inside the
          containing list.

     ‘#+@LST’
     ‘(ungexp-native-splicing LST)’
          Like the above, but refers to native builds of the objects
          listed in LST.

     G-expressions created by ‘gexp’ or ‘#~’ are run-time objects of the
     ‘gexp?’ type (see below).

 -- Scheme Syntax: with-imported-modules MODULES BODY...
     Mark the gexps defined in BODY... as requiring MODULES in their
     execution environment.

     Each item in MODULES can be the name of a module, such as ‘(guix
     build utils)’, or it can be a module name, followed by an arrow,
     followed by a file-like object:

          `((guix build utils)
            (guix gcrypt)
            ((guix config) => ,(scheme-file "config.scm"
                                            #~(define-module ...))))

     In the example above, the first two modules are taken from the
     search path, and the last one is created from the given file-like
     object.

     This form has _lexical_ scope: it has an effect on the gexps
     directly defined in BODY..., but not on those defined, say, in
     procedures called from BODY....

 -- Scheme Syntax: with-extensions EXTENSIONS BODY...
     Mark the gexps defined in BODY... as requiring EXTENSIONS in their
     build and execution environment.  EXTENSIONS is typically a list of
     package objects such as those defined in the ‘(gnu packages guile)’
     module.

     Concretely, the packages listed in EXTENSIONS are added to the load
     path while compiling imported modules in BODY...; they are also
     added to the load path of the gexp returned by BODY....

 -- Scheme Procedure: gexp? OBJ
     Return ‘#t’ if OBJ is a G-expression.

   G-expressions are meant to be written to disk, either as code
building some derivation, or as plain files in the store.  The monadic
procedures below allow you to do that (*note The Store Monad::, for more
information about monads).

 -- Monadic Procedure: gexp->derivation NAME EXP [#:system
          (%current-system)] [#:target #f] [#:graft? #t] [#:hash #f]
          [#:hash-algo #f] [#:recursive? #f] [#:env-vars '()] [#:modules
          '()] [#:module-path ‘%load-path’] [#:effective-version "2.2"]
          [#:references-graphs #f] [#:allowed-references #f]
          [#:disallowed-references #f] [#:leaked-env-vars #f]
          [#:script-name (string-append NAME "-builder")]
          [#:deprecation-warnings #f] [#:local-build? #f]
          [#:substitutable? #t] [#:properties '()] [#:guile-for-build
          #f]
     Return a derivation NAME that runs EXP (a gexp) with
     GUILE-FOR-BUILD (a derivation) on SYSTEM; EXP is stored in a file
     called SCRIPT-NAME.  When TARGET is true, it is used as the
     cross-compilation target triplet for packages referred to by EXP.

     MODULES is deprecated in favor of ‘with-imported-modules’.  Its
     meaning is to make MODULES available in the evaluation context of
     EXP; MODULES is a list of names of Guile modules searched in
     MODULE-PATH to be copied in the store, compiled, and made available
     in the load path during the execution of EXP—e.g., ‘((guix build
     utils) (guix build gnu-build-system))’.

     EFFECTIVE-VERSION determines the string to use when adding
     extensions of EXP (see ‘with-extensions’) to the search path—e.g.,
     ‘"2.2"’.

     GRAFT? determines whether packages referred to by EXP should be
     grafted when applicable.

     When REFERENCES-GRAPHS is true, it must be a list of tuples of one
     of the following forms:

          (FILE-NAME PACKAGE)
          (FILE-NAME PACKAGE OUTPUT)
          (FILE-NAME DERIVATION)
          (FILE-NAME DERIVATION OUTPUT)
          (FILE-NAME STORE-ITEM)

     The right-hand-side of each element of REFERENCES-GRAPHS is
     automatically made an input of the build process of EXP.  In the
     build environment, each FILE-NAME contains the reference graph of
     the corresponding item, in a simple text format.

     ALLOWED-REFERENCES must be either ‘#f’ or a list of output names
     and packages.  In the latter case, the list denotes store items
     that the result is allowed to refer to.  Any reference to another
     store item will lead to a build error.  Similarly for
     DISALLOWED-REFERENCES, which can list items that must not be
     referenced by the outputs.

     DEPRECATION-WARNINGS determines whether to show deprecation
     warnings while compiling modules.  It can be ‘#f’, ‘#t’, or
     ‘'detailed’.

     The other arguments are as for ‘derivation’ (*note Derivations::).

   The ‘local-file’, ‘plain-file’, ‘computed-file’, ‘program-file’, and
‘scheme-file’ procedures below return “file-like objects”.  That is,
when unquoted in a G-expression, these objects lead to a file in the
store.  Consider this G-expression:

     #~(system* #$(file-append glibc "/sbin/nscd") "-f"
                #$(local-file "/tmp/my-nscd.conf"))

   The effect here is to “intern” ‘/tmp/my-nscd.conf’ by copying it to
the store.  Once expanded, for instance via ‘gexp->derivation’, the
G-expression refers to that copy under ‘/gnu/store’; thus, modifying or
removing the file in ‘/tmp’ does not have any effect on what the
G-expression does.  ‘plain-file’ can be used similarly; it differs in
that the file content is directly passed as a string.

 -- Scheme Procedure: local-file FILE [NAME] [#:recursive? #f]
          [#:select? (const #t)]
     Return an object representing local file FILE to add to the store;
     this object can be used in a gexp.  If FILE is a literal string
     denoting a relative file name, it is looked up relative to the
     source file where it appears; if FILE is not a literal string, it
     is looked up relative to the current working directory at run time.
     FILE will be added to the store under NAME–by default the base name
     of FILE.

     When RECURSIVE? is true, the contents of FILE are added
     recursively; if FILE designates a flat file and RECURSIVE? is true,
     its contents are added, and its permission bits are kept.

     When RECURSIVE? is true, call ‘(SELECT? FILE STAT)’ for each
     directory entry, where FILE is the entry’s absolute file name and
     STAT is the result of ‘lstat’; exclude entries for which SELECT?
     does not return true.

     This is the declarative counterpart of the ‘interned-file’ monadic
     procedure (*note ‘interned-file’: The Store Monad.).

 -- Scheme Procedure: plain-file NAME CONTENT
     Return an object representing a text file called NAME with the
     given CONTENT (a string or a bytevector) to be added to the store.

     This is the declarative counterpart of ‘text-file’.

 -- Scheme Procedure: computed-file NAME GEXP [#:local-build? #t]
          [#:options '()]
     Return an object representing the store item NAME, a file or
     directory computed by GEXP.  When LOCAL-BUILD? is true (the
     default), the derivation is built locally.  OPTIONS is a list of
     additional arguments to pass to ‘gexp->derivation’.

     This is the declarative counterpart of ‘gexp->derivation’.

 -- Monadic Procedure: gexp->script NAME EXP [#:guile (default-guile)]
          [#:module-path %load-path] [#:system (%current-system)]
          [#:target #f]
     Return an executable script NAME that runs EXP using GUILE, with
     EXP’s imported modules in its search path.  Look up EXP’s modules
     in MODULE-PATH.

     The example below builds a script that simply invokes the ‘ls’
     command:

          (use-modules (guix gexp) (gnu packages base))

          (gexp->script "list-files"
                        #~(execl #$(file-append coreutils "/bin/ls")
                                 "ls"))

     When “running” it through the store (*note ‘run-with-store’: The
     Store Monad.), we obtain a derivation that produces an executable
     file ‘/gnu/store/...-list-files’ along these lines:

          #!/gnu/store/...-guile-2.0.11/bin/guile -ds
          !#
          (execl "/gnu/store/...-coreutils-8.22"/bin/ls" "ls")

 -- Scheme Procedure: program-file NAME EXP [#:guile #f] [#:module-path
          %load-path]
     Return an object representing the executable store item NAME that
     runs GEXP.  GUILE is the Guile package used to execute that script.
     Imported modules of GEXP are looked up in MODULE-PATH.

     This is the declarative counterpart of ‘gexp->script’.

 -- Monadic Procedure: gexp->file NAME EXP [#:set-load-path? #t]
          [#:module-path %load-path] [#:splice? #f] [#:guile
          (default-guile)]
     Return a derivation that builds a file NAME containing EXP.  When
     SPLICE? is true, EXP is considered to be a list of expressions that
     will be spliced in the resulting file.

     When SET-LOAD-PATH? is true, emit code in the resulting file to set
     ‘%load-path’ and ‘%load-compiled-path’ to honor EXP’s imported
     modules.  Look up EXP’s modules in MODULE-PATH.

     The resulting file holds references to all the dependencies of EXP
     or a subset thereof.

 -- Scheme Procedure: scheme-file NAME EXP [#:splice? #f]
          [#:set-load-path? #t]
     Return an object representing the Scheme file NAME that contains
     EXP.

     This is the declarative counterpart of ‘gexp->file’.

 -- Monadic Procedure: text-file* NAME TEXT ...
     Return as a monadic value a derivation that builds a text file
     containing all of TEXT.  TEXT may list, in addition to strings,
     objects of any type that can be used in a gexp: packages,
     derivations, local file objects, etc.  The resulting store file
     holds references to all these.

     This variant should be preferred over ‘text-file’ anytime the file
     to create will reference items from the store.  This is typically
     the case when building a configuration file that embeds store file
     names, like this:

          (define (profile.sh)
            ;; Return the name of a shell script in the store that
            ;; initializes the 'PATH' environment variable.
            (text-file* "profile.sh"
                        "export PATH=" coreutils "/bin:"
                        grep "/bin:" sed "/bin\n"))

     In this example, the resulting ‘/gnu/store/...-profile.sh’ file
     will reference COREUTILS, GREP, and SED, thereby preventing them
     from being garbage-collected during its lifetime.

 -- Scheme Procedure: mixed-text-file NAME TEXT ...
     Return an object representing store file NAME containing TEXT.
     TEXT is a sequence of strings and file-like objects, as in:

          (mixed-text-file "profile"
                           "export PATH=" coreutils "/bin:" grep "/bin")

     This is the declarative counterpart of ‘text-file*’.

 -- Scheme Procedure: file-union NAME FILES
     Return a ‘<computed-file>’ that builds a directory containing all
     of FILES.  Each item in FILES must be a two-element list where the
     first element is the file name to use in the new directory, and the
     second element is a gexp denoting the target file.  Here’s an
     example:

          (file-union "etc"
                      `(("hosts" ,(plain-file "hosts"
                                              "127.0.0.1 localhost"))
                        ("bashrc" ,(plain-file "bashrc"
                                               "alias ls='ls --color=auto'"))))

     This yields an ‘etc’ directory containing these two files.

 -- Scheme Procedure: directory-union NAME THINGS
     Return a directory that is the union of THINGS, where THINGS is a
     list of file-like objects denoting directories.  For example:

          (directory-union "guile+emacs" (list guile emacs))

     yields a directory that is the union of the ‘guile’ and ‘emacs’
     packages.

 -- Scheme Procedure: file-append OBJ SUFFIX ...
     Return a file-like object that expands to the concatenation of OBJ
     and SUFFIX, where OBJ is a lowerable object and each SUFFIX is a
     string.

     As an example, consider this gexp:

          (gexp->script "run-uname"
                        #~(system* #$(file-append coreutils
                                                  "/bin/uname")))

     The same effect could be achieved with:

          (gexp->script "run-uname"
                        #~(system* (string-append #$coreutils
                                                  "/bin/uname")))

     There is one difference though: in the ‘file-append’ case, the
     resulting script contains the absolute file name as a string,
     whereas in the second case, the resulting script contains a
     ‘(string-append ...)’ expression to construct the file name _at run
     time_.

 -- Scheme Syntax: let-system SYSTEM BODY...
 -- Scheme Syntax: let-system (SYSTEM TARGET) BODY...
     Bind SYSTEM to the currently targeted system—e.g.,
     ‘"x86_64-linux"’—within BODY.

     In the second case, additionally bind TARGET to the current
     cross-compilation target—a GNU triplet such as
     ‘"arm-linux-gnueabihf"’—or ‘#f’ if we are not cross-compiling.

     ‘let-system’ is useful in the occasional case where the object
     spliced into the gexp depends on the target system, as in this
     example:

          #~(system*
             #+(let-system system
                 (cond ((string-prefix? "armhf-" system)
                        (file-append qemu "/bin/qemu-system-arm"))
                       ((string-prefix? "x86_64-" system)
                        (file-append qemu "/bin/qemu-system-x86_64"))
                       (else
                        (error "dunno!"))))
             "-net" "user" #$image)

 -- Scheme Syntax: with-parameters ((PARAMETER VALUE) ...) EXP
     This macro is similar to the ‘parameterize’ form for
     dynamically-bound “parameters” (*note (guile)Parameters::).  The
     key difference is that it takes effect when the file-like object
     returned by EXP is lowered to a derivation or store item.

     A typical use of ‘with-parameters’ is to force the system in effect
     for a given object:

          (with-parameters ((%current-system "i686-linux"))
            coreutils)

     The example above returns an object that corresponds to the i686
     build of Coreutils, regardless of the current value of
     ‘%current-system’.

   Of course, in addition to gexps embedded in “host” code, there are
also modules containing build tools.  To make it clear that they are
meant to be used in the build stratum, these modules are kept in the
‘(guix build ...)’ name space.

   Internally, high-level objects are “lowered”, using their compiler,
to either derivations or store items.  For instance, lowering a package
yields a derivation, and lowering a ‘plain-file’ yields a store item.
This is achieved using the ‘lower-object’ monadic procedure.

 -- Monadic Procedure: lower-object OBJ [SYSTEM] [#:target #f]
     Return as a value in ‘%store-monad’ the derivation or store item
     corresponding to OBJ for SYSTEM, cross-compiling for TARGET if
     TARGET is true.  OBJ must be an object that has an associated gexp
     compiler, such as a ‘<package>’.

 -- Procedure: gexp->approximate-sexp GEXP
     Sometimes, it may be useful to convert a G-exp into a S-exp.  For
     example, some linters (*note Invoking guix lint::) peek into the
     build phases of a package to detect potential problems.  This
     conversion can be achieved with this procedure.  However, some
     information can be lost in the process.  More specifically,
     lowerable objects will be silently replaced with some arbitrary
     object – currently the list ‘(*approximate*)’, but this may change.

   ---------- Footnotes ----------

   (1) The term “stratum” in this context was coined by Manuel Serrano
et al. in the context of their work on Hop.  Oleg Kiselyov, who has
written insightful essays and code on this topic
(http://okmij.org/ftp/meta-programming/#meta-scheme), refers to this
kind of code generation as “staging”.


File: guix.info,  Node: Invoking guix repl,  Next: Using Guix Interactively,  Prev: G-Expressions,  Up: Programming Interface

9.13 Invoking ‘guix repl’
=========================

The ‘guix repl’ command makes it easier to program Guix in Guile by
launching a Guile “read-eval-print loop” (REPL) for interactive
programming (*note (guile)Using Guile Interactively::), or by running
Guile scripts (*note (guile)Running Guile Scripts::).  Compared to just
launching the ‘guile’ command, ‘guix repl’ guarantees that all the Guix
modules and all its dependencies are available in the search path.

   The general syntax is:

     guix repl OPTIONS [FILE ARGS]

   When a FILE argument is provided, FILE is executed as a Guile
scripts:

     guix repl my-script.scm

   To pass arguments to the script, use ‘--’ to prevent them from being
interpreted as arguments to ‘guix repl’ itself:

     guix repl -- my-script.scm --input=foo.txt

   To make a script executable directly from the shell, using the guix
executable that is on the user’s search path, add the following two
lines at the top of the script:

     #!/usr/bin/env -S guix repl --
     !#

   Without a file name argument, a Guile REPL is started, allowing for
interactive use (*note Using Guix Interactively::):

     $ guix repl
     scheme@(guile-user)> ,use (gnu packages base)
     scheme@(guile-user)> coreutils
     $1 = #<package coreutils@8.29 gnu/packages/base.scm:327 3e28300>

   In addition, ‘guix repl’ implements a simple machine-readable REPL
protocol for use by ‘(guix inferior)’, a facility to interact with
“inferiors”, separate processes running a potentially different revision
of Guix.

   The available options are as follows:

‘--type=TYPE’
‘-t TYPE’
     Start a REPL of the given TYPE, which can be one of the following:

     ‘guile’
          This is default, and it spawns a standard full-featured Guile
          REPL.
     ‘machine’
          Spawn a REPL that uses the machine-readable protocol.  This is
          the protocol that the ‘(guix inferior)’ module speaks.

‘--listen=ENDPOINT’
     By default, ‘guix repl’ reads from standard input and writes to
     standard output.  When this option is passed, it will instead
     listen for connections on ENDPOINT.  Here are examples of valid
     options:

     ‘--listen=tcp:37146’
          Accept connections on localhost on port 37146.

     ‘--listen=unix:/tmp/socket’
          Accept connections on the Unix-domain socket ‘/tmp/socket’.

‘--load-path=DIRECTORY’
‘-L DIRECTORY’
     Add DIRECTORY to the front of the package module search path (*note
     Package Modules::).

     This allows users to define their own packages and make them
     visible to the script or REPL.

‘-q’
     Inhibit loading of the ‘~/.guile’ file.  By default, that
     configuration file is loaded when spawning a ‘guile’ REPL.


File: guix.info,  Node: Using Guix Interactively,  Prev: Invoking guix repl,  Up: Programming Interface

9.14 Using Guix Interactively
=============================

The ‘guix repl’ command gives you access to a warm and friendly
“read-eval-print loop” (REPL) (*note Invoking guix repl::).  If you’re
getting into Guix programming—defining your own packages, writing
manifests, defining services for Guix System or Guix Home, etc.—you will
surely find it convenient to toy with ideas at the REPL.

   If you use Emacs, the most convenient way to do that is with Geiser
(*note The Perfect Setup::), but you do not have to use Emacs to enjoy
the REPL.  When using ‘guix repl’ or ‘guile’ in the terminal, we
recommend using Readline for completion and Colorized to get colorful
output.  To do that, you can run:

     guix install guile guile-readline guile-colorized

...  and then create a ‘.guile’ file in your home directory containing
this:

     (use-modules (ice-9 readline) (ice-9 colorized))

     (activate-readline)
     (activate-colorized)

   The REPL lets you evaluate Scheme code; you type a Scheme expression
at the prompt, and the REPL prints what it evaluates to:

     $ guix repl
     scheme@(guix-user)> (+ 2 3)
     $1 = 5
     scheme@(guix-user)> (string-append "a" "b")
     $2 = "ab"

   It becomes interesting when you start fiddling with Guix at the REPL.
The first thing you’ll want to do is to “import” the ‘(guix)’ module,
which gives access to the main part of the programming interface, and
perhaps a bunch of useful Guix modules.  You could type ‘(use-modules
(guix))’, which is valid Scheme code to import a module (*note
(guile)Using Guile Modules::), but the REPL provides the ‘use’ “command”
as a shorthand notation (*note (guile)REPL Commands::):

     scheme@(guix-user)> ,use (guix)
     scheme@(guix-user)> ,use (gnu packages base)

   Notice that REPL commands are introduced by a leading comma.  A REPL
command like ‘use’ is not valid Scheme code; it’s interpreted specially
by the REPL.

   Guix extends the Guile REPL with additional commands for convenience.
Among those, the ‘build’ command comes in handy: it ensures that the
given file-like object is built, building it if needed, and returns its
output file name(s).  In the example below, we build the ‘coreutils’ and
‘grep’ packages, as well as a “computed file” (*note ‘computed-file’:
G-Expressions.), and we use the ‘scandir’ procedure to list the files in
Grep’s ‘/bin’ directory:

     scheme@(guix-user)> ,build coreutils
     $1 = "/gnu/store/...-coreutils-8.32-debug"
     $2 = "/gnu/store/...-coreutils-8.32"
     scheme@(guix-user)> ,build grep
     $3 = "/gnu/store/...-grep-3.6"
     scheme@(guix-user)> ,build (computed-file "x" #~(mkdir #$output))
     building /gnu/store/...-x.drv...
     $4 = "/gnu/store/...-x"
     scheme@(guix-user)> ,use(ice-9 ftw)
     scheme@(guix-user)> (scandir (string-append $3 "/bin"))
     $5 = ("." ".." "egrep" "fgrep" "grep")

   At a lower-level, a useful command is ‘lower’: it takes a file-like
object and “lowers” it into a derivation (*note Derivations::) or a
store file:

     scheme@(guix-user)> ,lower grep
     $6 = #<derivation /gnu/store/...-grep-3.6.drv => /gnu/store/...-grep-3.6 7f0e639115f0>
     scheme@(guix-user)> ,lower (plain-file "x" "Hello!")
     $7 = "/gnu/store/...-x"

   The full list of REPL commands can be seen by typing ‘,help guix’ and
is given below for reference.

 -- REPL command: build OBJECT
     Lower OBJECT and build it if it’s not already built, returning its
     output file name(s).

 -- REPL command: lower OBJECT
     Lower OBJECT into a derivation or store file name and return it.

 -- REPL command: verbosity LEVEL
     Change build verbosity to LEVEL.

     This is similar to the ‘--verbosity’ command-line option (*note
     Common Build Options::): level 0 means total silence, level 1 shows
     build events only, and higher levels print build logs.

 -- REPL command: run-in-store EXP
     Run EXP, a monadic expresssion, through the store monad.  *Note The
     Store Monad::, for more information.

 -- REPL command: enter-store-monad
     Enter a new REPL to evaluate monadic expressions (*note The Store
     Monad::).  You can quit this “inner” REPL by typing ‘,q’.


File: guix.info,  Node: Utilities,  Next: Foreign Architectures,  Prev: Programming Interface,  Up: Top

10 Utilities
************

This section describes Guix command-line utilities.  Some of them are
primarily targeted at developers and users who write new package
definitions, while others are more generally useful.  They complement
the Scheme programming interface of Guix in a convenient way.

* Menu:

* Invoking guix build::         Building packages from the command line.
* Invoking guix edit::          Editing package definitions.
* Invoking guix download::      Downloading a file and printing its hash.
* Invoking guix hash::          Computing the cryptographic hash of a file.
* Invoking guix import::        Importing package definitions.
* Invoking guix refresh::       Updating package definitions.
* Invoking guix style::         Styling package definitions.
* Invoking guix lint::          Finding errors in package definitions.
* Invoking guix size::          Profiling disk usage.
* Invoking guix graph::         Visualizing the graph of packages.
* Invoking guix publish::       Sharing substitutes.
* Invoking guix challenge::     Challenging substitute servers.
* Invoking guix copy::          Copying to and from a remote store.
* Invoking guix container::     Process isolation.
* Invoking guix weather::       Assessing substitute availability.
* Invoking guix processes::     Listing client processes.


File: guix.info,  Node: Invoking guix build,  Next: Invoking guix edit,  Up: Utilities

10.1 Invoking ‘guix build’
==========================

The ‘guix build’ command builds packages or derivations and their
dependencies, and prints the resulting store paths.  Note that it does
not modify the user’s profile—this is the job of the ‘guix package’
command (*note Invoking guix package::).  Thus, it is mainly useful for
distribution developers.

   The general syntax is:

     guix build OPTIONS PACKAGE-OR-DERIVATION...

   As an example, the following command builds the latest versions of
Emacs and of Guile, displays their build logs, and finally displays the
resulting directories:

     guix build emacs guile

   Similarly, the following command builds all the available packages:

     guix build --quiet --keep-going \
       $(guix package -A | awk '{ print $1 "@" $2 }')

   PACKAGE-OR-DERIVATION may be either the name of a package found in
the software distribution such as ‘coreutils’ or ‘coreutils@8.20’, or a
derivation such as ‘/gnu/store/...-coreutils-8.19.drv’.  In the former
case, a package with the corresponding name (and optionally version) is
searched for among the GNU distribution modules (*note Package
Modules::).

   Alternatively, the ‘--expression’ option may be used to specify a
Scheme expression that evaluates to a package; this is useful when
disambiguating among several same-named packages or package variants is
needed.

   There may be zero or more OPTIONS.  The available options are
described in the subsections below.

* Menu:

* Common Build Options::        Build options for most commands.
* Package Transformation Options::  Creating variants of packages.
* Additional Build Options::    Options specific to ’guix build’.
* Debugging Build Failures::    Real life packaging experience.


File: guix.info,  Node: Common Build Options,  Next: Package Transformation Options,  Up: Invoking guix build

10.1.1 Common Build Options
---------------------------

A number of options that control the build process are common to ‘guix
build’ and other commands that can spawn builds, such as ‘guix package’
or ‘guix archive’.  These are the following:

‘--load-path=DIRECTORY’
‘-L DIRECTORY’
     Add DIRECTORY to the front of the package module search path (*note
     Package Modules::).

     This allows users to define their own packages and make them
     visible to the command-line tools.

‘--keep-failed’
‘-K’
     Keep the build tree of failed builds.  Thus, if a build fails, its
     build tree is kept under ‘/tmp’, in a directory whose name is shown
     at the end of the build log.  This is useful when debugging build
     issues.  *Note Debugging Build Failures::, for tips and tricks on
     how to debug build issues.

     This option implies ‘--no-offload’, and it has no effect when
     connecting to a remote daemon with a ‘guix://’ URI (*note the
     ‘GUIX_DAEMON_SOCKET’ variable: The Store.).

‘--keep-going’
‘-k’
     Keep going when some of the derivations fail to build; return only
     once all the builds have either completed or failed.

     The default behavior is to stop as soon as one of the specified
     derivations has failed.

‘--dry-run’
‘-n’
     Do not build the derivations.

‘--fallback’
     When substituting a pre-built binary fails, fall back to building
     packages locally (*note Substitution Failure::).

‘--substitute-urls=URLS’
     Consider URLS the whitespace-separated list of substitute source
     URLs, overriding the default list of URLs of ‘guix-daemon’ (*note
     ‘guix-daemon’ URLs: daemon-substitute-urls.).

     This means that substitutes may be downloaded from URLS, provided
     they are signed by a key authorized by the system administrator
     (*note Substitutes::).

     When URLS is the empty string, substitutes are effectively
     disabled.

‘--no-substitutes’
     Do not use substitutes for build products.  That is, always build
     things locally instead of allowing downloads of pre-built binaries
     (*note Substitutes::).

‘--no-grafts’
     Do not “graft” packages.  In practice, this means that package
     updates available as grafts are not applied.  *Note Security
     Updates::, for more information on grafts.

‘--rounds=N’
     Build each derivation N times in a row, and raise an error if
     consecutive build results are not bit-for-bit identical.

     This is a useful way to detect non-deterministic builds processes.
     Non-deterministic build processes are a problem because they make
     it practically impossible for users to _verify_ whether third-party
     binaries are genuine.  *Note Invoking guix challenge::, for more.

     When used in conjunction with ‘--keep-failed’, the differing output
     is kept in the store, under ‘/gnu/store/...-check’.  This makes it
     easy to look for differences between the two results.

‘--no-offload’
     Do not use offload builds to other machines (*note Daemon Offload
     Setup::).  That is, always build things locally instead of
     offloading builds to remote machines.

‘--max-silent-time=SECONDS’
     When the build or substitution process remains silent for more than
     SECONDS, terminate it and report a build failure.

     By default, the daemon’s setting is honored (*note
     ‘--max-silent-time’: Invoking guix-daemon.).

‘--timeout=SECONDS’
     Likewise, when the build or substitution process lasts for more
     than SECONDS, terminate it and report a build failure.

     By default, the daemon’s setting is honored (*note ‘--timeout’:
     Invoking guix-daemon.).

‘-v LEVEL’
‘--verbosity=LEVEL’
     Use the given verbosity LEVEL, an integer.  Choosing 0 means that
     no output is produced, 1 is for quiet output; 2 is similar to 1 but
     it additionally displays download URLs; 3 shows all the build log
     output on standard error.

‘--cores=N’
‘-c N’
     Allow the use of up to N CPU cores for the build.  The special
     value ‘0’ means to use as many CPU cores as available.

‘--max-jobs=N’
‘-M N’
     Allow at most N build jobs in parallel.  *Note ‘--max-jobs’:
     Invoking guix-daemon, for details about this option and the
     equivalent ‘guix-daemon’ option.

‘--debug=LEVEL’
     Produce debugging output coming from the build daemon.  LEVEL must
     be an integer between 0 and 5; higher means more verbose output.
     Setting a level of 4 or more may be helpful when debugging setup
     issues with the build daemon.

   Behind the scenes, ‘guix build’ is essentially an interface to the
‘package-derivation’ procedure of the ‘(guix packages)’ module, and to
the ‘build-derivations’ procedure of the ‘(guix derivations)’ module.

   In addition to options explicitly passed on the command line, ‘guix
build’ and other ‘guix’ commands that support building honor the
‘GUIX_BUILD_OPTIONS’ environment variable.

 -- Environment Variable: GUIX_BUILD_OPTIONS
     Users can define this variable to a list of command line options
     that will automatically be used by ‘guix build’ and other ‘guix’
     commands that can perform builds, as in the example below:

          $ export GUIX_BUILD_OPTIONS="--no-substitutes -c 2 -L /foo/bar"

     These options are parsed independently, and the result is appended
     to the parsed command-line options.


File: guix.info,  Node: Package Transformation Options,  Next: Additional Build Options,  Prev: Common Build Options,  Up: Invoking guix build

10.1.2 Package Transformation Options
-------------------------------------

Another set of command-line options supported by ‘guix build’ and also
‘guix package’ are “package transformation options”.  These are options
that make it possible to define “package variants”—for instance,
packages built from different source code.  This is a convenient way to
create customized packages on the fly without having to type in the
definitions of package variants (*note Defining Packages::).

   Package transformation options are preserved across upgrades: ‘guix
upgrade’ attempts to apply transformation options initially used when
creating the profile to the upgraded packages.

   The available options are listed below.  Most commands support them
and also support a ‘--help-transform’ option that lists all the
available options and a synopsis (these options are not shown in the
‘--help’ output for brevity).

‘--tune[=CPU]’
     Use versions of the packages marked as “tunable” optimized for CPU.
     When CPU is ‘native’, or when it is omitted, tune for the CPU on
     which the ‘guix’ command is running.

     Valid CPU names are those recognized by the underlying compiler, by
     default the GNU Compiler Collection.  On x86_64 processors, this
     includes CPU names such as ‘nehalem’, ‘haswell’, and ‘skylake’
     (*note ‘-march’: (gcc)x86 Options.).

     As new generations of CPUs come out, they augment the standard
     instruction set architecture (ISA) with additional instructions, in
     particular instructions for single-instruction/multiple-data (SIMD)
     parallel processing.  For example, while Core2 and Skylake CPUs
     both implement the x86_64 ISA, only the latter supports AVX2 SIMD
     instructions.

     The primary gain one can expect from ‘--tune’ is for programs that
     can make use of those SIMD capabilities _and_ that do not already
     have a mechanism to select the right optimized code at run time.
     Packages that have the ‘tunable?’ property set are considered
     “tunable packages” by the ‘--tune’ option; a package definition
     with the property set looks like this:

          (package
            (name "hello-simd")
            ;; ...

            ;; This package may benefit from SIMD extensions so
            ;; mark it as "tunable".
            (properties '((tunable? . #t))))

     Other packages are not considered tunable.  This allows Guix to use
     generic binaries in the cases where tuning for a specific CPU is
     unlikely to provide any gain.

     Tuned packages are built with ‘-march=CPU’; under the hood, the
     ‘-march’ option is passed to the actual wrapper by a compiler
     wrapper.  Since the build machine may not be able to run code for
     the target CPU micro-architecture, the test suite is not run when
     building a tuned package.

     To reduce rebuilds to the minimum, tuned packages are _grafted_
     onto packages that depend on them (*note grafts: Security
     Updates.).  Thus, using ‘--no-grafts’ cancels the effect of
     ‘--tune’.

     We call this technique “package multi-versioning”: several variants
     of tunable packages may be built, one for each CPU variant.  It is
     the coarse-grain counterpart of “function multi-versioning” as
     implemented by the GNU tool chain (*note (gcc)Function
     Multiversioning::).

‘--with-source=SOURCE’
‘--with-source=PACKAGE=SOURCE’
‘--with-source=PACKAGE@VERSION=SOURCE’
     Use SOURCE as the source of PACKAGE, and VERSION as its version
     number.  SOURCE must be a file name or a URL, as for ‘guix
     download’ (*note Invoking guix download::).

     When PACKAGE is omitted, it is taken to be the package name
     specified on the command line that matches the base of SOURCE—e.g.,
     if SOURCE is ‘/src/guile-2.0.10.tar.gz’, the corresponding package
     is ‘guile’.

     Likewise, when VERSION is omitted, the version string is inferred
     from SOURCE; in the previous example, it is ‘2.0.10’.

     This option allows users to try out versions of packages other than
     the one provided by the distribution.  The example below downloads
     ‘ed-1.7.tar.gz’ from a GNU mirror and uses that as the source for
     the ‘ed’ package:

          guix build ed --with-source=mirror://gnu/ed/ed-1.4.tar.gz

     As a developer, ‘--with-source’ makes it easy to test release
     candidates, and even to test their impact on packages that depend
     on them:

          guix build elogind --with-source=.../shepherd-0.9.0rc1.tar.gz

     ... or to build from a checkout in a pristine environment:

          $ git clone git://git.sv.gnu.org/guix.git
          $ guix build guix --with-source=guix@1.0=./guix

‘--with-input=PACKAGE=REPLACEMENT’
     Replace dependency on PACKAGE by a dependency on REPLACEMENT.
     PACKAGE must be a package name, and REPLACEMENT must be a package
     specification such as ‘guile’ or ‘guile@1.8’.

     For instance, the following command builds Guix, but replaces its
     dependency on the current stable version of Guile with a dependency
     on the legacy version of Guile, ‘guile@2.0’:

          guix build --with-input=guile=guile@2.0 guix

     This is a recursive, deep replacement.  So in this example, both
     ‘guix’ and its dependency ‘guile-json’ (which also depends on
     ‘guile’) get rebuilt against ‘guile@2.0’.

     This is implemented using the ‘package-input-rewriting’ Scheme
     procedure (*note ‘package-input-rewriting’: Defining Packages.).

‘--with-graft=PACKAGE=REPLACEMENT’
     This is similar to ‘--with-input’ but with an important difference:
     instead of rebuilding the whole dependency chain, REPLACEMENT is
     built and then “grafted” onto the binaries that were initially
     referring to PACKAGE.  *Note Security Updates::, for more
     information on grafts.

     For example, the command below grafts version 3.5.4 of GnuTLS onto
     Wget and all its dependencies, replacing references to the version
     of GnuTLS they currently refer to:

          guix build --with-graft=gnutls=gnutls@3.5.4 wget

     This has the advantage of being much faster than rebuilding
     everything.  But there is a caveat: it works if and only if PACKAGE
     and REPLACEMENT are strictly compatible—for example, if they
     provide a library, the application binary interface (ABI) of those
     libraries must be compatible.  If REPLACEMENT is somehow
     incompatible with PACKAGE, then the resulting package may be
     unusable.  Use with care!

‘--with-debug-info=PACKAGE’
     Build PACKAGE in a way that preserves its debugging info and graft
     it onto packages that depend on it.  This is useful if PACKAGE does
     not already provide debugging info as a ‘debug’ output (*note
     Installing Debugging Files::).

     For example, suppose you’re experiencing a crash in Inkscape and
     would like to see what’s up in GLib, a library deep down in
     Inkscape’s dependency graph.  GLib lacks a ‘debug’ output, so
     debugging is tough.  Fortunately, you rebuild GLib with debugging
     info and tack it on Inkscape:

          guix install inkscape --with-debug-info=glib

     Only GLib needs to be recompiled so this takes a reasonable amount
     of time.  *Note Installing Debugging Files::, for more info.

          Note: Under the hood, this option works by passing the
          ‘#:strip-binaries? #f’ to the build system of the package of
          interest (*note Build Systems::).  Most build systems support
          that option but some do not.  In that case, an error is
          raised.

          Likewise, if a C/C++ package is built without ‘-g’ (which is
          rarely the case), debugging info will remain unavailable even
          when ‘#:strip-binaries?’ is false.

‘--with-c-toolchain=PACKAGE=TOOLCHAIN’
     This option changes the compilation of PACKAGE and everything that
     depends on it so that they get built with TOOLCHAIN instead of the
     default GNU tool chain for C/C++.

     Consider this example:

          guix build octave-cli \
            --with-c-toolchain=fftw=gcc-toolchain@10 \
            --with-c-toolchain=fftwf=gcc-toolchain@10

     The command above builds a variant of the ‘fftw’ and ‘fftwf’
     packages using version 10 of ‘gcc-toolchain’ instead of the default
     tool chain, and then builds a variant of the GNU Octave
     command-line interface using them.  GNU Octave itself is also built
     with ‘gcc-toolchain@10’.

     This other example builds the Hardware Locality (‘hwloc’) library
     and its dependents up to ‘intel-mpi-benchmarks’ with the Clang C
     compiler:

          guix build --with-c-toolchain=hwloc=clang-toolchain \
                     intel-mpi-benchmarks

          Note: There can be application binary interface (ABI)
          incompatibilities among tool chains.  This is particularly
          true of the C++ standard library and run-time support
          libraries such as that of OpenMP.  By rebuilding all
          dependents with the same tool chain, ‘--with-c-toolchain’
          minimizes the risks of incompatibility but cannot entirely
          eliminate them.  Choose PACKAGE wisely.

‘--with-git-url=PACKAGE=URL’
     Build PACKAGE from the latest commit of the ‘master’ branch of the
     Git repository at URL.  Git sub-modules of the repository are
     fetched, recursively.

     For example, the following command builds the NumPy Python library
     against the latest commit of the master branch of Python itself:

          guix build python-numpy \
            --with-git-url=python=https://github.com/python/cpython

     This option can also be combined with ‘--with-branch’ or
     ‘--with-commit’ (see below).

     Obviously, since it uses the latest commit of the given branch, the
     result of such a command varies over time.  Nevertheless it is a
     convenient way to rebuild entire software stacks against the latest
     commit of one or more packages.  This is particularly useful in the
     context of continuous integration (CI).

     Checkouts are kept in a cache under ‘~/.cache/guix/checkouts’ to
     speed up consecutive accesses to the same repository.  You may want
     to clean it up once in a while to save disk space.

‘--with-branch=PACKAGE=BRANCH’
     Build PACKAGE from the latest commit of BRANCH.  If the ‘source’
     field of PACKAGE is an origin with the ‘git-fetch’ method (*note
     origin Reference::) or a ‘git-checkout’ object, the repository URL
     is taken from that ‘source’.  Otherwise you have to use
     ‘--with-git-url’ to specify the URL of the Git repository.

     For instance, the following command builds ‘guile-sqlite3’ from the
     latest commit of its ‘master’ branch, and then builds ‘guix’ (which
     depends on it) and ‘cuirass’ (which depends on ‘guix’) against this
     specific ‘guile-sqlite3’ build:

          guix build --with-branch=guile-sqlite3=master cuirass

‘--with-commit=PACKAGE=COMMIT’
     This is similar to ‘--with-branch’, except that it builds from
     COMMIT rather than the tip of a branch.  COMMIT must be a valid Git
     commit SHA1 identifier, a tag, or a ‘git describe’ style identifier
     such as ‘1.0-3-gabc123’.

‘--with-patch=PACKAGE=FILE’
     Add FILE to the list of patches applied to PACKAGE, where PACKAGE
     is a spec such as ‘python@3.8’ or ‘glibc’.  FILE must contain a
     patch; it is applied with the flags specified in the ‘origin’ of
     PACKAGE (*note origin Reference::), which by default includes ‘-p1’
     (*note (diffutils)patch Directories::).

     As an example, the command below rebuilds Coreutils with the GNU C
     Library (glibc) patched with the given patch:

          guix build coreutils --with-patch=glibc=./glibc-frob.patch

     In this example, glibc itself as well as everything that leads to
     Coreutils in the dependency graph is rebuilt.

‘--with-latest=PACKAGE’
     So you like living on the bleeding edge?  This option is for you!
     It replaces occurrences of PACKAGE in the dependency graph with its
     latest upstream version, as reported by ‘guix refresh’ (*note
     Invoking guix refresh::).

     It does so by determining the latest upstream release of PACKAGE
     (if possible), downloading it, and authenticating it _if_ it comes
     with an OpenPGP signature.

     As an example, the command below builds Guix against the latest
     version of Guile-JSON:

          guix build guix --with-latest=guile-json

     There are limitations.  First, in cases where the tool cannot or
     does not know how to authenticate source code, you are at risk of
     running malicious code; a warning is emitted in this case.  Second,
     this option simply changes the source used in the existing package
     definitions, which is not always sufficient: there might be
     additional dependencies that need to be added, patches to apply,
     and more generally the quality assurance work that Guix developers
     normally do will be missing.

     You’ve been warned!  In all the other cases, it’s a snappy way to
     stay on top.  We encourage you to submit patches updating the
     actual package definitions once you have successfully tested an
     upgrade (*note Contributing::).

‘--without-tests=PACKAGE’
     Build PACKAGE without running its tests.  This can be useful in
     situations where you want to skip the lengthy test suite of a
     intermediate package, or if a package’s test suite fails in a
     non-deterministic fashion.  It should be used with care because
     running the test suite is a good way to ensure a package is working
     as intended.

     Turning off tests leads to a different store item.  Consequently,
     when using this option, anything that depends on PACKAGE must be
     rebuilt, as in this example:

          guix install --without-tests=python python-notebook

     The command above installs ‘python-notebook’ on top of ‘python’
     built without running its test suite.  To do so, it also rebuilds
     everything that depends on ‘python’, including ‘python-notebook’
     itself.

     Internally, ‘--without-tests’ relies on changing the ‘#:tests?’
     option of a package’s ‘check’ phase (*note Build Systems::).  Note
     that some packages use a customized ‘check’ phase that does not
     respect a ‘#:tests? #f’ setting.  Therefore, ‘--without-tests’ has
     no effect on these packages.

   Wondering how to achieve the same effect using Scheme code, for
example in your manifest, or how to write your own package
transformation?  *Note Defining Package Variants::, for an overview of
the programming interfaces available.


File: guix.info,  Node: Additional Build Options,  Next: Debugging Build Failures,  Prev: Package Transformation Options,  Up: Invoking guix build

10.1.3 Additional Build Options
-------------------------------

The command-line options presented below are specific to ‘guix build’.

‘--quiet’
‘-q’
     Build quietly, without displaying the build log; this is equivalent
     to ‘--verbosity=0’.  Upon completion, the build log is kept in
     ‘/var’ (or similar) and can always be retrieved using the
     ‘--log-file’ option.

‘--file=FILE’
‘-f FILE’
     Build the package, derivation, or other file-like object that the
     code within FILE evaluates to (*note file-like objects:
     G-Expressions.).

     As an example, FILE might contain a package definition like this
     (*note Defining Packages::):

          (use-modules (guix)
                       (guix build-system gnu)
                       (guix licenses))

          (package
            (name "hello")
            (version "2.10")
            (source (origin
                      (method url-fetch)
                      (uri (string-append "mirror://gnu/hello/hello-" version
                                          ".tar.gz"))
                      (sha256
                       (base32
                        "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
            (build-system gnu-build-system)
            (synopsis "Hello, GNU world: An example GNU package")
            (description "Guess what GNU Hello prints!")
            (home-page "http://www.gnu.org/software/hello/")
            (license gpl3+))

     The FILE may also contain a JSON representation of one or more
     package definitions.  Running ‘guix build -f’ on ‘hello.json’ with
     the following contents would result in building the packages
     ‘myhello’ and ‘greeter’:

          [
            {
              "name": "myhello",
              "version": "2.10",
              "source": "mirror://gnu/hello/hello-2.10.tar.gz",
              "build-system": "gnu",
              "arguments": {
                "tests?": false
              }
              "home-page": "https://www.gnu.org/software/hello/",
              "synopsis": "Hello, GNU world: An example GNU package",
              "description": "GNU Hello prints a greeting.",
              "license": "GPL-3.0+",
              "native-inputs": ["gettext"]
            },
            {
              "name": "greeter",
              "version": "1.0",
              "source": "https://example.com/greeter-1.0.tar.gz",
              "build-system": "gnu",
              "arguments": {
                "test-target": "foo",
                "parallel-build?": false,
              },
              "home-page": "https://example.com/",
              "synopsis": "Greeter using GNU Hello",
              "description": "This is a wrapper around GNU Hello.",
              "license": "GPL-3.0+",
              "inputs": ["myhello", "hello"]
            }
          ]

‘--manifest=MANIFEST’
‘-m MANIFEST’
     Build all packages listed in the given MANIFEST (*note
     ‘--manifest’: profile-manifest.).

‘--expression=EXPR’
‘-e EXPR’
     Build the package or derivation EXPR evaluates to.

     For example, EXPR may be ‘(@ (gnu packages guile) guile-1.8)’,
     which unambiguously designates this specific variant of version 1.8
     of Guile.

     Alternatively, EXPR may be a G-expression, in which case it is used
     as a build program passed to ‘gexp->derivation’ (*note
     G-Expressions::).

     Lastly, EXPR may refer to a zero-argument monadic procedure (*note
     The Store Monad::).  The procedure must return a derivation as a
     monadic value, which is then passed through ‘run-with-store’.

‘--source’
‘-S’
     Build the source derivations of the packages, rather than the
     packages themselves.

     For instance, ‘guix build -S gcc’ returns something like
     ‘/gnu/store/...-gcc-4.7.2.tar.bz2’, which is the GCC source
     tarball.

     The returned source tarball is the result of applying any patches
     and code snippets specified in the package ‘origin’ (*note Defining
     Packages::).

     As with other derivations, the result of building a source
     derivation can be verified using the ‘--check’ option (*note
     build-check::).  This is useful to validate that a (potentially
     already built or substituted, thus cached) package source matches
     against its declared hash.

     Note that ‘guix build -S’ compiles the sources only of the
     specified packages.  They do not include the sources of statically
     linked dependencies and by themselves are insufficient for
     reproducing the packages.

‘--sources’
     Fetch and return the source of PACKAGE-OR-DERIVATION and all their
     dependencies, recursively.  This is a handy way to obtain a local
     copy of all the source code needed to build PACKAGES, allowing you
     to eventually build them even without network access.  It is an
     extension of the ‘--source’ option and can accept one of the
     following optional argument values:

     ‘package’
          This value causes the ‘--sources’ option to behave in the same
          way as the ‘--source’ option.

     ‘all’
          Build the source derivations of all packages, including any
          source that might be listed as ‘inputs’.  This is the default
          value.

               $ guix build --sources tzdata
               The following derivations will be built:
                  /gnu/store/...-tzdata2015b.tar.gz.drv
                  /gnu/store/...-tzcode2015b.tar.gz.drv

     ‘transitive’
          Build the source derivations of all packages, as well of all
          transitive inputs to the packages.  This can be used e.g. to
          prefetch package source for later offline building.

               $ guix build --sources=transitive tzdata
               The following derivations will be built:
                  /gnu/store/...-tzcode2015b.tar.gz.drv
                  /gnu/store/...-findutils-4.4.2.tar.xz.drv
                  /gnu/store/...-grep-2.21.tar.xz.drv
                  /gnu/store/...-coreutils-8.23.tar.xz.drv
                  /gnu/store/...-make-4.1.tar.xz.drv
                  /gnu/store/...-bash-4.3.tar.xz.drv
               ...

‘--system=SYSTEM’
‘-s SYSTEM’
     Attempt to build for SYSTEM—e.g., ‘i686-linux’—instead of the
     system type of the build host.  The ‘guix build’ command allows you
     to repeat this option several times, in which case it builds for
     all the specified systems; other commands ignore extraneous ‘-s’
     options.

          Note: The ‘--system’ flag is for _native_ compilation and must
          not be confused with cross-compilation.  See ‘--target’ below
          for information on cross-compilation.

     An example use of this is on Linux-based systems, which can emulate
     different personalities.  For instance, passing
     ‘--system=i686-linux’ on an ‘x86_64-linux’ system or
     ‘--system=armhf-linux’ on an ‘aarch64-linux’ system allows you to
     build packages in a complete 32-bit environment.

          Note: Building for an ‘armhf-linux’ system is unconditionally
          enabled on ‘aarch64-linux’ machines, although certain aarch64
          chipsets do not allow for this functionality, notably the
          ThunderX.

     Similarly, when transparent emulation with QEMU and ‘binfmt_misc’
     is enabled (*note ‘qemu-binfmt-service-type’: Virtualization
     Services.), you can build for any system for which a QEMU
     ‘binfmt_misc’ handler is installed.

     Builds for a system other than that of the machine you are using
     can also be offloaded to a remote machine of the right
     architecture.  *Note Daemon Offload Setup::, for more information
     on offloading.

‘--target=TRIPLET’
     Cross-build for TRIPLET, which must be a valid GNU triplet, such as
     ‘"aarch64-linux-gnu"’ (*note GNU configuration triplets:
     (autoconf)Specifying Target Triplets.).

‘--list-systems’
     List all the supported systems, that can be passed as an argument
     to ‘--system’.

‘--list-targets’
     List all the supported targets, that can be passed as an argument
     to ‘--target’.

‘--check’
     Rebuild PACKAGE-OR-DERIVATION, which are already available in the
     store, and raise an error if the build results are not bit-for-bit
     identical.

     This mechanism allows you to check whether previously installed
     substitutes are genuine (*note Substitutes::), or whether the build
     result of a package is deterministic.  *Note Invoking guix
     challenge::, for more background information and tools.

     When used in conjunction with ‘--keep-failed’, the differing output
     is kept in the store, under ‘/gnu/store/...-check’.  This makes it
     easy to look for differences between the two results.

‘--repair’
     Attempt to repair the specified store items, if they are corrupt,
     by re-downloading or rebuilding them.

     This operation is not atomic and thus restricted to ‘root’.

‘--derivations’
‘-d’
     Return the derivation paths, not the output paths, of the given
     packages.

‘--root=FILE’
‘-r FILE’
     Make FILE a symlink to the result, and register it as a garbage
     collector root.

     Consequently, the results of this ‘guix build’ invocation are
     protected from garbage collection until FILE is removed.  When that
     option is omitted, build results are eligible for garbage
     collection as soon as the build completes.  *Note Invoking guix
     gc::, for more on GC roots.

‘--log-file’
     Return the build log file names or URLs for the given
     PACKAGE-OR-DERIVATION, or raise an error if build logs are missing.

     This works regardless of how packages or derivations are specified.
     For instance, the following invocations are equivalent:

          guix build --log-file $(guix build -d guile)
          guix build --log-file $(guix build guile)
          guix build --log-file guile
          guix build --log-file -e '(@ (gnu packages guile) guile-2.0)'

     If a log is unavailable locally, and unless ‘--no-substitutes’ is
     passed, the command looks for a corresponding log on one of the
     substitute servers (as specified with ‘--substitute-urls’).

     So for instance, imagine you want to see the build log of GDB on
     ‘aarch64’, but you are actually on an ‘x86_64’ machine:

          $ guix build --log-file gdb -s aarch64-linux
          https://ci.guix.gnu.org/log/...-gdb-7.10

     You can freely access a huge library of build logs!


File: guix.info,  Node: Debugging Build Failures,  Prev: Additional Build Options,  Up: Invoking guix build

10.1.4 Debugging Build Failures
-------------------------------

When defining a new package (*note Defining Packages::), you will
probably find yourself spending some time debugging and tweaking the
build until it succeeds.  To do that, you need to operate the build
commands yourself in an environment as close as possible to the one the
build daemon uses.

   To that end, the first thing to do is to use the ‘--keep-failed’ or
‘-K’ option of ‘guix build’, which will keep the failed build tree in
‘/tmp’ or whatever directory you specified as ‘TMPDIR’ (*note
‘--keep-failed’: Common Build Options.).

   From there on, you can ‘cd’ to the failed build tree and source the
‘environment-variables’ file, which contains all the environment
variable definitions that were in place when the build failed.  So let’s
say you’re debugging a build failure in package ‘foo’; a typical session
would look like this:

     $ guix build foo -K
     ... build fails
     $ cd /tmp/guix-build-foo.drv-0
     $ source ./environment-variables
     $ cd foo-1.2

   Now, you can invoke commands as if you were the daemon (almost) and
troubleshoot your build process.

   Sometimes it happens that, for example, a package’s tests pass when
you run them manually but they fail when the daemon runs them.  This can
happen because the daemon runs builds in containers where, unlike in our
environment above, network access is missing, ‘/bin/sh’ does not exist,
etc.  (*note Build Environment Setup::).

   In such cases, you may need to run inspect the build process from
within a container similar to the one the build daemon creates:

     $ guix build -K foo
     ...
     $ cd /tmp/guix-build-foo.drv-0
     $ guix shell --no-grafts -C -D foo strace gdb
     [env]# source ./environment-variables
     [env]# cd foo-1.2

   Here, ‘guix shell -C’ creates a container and spawns a new shell in
it (*note Invoking guix shell::).  The ‘strace gdb’ part adds the
‘strace’ and ‘gdb’ commands to the container, which you may find handy
while debugging.  The ‘--no-grafts’ option makes sure we get the exact
same environment, with ungrafted packages (*note Security Updates::, for
more info on grafts).

   To get closer to a container like that used by the build daemon, we
can remove ‘/bin/sh’:

     [env]# rm /bin/sh

   (Don’t worry, this is harmless: this is all happening in the
throw-away container created by ‘guix shell’.)

   The ‘strace’ command is probably not in the search path, but we can
run:

     [env]# $GUIX_ENVIRONMENT/bin/strace -f -o log make check

   In this way, not only you will have reproduced the environment
variables the daemon uses, you will also be running the build process in
a container similar to the one the daemon uses.


File: guix.info,  Node: Invoking guix edit,  Next: Invoking guix download,  Prev: Invoking guix build,  Up: Utilities

10.2 Invoking ‘guix edit’
=========================

So many packages, so many source files!  The ‘guix edit’ command
facilitates the life of users and packagers by pointing their editor at
the source file containing the definition of the specified packages.
For instance:

     guix edit gcc@4.9 vim

launches the program specified in the ‘VISUAL’ or in the ‘EDITOR’
environment variable to view the recipe of GCC 4.9.3 and that of Vim.

   If you are using a Guix Git checkout (*note Building from Git::), or
have created your own packages on ‘GUIX_PACKAGE_PATH’ (*note Package
Modules::), you will be able to edit the package recipes.  In other
cases, you will be able to examine the read-only recipes for packages
currently in the store.

   Instead of ‘GUIX_PACKAGE_PATH’, the command-line option
‘--load-path=DIRECTORY’ (or in short ‘-L DIRECTORY’) allows you to add
DIRECTORY to the front of the package module search path and so make
your own packages visible.


File: guix.info,  Node: Invoking guix download,  Next: Invoking guix hash,  Prev: Invoking guix edit,  Up: Utilities

10.3 Invoking ‘guix download’
=============================

When writing a package definition, developers typically need to download
a source tarball, compute its SHA256 hash, and write that hash in the
package definition (*note Defining Packages::).  The ‘guix download’
tool helps with this task: it downloads a file from the given URI, adds
it to the store, and prints both its file name in the store and its
SHA256 hash.

   The fact that the downloaded file is added to the store saves
bandwidth: when the developer eventually tries to build the newly
defined package with ‘guix build’, the source tarball will not have to
be downloaded again because it is already in the store.  It is also a
convenient way to temporarily stash files, which may be deleted
eventually (*note Invoking guix gc::).

   The ‘guix download’ command supports the same URIs as used in package
definitions.  In particular, it supports ‘mirror://’ URIs.  ‘https’ URIs
(HTTP over TLS) are supported _provided_ the Guile bindings for GnuTLS
are available in the user’s environment; when they are not available, an
error is raised.  *Note how to install the GnuTLS bindings for Guile:
(gnutls-guile)Guile Preparations, for more information.

   ‘guix download’ verifies HTTPS server certificates by loading the
certificates of X.509 authorities from the directory pointed to by the
‘SSL_CERT_DIR’ environment variable (*note X.509 Certificates::), unless
‘--no-check-certificate’ is used.

   The following options are available:

‘--hash=ALGORITHM’
‘-H ALGORITHM’
     Compute a hash using the specified ALGORITHM.  *Note Invoking guix
     hash::, for more information.

‘--format=FMT’
‘-f FMT’
     Write the hash in the format specified by FMT.  For more
     information on the valid values for FMT, *note Invoking guix
     hash::.

‘--no-check-certificate’
     Do not validate the X.509 certificates of HTTPS servers.

     When using this option, you have _absolutely no guarantee_ that you
     are communicating with the authentic server responsible for the
     given URL, which makes you vulnerable to “man-in-the-middle”
     attacks.

‘--output=FILE’
‘-o FILE’
     Save the downloaded file to FILE instead of adding it to the store.


File: guix.info,  Node: Invoking guix hash,  Next: Invoking guix import,  Prev: Invoking guix download,  Up: Utilities

10.4 Invoking ‘guix hash’
=========================

The ‘guix hash’ command computes the hash of a file.  It is primarily a
convenience tool for anyone contributing to the distribution: it
computes the cryptographic hash of one or more files, which can be used
in the definition of a package (*note Defining Packages::).

   The general syntax is:

     guix hash OPTION FILE ...

   When FILE is ‘-’ (a hyphen), ‘guix hash’ computes the hash of data
read from standard input.  ‘guix hash’ has the following options:

‘--hash=ALGORITHM’
‘-H ALGORITHM’
     Compute a hash using the specified ALGORITHM, ‘sha256’ by default.

     ALGORITHM must be the name of a cryptographic hash algorithm
     supported by Libgcrypt via Guile-Gcrypt—e.g., ‘sha512’ or
     ‘sha3-256’ (*note (guile-gcrypt)Hash Functions::).

‘--format=FMT’
‘-f FMT’
     Write the hash in the format specified by FMT.

     Supported formats: ‘base64’, ‘nix-base32’, ‘base32’, ‘base16’
     (‘hex’ and ‘hexadecimal’ can be used as well).

     If the ‘--format’ option is not specified, ‘guix hash’ will output
     the hash in ‘nix-base32’.  This representation is used in the
     definitions of packages.

‘--recursive’
‘-r’
     The ‘--recursive’ option is deprecated in favor of
     ‘--serializer=nar’ (see below); ‘-r’ remains accepted as a
     convenient shorthand.

‘--serializer=TYPE’
‘-S TYPE’
     Compute the hash on FILE using TYPE serialization.

     TYPE may be one of the following:

     ‘none’
          This is the default: it computes the hash of a file’s
          contents.

     ‘nar’
          Compute the hash of a “normalized archive” (or “nar”)
          containing FILE, including its children if it is a directory.
          Some of the metadata of FILE is part of the archive; for
          instance, when FILE is a regular file, the hash is different
          depending on whether FILE is executable or not.  Metadata such
          as time stamps have no impact on the hash (*note Invoking guix
          archive::, for more info on the nar format).

     ‘git’
          Compute the hash of the file or directory as a Git “tree”,
          following the same method as the Git version control system.

‘--exclude-vcs’
‘-x’
     When combined with ‘--recursive’, exclude version control system
     directories (‘.bzr’, ‘.git’, ‘.hg’, etc.).

     As an example, here is how you would compute the hash of a Git
     checkout, which is useful when using the ‘git-fetch’ method (*note
     origin Reference::):

          $ git clone http://example.org/foo.git
          $ cd foo
          $ guix hash -x --serializer=nar .


File: guix.info,  Node: Invoking guix import,  Next: Invoking guix refresh,  Prev: Invoking guix hash,  Up: Utilities

10.5 Invoking ‘guix import’
===========================

The ‘guix import’ command is useful for people who would like to add a
package to the distribution with as little work as possible—a legitimate
demand.  The command knows of a few repositories from which it can
“import” package metadata.  The result is a package definition, or a
template thereof, in the format we know (*note Defining Packages::).

   The general syntax is:

     guix import IMPORTER OPTIONS...

   IMPORTER specifies the source from which to import package metadata,
and OPTIONS specifies a package identifier and other options specific to
IMPORTER.

   Some of the importers rely on the ability to run the ‘gpgv’ command.
For these, GnuPG must be installed and in ‘$PATH’; run ‘guix install
gnupg’ if needed.

   Currently, the available “importers” are:

‘gnu’
     Import metadata for the given GNU package.  This provides a
     template for the latest version of that GNU package, including the
     hash of its source tarball, and its canonical synopsis and
     description.

     Additional information such as the package dependencies and its
     license needs to be figured out manually.

     For example, the following command returns a package definition for
     GNU Hello:

          guix import gnu hello

     Specific command-line options are:

     ‘--key-download=POLICY’
          As for ‘guix refresh’, specify the policy to handle missing
          OpenPGP keys when verifying the package signature.  *Note
          ‘--key-download’: Invoking guix refresh.

‘pypi’
     Import metadata from the Python Package Index
     (https://pypi.python.org/).  Information is taken from the
     JSON-formatted description available at ‘pypi.python.org’ and
     usually includes all the relevant information, including package
     dependencies.  For maximum efficiency, it is recommended to install
     the ‘unzip’ utility, so that the importer can unzip Python wheels
     and gather data from them.

     The command below imports metadata for the latest version of the
     ‘itsdangerous’ Python package:

          guix import pypi itsdangerous

     You can also ask for a specific version:

          guix import pypi itsdangerous@1.1.0

     ‘--recursive’
     ‘-r’
          Traverse the dependency graph of the given upstream package
          recursively and generate package expressions for all those
          packages that are not yet in Guix.

‘gem’
     Import metadata from RubyGems (https://rubygems.org/).  Information
     is taken from the JSON-formatted description available at
     ‘rubygems.org’ and includes most relevant information, including
     runtime dependencies.  There are some caveats, however.  The
     metadata doesn’t distinguish between synopses and descriptions, so
     the same string is used for both fields.  Additionally, the details
     of non-Ruby dependencies required to build native extensions is
     unavailable and left as an exercise to the packager.

     The command below imports metadata for the ‘rails’ Ruby package:

          guix import gem rails

     You can also ask for a specific version:

          guix import gem rails@7.0.4

     ‘--recursive’
     ‘-r’
          Traverse the dependency graph of the given upstream package
          recursively and generate package expressions for all those
          packages that are not yet in Guix.

‘minetest’
     Import metadata from ContentDB (https://content.minetest.net).
     Information is taken from the JSON-formatted metadata provided
     through ContentDB’s API (https://content.minetest.net/help/api/)
     and includes most relevant information, including dependencies.
     There are some caveats, however.  The license information is often
     incomplete.  The commit hash is sometimes missing.  The
     descriptions are in the Markdown format, but Guix uses Texinfo
     instead.  Texture packs and subgames are unsupported.

     The command below imports metadata for the Mesecons mod by Jeija:

          guix import minetest Jeija/mesecons

     The author name can also be left out:

          guix import minetest mesecons

     ‘--recursive’
     ‘-r’
          Traverse the dependency graph of the given upstream package
          recursively and generate package expressions for all those
          packages that are not yet in Guix.

‘cpan’
     Import metadata from MetaCPAN (https://www.metacpan.org/).
     Information is taken from the JSON-formatted metadata provided
     through MetaCPAN’s API (https://fastapi.metacpan.org/) and includes
     most relevant information, such as module dependencies.  License
     information should be checked closely.  If Perl is available in the
     store, then the ‘corelist’ utility will be used to filter core
     modules out of the list of dependencies.

     The command command below imports metadata for the Acme::Boolean
     Perl module:

          guix import cpan Acme::Boolean

‘cran’
     Import metadata from CRAN (https://cran.r-project.org/), the
     central repository for the GNU R statistical and graphical
     environment (https://r-project.org).

     Information is extracted from the ‘DESCRIPTION’ file of the
     package.

     The command command below imports metadata for the Cairo R package:

          guix import cran Cairo

     You can also ask for a specific version:

          guix import cran rasterVis@0.50.3

     When ‘--recursive’ is added, the importer will traverse the
     dependency graph of the given upstream package recursively and
     generate package expressions for all those packages that are not
     yet in Guix.

     When ‘--style=specification’ is added, the importer will generate
     package definitions whose inputs are package specifications instead
     of references to package variables.  This is useful when generated
     package definitions are to be appended to existing user modules, as
     the list of used package modules need not be changed.  The default
     is ‘--style=variable’.

     When ‘--archive=bioconductor’ is added, metadata is imported from
     Bioconductor (https://www.bioconductor.org/), a repository of R
     packages for the analysis and comprehension of high-throughput
     genomic data in bioinformatics.

     Information is extracted from the ‘DESCRIPTION’ file contained in
     the package archive.

     The command below imports metadata for the GenomicRanges R package:

          guix import cran --archive=bioconductor GenomicRanges

     Finally, you can also import R packages that have not yet been
     published on CRAN or Bioconductor as long as they are in a git
     repository.  Use ‘--archive=git’ followed by the URL of the git
     repository:

          guix import cran --archive=git https://github.com/immunogenomics/harmony

‘texlive’
     Import TeX package information from the TeX Live package database
     for TeX packages that are part of the TeX Live distribution
     (https://www.tug.org/texlive/).

     Information about the package is obtained from the TeX Live package
     database, a plain text file that is included in the ‘texlive-bin’
     package.  The source code is downloaded from possibly multiple
     locations in the SVN repository of the Tex Live project.

     The command command below imports metadata for the ‘fontspec’ TeX
     package:

          guix import texlive fontspec

‘json’
     Import package metadata from a local JSON file.  Consider the
     following example package definition in JSON format:

          {
            "name": "hello",
            "version": "2.10",
            "source": "mirror://gnu/hello/hello-2.10.tar.gz",
            "build-system": "gnu",
            "home-page": "https://www.gnu.org/software/hello/",
            "synopsis": "Hello, GNU world: An example GNU package",
            "description": "GNU Hello prints a greeting.",
            "license": "GPL-3.0+",
            "native-inputs": ["gettext"]
          }

     The field names are the same as for the ‘<package>’ record (*Note
     Defining Packages::).  References to other packages are provided as
     JSON lists of quoted package specification strings such as ‘guile’
     or ‘guile@2.0’.

     The importer also supports a more explicit source definition using
     the common fields for ‘<origin>’ records:

          {
            ...
            "source": {
              "method": "url-fetch",
              "uri": "mirror://gnu/hello/hello-2.10.tar.gz",
              "sha256": {
                "base32": "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"
              }
            }
            ...
          }

     The command below reads metadata from the JSON file ‘hello.json’
     and outputs a package expression:

          guix import json hello.json

‘hackage’
     Import metadata from the Haskell community’s central package
     archive Hackage (https://hackage.haskell.org/).  Information is
     taken from Cabal files and includes all the relevant information,
     including package dependencies.

     Specific command-line options are:

     ‘--stdin’
     ‘-s’
          Read a Cabal file from standard input.
     ‘--no-test-dependencies’
     ‘-t’
          Do not include dependencies required only by the test suites.
     ‘--cabal-environment=ALIST’
     ‘-e ALIST’
          ALIST is a Scheme alist defining the environment in which the
          Cabal conditionals are evaluated.  The accepted keys are:
          ‘os’, ‘arch’, ‘impl’ and a string representing the name of a
          flag.  The value associated with a flag has to be either the
          symbol ‘true’ or ‘false’.  The value associated with other
          keys has to conform to the Cabal file format definition.  The
          default value associated with the keys ‘os’, ‘arch’ and ‘impl’
          is ‘linux’, ‘x86_64’ and ‘ghc’, respectively.
     ‘--recursive’
     ‘-r’
          Traverse the dependency graph of the given upstream package
          recursively and generate package expressions for all those
          packages that are not yet in Guix.

     The command below imports metadata for the latest version of the
     HTTP Haskell package without including test dependencies and
     specifying the value of the flag ‘network-uri’ as ‘false’:

          guix import hackage -t -e "'((\"network-uri\" . false))" HTTP

     A specific package version may optionally be specified by following
     the package name by an at-sign and a version number as in the
     following example:

          guix import hackage mtl@2.1.3.1

‘stackage’
     The ‘stackage’ importer is a wrapper around the ‘hackage’ one.  It
     takes a package name, looks up the package version included in a
     long-term support (LTS) Stackage (https://www.stackage.org) release
     and uses the ‘hackage’ importer to retrieve its metadata.  Note
     that it is up to you to select an LTS release compatible with the
     GHC compiler used by Guix.

     Specific command-line options are:

     ‘--no-test-dependencies’
     ‘-t’
          Do not include dependencies required only by the test suites.
     ‘--lts-version=VERSION’
     ‘-l VERSION’
          VERSION is the desired LTS release version.  If omitted the
          latest release is used.
     ‘--recursive’
     ‘-r’
          Traverse the dependency graph of the given upstream package
          recursively and generate package expressions for all those
          packages that are not yet in Guix.

     The command below imports metadata for the HTTP Haskell package
     included in the LTS Stackage release version 7.18:

          guix import stackage --lts-version=7.18 HTTP

‘elpa’
     Import metadata from an Emacs Lisp Package Archive (ELPA) package
     repository (*note (emacs)Packages::).

     Specific command-line options are:

     ‘--archive=REPO’
     ‘-a REPO’
          REPO identifies the archive repository from which to retrieve
          the information.  Currently the supported repositories and
          their identifiers are:
             - GNU (https://elpa.gnu.org/packages), selected by the
               ‘gnu’ identifier.  This is the default.

               Packages from ‘elpa.gnu.org’ are signed with one of the
               keys contained in the GnuPG keyring at
               ‘share/emacs/25.1/etc/package-keyring.gpg’ (or similar)
               in the ‘emacs’ package (*note ELPA package signatures:
               (emacs)Package Installation.).

             - NonGNU (https://elpa.nongnu.org/nongnu/), selected by the
               ‘nongnu’ identifier.

             - MELPA-Stable (https://stable.melpa.org/packages),
               selected by the ‘melpa-stable’ identifier.

             - MELPA (https://melpa.org/packages), selected by the
               ‘melpa’ identifier.

     ‘--recursive’
     ‘-r’
          Traverse the dependency graph of the given upstream package
          recursively and generate package expressions for all those
          packages that are not yet in Guix.

‘crate’
     Import metadata from the crates.io Rust package repository
     crates.io (https://crates.io), as in this example:

          guix import crate blake2-rfc

     The crate importer also allows you to specify a version string:

          guix import crate constant-time-eq@0.1.0

     Additional options include:

     ‘--recursive’
     ‘-r’
          Traverse the dependency graph of the given upstream package
          recursively and generate package expressions for all those
          packages that are not yet in Guix.

‘elm’
     Import metadata from the Elm package repository
     package.elm-lang.org (https://package.elm-lang.org), as in this
     example:

          guix import elm elm-explorations/webgl

     The Elm importer also allows you to specify a version string:

          guix import elm elm-explorations/webgl@1.1.3

     Additional options include:

     ‘--recursive’
     ‘-r’
          Traverse the dependency graph of the given upstream package
          recursively and generate package expressions for all those
          packages that are not yet in Guix.

‘opam’
     Import metadata from the OPAM (https://opam.ocaml.org/) package
     repository used by the OCaml community.

     Additional options include:

     ‘--recursive’
     ‘-r’
          Traverse the dependency graph of the given upstream package
          recursively and generate package expressions for all those
          packages that are not yet in Guix.
     ‘--repo’
          By default, packages are searched in the official OPAM
          repository.  This option, which can be used more than once,
          lets you add other repositories which will be searched for
          packages.  It accepts as valid arguments:

             • the name of a known repository - can be one of ‘opam’,
               ‘coq’ (equivalent to ‘coq-released’), ‘coq-core-dev’,
               ‘coq-extra-dev’ or ‘grew’.
             • the URL of a repository as expected by the ‘opam
               repository add’ command (for instance, the URL equivalent
               of the above ‘opam’ name would be
               <https://opam.ocaml.org>).
             • the path to a local copy of a repository (a directory
               containing a ‘packages/’ sub-directory).

          Repositories are assumed to be passed to this option by order
          of preference.  The additional repositories will not replace
          the default ‘opam’ repository, which is always kept as a
          fallback.

          Also, please note that versions are not compared across
          repositories.  The first repository (from left to right) that
          has at least one version of a given package will prevail over
          any others, and the version imported will be the latest one
          found _in this repository only_.

‘go’
     Import metadata for a Go module using proxy.golang.org
     (https://proxy.golang.org).

          guix import go gopkg.in/yaml.v2

     It is possible to use a package specification with a ‘@VERSION’
     suffix to import a specific version.

     Additional options include:

     ‘--recursive’
     ‘-r’
          Traverse the dependency graph of the given upstream package
          recursively and generate package expressions for all those
          packages that are not yet in Guix.
     ‘--pin-versions’
          When using this option, the importer preserves the exact
          versions of the Go modules dependencies instead of using their
          latest available versions.  This can be useful when attempting
          to import packages that recursively depend on former versions
          of themselves to build.  When using this mode, the symbol of
          the package is made by appending the version to its name, so
          that multiple versions of the same package can coexist.

‘egg’
     Import metadata for CHICKEN eggs (https://wiki.call-cc.org/eggs).
     The information is taken from ‘PACKAGE.egg’ files found in the
     eggs-5-all (git://code.call-cc.org/eggs-5-all) Git repository.
     However, it does not provide all the information that we need,
     there is no “description” field, and the licenses used are not
     always precise (BSD is often used instead of BSD-N).

          guix import egg sourcehut

     You can also ask for a specific version:

          guix import egg arrays@1.0

     Additional options include:
     ‘--recursive’
     ‘-r’
          Traverse the dependency graph of the given upstream package
          recursively and generate package expressions for all those
          packages that are not yet in Guix.

‘hexpm’
     Import metadata from the hex.pm Erlang and Elixir package
     repository hex.pm (https://hex.pm), as in this example:

          guix import hexpm stun

     The importer tries to determine the build system used by the
     package.

     The hexpm importer also allows you to specify a version string:

          guix import hexpm cf@0.3.0

     Additional options include:

     ‘--recursive’
     ‘-r’
          Traverse the dependency graph of the given upstream package
          recursively and generate package expressions for all those
          packages that are not yet in Guix.

   The structure of the ‘guix import’ code is modular.  It would be
useful to have more importers for other package formats, and your help
is welcome here (*note Contributing::).


File: guix.info,  Node: Invoking guix refresh,  Next: Invoking guix style,  Prev: Invoking guix import,  Up: Utilities

10.6 Invoking ‘guix refresh’
============================

The primary audience of the ‘guix refresh’ command is packagers.  As a
user, you may be interested in the ‘--with-latest’ option, which can
bring you package update superpowers built upon ‘guix refresh’ (*note
‘--with-latest’: Package Transformation Options.).  By default, ‘guix
refresh’ reports any packages provided by the distribution that are
outdated compared to the latest upstream version, like this:

     $ guix refresh
     gnu/packages/gettext.scm:29:13: gettext would be upgraded from 0.18.1.1 to 0.18.2.1
     gnu/packages/glib.scm:77:12: glib would be upgraded from 2.34.3 to 2.37.0

   Alternatively, one can specify packages to consider, in which case a
warning is emitted for packages that lack an updater:

     $ guix refresh coreutils guile guile-ssh
     gnu/packages/ssh.scm:205:2: warning: no updater for guile-ssh
     gnu/packages/guile.scm:136:12: guile would be upgraded from 2.0.12 to 2.0.13

   ‘guix refresh’ browses the upstream repository of each package and
determines the highest version number of the releases therein.  The
command knows how to update specific types of packages: GNU packages,
ELPA packages, etc.—see the documentation for ‘--type’ below.  There are
many packages, though, for which it lacks a method to determine whether
a new upstream release is available.  However, the mechanism is
extensible, so feel free to get in touch with us to add a new method!

‘--recursive’
     Consider the packages specified, and all the packages upon which
     they depend.

          $ guix refresh --recursive coreutils
          gnu/packages/acl.scm:40:13: acl would be upgraded from 2.2.53 to 2.3.1
          gnu/packages/m4.scm:30:12: 1.4.18 is already the latest version of m4
          gnu/packages/xml.scm:68:2: warning: no updater for expat
          gnu/packages/multiprecision.scm:40:12: 6.1.2 is already the latest version of gmp
          ...

   Sometimes the upstream name differs from the package name used in
Guix, and ‘guix refresh’ needs a little help.  Most updaters honor the
‘upstream-name’ property in package definitions, which can be used to
that effect:

     (define-public network-manager
       (package
         (name "network-manager")
         ;; ...
         (properties '((upstream-name . "NetworkManager")))))

   When passed ‘--update’, it modifies distribution source files to
update the version numbers and source tarball hashes of those package
recipes (*note Defining Packages::).  This is achieved by downloading
each package’s latest source tarball and its associated OpenPGP
signature, authenticating the downloaded tarball against its signature
using ‘gpgv’, and finally computing its hash—note that GnuPG must be
installed and in ‘$PATH’; run ‘guix install gnupg’ if needed.

   When the public key used to sign the tarball is missing from the
user’s keyring, an attempt is made to automatically retrieve it from a
public key server; when this is successful, the key is added to the
user’s keyring; otherwise, ‘guix refresh’ reports an error.

   The following options are supported:

‘--expression=EXPR’
‘-e EXPR’
     Consider the package EXPR evaluates to.

     This is useful to precisely refer to a package, as in this example:

          guix refresh -l -e '(@@ (gnu packages commencement) glibc-final)'

     This command lists the dependents of the “final” libc (essentially
     all the packages).

‘--update’
‘-u’
     Update distribution source files (package recipes) in place.  This
     is usually run from a checkout of the Guix source tree (*note
     Running Guix Before It Is Installed::):

          $ ./pre-inst-env guix refresh -s non-core -u

     *Note Defining Packages::, for more information on package
     definitions.

‘--select=[SUBSET]’
‘-s SUBSET’
     Select all the packages in SUBSET, one of ‘core’ or ‘non-core’.

     The ‘core’ subset refers to all the packages at the core of the
     distribution—i.e., packages that are used to build “everything
     else”.  This includes GCC, libc, Binutils, Bash, etc.  Usually,
     changing one of these packages in the distribution entails a
     rebuild of all the others.  Thus, such updates are an inconvenience
     to users in terms of build time or bandwidth used to achieve the
     upgrade.

     The ‘non-core’ subset refers to the remaining packages.  It is
     typically useful in cases where an update of the core packages
     would be inconvenient.

‘--manifest=FILE’
‘-m FILE’
     Select all the packages from the manifest in FILE.  This is useful
     to check if any packages of the user manifest can be updated.

‘--type=UPDATER’
‘-t UPDATER’
     Select only packages handled by UPDATER (may be a comma-separated
     list of updaters).  Currently, UPDATER may be one of:

     ‘gnu’
          the updater for GNU packages;
     ‘savannah’
          the updater for packages hosted at Savannah
          (https://savannah.gnu.org);
     ‘sourceforge’
          the updater for packages hosted at SourceForge
          (https://sourceforge.net);
     ‘gnome’
          the updater for GNOME packages;
     ‘kde’
          the updater for KDE packages;
     ‘xorg’
          the updater for X.org packages;
     ‘kernel.org’
          the updater for packages hosted on kernel.org;
     ‘egg’
          the updater for Egg (https://wiki.call-cc.org/eggs/) packages;
     ‘elpa’
          the updater for ELPA (https://elpa.gnu.org/) packages;
     ‘cran’
          the updater for CRAN (https://cran.r-project.org/) packages;
     ‘bioconductor’
          the updater for Bioconductor (https://www.bioconductor.org/) R
          packages;
     ‘cpan’
          the updater for CPAN (https://www.cpan.org/) packages;
     ‘pypi’
          the updater for PyPI (https://pypi.python.org) packages.
     ‘gem’
          the updater for RubyGems (https://rubygems.org) packages.
     ‘github’
          the updater for GitHub (https://github.com) packages.
     ‘hackage’
          the updater for Hackage (https://hackage.haskell.org)
          packages.
     ‘stackage’
          the updater for Stackage (https://www.stackage.org) packages.
     ‘crate’
          the updater for Crates (https://crates.io) packages.
     ‘launchpad’
          the updater for Launchpad (https://launchpad.net) packages.
     ‘generic-html’
          a generic updater that crawls the HTML page where the source
          tarball of the package is hosted, when applicable.

     ‘generic-git’
          a generic updater for packages hosted on Git repositories.  It
          tries to be smart about parsing Git tag names, but if it is
          not able to parse the tag name and compare tags correctly,
          users can define the following properties for a package.

             • ‘release-tag-prefix’: a regular expression for matching a
               prefix of the tag name.

             • ‘release-tag-suffix’: a regular expression for matching a
               suffix of the tag name.

             • ‘release-tag-version-delimiter’: a string used as the
               delimiter in the tag name for separating the numbers of
               the version.

             • ‘accept-pre-releases’: by default, the updater will
               ignore pre-releases; to make it also look for
               pre-releases, set the this property to ‘#t’.

               (package
                 (name "foo")
                 ;; ...
                 (properties
                   '((release-tag-prefix . "^release0-")
                     (release-tag-suffix . "[a-z]?$")
                     (release-tag-version-delimiter . ":"))))

     For instance, the following command only checks for updates of
     Emacs packages hosted at ‘elpa.gnu.org’ and for updates of CRAN
     packages:

          $ guix refresh --type=elpa,cran
          gnu/packages/statistics.scm:819:13: r-testthat would be upgraded from 0.10.0 to 0.11.0
          gnu/packages/emacs.scm:856:13: emacs-auctex would be upgraded from 11.88.6 to 11.88.9

‘--list-updaters’
     List available updaters and exit (see ‘--type’ above).

     For each updater, display the fraction of packages it covers; at
     the end, display the fraction of packages covered by all these
     updaters.

   In addition, ‘guix refresh’ can be passed one or more package names,
as in this example:

     $ ./pre-inst-env guix refresh -u emacs idutils gcc@4.8

The command above specifically updates the ‘emacs’ and ‘idutils’
packages.  The ‘--select’ option would have no effect in this case.  You
might also want to update definitions that correspond to the packages
installed in your profile:

     $ ./pre-inst-env guix refresh -u \
            $(guix package --list-installed | cut -f1)

   When considering whether to upgrade a package, it is sometimes
convenient to know which packages would be affected by the upgrade and
should be checked for compatibility.  For this the following option may
be used when passing ‘guix refresh’ one or more package names:

‘--list-dependent’
‘-l’
     List top-level dependent packages that would need to be rebuilt as
     a result of upgrading one or more packages.

     *Note the ‘reverse-package’ type of ‘guix graph’: Invoking guix
     graph, for information on how to visualize the list of dependents
     of a package.

   Be aware that the ‘--list-dependent’ option only _approximates_ the
rebuilds that would be required as a result of an upgrade.  More
rebuilds might be required under some circumstances.

     $ guix refresh --list-dependent flex
     Building the following 120 packages would ensure 213 dependent packages are rebuilt:
     hop@2.4.0 emacs-geiser@0.13 notmuch@0.18 mu@0.9.9.5 cflow@1.4 idutils@4.6 ...

   The command above lists a set of packages that could be built to
check for compatibility with an upgraded ‘flex’ package.

‘--list-transitive’
     List all the packages which one or more packages depend upon.

          $ guix refresh --list-transitive flex
          flex@2.6.4 depends on the following 25 packages: perl@5.28.0 help2man@1.47.6
          bison@3.0.5 indent@2.2.10 tar@1.30 gzip@1.9 bzip2@1.0.6 xz@5.2.4 file@5.33 ...

   The command above lists a set of packages which, when changed, would
cause ‘flex’ to be rebuilt.

   The following options can be used to customize GnuPG operation:

‘--gpg=COMMAND’
     Use COMMAND as the GnuPG 2.x command.  COMMAND is searched for in
     ‘$PATH’.

‘--keyring=FILE’
     Use FILE as the keyring for upstream keys.  FILE must be in the
     “keybox format”.  Keybox files usually have a name ending in ‘.kbx’
     and the GNU Privacy Guard (GPG) can manipulate these files (*note
     ‘kbxutil’: (gnupg)kbxutil, for information on a tool to manipulate
     keybox files).

     When this option is omitted, ‘guix refresh’ uses
     ‘~/.config/guix/upstream/trustedkeys.kbx’ as the keyring for
     upstream signing keys.  OpenPGP signatures are checked against keys
     from this keyring; missing keys are downloaded to this keyring as
     well (see ‘--key-download’ below).

     You can export keys from your default GPG keyring into a keybox
     file using commands like this one:

          gpg --export rms@gnu.org | kbxutil --import-openpgp >> mykeyring.kbx

     Likewise, you can fetch keys to a specific keybox file like this:

          gpg --no-default-keyring --keyring mykeyring.kbx \
            --recv-keys 3CE464558A84FDC69DB40CFB090B11993D9AEBB5

     *Note ‘--keyring’: (gnupg)GPG Configuration Options, for more
     information on GPG’s ‘--keyring’ option.

‘--key-download=POLICY’
     Handle missing OpenPGP keys according to POLICY, which may be one
     of:

     ‘always’
          Always download missing OpenPGP keys from the key server, and
          add them to the user’s GnuPG keyring.

     ‘never’
          Never try to download missing OpenPGP keys.  Instead just bail
          out.

     ‘interactive’
          When a package signed with an unknown OpenPGP key is
          encountered, ask the user whether to download it or not.  This
          is the default behavior.

‘--key-server=HOST’
     Use HOST as the OpenPGP key server when importing a public key.

‘--load-path=DIRECTORY’
‘-L DIRECTORY’
     Add DIRECTORY to the front of the package module search path (*note
     Package Modules::).

     This allows users to define their own packages and make them
     visible to the command-line tools.

   The ‘github’ updater uses the GitHub API
(https://developer.github.com/v3/) to query for new releases.  When used
repeatedly e.g. when refreshing all packages, GitHub will eventually
refuse to answer any further API requests.  By default 60 API requests
per hour are allowed, and a full refresh on all GitHub packages in Guix
requires more than this.  Authentication with GitHub through the use of
an API token alleviates these limits.  To use an API token, set the
environment variable ‘GUIX_GITHUB_TOKEN’ to a token procured from
<https://github.com/settings/tokens> or otherwise.


File: guix.info,  Node: Invoking guix style,  Next: Invoking guix lint,  Prev: Invoking guix refresh,  Up: Utilities

10.7 Invoking ‘guix style’
==========================

The ‘guix style’ command helps users and packagers alike style their
package definitions and configuration files according to the latest
fashionable trends.  It can either reformat whole files, with the
‘--whole-file’ option, or apply specific “styling rules” to individual
package definitions.  The command currently provides the following
styling rules:

   • formatting package definitions according to the project’s
     conventions (*note Formatting Code::);

   • rewriting package inputs to the “new style”, as explained below.

   The way package inputs are written is going through a transition
(*note package Reference::, for more on package inputs).  Until version
1.3.0, package inputs were written using the “old style”, where each
input was given an explicit label, most of the time the package name:

     (package
       ;; ...
       ;; The "old style" (deprecated).
       (inputs `(("libunistring" ,libunistring)
                 ("libffi" ,libffi))))

   Today, the old style is deprecated and the preferred style looks like
this:

     (package
       ;; ...
       ;; The "new style".
       (inputs (list libunistring libffi)))

   Likewise, uses of ‘alist-delete’ and friends to manipulate inputs is
now deprecated in favor of ‘modify-inputs’ (*note Defining Package
Variants::, for more info on ‘modify-inputs’).

   In the vast majority of cases, this is a purely mechanical change on
the surface syntax that does not even incur a package rebuild.  Running
‘guix style -S inputs’ can do that for you, whether you’re working on
packages in Guix proper or in an external channel.

   The general syntax is:

     guix style [OPTIONS] PACKAGE...

   This causes ‘guix style’ to analyze and rewrite the definition of
PACKAGE... or, when PACKAGE is omitted, of _all_ the packages.  The
‘--styling’ or ‘-S’ option allows you to select the style rule, the
default rule being ‘format’—see below.

   To reformat entire source files, the syntax is:

     guix style --whole-file FILE...

   The available options are listed below.

‘--dry-run’
‘-n’
     Show source file locations that would be edited but do not modify
     them.

‘--whole-file’
‘-f’
     Reformat the given files in their entirety.  In that case,
     subsequent arguments are interpreted as file names (rather than
     package names), and the ‘--styling’ option has no effect.

     As an example, here is how you might reformat your operating system
     configuration (you need write permissions for the file):

          guix style -f /etc/config.scm

‘--styling=RULE’
‘-S RULE’
     Apply RULE, one of the following styling rules:

     ‘format’
          Format the given package definition(s)—this is the default
          styling rule.  For example, a packager running Guix on a
          checkout (*note Running Guix Before It Is Installed::) might
          want to reformat the definition of the Coreutils package like
          so:

               ./pre-inst-env guix style coreutils

     ‘inputs’
          Rewrite package inputs to the “new style”, as described above.
          This is how you would rewrite inputs of package ‘whatnot’ in
          your own channel:

               guix style -L ~/my/channel -S inputs whatnot

          Rewriting is done in a conservative way: preserving comments
          and bailing out if it cannot make sense of the code that
          appears in an inputs field.  The ‘--input-simplification’
          option described below provides fine-grain control over when
          inputs should be simplified.

‘--list-stylings’
‘-l’
     List and describe the available styling rules and exit.

‘--load-path=DIRECTORY’
‘-L DIRECTORY’
     Add DIRECTORY to the front of the package module search path (*note
     Package Modules::).

‘--expression=EXPR’
‘-e EXPR’
     Style the package EXPR evaluates to.

     For example, running:

          guix style -e '(@ (gnu packages gcc) gcc-5)'

     styles the ‘gcc-5’ package definition.

‘--input-simplification=POLICY’
     When using the ‘inputs’ styling rule, with ‘-S inputs’, this option
     specifies the package input simplification policy for cases where
     an input label does not match the corresponding package name.
     POLICY may be one of the following:

     ‘silent’
          Simplify inputs only when the change is “silent”, meaning that
          the package does not need to be rebuilt (its derivation is
          unchanged).

     ‘safe’
          Simplify inputs only when that is “safe” to do: the package
          might need to be rebuilt, but the change is known to have no
          observable effect.

     ‘always’
          Simplify inputs even when input labels do not match package
          names, and even if that might have an observable effect.

     The default is ‘silent’, meaning that input simplifications do not
     trigger any package rebuild.


File: guix.info,  Node: Invoking guix lint,  Next: Invoking guix size,  Prev: Invoking guix style,  Up: Utilities

10.8 Invoking ‘guix lint’
=========================

The ‘guix lint’ command is meant to help package developers avoid common
errors and use a consistent style.  It runs a number of checks on a
given set of packages in order to find common mistakes in their
definitions.  Available “checkers” include (see ‘--list-checkers’ for a
complete list):

‘synopsis’
‘description’
     Validate certain typographical and stylistic rules about package
     descriptions and synopses.

‘inputs-should-be-native’
     Identify inputs that should most likely be native inputs.

‘source’
‘home-page’
‘mirror-url’
‘github-url’
‘source-file-name’
     Probe ‘home-page’ and ‘source’ URLs and report those that are
     invalid.  Suggest a ‘mirror://’ URL when applicable.  If the
     ‘source’ URL redirects to a GitHub URL, recommend usage of the
     GitHub URL.  Check that the source file name is meaningful, e.g. is
     not just a version number or “git-checkout”, without a declared
     ‘file-name’ (*note origin Reference::).

‘source-unstable-tarball’
     Parse the ‘source’ URL to determine if a tarball from GitHub is
     autogenerated or if it is a release tarball.  Unfortunately
     GitHub’s autogenerated tarballs are sometimes regenerated.

‘derivation’
     Check that the derivation of the given packages can be successfully
     computed for all the supported systems (*note Derivations::).

‘profile-collisions’
     Check whether installing the given packages in a profile would lead
     to collisions.  Collisions occur when several packages with the
     same name but a different version or a different store file name
     are propagated.  *Note ‘propagated-inputs’: package Reference, for
     more information on propagated inputs.

‘archival’
     Checks whether the package’s source code is archived at Software
     Heritage (https://www.softwareheritage.org).

     When the source code that is not archived comes from a
     version-control system (VCS)—e.g., it’s obtained with ‘git-fetch’,
     send Software Heritage a “save” request so that it eventually
     archives it.  This ensures that the source will remain available in
     the long term, and that Guix can fall back to Software Heritage
     should the source code disappear from its original host.  The
     status of recent “save” requests can be viewed on-line
     (https://archive.softwareheritage.org/save/#requests).

     When source code is a tarball obtained with ‘url-fetch’, simply
     print a message when it is not archived.  As of this writing,
     Software Heritage does not allow requests to save arbitrary
     tarballs; we are working on ways to ensure that non-VCS source code
     is also archived.

     Software Heritage limits the request rate per IP address
     (https://archive.softwareheritage.org/api/#rate-limiting).  When
     the limit is reached, ‘guix lint’ prints a message and the
     ‘archival’ checker stops doing anything until that limit has been
     reset.

‘cve’
     Report known vulnerabilities found in the Common Vulnerabilities
     and Exposures (CVE) databases of the current and past year
     published by the US NIST (https://nvd.nist.gov/vuln/data-feeds).

     To view information about a particular vulnerability, visit pages
     such as:

        • 
          ‘https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-YYYY-ABCD’
        • ‘https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-YYYY-ABCD’

     where ‘CVE-YYYY-ABCD’ is the CVE identifier—e.g., ‘CVE-2015-7554’.

     Package developers can specify in package recipes the Common
     Platform Enumeration (CPE) (https://nvd.nist.gov/products/cpe) name
     and version of the package when they differ from the name or
     version that Guix uses, as in this example:

          (package
            (name "grub")
            ;; ...
            ;; CPE calls this package "grub2".
            (properties '((cpe-name . "grub2")
                          (cpe-version . "2.3"))))

     Some entries in the CVE database do not specify which version of a
     package they apply to, and would thus “stick around” forever.
     Package developers who found CVE alerts and verified they can be
     ignored can declare them as in this example:

          (package
            (name "t1lib")
            ;; ...
            ;; These CVEs no longer apply and can be safely ignored.
            (properties `((lint-hidden-cve . ("CVE-2011-0433"
                                              "CVE-2011-1553"
                                              "CVE-2011-1554"
                                              "CVE-2011-5244")))))

‘formatting’
     Warn about obvious source code formatting issues: trailing white
     space, use of tabulations, etc.

‘input-labels’
     Report old-style input labels that do not match the name of the
     corresponding package.  This aims to help migrate from the “old
     input style”.  *Note package Reference::, for more information on
     package inputs and input styles.  *Note Invoking guix style::, on
     how to migrate to the new style.

   The general syntax is:

     guix lint OPTIONS PACKAGE...

   If no package is given on the command line, then all packages are
checked.  The OPTIONS may be zero or more of the following:

‘--list-checkers’
‘-l’
     List and describe all the available checkers that will be run on
     packages and exit.

‘--checkers’
‘-c’
     Only enable the checkers specified in a comma-separated list using
     the names returned by ‘--list-checkers’.

‘--exclude’
‘-x’
     Only disable the checkers specified in a comma-separated list using
     the names returned by ‘--list-checkers’.

‘--expression=EXPR’
‘-e EXPR’
     Consider the package EXPR evaluates to.

     This is useful to unambiguously designate packages, as in this
     example:

          guix lint -c archival -e '(@ (gnu packages guile) guile-3.0)'

‘--no-network’
‘-n’
     Only enable the checkers that do not depend on Internet access.

‘--load-path=DIRECTORY’
‘-L DIRECTORY’
     Add DIRECTORY to the front of the package module search path (*note
     Package Modules::).

     This allows users to define their own packages and make them
     visible to the command-line tools.


File: guix.info,  Node: Invoking guix size,  Next: Invoking guix graph,  Prev: Invoking guix lint,  Up: Utilities

10.9 Invoking ‘guix size’
=========================

The ‘guix size’ command helps package developers profile the disk usage
of packages.  It is easy to overlook the impact of an additional
dependency added to a package, or the impact of using a single output
for a package that could easily be split (*note Packages with Multiple
Outputs::).  Such are the typical issues that ‘guix size’ can highlight.

   The command can be passed one or more package specifications such as
‘gcc@4.8’ or ‘guile:debug’, or a file name in the store.  Consider this
example:

     $ guix size coreutils
     store item                               total    self
     /gnu/store/...-gcc-5.5.0-lib           60.4    30.1  38.1%
     /gnu/store/...-glibc-2.27              30.3    28.8  36.6%
     /gnu/store/...-coreutils-8.28          78.9    15.0  19.0%
     /gnu/store/...-gmp-6.1.2               63.1     2.7   3.4%
     /gnu/store/...-bash-static-4.4.12       1.5     1.5   1.9%
     /gnu/store/...-acl-2.2.52              61.1     0.4   0.5%
     /gnu/store/...-attr-2.4.47             60.6     0.2   0.3%
     /gnu/store/...-libcap-2.25             60.5     0.2   0.2%
     total: 78.9 MiB

   The store items listed here constitute the “transitive closure” of
Coreutils—i.e., Coreutils and all its dependencies, recursively—as would
be returned by:

     $ guix gc -R /gnu/store/...-coreutils-8.23

   Here the output shows three columns next to store items.  The first
column, labeled “total”, shows the size in mebibytes (MiB) of the
closure of the store item—that is, its own size plus the size of all its
dependencies.  The next column, labeled “self”, shows the size of the
item itself.  The last column shows the ratio of the size of the item
itself to the space occupied by all the items listed here.

   In this example, we see that the closure of Coreutils weighs in at
79 MiB, most of which is taken by libc and GCC’s run-time support
libraries.  (That libc and GCC’s libraries represent a large fraction of
the closure is not a problem per se because they are always available on
the system anyway.)

   Since the command also accepts store file names, assessing the size
of a build result is straightforward:

     guix size $(guix system build config.scm)

   When the package(s) passed to ‘guix size’ are available in the
store(1), ‘guix size’ queries the daemon to determine its dependencies,
and measures its size in the store, similar to ‘du -ms --apparent-size’
(*note (coreutils)du invocation::).

   When the given packages are _not_ in the store, ‘guix size’ reports
information based on the available substitutes (*note Substitutes::).
This makes it possible it to profile disk usage of store items that are
not even on disk, only available remotely.

   You can also specify several package names:

     $ guix size coreutils grep sed bash
     store item                               total    self
     /gnu/store/...-coreutils-8.24          77.8    13.8  13.4%
     /gnu/store/...-grep-2.22               73.1     0.8   0.8%
     /gnu/store/...-bash-4.3.42             72.3     4.7   4.6%
     /gnu/store/...-readline-6.3            67.6     1.2   1.2%
     ...
     total: 102.3 MiB

In this example we see that the combination of the four packages takes
102.3 MiB in total, which is much less than the sum of each closure
since they have a lot of dependencies in common.

   When looking at the profile returned by ‘guix size’, you may find
yourself wondering why a given package shows up in the profile at all.
To understand it, you can use ‘guix graph --path -t references’ to
display the shortest path between the two packages (*note Invoking guix
graph::).

   The available options are:

‘--substitute-urls=URLS’
     Use substitute information from URLS.  *Note the same option for
     ‘guix build’: client-substitute-urls.

‘--sort=KEY’
     Sort lines according to KEY, one of the following options:

     ‘self’
          the size of each item (the default);
     ‘closure’
          the total size of the item’s closure.

‘--map-file=FILE’
     Write a graphical map of disk usage in PNG format to FILE.

     For the example above, the map looks like this:

[image src="images/coreutils-size-map.png" alt="map of Coreutils disk usage
produced by guix size"]

     This option requires that Guile-Charting
     (https://wingolog.org/software/guile-charting/) be installed and
     visible in Guile’s module search path.  When that is not the case,
     ‘guix size’ fails as it tries to load it.

‘--system=SYSTEM’
‘-s SYSTEM’
     Consider packages for SYSTEM—e.g., ‘x86_64-linux’.

‘--load-path=DIRECTORY’
‘-L DIRECTORY’
     Add DIRECTORY to the front of the package module search path (*note
     Package Modules::).

     This allows users to define their own packages and make them
     visible to the command-line tools.

   ---------- Footnotes ----------

   (1) More precisely, ‘guix size’ looks for the _ungrafted_ variant of
the given package(s), as returned by ‘guix build PACKAGE --no-grafts’.
*Note Security Updates::, for information on grafts.


File: guix.info,  Node: Invoking guix graph,  Next: Invoking guix publish,  Prev: Invoking guix size,  Up: Utilities

10.10 Invoking ‘guix graph’
===========================

Packages and their dependencies form a “graph”, specifically a directed
acyclic graph (DAG). It can quickly become difficult to have a mental
model of the package DAG, so the ‘guix graph’ command provides a visual
representation of the DAG.  By default, ‘guix graph’ emits a DAG
representation in the input format of Graphviz
(https://www.graphviz.org/), so its output can be passed directly to the
‘dot’ command of Graphviz.  It can also emit an HTML page with embedded
JavaScript code to display a “chord diagram” in a Web browser, using the
d3.js (https://d3js.org/) library, or emit Cypher queries to construct a
graph in a graph database supporting the openCypher
(https://www.opencypher.org/) query language.  With ‘--path’, it simply
displays the shortest path between two packages.  The general syntax is:

     guix graph OPTIONS PACKAGE...

   For example, the following command generates a PDF file representing
the package DAG for the GNU Core Utilities, showing its build-time
dependencies:

     guix graph coreutils | dot -Tpdf > dag.pdf

   The output looks like this:

[image src="images/coreutils-graph.png" alt="Dependency graph of the GNU Coreutils"]

   Nice little graph, no?

   You may find it more pleasant to navigate the graph interactively
with ‘xdot’ (from the ‘xdot’ package):

     guix graph coreutils | xdot -

   But there is more than one graph!  The one above is concise: it is
the graph of package objects, omitting implicit inputs such as GCC,
libc, grep, etc.  It is often useful to have such a concise graph, but
sometimes one may want to see more details.  ‘guix graph’ supports
several types of graphs, allowing you to choose the level of detail:

‘package’
     This is the default type used in the example above.  It shows the
     DAG of package objects, excluding implicit dependencies.  It is
     concise, but filters out many details.

‘reverse-package’
     This shows the _reverse_ DAG of packages.  For example:

          guix graph --type=reverse-package ocaml

     ... yields the graph of packages that _explicitly_ depend on OCaml
     (if you are also interested in cases where OCaml is an implicit
     dependency, see ‘reverse-bag’ below).

     Note that for core packages this can yield huge graphs.  If all you
     want is to know the number of packages that depend on a given
     package, use ‘guix refresh --list-dependent’ (*note
     ‘--list-dependent’: Invoking guix refresh.).

‘bag-emerged’
     This is the package DAG, _including_ implicit inputs.

     For instance, the following command:

          guix graph --type=bag-emerged coreutils

     ... yields this bigger graph:

[image src="images/coreutils-bag-graph.png" alt="Detailed dependency graph of the GNU Coreutils"]

     At the bottom of the graph, we see all the implicit inputs of
     GNU-BUILD-SYSTEM (*note ‘gnu-build-system’: Build Systems.).

     Now, note that the dependencies of these implicit inputs—that is,
     the “bootstrap dependencies” (*note Bootstrapping::)—are not shown
     here, for conciseness.

‘bag’
     Similar to ‘bag-emerged’, but this time including all the bootstrap
     dependencies.

‘bag-with-origins’
     Similar to ‘bag’, but also showing origins and their dependencies.

‘reverse-bag’
     This shows the _reverse_ DAG of packages.  Unlike
     ‘reverse-package’, it also takes implicit dependencies into
     account.  For example:

          guix graph -t reverse-bag dune

     ... yields the graph of all packages that depend on Dune, directly
     or indirectly.  Since Dune is an _implicit_ dependency of many
     packages via ‘dune-build-system’, this shows a large number of
     packages, whereas ‘reverse-package’ would show very few if any.

‘derivation’
     This is the most detailed representation: It shows the DAG of
     derivations (*note Derivations::) and plain store items.  Compared
     to the above representation, many additional nodes are visible,
     including build scripts, patches, Guile modules, etc.

     For this type of graph, it is also possible to pass a ‘.drv’ file
     name instead of a package name, as in:

          guix graph -t derivation $(guix system build -d my-config.scm)

‘module’
     This is the graph of “package modules” (*note Package Modules::).
     For example, the following command shows the graph for the package
     module that defines the ‘guile’ package:

          guix graph -t module guile | xdot -

   All the types above correspond to _build-time dependencies_.  The
following graph type represents the _run-time dependencies_:

‘references’
     This is the graph of “references” of a package output, as returned
     by ‘guix gc --references’ (*note Invoking guix gc::).

     If the given package output is not available in the store, ‘guix
     graph’ attempts to obtain dependency information from substitutes.

     Here you can also pass a store file name instead of a package name.
     For example, the command below produces the reference graph of your
     profile (which can be big!):

          guix graph -t references $(readlink -f ~/.guix-profile)

‘referrers’
     This is the graph of the “referrers” of a store item, as returned
     by ‘guix gc --referrers’ (*note Invoking guix gc::).

     This relies exclusively on local information from your store.  For
     instance, let us suppose that the current Inkscape is available in
     10 profiles on your machine; ‘guix graph -t referrers inkscape’
     will show a graph rooted at Inkscape and with those 10 profiles
     linked to it.

     It can help determine what is preventing a store item from being
     garbage collected.

   Often, the graph of the package you are interested in does not fit on
your screen, and anyway all you want to know is _why_ that package
actually depends on some seemingly unrelated package.  The ‘--path’
option instructs ‘guix graph’ to display the shortest path between two
packages (or derivations, or store items, etc.):

     $ guix graph --path emacs libunistring
     emacs@26.3
     mailutils@3.9
     libunistring@0.9.10
     $ guix graph --path -t derivation emacs libunistring
     /gnu/store/...-emacs-26.3.drv
     /gnu/store/...-mailutils-3.9.drv
     /gnu/store/...-libunistring-0.9.10.drv
     $ guix graph --path -t references emacs libunistring
     /gnu/store/...-emacs-26.3
     /gnu/store/...-libidn2-2.2.0
     /gnu/store/...-libunistring-0.9.10

   Sometimes you still want to visualize the graph but would like to
trim it so it can actually be displayed.  One way to do it is via the
‘--max-depth’ (or ‘-M’) option, which lets you specify the maximum depth
of the graph.  In the example below, we visualize only ‘libreoffice’ and
the nodes whose distance to ‘libreoffice’ is at most 2:

     guix graph -M 2 libreoffice | xdot -f fdp -

   Mind you, that’s still a big ball of spaghetti, but at least ‘dot’
can render it quickly and it can be browsed somewhat.

   The available options are the following:

‘--type=TYPE’
‘-t TYPE’
     Produce a graph output of TYPE, where TYPE must be one of the
     values listed above.

‘--list-types’
     List the supported graph types.

‘--backend=BACKEND’
‘-b BACKEND’
     Produce a graph using the selected BACKEND.

‘--list-backends’
     List the supported graph backends.

     Currently, the available backends are Graphviz and d3.js.

‘--path’
     Display the shortest path between two nodes of the type specified
     by ‘--type’.  The example below shows the shortest path between
     ‘libreoffice’ and ‘llvm’ according to the references of
     ‘libreoffice’:

          $ guix graph --path -t references libreoffice llvm
          /gnu/store/...-libreoffice-6.4.2.2
          /gnu/store/...-libepoxy-1.5.4
          /gnu/store/...-mesa-19.3.4
          /gnu/store/...-llvm-9.0.1

‘--expression=EXPR’
‘-e EXPR’
     Consider the package EXPR evaluates to.

     This is useful to precisely refer to a package, as in this example:

          guix graph -e '(@@ (gnu packages commencement) gnu-make-final)'

‘--system=SYSTEM’
‘-s SYSTEM’
     Display the graph for SYSTEM—e.g., ‘i686-linux’.

     The package dependency graph is largely architecture-independent,
     but there are some architecture-dependent bits that this option
     allows you to visualize.

‘--load-path=DIRECTORY’
‘-L DIRECTORY’
     Add DIRECTORY to the front of the package module search path (*note
     Package Modules::).

     This allows users to define their own packages and make them
     visible to the command-line tools.

   On top of that, ‘guix graph’ supports all the usual package
transformation options (*note Package Transformation Options::).  This
makes it easy to view the effect of a graph-rewriting transformation
such as ‘--with-input’.  For example, the command below outputs the
graph of ‘git’ once ‘openssl’ has been replaced by ‘libressl’ everywhere
in the graph:

     guix graph git --with-input=openssl=libressl

   So many possibilities, so much fun!


File: guix.info,  Node: Invoking guix publish,  Next: Invoking guix challenge,  Prev: Invoking guix graph,  Up: Utilities

10.11 Invoking ‘guix publish’
=============================

The purpose of ‘guix publish’ is to enable users to easily share their
store with others, who can then use it as a substitute server (*note
Substitutes::).

   When ‘guix publish’ runs, it spawns an HTTP server which allows
anyone with network access to obtain substitutes from it.  This means
that any machine running Guix can also act as if it were a build farm,
since the HTTP interface is compatible with Cuirass, the software behind
the ‘ci.guix.gnu.org’ build farm.

   For security, each substitute is signed, allowing recipients to check
their authenticity and integrity (*note Substitutes::).  Because ‘guix
publish’ uses the signing key of the system, which is only readable by
the system administrator, it must be started as root; the ‘--user’
option makes it drop root privileges early on.

   The signing key pair must be generated before ‘guix publish’ is
launched, using ‘guix archive --generate-key’ (*note Invoking guix
archive::).

   When the ‘--advertise’ option is passed, the server advertises its
availability on the local network using multicast DNS (mDNS) and DNS
service discovery (DNS-SD), currently via Guile-Avahi (*note
(guile-avahi)Top::).

   The general syntax is:

     guix publish OPTIONS...

   Running ‘guix publish’ without any additional arguments will spawn an
HTTP server on port 8080:

     guix publish

   ‘guix publish’ can also be started following the systemd “socket
activation” protocol (*note ‘make-systemd-constructor’:
(shepherd)Service De- and Constructors.).

   Once a publishing server has been authorized, the daemon may download
substitutes from it.  *Note Getting Substitutes from Other Servers::.

   By default, ‘guix publish’ compresses archives on the fly as it
serves them.  This “on-the-fly” mode is convenient in that it requires
no setup and is immediately available.  However, when serving lots of
clients, we recommend using the ‘--cache’ option, which enables caching
of the archives before they are sent to clients—see below for details.
The ‘guix weather’ command provides a handy way to check what a server
provides (*note Invoking guix weather::).

   As a bonus, ‘guix publish’ also serves as a content-addressed mirror
for source files referenced in ‘origin’ records (*note origin
Reference::).  For instance, assuming ‘guix publish’ is running on
‘example.org’, the following URL returns the raw ‘hello-2.10.tar.gz’
file with the given SHA256 hash (represented in ‘nix-base32’ format,
*note Invoking guix hash::):

     http://example.org/file/hello-2.10.tar.gz/sha256/0ssi1...ndq1i

   Obviously, these URLs only work for files that are in the store; in
other cases, they return 404 (“Not Found”).

   Build logs are available from ‘/log’ URLs like:

     http://example.org/log/gwspk...-guile-2.2.3

When ‘guix-daemon’ is configured to save compressed build logs, as is
the case by default (*note Invoking guix-daemon::), ‘/log’ URLs return
the compressed log as-is, with an appropriate ‘Content-Type’ and/or
‘Content-Encoding’ header.  We recommend running ‘guix-daemon’ with
‘--log-compression=gzip’ since Web browsers can automatically decompress
it, which is not the case with Bzip2 compression.

   The following options are available:

‘--port=PORT’
‘-p PORT’
     Listen for HTTP requests on PORT.

‘--listen=HOST’
     Listen on the network interface for HOST.  The default is to accept
     connections from any interface.

‘--user=USER’
‘-u USER’
     Change privileges to USER as soon as possible—i.e., once the server
     socket is open and the signing key has been read.

‘--compression[=METHOD[:LEVEL]]’
‘-C [METHOD[:LEVEL]]’
     Compress data using the given METHOD and LEVEL.  METHOD is one of
     ‘lzip’, ‘zstd’, and ‘gzip’; when METHOD is omitted, ‘gzip’ is used.

     When LEVEL is zero, disable compression.  The range 1 to 9
     corresponds to different compression levels: 1 is the fastest, and
     9 is the best (CPU-intensive).  The default is 3.

     Usually, ‘lzip’ compresses noticeably better than ‘gzip’ for a
     small increase in CPU usage; see benchmarks on the lzip Web page
     (https://nongnu.org/lzip/lzip_benchmark.html).  However, ‘lzip’
     achieves low decompression throughput (on the order of 50 MiB/s on
     modern hardware), which can be a bottleneck for someone who
     downloads over a fast network connection.

     The compression ratio of ‘zstd’ is between that of ‘lzip’ and that
     of ‘gzip’; its main advantage is a high decompression speed
     (https://facebook.github.io/zstd/).

     Unless ‘--cache’ is used, compression occurs on the fly and the
     compressed streams are not cached.  Thus, to reduce load on the
     machine that runs ‘guix publish’, it may be a good idea to choose a
     low compression level, to run ‘guix publish’ behind a caching
     proxy, or to use ‘--cache’.  Using ‘--cache’ has the advantage that
     it allows ‘guix publish’ to add ‘Content-Length’ HTTP header to its
     responses.

     This option can be repeated, in which case every substitute gets
     compressed using all the selected methods, and all of them are
     advertised.  This is useful when users may not support all the
     compression methods: they can select the one they support.

‘--cache=DIRECTORY’
‘-c DIRECTORY’
     Cache archives and meta-data (‘.narinfo’ URLs) to DIRECTORY and
     only serve archives that are in cache.

     When this option is omitted, archives and meta-data are created
     on-the-fly.  This can reduce the available bandwidth, especially
     when compression is enabled, since this may become CPU-bound.
     Another drawback of the default mode is that the length of archives
     is not known in advance, so ‘guix publish’ does not add a
     ‘Content-Length’ HTTP header to its responses, which in turn
     prevents clients from knowing the amount of data being downloaded.

     Conversely, when ‘--cache’ is used, the first request for a store
     item (via a ‘.narinfo’ URL) triggers a background process to “bake”
     the archive—computing its ‘.narinfo’ and compressing the archive,
     if needed.  Once the archive is cached in DIRECTORY, subsequent
     requests succeed and are served directly from the cache, which
     guarantees that clients get the best possible bandwidth.

     That first ‘.narinfo’ request nonetheless returns 200, provided the
     requested store item is “small enough”, below the cache bypass
     threshold—see ‘--cache-bypass-threshold’ below.  That way, clients
     do not have to wait until the archive is baked.  For larger store
     items, the first ‘.narinfo’ request returns 404, meaning that
     clients have to wait until the archive is baked.

     The “baking” process is performed by worker threads.  By default,
     one thread per CPU core is created, but this can be customized.
     See ‘--workers’ below.

     When ‘--ttl’ is used, cached entries are automatically deleted when
     they have expired.

‘--workers=N’
     When ‘--cache’ is used, request the allocation of N worker threads
     to “bake” archives.

‘--ttl=TTL’
     Produce ‘Cache-Control’ HTTP headers that advertise a time-to-live
     (TTL) of TTL.  TTL must denote a duration: ‘5d’ means 5 days, ‘1m’
     means 1 month, and so on.

     This allows the user’s Guix to keep substitute information in cache
     for TTL.  However, note that ‘guix publish’ does not itself
     guarantee that the store items it provides will indeed remain
     available for as long as TTL.

     Additionally, when ‘--cache’ is used, cached entries that have not
     been accessed for TTL and that no longer have a corresponding item
     in the store, may be deleted.

‘--negative-ttl=TTL’
     Similarly produce ‘Cache-Control’ HTTP headers to advertise the
     time-to-live (TTL) of _negative_ lookups—missing store items, for
     which the HTTP 404 code is returned.  By default, no negative TTL
     is advertised.

     This parameter can help adjust server load and substitute latency
     by instructing cooperating clients to be more or less patient when
     a store item is missing.

‘--cache-bypass-threshold=SIZE’
     When used in conjunction with ‘--cache’, store items smaller than
     SIZE are immediately available, even when they are not yet in
     cache.  SIZE is a size in bytes, or it can be suffixed by ‘M’ for
     megabytes and so on.  The default is ‘10M’.

     “Cache bypass” allows you to reduce the publication delay for
     clients at the expense of possibly additional I/O and CPU use on
     the server side: depending on the client access patterns, those
     store items can end up being baked several times until a copy is
     available in cache.

     Increasing the threshold may be useful for sites that have few
     users, or to guarantee that users get substitutes even for store
     items that are not popular.

‘--nar-path=PATH’
     Use PATH as the prefix for the URLs of “nar” files (*note
     normalized archives: Invoking guix archive.).

     By default, nars are served at a URL such as
     ‘/nar/gzip/...-coreutils-8.25’.  This option allows you to change
     the ‘/nar’ part to PATH.

‘--public-key=FILE’
‘--private-key=FILE’
     Use the specific FILEs as the public/private key pair used to sign
     the store items being published.

     The files must correspond to the same key pair (the private key is
     used for signing and the public key is merely advertised in the
     signature metadata).  They must contain keys in the canonical
     s-expression format as produced by ‘guix archive --generate-key’
     (*note Invoking guix archive::).  By default,
     ‘/etc/guix/signing-key.pub’ and ‘/etc/guix/signing-key.sec’ are
     used.

‘--repl[=PORT]’
‘-r [PORT]’
     Spawn a Guile REPL server (*note (guile)REPL Servers::) on PORT
     (37146 by default).  This is used primarily for debugging a running
     ‘guix publish’ server.

   Enabling ‘guix publish’ on Guix System is a one-liner: just
instantiate a ‘guix-publish-service-type’ service in the ‘services’
field of the ‘operating-system’ declaration (*note
‘guix-publish-service-type’: guix-publish-service-type.).

   If you are instead running Guix on a “foreign distro”, follow these
instructions:

   • If your host distro uses the systemd init system:

          # ln -s ~root/.guix-profile/lib/systemd/system/guix-publish.service \
                  /etc/systemd/system/
          # systemctl start guix-publish && systemctl enable guix-publish

   • If your host distro uses the Upstart init system:

          # ln -s ~root/.guix-profile/lib/upstart/system/guix-publish.conf /etc/init/
          # start guix-publish

   • Otherwise, proceed similarly with your distro’s init system.

