Flakes in the nix store do not retain references to their inputs

[tejing1]

Describe the bug

Many nixos + flakes users keep a reference to self in their nixos generations, for traceability, however this does not ensure that inputs to the flake are kept in the closure. For most inputs, in most cases, this simply means they need to be re-downloaded from github or wherever they came from, however for path: inputs, and for inputs where the source has been taken down for some reason, a user may be unpleasantly surprised to find that they do not, in fact, have all the nix code needed to reproduce their system.

Steps To Reproduce
(Note the dependence on PWD in flake2's inputs. Adjust as needed if reproducing.)

$ find
.
./flake2
./flake2/flake.nix
./flake1
./flake1/flake.nix
$ cat flake1/flake.nix
{
  outputs = { self, ... }: {
    foo = 1;
  };
}
$ cat flake2/flake.nix
{
  inputs.nixpkgs.url = github:nixos/nixpkgs/nixos-22.05;
  inputs.flake1.url = path:/mnt/persist/share/data/tejing/work/tmp/flake1;
  outputs = {self, nixpkgs, flake1}: {
    inherit (flake1) foo;
    result = nixpkgs.legacyPackages.x86_64-linux.runCommand "flake2-result" { inherit (self) outPath; } ''
      mkdir $out
      ln -s $outPath $out/self
    '';
  };
}
$ nix eval ./flake2\#foo
warning: creating lock file '/mnt/persist/share/data/tejing/work/tmp/flake2/flake.lock'
1
$ nix build ./flake2\#result
$ nix eval $(realpath ./result/self)\#foo
1
$ mv flake1 flake1.bak
$ nix-store --gc
finding garbage collector roots...
deleting garbage...
deleting '/nix/store/8igg0ivjv0y8c04ac7b78n0zgmg1i987-source'
... SNIP ...
deleting '/nix/store/wj6j8lrdlind44n7vqn864ga7y802vc7-libunistring-1.0'
deleting unused links...
note: currently hard linking saves 4783.59 MiB
32 store paths deleted, 73.96 MiB freed
$ nix eval $(realpath ./result/self)\#foo
error (ignored): error: end of string reached
error: getting status of '/mnt/persist/share/data/tejing/work/tmp/flake1': No such file or directory
(use '--show-trace' to show detailed location information)

Expected behavior

I would expect that final nix eval $(realpath ./result/self)\#foo to successfully produce 1, just as it did before the gc, indicating that saving self in the result output had in fact kept everything necessary to re-evaluate the flake.

nix-env --version output

$ nix-env --version
nix-env (Nix) 2.8.1

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/what-would-you-change-in-nix-or-nixos/21086/3

[tpwrules]

Related, including possible solutions: #4250

[tejing1]

For anyone looking to work around this problem, this may be helpful: tejing1/nixos-config@4767d88

[a-h]

If you're looking at @tejing1's config and wondering how to adapt it for your use in flakes, this example I wrote might help.

https://github.com/a-h/nix-copy-flake-inputs-to-store/blob/main/flake.nix

{
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-23.11";
    nixos-generators = {
      url = "github:nix-community/nixos-generators";
      inputs.nixpkgs.follows = "nixpkgs";
    };
    xc = {
      url = "github:joerdav/xc";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };
  outputs = { self, nixpkgs, xc, ... }:
    let
      pkgsForSystem = system: import nixpkgs {
        inherit system;
        overlays = [
          (final: prev: { xc = xc.packages.${system}.xc; })
        ];
      };
      flakeClosureRefForSystem = { system, pkgs }: (import ./flakeClosureRef.nix {
        pkgs = pkgs;
        lib = nixpkgs.lib;
      });
      listFlakeInputsForSystem = { system, pkgs }: pkgs.writeShellScriptBin "list-flake-inputs" ''
        cat ${((flakeClosureRefForSystem { inherit system pkgs; }) self)}
      '';
      allSystems = [ "x86_64-linux" "aarch64-linux" "x86_64-darwin" "aarch64-darwin" ];
      forAllSystems = f: nixpkgs.lib.genAttrs allSystems (system: f {
        system = system;
        pkgs = pkgsForSystem system;
      });
    in
    {
      # Add the flake input reference to the output set so that we can see it in the repl.
      #
      # Load the repl with:
      #   nix repl
      # Inside the repl, load the flake:
      #   :lf .
      # View the derivation:
      #   outputs.flakeInputReference.x86_64-linux
      # Then build it.
      #   :b outputs.flakeInputReference.x86_64-linux
      #
      # The store path will be printed.
      #
      # cat the store path to see the contents. If you inspect the directories, you'll see
      # that the directories contain the source code of all flake inputs.
      flakeInputReference = forAllSystems ({ system, pkgs }: {
        default = ((flakeClosureRefForSystem { inherit system pkgs; }) self);
      });

      # `nix develop` provides a shell containing development tools.
      devShells = forAllSystems ({ system, pkgs }: {
        default = pkgs.mkShell {
          buildInputs = [
            # Bring in xc as an overlay applied within pkgsForSystem.
            # So instead of xc.packages.${system}.xc, we can use pkgs.xc.
            pkgs.xc

            # Ensure that the recursive tree of flake inputs are added to the Nix store.
            # You can list the flake inputs with the `list-flake-inputs` command.
            (listFlakeInputsForSystem { inherit system pkgs; })
          ];
        };
      });
    };
}

If you want to build without an internet connection later as per #4250 you may find this useful.

[dezren39]

hi @a-h i'm sorry can you help me understand how to use this for nixosConfiguration inputs?

[tejing1]

For a nixos config, my original example is closer to what you want.

Assuming you pass inputs through specialArgs, putting this in a file and adding it to imports should do it:

{ lib, pkgs, inputs, ... }:

let
  inherit (builtins) concatMap attrValues concatStringsSep;
  inherit (lib) unique;
  flakesClosure = flakes: if flakes == [] then [] else unique (flakes ++ flakesClosure (concatMap (flake: if flake ? inputs then attrValues flake.inputs else []) flakes));
  flakeClosureRef = flake: pkgs.writeText "flake-closure" (concatStringsSep "\n" (flakesClosure [ flake ]) + "\n");
in
{
  system.extraDependencies = [ (flakeClosureRef inputs.self) ];
}