stdenv-darwin: now with 50% less LLVM!

[Gaelan]

Motivation for this change

On Darwin, stdenv was building LLVM twice, with the only difference being that one used a stage 3 libxml2 and the other used one from stage 4. Unless I'm missing something (which is quite possible!), there's no reason to do this, and we can save a bunch of build time by just using the one LLVM.

I tested this by building stdenv and using it to build GNU hello, and also checked to make sure bootstrap-tools didn't end up in stdenv's closure.

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

[Gaelan]

cc @LnL7

[matthewbauer]

Nice work!

[cole-h]

This should probably be rebased against staging.

@ofborg eval

[Gaelan]

Now against staging.

[LnL7]

The goal here is to build all of the stdenv tooling from scratch without any references to the tools used to bootstrap it. Secondly some of the overrides are there to reduce the tree needed to do this, by default certain packages like sqlite would be included by python/cmake while they are not actually needed for example.

Assuming libxml2 from stage 3 and any other dependencies of that first llvm already don't have references anymore, then there shouldn't be any reason to build another one. If this is the case I wonder if we could even get rid of a stage.

Could you double check this?

$ nix-store -qR $(nix-build -A stdenv.__bootPackages.stdenv) | grep bootstrap
/nix/store/2ws9cmamvr7xyvdg4d2nnd1bmr1zjrrq-bootstrap-tools
/nix/store/vvk6w4jdwm5m5r1f6mlvqjx4p1lpmik8-bootstrap-stage4-binutils-wrapper-
/nix/store/xkl1gvhj841qq3m0abxlg9w06dpx11hh-bootstrap-stage4-clang-wrapper-
/nix/store/1rkmy85jsvqq9bnbcz4fwm246n4h9r3a-bootstrap-stage4-stdenv-darwin

$ nix-store -qR $(nix-build -A stdenv) | grep bootstrap

[Gaelan]

$ git checkout single-llvm
Already on 'single-llvm'
Your branch is up to date with 'Gaelan/single-llvm'.

$ nix-store -qR $(nix-build . -A stdenv.__bootPackages.stdenv) | grep bootstrap
/nix/store/2ws9cmamvr7xyvdg4d2nnd1bmr1zjrrq-bootstrap-tools
/nix/store/9aa2rfiacjka8kvrg17ryyb6lhk5n62x-bootstrap-stage4-binutils-wrapper-
/nix/store/pm9jimjpvjpq0v9jsyswz5h35mx2bxi7-bootstrap-stage4-clang-wrapper-
/nix/store/rn1r46nyp4qfjhjbyysl91l0g1rv0q2l-bootstrap-stage4-stdenv-darwin

$ nix-store -qR $(nix-build . -A stdenv) | grep bootstrap

[Gaelan]

Is anything blocking this at this point?

[matthewbauer]

Nope- merged to staging. If you could be on alert for any breakages from this that would be good though. Occasionally staging-next will get stuck because we don’t know why something broke.