cabal-install 3.14 linking fails with "shared object file not found"

[brandonchinn178]

Describe the bug

In my project, if I use the latest version of cabal-install in CI (3.14), I get a failure, but CI passes when pinning cabal-install to 3.12. PR: brandonchinn178/aeson-schemas#99. My project seems like a pretty normal library+executable+benchmark project, without anything special like compiler plugins. I'm running a simple cabal build && cabal exec -- cabal test. I can't repro locally.

CI logs:

• 3.12 passes: https://github.com/brandonchinn178/aeson-schemas/actions/runs/12850695693/job/35830628997
• 3.14 fails: https://github.com/brandonchinn178/aeson-schemas/actions/runs/12850698146/job/35830632450

One difference in 3.14 is it says "cabal_macros.h changed", presumably why it's rebuilding. Printing out cabal_macros.h before and after, I think the change is happy is not included after cabal build, but is included in the subsequent cabal exec -- cabal test. I don't use or declare happy anywhere in my cabal file, but it looks like a dependency, haskell-src-exts, uses it.

I do see this error message reported in other issues, including #8580 which I reported, but I'm reporting this separately because it looks like a possible regression between 3.12 and 3.14, instead of something I'm doing wrong.

To Reproduce
Steps to reproduce the behavior:

$ cabal build && cabal exec -- cabal test

Expected behavior
Should succeed

System information

• Operating system
• cabal, ghc versions

Additional context
Add any other context about the problem here.

[mpickering]

I can reproduce this locally.

[mpickering]

Is there a reason in particular you are running cabal exec -- cabal test rather than just cabal test?

[mpickering]

The issue appears to be that when cabal exec -- cabal test is run, there is an environment file in scope, this causes the .so to have more dependencies than it did initially (since more -package args are passed). cabal then sets the rpath, but only to actual dependencies (not everything in the environment file).

Then the .so is attempted to be loaded, the shared library fails to be found because it is not on the rpath.

[brandonchinn178]

Yes, I believe this test needs cabal exec because it shells out to ghc:

https://github.com/brandonchinn178/aeson-schemas/blob/main/test%2FTests%2FGetQQ.hs#L414

There might be other tests too. Removing cabal exec should see failures

[mpickering]

The cabal_macros.h file changes for the same reason, the ghc invocation picks up the environment file and hence there are extra -package arguments to ghc.

[mpickering]

@brandonchinn178 I can't reproduce with cabal-install at commit - 5a21af6 .. possibly already fixed. With 3.14.1.0 I see "configuration changed" as the reason it has decided to rebuild.

Perhaps another way to implement this would be cabal exec -- $(cabal list-bin aeson-schemas-test)

The configuration changed message is just a generic message which says that ElaboratedConfiguredPackage is different somehow.

[mpickering]

Good news is I can reproduce with a locally built 3.14.1.1 cabal-install.

[mpickering]

The reason why it thinks that the configuration has changed is because when cabal test runs inside cabal exec the PATH to happy is configured.

"/nix/store/2b5jf6vy42z8jinw0n023fld1b9xf5xm-ghc-9.  "/nix/store/2b5jf6vy42z8jinw0n023fld1b9xf5xm-ghc-9. 
10.1/bin/ghc"),                                      10.1/bin/ghc"),                                     
("ghc-pkg",                                          ("ghc-pkg",                                         
"/nix/store/2b5jf6vy42z8jinw0n023fld1b9xf5xm-ghc-9.  "/nix/store/2b5jf6vy42z8jinw0n023fld1b9xf5xm-ghc-9. 
10.1/bin/ghc-pkg-9.10.1"),                           10.1/bin/ghc-pkg-9.10.1"),                          
("haddock",                                          ("haddock",                                         
"/nix/store/2b5jf6vy42z8jinw0n023fld1b9xf5xm-ghc-9.  "/nix/store/2b5jf6vy42z8jinw0n023fld1b9xf5xm-ghc-9. 
10.1/bin/haddock-ghc-9.10.1"),                       10.1/bin/haddock-ghc-9.10.1"),                      
("happy",                                                                                                
"/tmp/nix-shell-891311-0/tmp.8XPSHwULXW/aeson-schem                                                      
as/new_store/ghc-9.10.1-inplace/happy-2.1.5-e-happy                                                      
-c05fa464c1d118eac060d2bb66deb7ee67bdbead053d4a468e                                                      
f31979c7e04537/bin/happy"),                                                                              
("hpc",                                              ("hpc",                                             
"/nix/store/2b5jf6vy42z8jinw0n023fld1b9xf5xm-ghc-9.  "/nix/store/2b5jf6vy42z8jinw0n023fld1b9xf5xm-ghc-9. 
10.1/bin/hpc-ghc-9.10.1"),                           10.1/bin/hpc-ghc-9.10.1"),                          
("hsc2hs",                                           ("hsc2hs",                                          
"/nix/store/2b5jf6vy42z8jinw0n023fld1b9xf5xm-ghc-9.  "/nix/store/2b5jf6vy42z8jinw0n023fld1b9xf5xm-ghc-9. 
10.1/bin/hsc2hs-ghc-9.10.1"),                        10.1/bin/hsc2hs-ghc-9.10.1"),                       
("ld",                                               ("ld",                                              
---                                                  ---                                                 
UnitId "first-class-families-0.8.1.0-5dad8c3d5d9563  UnitId "first-class-families-0.8.1.0-5dad8c3d5d9563 
576e1f44ad47c8184ecbe7c5145b991652b7f6c9cfb29edd5c"  576e1f44ad47c8184ecbe7c5145b991652b7f6c9cfb29edd5c" 
,                                                    ,                                                   
UnitId "hashable-1.5.0.0-bbf1f845af13b94075c573458a  UnitId "hashable-1.5.0.0-bbf1f845af13b94075c573458a 
b31dee318bda01c596ac2f915a9df45de776d3",             b31dee318bda01c596ac2f915a9df45de776d3",            
UnitId "megaparsec-9.7.0-627834fd7888b070e48135ff5d  UnitId "megaparsec-9.7.0-627834fd7888b070e48135ff5d 
2f9de070a4a86da0683b0b419a2044853aeefd",             2f9de070a4a86da0683b0b419a2044853aeefd",            
UnitId "template-haskell-2.22.0.0-inplace",          UnitId "template-haskell-2.22.0.0-inplace",         
UnitId "text-2.1.1-inplace",                         UnitId "text-2.1.1-inplace",                        
UnitId "unordered-containers-0.2.20-514b90ab4cd1751  UnitId "unordered-containers-0.2.20-514b90ab4cd1751 
972e07befc98d14e821b236a1f1757c58e46c6fd38d46e31c"]  972e07befc98d14e821b236a1f1757c58e46c6fd38d46e31c"] 
})},                                                 })})    

[mpickering]

Therefore I think this is the same issue as #10692 which was fixed by #10731.

However, the real issue is arguably that GHC called by cabal-install is getting affected by the existence of environment files. cabal-install already has logic to avoid getting affected by GHC_PACKAGE_PATH, so it should also be isolated by environment files existing.

[brandonchinn178]

I think that makes sense, thanks for investigating! Are there any plans for 3.14.1.2?