Use public API for Pkg stdlibs

[fonsp]

Using JuliaLang/Pkg.jl#4081

Try this Pull Request!

Open Julia and type:

julia> import Pkg
julia> Pkg.activate(temp=true)
julia> Pkg.add(url="https://github.com/fonsp/Pluto.jl", rev="pkg-stdlibs-public-api")
julia> using Pluto

[pankgeorg]

What a Standard Library is is not really well defined, in conjunction with the (internal) method Base.in_sysimage. Being in the readdir(Sys.STDLIB) doesn't necessarily mean it's in the sysimage (especially with the concept of "Upgradeable" Stdlibs). And vice versa, the other way around is also not stable. I think readdir(Sys.STDLIB) means "Pkg should not resolve this independently" and Base.in_sysimage means "Pkg should pin the version to the sysimage version and we should never Pkg.add it". But I may be wrong. The semantics of Pkg in the presence of a sysimage (and now upgradable libs) are a bit fuzzy in my head.

[fonsp]

In Pluto's case, stdlib means:

• Problems with instantiation cross-Julia-versions can be fixed by removing stdlibs' compat bounds
• Don't bother with getting versions from registry, version resolution is too difficult to understand
• You don't need to restart the notebook if you remove the import – stdlib loading has no side effects (this is true for many non stdlib packages too but we dont know)

[pankgeorg]

In Pluto's case, stdlib means:

• Problems with instantiation cross-Julia-versions can be fixed by removing stdlibs' compat bounds
• Don't bother with getting versions from registry, version resolution is too difficult to understand
• You don't need to restart the notebook if you remove the import – stdlib loading has no side effects (this is true for many non stdlib packages too but we dont know)

For this definition, Stdlibs are the more or less the same as sysimage packages. I think. And I'd say that readdir(Sys.STDLIB) isn't sufficient for the latter (because you don't necessarily put the packages of the sysimage there). But it should be good for most cases. System image depot layout is a bit underdefined.

[fonsp]

It's the same as before though! I think we just want a very permissive list that includes anything that has ever been any kind of stdlib:

[fonsp]

(We should add tests for that statement)

[pankgeorg]

Agreed. I'm thinking of the sysimage case, for which what we do is not very well defined. PlutoPkg kind of works on recent versions, even without deactivation. But I think if the semantics were clear, we could identify these packages better in the sysimage case and it could work better (i.e. not try to add a package already in the sysimage, which takes 20 minutes but works)

[fonsp]

Yes! We should start with a draft about the expected behaviour for sysimage packages in Pluto. Mostly need to figure out:

• What do users see in the GUI
• What happens to the Project.toml deps and compat
• Which Pkg commands get called by Pluto when adding an import