Does not support JULIA_CPU_TARGET correctly

[simonbyrne]

To reproduce:

• use Julia 1.9.0
• set JULIA_CPU_TARGET to an earlier (but compatible) generation than the current processor (e.g. I'm on a skylake processor, but set JULIA_CPU_TARGET=broadwell)
• try to install VectorizationBase:

├ JULIA_CPU_TARGET=broadwell julia --project=. -e 'using Pkg; Pkg.add("VectorizationBase"); using VectorizationBase'
   Resolving package versions...
    Updating `/central/home/spjbyrne/misc/hostcpu/Project.toml`
  [3d5dd08c] + VectorizationBase v0.21.64
    Updating `/central/home/spjbyrne/misc/hostcpu/Manifest.toml`
  [79e6a3ab] + Adapt v3.6.2
  [4fba245c] + ArrayInterface v7.4.8
  [62783981] + BitTwiddlingConvenienceFunctions v0.1.5
  [2a0fbf3d] + CPUSummary v0.2.3
  [34da2185] + Compat v4.6.1
  [adafc99b] + CpuId v0.3.1
  [3e5b6fbb] + HostCPUFeatures v0.1.14
  [615f187c] + IfElse v0.1.1
  [10f19ff3] + LayoutPointers v0.1.14
  [d125e4d3] + ManualMemory v0.1.8
  [aea7be01] + PrecompileTools v1.1.2
  [21216c6a] + Preferences v1.4.0
  [ae029012] + Requires v1.3.0
  [94e857df] + SIMDTypes v0.1.0
  [66db9d55] + SnoopPrecompile v1.0.3
  [aedffcd0] + Static v0.8.7
  [0d7ed370] + StaticArrayInterface v1.4.0
  [3d5dd08c] + VectorizationBase v0.21.64
  [0dad84c5] + ArgTools v1.1.1
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [f43a241f] + Downloads v1.6.0
  [7b1f6079] + FileWatching
  [b77e0a4c] + InteractiveUtils
  [b27032c2] + LibCURL v0.6.3
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [ca575930] + NetworkOptions v1.2.0
  [44cfe95a] + Pkg v1.9.0
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization
  [6462fe0b] + Sockets
  [2f01184e] + SparseArrays
  [4607b0f0] + SuiteSparse
  [fa267f1f] + TOML v1.0.3
  [a4e569a6] + Tar v1.10.0
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll v1.0.2+0
  [deac9b47] + LibCURL_jll v7.84.0+0
  [29816b5a] + LibSSH2_jll v1.10.2+0
  [c8ffd9c3] + MbedTLS_jll v2.28.2+0
  [14a3606d] + MozillaCACerts_jll v2022.10.11
  [4536629a] + OpenBLAS_jll v0.3.21+4
  [bea87d4a] + SuiteSparse_jll v5.10.1+6
  [83775a58] + Zlib_jll v1.2.13+0
  [8e850b90] + libblastrampoline_jll v5.7.0+0
  [8e850ede] + nghttp2_jll v1.48.0+0
  [3f19e933] + p7zip_jll v17.4.0+0
ERROR: LoadError: InitError: Evaluation into the closed module `HostCPUFeatures` breaks incremental compilation because the side effects will not be permanent. This is likely due to some other module mutating `HostCPUFeatures` with `eval` during precompilation - don't do this.
Stacktrace:
  [1] eval
    @ ./boot.jl:370 [inlined]
  [2] setfeaturefalse(s::Symbol)
    @ HostCPUFeatures ~/.julia/packages/HostCPUFeatures/fielO/src/cpu_info_x86.jl:36
  [3] make_generic(target::String)
    @ HostCPUFeatures ~/.julia/packages/HostCPUFeatures/fielO/src/cpu_info_x86.jl:74
  [4] __init__()
    @ HostCPUFeatures ~/.julia/packages/HostCPUFeatures/fielO/src/HostCPUFeatures.jl:49
  [5] register_restored_modules(sv::Core.SimpleVector, pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:1074
  [6] _include_from_serialized(pkg::Base.PkgId, path::String, ocachepath::String, depmods::Vector{Any})
    @ Base ./loading.jl:1020
  [7] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128)
    @ Base ./loading.jl:1471
  [8] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1748
  [9] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1625
 [10] macro expansion
    @ ./loading.jl:1613 [inlined]
 [11] macro expansion
    @ ./lock.jl:267 [inlined]
 [12] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1576
 [13] include
    @ ./Base.jl:457 [inlined]
 [14] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base ./loading.jl:2010
 [15] top-level scope
    @ stdin:2
during initialization of module HostCPUFeatures
in expression starting at /home/spjbyrne/.julia/packages/VectorizationBase/0dXyA/src/VectorizationBase.jl:1
in expression starting at stdin:2
ERROR: Failed to precompile VectorizationBase [3d5dd08c-fd9d-11e8-17fa-ed2836048c2f] to "/home/spjbyrne/.julia/compiled/v1.9/VectorizationBase/jl_AtyiqF".
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
   @ Base ./loading.jl:2260
 [3] compilecache
   @ ./loading.jl:2127 [inlined]
 [4] _require(pkg::Base.PkgId, env::String)
   @ Base ./loading.jl:1770
 [5] _require_prelocked(uuidkey::Base.PkgId, env::String)
   @ Base ./loading.jl:1625
 [6] macro expansion
   @ ./loading.jl:1613 [inlined]
 [7] macro expansion
   @ ./lock.jl:267 [inlined]
 [8] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1576

https://docs.julialang.org/en/v1/devdocs/sysimg/#Specifying-multiple-system-image-targets

Unfortunately since this package is ultimately depended on by a huge number of packages, this completely breaks pkgimages for me

[simonbyrne]

@vchuravy any ideas on how one could make this package pkgimage compatible?

[chriselrod]

Long term, the plan is to deprecate this stack.
So anything that isn't a quick fix that also solves a ton of other problems probably isn't worth it.

[chriselrod]

The problem is that if it is generating, for example, AVX512 code, you probably can't run it on a target without it.

So things like a Beowulf cluster with a shared filesystem are tricky.

[simonbyrne]

Yeah, it's a heterogeneous cluster, which I'm trying to build on the lowest common arch.

[vchuravy]

how one could make this package pkgimage compatible?

One would need to deeply integrate with multi-versioning...

I will note that this is not package-image specific, but will also happen with package-compiler

[chriselrod]

I suggest deving the package, deleting the __init__ function as well as

HostCPUFeatures.jl/src/cpu_info.jl

Line 42 in cc4fe3d

set_features!()

HostCPUFeatures.jl/src/cpu_info.jl

Line 65 in cc4fe3d

define_cpu_name()

(maybe just delete the entire file/remove the include).
And then simple call make_generic("broadwell"), or whatever the lowest common arch is.

HostCPUFeatures.jl/src/cpu_info_x86.jl

Line 68 in cc4fe3d

elseif occursin("znver", target) || occursin("lake", target) || occursin("well", target)

[mkitti]

What if we do not run this code during precompilation?

if ccall(:jl_generating_output, Cint, ()) == 0
    # run code that should not run during precompile here
end

Or we just move that call up to the top of the function? Perhaps I'm misunderstanding the issue.

function __init__()
  ccall(:jl_generating_output, Cint, ()) == 1 && return
  if Sys.ARCH === :x86_64 || Sys.ARCH === :i686
    target = Base.unsafe_string(Base.JLOptions().cpu_target)
    occursin("native",  target) || return make_generic(target)
  end
  BASELINE_CPU_NAME == Sys.CPU_NAME::String || redefine()
  return nothing
end

[chriselrod]

I'm not sure why this is after the make_generic codeblock:

HostCPUFeatures.jl/src/HostCPUFeatures.jl

Lines 47 to 51 in cc4fe3d

	if Sys.ARCH === :x86_64 || Sys.ARCH === :i686
	target = Base.unsafe_string(Base.JLOptions().cpu_target)
	occursin("native", target) || return make_generic(target)
	end
	ccall(:jl_generating_output, Cint, ()) == 1 && return

[chriselrod]

Want to make a PR and bump the patch version?

[mkitti]

ok, give me a minute

[mkitti]

See #13