__wasilibc_unmodified_upstream for standalone builds?

[gitonthescene]

Hi, I'm new to WASI and am happy to be redirected to documentation or a chat server or whatever. I've taken over a project that builds both as a C executable and a wasm. It's currently built as a standalone implementing functions like write() and mmap() but it was using system headers for the WASM build. My idea was that WASI would be a better platform agnostic place to find these headers.

Because it's a standalone I've put -D__wasilibc_unmodified_upstream on the command line, but that leads me to require bits/errno.h which is not available for my standalone. I created one as a workaround which includes the following:

#include <__errno.h>
#include <__errno_values.h>

This works fairly well but then I run an issue with fcntl not including __header_fcntl.h, which I worked around similarly.

I also had issues with multiple definitions for struct timespec here and here? (I think).

My list of flags ended up looking like this:

O_WASM=@opts -Oz -nostdlib -ffreestanding --target=wasm32-wasi -U __SIZEOF_INT128__ -Dwasm -D__wasilibc_unmodified_upstream -D_WASI_EMULATED_MMAN -D__DEFINED_struct_timespec -Wno-implicit-function-declaration -Wno-visibility --sysroot=$(WASI_SDK_PATH)/share/wasi-sysroot -I.

I could get it to build, but this all feels fairly hacky. Am I wrong? Is there a better place to ask for guidance?

Thanks for your time,
-Doug

[pchickey]

Its better to think of __wasilibc_unmodified_upstream as a marker in the source code. It should never be defined: its expected that defining it will break everything. It exists as a marker so that wasi-libc's maintainers can better read and understand how diffs from upstream musl when if they are being pulled down into this repo. (It is of limited use even for that, since some changes got so complex that the unmodified_upstream has been deleted.)

wasi-libc hasnt tried to make its headers cross-platform compatible. I'm not aware of any libc for wasi which has.

[gitonthescene]

For clarity are you saying that the plan to use wasi for headers only is untenable?

[pchickey]

Correct. You shouldn't compile object files against a sysroot for one libc, and then link them with another libc, because the size and layout of types may be different. For example, your wasm32 libc will define uintptr_t as 32 bits wide, and a native libc for a 64 bit operating system will define it as 64 bits wide. So, header files don't really provide a reusable abstraction, except in certain libc implementations (like glibc) where ABI compatibility between minor versions of the same libc is explicitly supported.

[gitonthescene]

Does this also mean that you can never provide your own implementation of specific functions in libc? I.e. it’s all or nothing?

[pchickey]

In the general case, no, there's not a safe way to provide your own implementation of any functions from a libc and be sure that won't cause a conflict with any implementation of any libc. In specific cases, you can read the source code of a specific libc for the specific functionality you want to provide your own implementation of, and convince yourself that it is safe. This is how the so-called "LD_PRELOAD tricks" works - you can override certain functionality of libc in order to build e.g. valgrind, but the implementation will be specialized to a certain libc on a certain target, and may not be possible to port to other targets or libcs (and indeed you can't port valgrind to wasm, because it depends on virtual memory implementation details that wasm does not support).