main: add support for LLVM 11 #1056

aykevl · 2020-04-14T20:50:23Z

This can be useful to test improvements in LLVM master and to make it
possible to support LLVM 11 for the most part already before the next
release. That also allows catching LLVM bugs early to fix them upstream.

Note that tests do not yet pass for this LLVM version, but the TinyGo
compiler can be built with the binaries from apt.llvm.org (at the time
of making this commit).

One concrete example of what I want to do with this change is to test/improve AVR support: upstream master has a number of improvements that aren't included yet in LLVM 10.

Note that #1055 should be merged first.

yevsia · 2020-04-14T22:30:45Z

Try to fix this problem #918 please, if it's possible in new LLVM...

aykevl · 2020-04-15T12:51:00Z

@yevsia the LLVM version and #918 are unrelated. You're very welcome to add a patch to implement pullups on AVR.

deadprogram · 2020-06-09T18:50:47Z

Looks like a rebase is needed...

QuLogic · 2020-08-23T04:12:28Z

In Fedora, we are already starting to use LLVM 11 for Fedora 33. I started testing out TinyGo 0.14.1 which obviously doesn't yet support it. I pulled the llvm11 branch from go-llvm, which is fortunately sufficient to build and run its tests. I have not done any deep investigation in these, but for TinyGo itself, there remain a few problems:

The old commit of wasi-libc is a bit broken with clang 11:
- Differences in predefined_macros.txt; this is fixed upstream by Fix -std=gnu17 when generating predefined_macros.txt WebAssembly/wasi-libc#187.
- Differences in undefined-symbols.txt; I see some upstream comments that that file won't be needed (Add pthread.h in wasi sysroot to support pthread WebAssembly/wasi-libc#209 (comment)), but haven't found any changes to that effect yet.
- Various warnings from incorrect functions calls; these are probably fixed by upstream's update to musl 1.1.24 and/or musl 1.2.0, but since these are non-fatal, I did not confirm or test yet.

Some smoke tests fail:

for circuitplay-express, pca10040, microbit, microbit-s110v8, itsybitsy-m0, itsybitsy-m4, feather-m0, trinket-m0, clue_alpha, pybadge, metro-m4-airlift, pyportal, pca10056-s140v7, reelboard-s140v7, wioterminal, pygamer, xiao, feather-nrf52840 with errors similar to:
```
tinygo build -size short -o test.hex -target=circuitplay-express examples/i2s
   code    data     bss |   flash     ram
   8888       8    4264 |    8896    4272
error: ROM segments are non-contiguous: /tmp/tinygo064621349/main
```

for gameboy-advance with link errors:

tinygo build -size short -o test.gba -target=gameboy-advance     examples/gba-display || true
ld.lld: warning: lld uses blx instruction, no object with architecture supporting feature detected
ld.lld: error: no memory region specified for section '.text.examples/gba-display.main'
ld.lld: error: no memory region specified for section '.text.handleInterrupt'
ld.lld: error: no memory region specified for section '.text.main'
ld.lld: error: no memory region specified for section '.text.memset'
error: failed to link /tmp/tinygo264710633/main: exit status 1

for nintendo-switch with a GC error:

tinygo build             -o test.elf -target=nintendoswitch      examples/serial || true
musttail call must precede a ret with an optional bitcast
  musttail call fastcc void @"(*internal/task.Queue).Push"(%"internal/task.Task"* nonnull %task.current1), !dbg !1213
error: GC pass caused a verification failure

wasm-ld doesn't understand --no-threads; it looks like this was a simple rename: https://reviews.llvm.org/D76885 Naturally, they said "--no-threads is used rarely."... This is a straightforward switch to --threads=1, though wasi-sdk removed it entirely, so it may not be necessary.

niaow · 2020-08-23T04:16:54Z

ld.lld: warning: lld uses blx instruction, no object with architecture supporting feature detected

Huh, I thought ditching thumb solved this, but maybe not?

musttail call must precede a ret with an optional bitcast

I don't see how tinygo could create this by itself? Unfortunately, this looks like it could just be an LLVM bug?

QuLogic · 2020-08-23T04:25:56Z

Surprisingly, updating to latest commit of wasi-libc (WebAssembly/wasi-libc@215adc8) 'just worked', and was able to compile. I'm not sure exactly which commit fixed it and if you want update directly to it (there are many large updates like the above-mentioned musl updates). This plus the wasm-ld ldflags change are enough to fix wasm builds. Unfortunately, go test fails with:

error: <unknown>:0:0: in function runtime.chanSend.resume void (%runtime.chanSend.Frame*): WebAssembly 'tail-call' feature not enabled

QuLogic · 2020-08-23T06:34:47Z

ld.lld: warning: lld uses blx instruction, no object with architecture supporting feature detected
Huh, I thought ditching thumb solved this, but maybe not?

I reverted Clang/LLD/LLVM to 10 and this warning still happens, so I guess not? It's not specific to 11 at least.

QuLogic · 2020-08-24T01:58:57Z

Differences in undefined-symbols.txt; I see some upstream comments that that file won't be needed (WebAssembly/wasi-libc#209 (comment)), but haven't found any changes to that effect yet.

I missed it earlier, but this one is fixed by WebAssembly/wasi-libc#152.

So the minimum commit to avoid the two diff errors is WebAssembly/wasi-libc@38b930a, which is about halfway between what TinyGo uses and the latest commit.

QuLogic · 2020-08-24T02:33:37Z

The musttail assert seems to have come from https://reviews.llvm.org/D79258, but I dumped ssa (-dumpssa) and IR (-printir) and see no mention of it in tinygo's stuff. You may be right that it's an LLVM bug.

aykevl · 2020-08-24T09:43:02Z

Looking into this now.

aykevl · 2020-08-24T11:29:32Z

Right now I'm hitting this:

error: <unknown>:0:0: in function runtime.chanSend.resume void (%runtime.chanSend.Frame*): WebAssembly 'tail-call' feature not enabled

... which I don't quite understand.

aykevl · 2020-08-24T11:39:57Z

I have made a separate PR to update the wasi-libc dependency: #1309

QuLogic · 2020-09-14T08:00:07Z

The tail-call thing seems to have to do with the channel.go test and compiling IR to an object file. That is, if you create bitcode or IR, then it is fine with it:

$ go run . build -target wasm -wasm-abi=js -print-stacks -o channel.o testdata/channel.go                                
error: scheduler.go:76:15: in function runtime.chanSend.resume void (%runtime.chanSend.Frame*): WebAssembly 'tail-call' feature not enabled

exit status 1

$ go run . build -target wasm -wasm-abi=js -print-stacks -o channel.bc testdata/channel.go

$ go run . build -target wasm -wasm-abi=js -print-stacks -o channel.ll testdata/channel.go

There seem to be many wasm features that are not listed in targets/wasm.json, but even still I was not able to figure out how to get wasm-ld to enable them. I could pass complete garbage to the --features option and it wouldn't complain. So this didn't work:

$ wasm-ld --features=+tail-call --allow-undefined --stack-first --export-all --no-demangle $TINYGOROOT/lib/wasi-libc/sysroot/lib/wasm32-wasi/libc.a channel.bc -o channel.wasm
wasm-ld: error: scheduler.go:76:15: in function runtime.chanSend.resume void (%runtime.chanSend.Frame*): WebAssembly 'tail-call' feature not enabled
wasm-ld: error: scheduler.go:76:15: in function runtime.chanSelect.resume void (%runtime.chanSelect.Frame*): WebAssembly 'tail-call' feature not enabled
wasm-ld: error: scheduler.go:76:15: in function runtime.chanRecv.resume void (%runtime.chanRecv.Frame*): WebAssembly 'tail-call' feature not enabled

I was able to get a further using llc, as it has a --disable-tail-calls option:

$ llc --filetype=obj --march=wasm32 --disable-tail-calls -o channel.o channel.ll  # also accept channel.bc

$ wasm-ld --allow-undefined --stack-first --export-all --no-demangle $TINYGOROOT/lib/wasi-libc/sysroot/lib/wasm32-wasi/libc.a -o channel.wasm channel.o

$ node targets/wasm_exec.js channel.wasm 
len, cap of channel: 1 2 false
len, cap of channel: 0 0 false
recv from open channel: 1 true
received num: 2
received num: 3
slept
received num: 4
received num: 5
received num: 6
received num: 7
received num: 8
recv from closed channel: 0 false
complex128: (+7.000000e+000+1.050000e+001i)
sum of n: 149
sum: 25
sum: 29
sum: 33
sum(100): 4950
deadlocking
select no-op
after no-op
sum: 5
did send one
select one n: 0
select n from chan: 55
select n from closed chan: 0
select send
sum: 235
non-concurrent channel recieve: 1
non-concurrent channel recieve: 2
closed buffered channel recieve: 3
closed buffered channel recieve: 4
closed buffered channel recieve: 0
hybrid buffered channel recieve: 2
blocking select sum: 3

So if you can figure out how to transform that --disable-tail-calls from llc into something that affects LLVMTargetMachineEmitToMemoryBuffer, then I think everything should work again.

QuLogic · 2020-09-14T08:15:47Z

Another path with llc that I have not totally figured out yet, its -mcpu option has a bleeding-edge CPU that seems able to enable all features:

$ llc --filetype=obj --march=wasm32 -mcpu=bleeding-edge -o channel.o channel.ll

but it maybe enables too much:

$ wasm-ld --allow-undefined --stack-first --export-all --no-demangle $TINYGOROOT/lib/wasi-libc/sysroot/lib/wasm32-wasi/libc.a -o channel.wasm channel.o
wasm-ld: error: 'atomics' feature is used by channel.o, so --shared-memory must be used

and that then requires a larger memory size:

$ wasm-ld --shared-memory --allow-undefined --stack-first --export-all --no-demangle $TINYGOROOT/lib/wasi-libc/sysroot/lib/wasm32-wasi/libc.a -o channel.wasm channel.o
wasm-ld: error: maximum memory too small, 67344 bytes needed

which compiles, but doesn't run:

$ wasm-ld --shared-memory --max-memory=131072 --allow-undefined --stack-first --export-all --no-demangle $TINYGOROOT/lib/wasi-libc/sysroot/lib/wasm32-wasi/libc.a -o channel.wasm channel.o

$ node targets/wasm_exec.js channel.wasm 
[CompileError: WebAssembly.instantiate(): invalid memory limits flags @+718]

I don't know what to fix there, but it does suggest that if you could figure out how to enable features like tail-call, but not the other stuff like atomics, you might be able compile and run.

aykevl · 2020-09-14T18:29:05Z

@QuLogic thanks a lot for all the research you did! Yes, it appears to be a target feature and "features": ["+tail-call"] does indeed work, but I don't think that's a solution: Chrome hasn't even added support for tail calls. Instead, we need to convince LLVM to not try to emit tail calls.

aykevl · 2020-09-14T18:30:08Z

Also, it's not a linker issue but a compiler issue. This is how you can reproduce it without invoking the linker:

tinygo build -o test.o -target=wasm ./testdata/channel.go

This can be useful to test improvements in LLVM master and to make it possible to support LLVM 11 for the most part already before the next release. That also allows catching LLVM bugs early to fix them upstream. Note that tests do not yet pass for this LLVM version, but the TinyGo compiler can be built with the binaries from apt.llvm.org (at the time of making this commit).

aykevl · 2020-09-14T18:53:00Z

I think I have a solution, let's see what CI thinks of it.

aykevl · 2020-09-14T19:15:59Z

Tests passed, so this should be ready for merging. The only thing I noticed was that the commit message is wrong (it says tests don't pass), if approved I can fix that while merging (or the one who merges it can fix it during "squash and merge").

In essence, to fix that last error I added a "disable-tail-calls"="true" attribute to every function when compiling to WebAssembly, which does exactly what it says.

aykevl · 2020-09-14T19:46:27Z

I compared code size and while it obviously changed, it went both ways. Some programs increased in size, others decreased.

QuLogic · 2020-09-14T19:51:04Z

Do you mean with LLVM11, or with/without tail calls?

aykevl · 2020-09-14T23:32:52Z

With LLVM 11. With/without tail calls isn't really a useful statistic as it isn't enabled by default in any major browser and it wasn't enabled in LLVM 10 anyway (otherwise wasm output would not have worked in a browser).

QuLogic · 2020-09-16T10:09:24Z

for nintendo-switch with a GC error:

tinygo build             -o test.elf -target=nintendoswitch      examples/serial || true
musttail call must precede a ret with an optional bitcast
  musttail call fastcc void @"(*internal/task.Queue).Push"(%"internal/task.Task"* nonnull %task.current1), !dbg !1213
error: GC pass caused a verification failure

This one is fixed by #1368.

Then this PR fixes everything else with LLVM 11.

QuLogic · 2020-10-04T23:16:42Z

I suppose you're waiting for the final LLVM 11 release, but to let you know, I've already been building this against LLVM 11 on Rawhide and it's worked fine for all tests.

deadprogram · 2020-10-12T17:21:11Z

LLVM 11 has just been released: https://lists.llvm.org/pipermail/llvm-announce/2020-October/000089.html

aykevl · 2020-10-13T17:05:49Z

I'm not waiting for anything - I'd like to see this PR merged. Is there anything blocking it?

deadprogram · 2020-10-13T18:23:42Z

No blockers from my side, I misunderstood the status.

Now merging, thanks @aykevl and @QuLogic

deadprogram · 2020-10-13T18:33:52Z

Getting CI failures such as

wasm-ld-11: error: unknown argument: --no-threads

which appears in https://app.circleci.com/pipelines/github/tinygo-org/tinygo/3773/workflows/2ff8b6ab-e33b-4946-81e3-b68f66dae25d/jobs/19529

QuLogic · 2020-10-13T19:30:49Z

That should have been removed by #1308?

Also, don't forget to merge the llvm11 branch in https://github.com/tinygo-org/go-llvm

deadprogram · 2020-10-13T20:27:04Z

Just added #1437 which should correct that build error, I think? Just waiting for CI to finish.

deadprogram · 2020-10-13T20:44:16Z

Yes, confirm that #1437 corrects that build error.

aykevl · 2020-10-14T11:59:37Z

That should have been removed by #1308?

That was for targets/wasm.json. Presumably targets/wasi.json was a copy of it, so also copied --no-threads.
Merged the PR so the dev branch should be green again.

aykevl mentioned this pull request Apr 14, 2020

main: support gdb debugging with AVR #1057

Merged

aykevl mentioned this pull request Apr 26, 2020

Unable to build ultrasonic sensor distance reading code tinygo-org/drivers#148

Closed

aykevl force-pushed the llvm11 branch from 2b908c3 to fb00d61 Compare May 3, 2020 22:55

aykevl force-pushed the llvm11 branch 2 times, most recently from f463f0b to 424ee98 Compare June 9, 2020 12:20

aykevl force-pushed the llvm11 branch 2 times, most recently from 63bb31a to ab71f83 Compare June 10, 2020 18:53

aykevl force-pushed the llvm11 branch from ab71f83 to f7f2082 Compare June 26, 2020 23:20

niaow added this to the v0.15 milestone Jun 27, 2020

aykevl mentioned this pull request Jul 20, 2020

runtime error when formatting a large int64 #1254

Closed

aykevl force-pushed the llvm11 branch from f7f2082 to 38e9f40 Compare August 24, 2020 09:36

aykevl force-pushed the dev branch from 581be0f to 32c7f3b Compare August 24, 2020 10:05

aykevl force-pushed the llvm11 branch 2 times, most recently from 0d82645 to 43ac6c2 Compare August 24, 2020 11:23

aykevl changed the title ~~main: add initial support for (in-development) LLVM 11~~ main: add support for LLVM 11 Aug 24, 2020

aykevl force-pushed the llvm11 branch from 43ac6c2 to cc3ec22 Compare September 14, 2020 18:52

aykevl requested a review from deadprogram September 25, 2020 15:05

aykevl mentioned this pull request Oct 13, 2020

Port to LLVM 11 #1435

Closed

deadprogram merged commit b40f250 into dev Oct 13, 2020

deadprogram deleted the llvm11 branch October 13, 2020 21:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

main: add support for LLVM 11 #1056

main: add support for LLVM 11 #1056

aykevl commented Apr 14, 2020

yevsia commented Apr 14, 2020 •

edited

Loading

aykevl commented Apr 15, 2020

deadprogram commented Jun 9, 2020

QuLogic commented Aug 23, 2020 •

edited

Loading

niaow commented Aug 23, 2020

QuLogic commented Aug 23, 2020

QuLogic commented Aug 23, 2020

QuLogic commented Aug 24, 2020

QuLogic commented Aug 24, 2020

aykevl commented Aug 24, 2020

aykevl commented Aug 24, 2020

aykevl commented Aug 24, 2020

QuLogic commented Sep 14, 2020

QuLogic commented Sep 14, 2020

aykevl commented Sep 14, 2020

aykevl commented Sep 14, 2020

aykevl commented Sep 14, 2020

aykevl commented Sep 14, 2020 •

edited

Loading

aykevl commented Sep 14, 2020

QuLogic commented Sep 14, 2020

aykevl commented Sep 14, 2020

QuLogic commented Sep 16, 2020 •

edited

Loading

QuLogic commented Oct 4, 2020

deadprogram commented Oct 12, 2020

aykevl commented Oct 13, 2020

deadprogram commented Oct 13, 2020

deadprogram commented Oct 13, 2020

QuLogic commented Oct 13, 2020

deadprogram commented Oct 13, 2020

deadprogram commented Oct 13, 2020

aykevl commented Oct 14, 2020

main: add support for LLVM 11 #1056

main: add support for LLVM 11 #1056

Conversation

aykevl commented Apr 14, 2020

yevsia commented Apr 14, 2020 • edited Loading

aykevl commented Apr 15, 2020

deadprogram commented Jun 9, 2020

QuLogic commented Aug 23, 2020 • edited Loading

niaow commented Aug 23, 2020

QuLogic commented Aug 23, 2020

QuLogic commented Aug 23, 2020

QuLogic commented Aug 24, 2020

QuLogic commented Aug 24, 2020

aykevl commented Aug 24, 2020

aykevl commented Aug 24, 2020

aykevl commented Aug 24, 2020

QuLogic commented Sep 14, 2020

QuLogic commented Sep 14, 2020

aykevl commented Sep 14, 2020

aykevl commented Sep 14, 2020

aykevl commented Sep 14, 2020

aykevl commented Sep 14, 2020 • edited Loading

aykevl commented Sep 14, 2020

QuLogic commented Sep 14, 2020

aykevl commented Sep 14, 2020

QuLogic commented Sep 16, 2020 • edited Loading

QuLogic commented Oct 4, 2020

deadprogram commented Oct 12, 2020

aykevl commented Oct 13, 2020

deadprogram commented Oct 13, 2020

deadprogram commented Oct 13, 2020

QuLogic commented Oct 13, 2020

deadprogram commented Oct 13, 2020

deadprogram commented Oct 13, 2020

aykevl commented Oct 14, 2020

yevsia commented Apr 14, 2020 •

edited

Loading

QuLogic commented Aug 23, 2020 •

edited

Loading

aykevl commented Sep 14, 2020 •

edited

Loading

QuLogic commented Sep 16, 2020 •

edited

Loading