all: support Go 1.12

[aykevl]

Fixes #216.

[aykevl]

This PR is now in a state that tests pass for all platforms (including WebAssembly) except for microcontrollers. If needed, I can make a PR with just these changes (except for the last) so that at least the wasm target is supported with Go 1.12.

[deadprogram]

Seems like awaiting the MCU functionality would be better to avoid having 2 versions in active release?

[aykevl]

Oh, no, we don't need multiple releases for that. It only means that TinyGo will still be able to produce wasm files when users have Go 1.12 installed.

[aykevl]

My plan for good Go 1.12 support is the following:

1. fix a bug that results in a compiler stack overflow with recursive types in interfaces:

   type node struct {
       next *node
   }
   var root interface{} = node{}

2. let MCU targets pretend they are nacl/arm instead of wasm/js

3. rebase this PR on top of that

It looks like pretending to be wasm/js on MCUs gets too difficult. GOOS=nacl is also much more stable as it has been around for a long time now, so upgrades will need less work. We can then pretend that GOOS=nacl is the OS-less target we've wanted for a while, as I doubt anyone is still seriously interested in Chrome Native Client support. (Note: Native Client is still supported in Go as it is used in play.golang.org).

[deadprogram]

That sounds like a very reasonable plan. As long as playground exists, then we should have a fixed target.

[bradfitz]

Be aware that we plan to remove Go's Native Client (nacl) port soonish here. Probably in Go 1.14. We thought Go was the only user for the playground, but we want to move the playground to gvisor and then we'd have no need for it. It's an annoying maintenance burden on us to maintain for ~zero users or future.

What do you want it for?

Upstream tracking bug is golang/go#30439

[aykevl]

@bradfitz thank you for the heads up! I was kind of afraid this would happen at some point, as GOOS=nacl is so little used.

What do you want it for?

The main target for TinyGo is bare metal systems (microcontrollers). So far, we've pretended to be GOOS=js GOARCH=wasm for these systems and added extra build tags so our runtime and machine packages know which chip it really is. For example, the BBC micro:bit uses build tags microbit, nrf51822, nrf51, nrf, tinygo.arm, js, wasm. Read the reasoning behind it here. Basically, the stdlib breaks when we don't use one of the supported GOOS/GOARCH combinations.

With Go 1.12, there were some changes to the syscall/js package which is a dependency of many stdlib packages, so has to be supported. Unfortunately, the changes in Go 1.12 to syscall/js are hard to support on MCUs, and to avoid breakage in the future I decided I wanted to get rid of this and use GOOS=nacl GOARCH=arm instead. It seemed like a good idea because it is also not a real OS so is easier to support.

Now I'm not so sure what to do. Ideally we'd like to have some non-OS or custom OS support in the stdlib. For example, we could have GOOS=posix that expects to be linked to a generic libc without making too much assumptions about the host OS except for POSIX compatibility. Or maybe make sure that the stdlib does not break with a custom GOOS (like GOOS=none) and just assumes the syscall package implements all required system calls.
I can see other uses of something like this. One is to use GOARCH=wasm outside of a browser. This is being done outside of the Go ecosystem in wasmer and olin (and others) and has been requested from Go as well in golang/go#27766. We've also had people using Go compiled with TinyGo to use outside of a browser, which kind of worked but wasn't working as well as it could because the stdlib doesn't support this at all.

[bradfitz]

Okay, so you don't actually need to use nacl at all? You just want to use its build tags to select different files?

[aykevl]

Okay, so you don't actually need to use nacl at all? You just want to use its build tags to select different files?

Basically, yes. I just want to get stdlib packages to compile and somehow do OS calls (like syscall.Write) with the least amount of hacks.

[aykevl]

Rebased on top of dev.
When Travis succeeds, I think this PR is ready.

Current status:

• Go 1.11 and go 1.12 are supported on Linux. They were already supported, but now we're actually testing them in CircleCI to make sure we don't accidentally break compatibility with Go 1.11. So, nothing changed here except that tests were added (and a dependency was updated).
• On macOS, both Go 1.11 and Go 1.12 should be supported. The only exception is that TinyGo is unable to build native binaries for macOS when using the Go 1.12 standard library because the syscall package changed in a big way: they now call the C standard library instead of doing direct system calls. While this is a good thing in general, it was implemented using all the magic that cgo uses behind the scenes and is thus not trivial to support.
  Automated tests for macOS still run with Go 1.11 until this compatibility issue has been fixed.

[aykevl]

@bradfitz we switched towards pretending to be linux/arm on bare metal systems and overriding the syscall package to avoid an incompatibility with 16-bit uintptrs on AVR. This seems to work so far, but it's not ideal. However, I think linux/arm is less likely to change in significant ways than js/wasm. See #247 for more details.
The effect is that we have GOOS=linux GOARCH=arm on bare-metal ARM and AVR chips, which is kind of weird but it seems to work. A true bare metal target would still be very useful, or a standard library that doesn't completely break down with non-standard GOOS/GOARCH build tags.

[free1139]

source code(Thu Mar 7 15:17:18 2019 +0100@46bc253)

github.com/tinygo-org/tinygo/src/examples/wasm

go version

go version go1.12.1 linux/amd64

build command

GOROOT=/usr/local/go ../../../tinygo build -o wasm.wasm wasm.go

build failed

wasm-ld-8: error: /tmp/tinygo799621561/main.o: Unexpected metadata version: 1 (Expected: 2)
error: failed to link /tmp/tinygo799621561/main: exit status 1

[aykevl]

@free1139 I don't think this is related to Go 1.12 support but to the LLVM version. I've seen this error before but I don't remember the exact cause. It was somehow related to a mismatch in LLVM version.

[justinclift]

A true bare metal target would still be very useful, or a standard library that doesn't completely break down with non-standard GOOS/GOARCH build tags.

Probably best to create an issue about it in the Go repo, so it can be discussed. Go 2 planning was happening a while back, so it might not be too late to get it into the minds of people working on that as well. 😄

[aykevl]

It would be very useful to decouple the stdlib from the compiler/runtime, and in fact @ianlancetaylor has written about it before. See this page for more details:
https://github.com/tinygo-org/tinygo/wiki/Go-pain-points#standard-library

[bradfitz]

Yes, please open a Go bug with your problems/requirements. This need not be a Go 2 issue at all.

[deadprogram]

What is the current status of this PR? Sorry, I did not determine if there was a clear resolution regarding the code in this specific PR. Thanks.

[aykevl]

From my POV it can be merged. That would be very useful, as it blocks some other changes.

[deadprogram]

Passes all CI tests on both Travis and CircleCI, so appears to do what we expect. Now merging.

[aykevl]

@bradfitz I have opened golang/go#31330.