Blockchain call extension

[nhynes]

Most blockchain functions map well to WASI except for call/transact, which is an RPC augmented with the ability to transfer the platform's native token (e.g., Eth) from the caller to the callee. It's possible to do this out-of band, but not efficiently: it'd require making two transactions, passing around receipts, and handling rollbacks/refunds.

As long as we're going the WASI module route, it might be worth adding a single function to the "blockchain" module to support Ethereum-like blockchains:

__wasi_blockchain_transact(
	callee_addr: *const u8,
	value: u64,
	input: *const u8,
	input_len: u64,
	fd: *mut __wasi_fd_t
) -> __wasi_errno_t;

• callee_addr is a fixed-length byte sequence with length known by the platform. For instance, Ethereum addresses are 20 bytes long.
• value is the amount of native token to transfer from the calling account to the callee
• input/input_len - a C slice containing the input to the transaction. input_len = 0 for simple balance transfer.
• fd - a capability to read the output of the transaction

The errno result would be

• success: ESUCCESS
• no account at callee address: EADDRNOTAVAIL
• insufficient funds for value transfer: ENOBAL (this one would have to be assigned a number
• anything else: application specific
• (running out of computation resources, or gas, is unrecoverable and results in the process aborting)

Upon ESUCCESS, fd would be of type SOCKET_STREAM and have read-only pipe
semantics.

This proposal would support most existing smart contract platforms including those with a synchronous RPC model (e.g. Ethereum, EOS) and those with asynchronous cross-shard RPCs (e.g, Near).

Note: left out of this call function, but perhaps desirable, is the amount of gas allocated to the sub-transaction. The alternative is to simply give the callee the full amount of remaining gas. One would presumably not call untrusted code, so there should be no need to limit gas for security reasons.

[devsnek]

why does this need to exist in wasi? i don't think a standardized api for cryptocurrencies is related to wasi's goals at all.

[nhynes]

i don't think a standardized api for cryptocurrencies is related to wasi's goals at all.

Where did I say cryptocurrency? Blockchain is a decentralized computation platform that benefits greatly from using WASI. As a modular systems interface, one would think that it's in the purview of WASI to consider modules for novel systems.

Please do not conflate (your personal opinions on) cryptocurrency and blockchain.

[devsnek]

the thing that is representing a transaction of tokens seems out of place for wasi. crytographic functions to create chained signatures seems like something that could be in a wasi crypto module.

[nhynes]

crytographic [sic] functions to create chained signatures

I don't see how that applies. A call in a smart contract platform is just an RPC with value. Unlike the input which is passed to the callee, the value parameter is handled by the runtime before the callee is executed. The additional information is why a plain socket won't work. This could be done at the application layer, surely, but it helps cross-chain interoperability if there's a runtime standard.

[devsnek]

ok so how I'm currently thinking about this is like, a js canvas api to draw a line vs a function that draws a histogram. Certainly drawing histograms is useful, and standardization would make it easier for everyone to draw histograms, but I wouldn't put that histogram function in the js canvas api. it appears to me that this transaction api with its ENOBAL is more of an analogue to the histogram function than to the line function.

[nhynes]

Okay, so imagine a world in which the canvas API were modular and the histogram function were in a plotting module. The plotting module would make it significantly easier for researchers to share results; everyone else could be blissfully unaware of its existence. That's what this proposal is.

If you don't like the name ENOBAL, it could just as easily be called ERUNTIME and multiplexed for use in, say, the CUDA module whenever that happens. If you don't like the idea of adding more errors at all, then we could work to standardize what existing error should be used (but then why make errno_t 16 bits?). However, the "call with value" functionality is as essential to blockchain as the equivalent method is to JS fetch.

At this point, we should wait for someone else in the subgroup who works with blockchain to chime in. This discussion is becoming one of whether you think blockchain WASI is useful, which is a separate matter entirely.

[devsnek]

to be clear, I don't doubt that wasi is valuable to the blockchain, or that this API is valuable to blockchain, I just don't yet understand why this api needs to be in the wasi namespace.

[nhynes]

why this api needs to be in the wasi namespace

Well, it wasn't meant to be in the namespace any more than the graphics or (eventually) bluetooth modules would be. The reason for proposing an optional module is that the function depends on WASI file descriptor semantics. A capability-based model is (un)arguably the correct way to architect a secure blockchain platform. Moreover the WASI async proposal, when it arrives, will be a perfect fit for RPCs in high-performance sharded/asynchronous blockchains.

Overall, we're seeing a mass movement of blockchain runtimes to Wasm. WASI is an excellent replacement for the ad hoc exports that are currently in use. A standard for blockchain applications would enable platform developers to collaborate on robust, shared runtime components and make it easy for users to leverage the unique features of each platform. I suppose, then, that the real question is how to streamline this process so that it does not disrupt the core standardization efforts.

[wanderer]

I think we can come up with a more general interface that can satisfy the of a blockchain embedding. The prerequisites probably should be broken up into separate issues, but here is the route I was thinking.

We should use wasm's funcRefs directly. And extend WASI's file APIs to be able to store funcRefs like files. instead of having callee_addr: *const u8, an Ethereum like blockchain embedding could use __wasi_path_open(root_fd, callee_addr/<entryPoint_funcRef>). After getting a fd to the entryPoint_funcRef there would have to be a corresponding methods to load the funcRef into the anyfunc table which would then allow caller to just use call_indirect. Gas and Value transfers could be added as independent interfaces. The Gas interface for example could look like

get_gas_budget

gets the amount of gas that will be spent when the function is called.

Parameters

• funcRef i32ref

Returns

• i32 the amount of gas to be spent

set_gas_budget

sets the amount of gas to be spent when the function is called.

Parameters

• funcRef i32ref
• gas i32

A capability-based model is (un)arguably the correct way to architect a secure blockchain platform.

I agree with this but Ethereum's model that allow any contract to talk to anyother contract complete breaks the capability model and we should avoid at all cost baking it into any interface.

[dumblob]

And extend WASI's file APIs to be able to store funcRefs like files.

File (or other tree-like, DB-like, ... you name it) access APIs might in the end not be part of core WASI (i.e. in the WASI namespace), but rather an optional module - pretty much as @devsnek already said above.

[wanderer]

@dumblob thats a good point, how to persistently store fds, funcRefs or any other capability should be independent of other associated interfaces

[dumblob]

how to persistently store

Not just persistently, but "store to a storage in any way" 😉.

[nhynes]

We should use wasm's funcRefs directly. And extend WASI's file APIs to be able to store funcRefs like files

Neat! I like this idea. Let me try to reiterate it to see if I have it right. Please correct me if I'm wrong.

import!(other_service.abi); // generates client impl

fn main() {
	let client =  OtherService::at(<address>); // -> path_open
	let params = Context::new().with_gas(42); // nothing yet
    let output = client.method(&params, arg1, arg2);
    //! -> obtain funcref, unpack params into funcref via external API, call funcref to get data
    // -> fd_close
}

Is this along the lines what you were thinking? One question: how does one actually get the funcref? The only output of open is an fd, so there'll need to be a way to get one from the other.

Another (unrelated question): we would ideally have something like WebIDL for a blockchain embedding. I've made an tool that automatically generates an IDL from rust source code. Right now it's a simple custom JSON format that can be used to generate a client. If you had any other ideas, I'm all ears.

[wanderer]

Is this along the lines what you were thinking?
it looks close

One question: how does one actually get the funcref? The only output of open is an fd, so there'll need to be a way to get one from the other.

yep! so currently the are 8 file types so we could just add another one func_ref and have methods for loading a fd of the type func_ref into the anyfunc table. so for example...

funcref_set : (slot : int, func : fd) -> ()
Where slot is the index in the anyfunc table to put the function references

funcref_get : (slot : int) -> (fd)
funcref_get would do the opposite and load a function reference from the anyfunc table return it as a file descriptor which could be saved with path_link

But this would change once we have reference-types, then we can have proper funcref and we would just an a casting functions from fd -> funcRef.

[nhynes]

Cool, makes sense.

methods for loading a fd of the type func_ref into the anyfunc table

Hmm, are these functions actually required? What if the funcref described by the fd were automatically loaded into the anyfunc table? I can't quite think of a scenario in which a module would want to obtain a funcref but not use it. Maybe if it wanted to hand the capability off to another module, but then why not just return the anyfunc ref directly? Indeed, if this were the case, we could already do something like this (associated .wat).

The only reason I'd be averse to more exports is because updating the WASI runtime requires hard forking, which is somewhat less than fun. That and it's nice to not maintain deprecated exports in perpetuity :)

[maxzaver]

I don't think that blockchain functions map well to the standard WASI modules, like core and processes, and I suggest we keep all of them in wasi-smart-contract or wasi-blockchain module. Here is the breakdown:

Filesystem

Most, if not all blockchains, implement their state storage as Merkle tree or trie. Merkle trees are key-value storages that allow getting and setting values based on the keys, as well as retrieving data in batches using the prefix and traversing neighboring keys.

Performance drawbacks

The blockchain developers and smart contract developers usually try to minimize the IO, so we cannot save the entire state of a contract as a single key-value entry (because every single mutation of a subset of the state will result in the entire state being written back). Instead, the developers of the smart contract are required to design their code with KV storage in mind.

Going against the intention of WASI

Emulating POSIX Filesystem on KV storages will create the glue code which is often going to be different for different blockchains. But the whole point of WASI was to avoid such glue code in the first place. Specifically, the glue code proposed by the OP will be similar to the JS glue code generated by Emscripten:

see the original WASI proposal

In addition to simple read/write we will require the glue code for the following:

• Folder structure and filepaths. They do not exist for KV storage, and therefore would require glue code that emulates them;
• Access rights. It makes more sense to set access rights directly, based on the prefix of the KV storage, oppose setting them on emulated directories;

Also, it seems like having smart contracts/blockchain as an entire separate WASI module was envisioned from the beginning as depicted in the announcement:

[nhynes]

w.r.t. the above comment, the current proposal does not relate to storage. However, I agree that exposing the MKVS in the blockchain module is very appropriate.

smart contract are required to design their code with KV storage in mind

That's not necessarily desirable. Developers who are highly performance sensitive will prefer to use the MKVS directly, but many--especially those trying to port existing applications--will appreciate POSIX semantics. You can look to GFS, CephFS, and S3 FUSE as examples. Users of higher-level frameworks mostly won't even touch keys or paths, so subsets of (elements of) values can be stored with prefixes that are efficient but opaque to the user. And, of course, embedders can simply choose not to provide the WASI fs module if they don't want to include glue code.

[devsnek]

fwiw I'm not entirely sure in what context func refs are being used, but you probably don't need to do the weird functions from open() thing. a funcref table is generally not a concept for direct interaction from libraries, but rather a very low level thing abstracted by the language you are compiling from, usually as normal function references for that language.

[pepyakin]

As far as I can see, the smart-contract models landscape hasn't yet been explored completely. It is yet to be seen if the ethereum-like/account-based approach is the most appropriate one.

And even if we assume that it is, then we will stumble upon the next problem. Here is a few questions:

• What type of an address the call function should work on? Should it be 20 bytes hash as in Ethereum or should it be a 32 bytes hash as in Substrate/Polkadot, or should it be a 4/8 byte-sized integer index? Or should it be opaque byte buffer which length, contents and semantics are host-specific?
• why value is not i128 or i256? I know there are chains that support i128. Ethereum itself support i256, isn't it?
• what semantics would create have? Will it work as CREATE or CREATE2? If the latter, should it pass seed/nonce (e.g. for creating several contracts from one sender)? How will be the code supplied? By passing Wasm code directly or by specifying some kind of identifier of the code saved somewhere?

Trying answering these questions might help to illustrate might points.

First problem I see deciding which option to choose, is efficiency.

If we settle on one single option, then other platforms will not be able to implement this or will suffer some from, most likely major, inefficiencies. If we settle on some generic option, then it might work (although I am not sure how exactly), but every contract and/or its contract runtime will have to pay this additional price of abstraction, even though that the contract could have avoided this overhead if compiled to the target chain.

Considering that there is a limit on how much work can be done within a particular time unit in a chain and the price of the transaction is directly proprotional to the amount of executed code, I really doubt that it really makes sense.

Another problem that comes to mind, security.

Let's say that we somehow managed to agree on some API. It should be clear by now that is should abstract quite some details. However, IMO, this might be problematic for a contract writer, since not all details can be abstracted without leaking and now the contract writer might need to account for these, now hidden, details. Even if the contract writer did his job exceptionally it still doesn't guarantee security since the users might try to deploy the contract on a new chain (or upgrade of an existing one). In Ethereum disasters because of such small unaccounted for details still happen even though the contract writers mostly know their target platform. I say mostly, because there are things like EIP-EIP-1283 (which was rejected shortly before the hard-fork because of such details, but still could have been included in) or things like state-rent, which in the worst case destroy half.

Furthermore, glue code that is required to support such abstractions inflates the trusted computing base.

Third, I am not sure what benefits would we get from that.

What is the selling point of WASI for smart-contracts?

1. To deploy a smart-contract without recompilation between different chains? Why would one want that? Contracts are way less useful without source code in most of the cases. There might be some benefits, but I am not sure if people would be willing to pay such a big price for that.
2. I am not sure about the greater WASI and of the idea of porting existing applications. Simply existing apps weren't designed with blockchain constraints in mind. We can't even compile most of the apps in browser because the model is rather different (e.g. take for example the browser main thread). Smart-contract platforms are way more exotic and more resemble constrained embedded platforms. For example, in my company experience we noticed that human-readable "String" shouldn't be used on chain in 99% cases usually can be moved out off-chain. Only the most critical logic should be on-chain (otherwise it would be extremely wasteful) and the existing applications weren't written with this in mind.
3. One could think, that some low-level part of a smart-contract platform could be extracted biased towards some particular smart-contract model (say, KV-storage with fixed-sized keys and values). And that might be true, this way we might have some libraries that work on that level. However, I am not sure that it should be WASI. We can get away by simple conventions.

Bottom line:

If we manage to come up with some blockchain standard, it might be one or most of the below:

1. be hard to program against,
2. be less efficient, more expensive to use,
3. drag the evolution of a smart-contract platform,
4. less secure to use or to program,
5. doesn't yield substantial benefit,

[nhynes]

Thanks for the comment; you raise some very pertinent questions. However, I think that your perspective misses that abstractions like WASI foster productive platform and developer ecosystems. For example, although EVM is efficient, the tooling leaves much to be desired: Solidity is a.s. nobody's preferred programming language and there still does not exist a GDB-like tool for EVM. On the other hand, WASI libc already enables writing blockchain applications using any LLVM frontend and is a basis for building tools like an in-browser blockchain simulator.

Smart-contract platforms are way more exotic and more resemble constrained embedded platforms

I disagree. You're talking about current smart-contract platforms. The current wave of blockchain platforms (e.g., Algorand, Dfinity, Near, EOS) promise to offer vastly more scalability. As with any properly designed system, let's start with the correct interfaces and work backwards from there.

deploy a smart-contract without recompilation between different chains

I'm going to go out on a limb and suggest that nobody (aside from maybe the platform, itself) is a fan of vendor lock-in. In fact, that's why we see products like Terraform and Serverless; and tools like Keras, Java, and Wasm that smooth over details of the underlying platform.

From another point of view, having a common interface allows creating shared infrastructure. After all, how many ways should there really be to express ERC-{20, 223, 721}? The Cartesian product of all platforms and all standard contracts is only going to become more unwieldy. Having a low-level WASI way to write these not only obviates the need for the developer to learn Yet Another Blockchain DSL, but also improves security since there are more eyes on/transactions hitting each community-blessed contract.

As another point on the topic of security, having a well-defined standard for runtime semantics would improve rather than weaken security since it would have benefited from the feedback of a larger organization, and developers could rely on the robust mental models they have for normal program execution.

I am not sure about the greater WASI and of the idea of porting existing applications

You could have said the exact same thing about Emscripten, but here were are, discussing the merits of its successor. I'll give you a very concrete use-case for a WASI runtime: compiling a JavaScript VM. As great as Rust is, most developers would prefer to use JavaScript. Compiling a JS engine to Wasm benefits from the security guarantees of linear memory. Doing this using Emscripten is a huge pain, but is much easier with WASI. You'll balk at the idea of running a VM inside of a VM, but I'm willing to bet that a blockchain that safely pulls off a JavaScript runtime with decent performance will become wildly popular. Indeed, the Agoric platform is already making this bet (and adds in capabilities, which are another selling point of WASI).

Just because you don't currently see the value of a common interface doesn't mean it's not worth having. We'll probably end up seeing one anyway for the same reasons that we have e.g., Terraform. Doing this proactively in a community setting is preferable to waiting for a company or small team to fill the void. At the very least, we can start writing shared tooling now.

In summary, it's always possible to dial efficiency up to 11 by creating a vertically integrated system, but it comes at the cost of usability outside of that vertical. If your platform is constrained enough such that WASI is unnecessary (c.f. Libra), then, by all means, please don't use it. For more general purpose applications, WASI is a very useful standard. I would hope that we can work together to answer the questions of what call, value, and create, among other things, should look like.

[pepyakin]

Don't get me wrong. I like the idea of standardization, the common set of tooling and having only one single way of doing something (e.g ERC-20). And of course I undestand the downsides of lock-in. WASI as a standard helps, as you said, to smooth out these non-essential differences and provide a good abstraction over things like file system. But note that file systems are around for at least half a century and most of the modern systems have more or less the same notion of them. More or less the same you can say about other components of application environment such as networking or graphics.

But I really unsure that you can say anything like that about smart-contract platforms. At the moment we have 1.5 working examples of smart-contract platforms. If you allow me, I'd compare it that if we were in 1970 and were trying to figure out which logic works best: binary, ternary or some 3rd option. Whether byte or word should be 8, 12, 36 or 60 bit. Thankfully, we have already passed this wild-west days for general computing and now we e.g. can assume that the byte is 8 bits and can do better and more common abstractions based on it. However, I do believe that we are not yet there w.r.t to smart-contract platforms.

Repeating myself: I am not clear if the current Ethereum-like account-based model is the best.
Looking at discussions around Eth 2.0 it is not yet clear to me whether they use this model themselves as primary model. Commiting to the standard prematurely might hinder the evolution and experimentation in the field. While it is possible to define some sort of standard, as I mentioned I think it would be near to impossible to create a common denominator which would abstract details of each platform without leaking important details, with good level of efficiency.

But let's assume that such standard were created. You wrote:

It's always possible to dial efficiency up to 11 by creating a vertically integrated system, but it comes at the cost of usability outside of that vertical.

This kind of worries me and reinforces my point to a some degree, because we can end-up that this standard will be only adopted by chains only designed for using with this standard. For other chains it will be either prohibitory costly (in terms of efficiency, usability or anything) or merely impossible to adopt such a standard.

At the best case we can end up, with a few chains that adopted this standard depending on the genericity of it. If the standard is too generic, then the details will leak. If the standard is too narrow, then we might end up with essentially a single smart-contract model (since there is not much sense in making copies of a chain with the same capabilities), that we would be able to fairly call the WASI-chain : )

Building the standard without the necessary exploration in the field we will most likely find ourselves in the same situation as in the famous comic strip from XKCD.

[maxzaver]

This proposal would support most existing smart contract platforms including those with a synchronous RPC model (e.g. Ethereum, EOS) and those with asynchronous cross-shard RPCs (e.g, Near).

To clarify, according to Ewasm folks, it is still not decided for Ethereum 2.0 whether calls (both in-shard and cross-shard) are going to be synchronous or asynchronous, see: https://youtu.be/iwU10WkWSBY?t=1469

At least for cross-shard calls the current plan is to go with receipts, which AFAIU implies asynchronous calls: https://youtu.be/iwU10WkWSBY?t=656

In general, it would be pretty hard to implement sync cross-shard calls for any sharded blockchain using WASM, because that would require dumping WASM stack and reloading it back. So async calls is a must have.

[bhack]

/cc @ewasm