Native methods for CSEC machine

[mattcce]

Introduction

Native methods in Java provide FFI functionality to call foreign functions. Native methods are especially useful in the CSE machine to implement primitive functions, and in some cases, are the only real way to do so.

Motivation

The motivation for implementing native methods initially stemmed from Object.hashCode, which itself is a native method.

Native methods have clear uses for some other cases as well, like with displaying or printing to standard output. Other specific uses for native methods are presently unclear, but having the ability to break out of the CSEC machine is certainly not a detriment.

Implementation

Notation note: we refer generally to a Java syntactic native method by the term native method, and the foreign function that is eventually called when the native method is invoked by the term foreign function.

CSEC Handling

Two control items (labelled C1, C2) have been modified to allow for native methods to be properly handled.

(C1) Method Declarations (AST Node MethodDeclaration) will now dynamically link a foreign function to a native method declaration. Closures (the Closure type) now distinguishes between three types of declarations (previously two), with the newest being NativeDeclaration.

Constructors cannot be native in Java. NativeDeclarations mirror the MethodDeclaration type, except it does not have a method body but instead hold a pointer to the resolved foreign function.

Implementation note: even though MethodDeclaration and ConstructorDeclaration, both of which are contained within the Closure type, are defined in the AST (in ast/types/classes.ts), we instead choose to define NativeDeclaration in ec-evaluator/types.ts instead, to localise these changes.

(C2) Invocations (Instruction invoke) will now distinguish the new Closure type, and handle it specially. In particular, retains most of the usual functionality, and will appropriately initialise a new environment frame for the function to be executed (see native method architecture below), and push a return instruction before directly calling the foreign function with the full CSEC machine state.

Native Method Architecture

We offer full and complete power to native methods, passing in the entire control, stash, and environment to foreign functions they call. This matches the power that foreign functions are expected to have.

The CSEC machine still handles argument resolution and returning. This is because FFI calls should function like normal functions to the CSEC machine, except that it ignores whatever happens while the foreign function is running. To this end, common foreign functions are expected to:

• collect/find its arguments from the environment supplied to it, if any,
• push a result of the valid type onto the top of the stash.

Note that the foreign function, using common utility methods that the rest of the CSEC machine uses, will have to appropriately deserialise and serialise the values. In particular, it must be able to destructure AST nodes that it expects and reconstruct valid AST nodes that the CSEC machine expects.

Because the full control, stash, and environment is passed to the foreign function, this allows foreign functions to effectively do anything to the CSEC machine state. However, foreign functions are generally not permitted to expect anything except its arguments being defined in the current environment.

The dictionary used uses fully-qualified method descriptors to identify the correct foreign function. These are of the form:

class::identifier ( p1Type p1Identifier, … ): returnType

Note that foreign functions always have the same type:

({ control, stash, environment }: { control: Control; stash: Stash; environment: Environment }) => void

In particular, foreign functions always return nothing. Their results are to be injected directly into the stash that is provided, for use by the CSEC machine (and the program it is currently running).

The identifier is included in such a symbolic reference for the express purpose of making foreign functions easier to implement. Because foreign functions must retrieve their arguments from the environment, their implementations must be aware of the correct parameter identifiers.

[github-actions]

Coverage report

St.❔	Category	Percentage	Covered / Total
🟡	Statements	73.91% (-0.04% 🔻)	7205/9748
🟡	Branches	60.52% (+0.13% 🔼)	2370/3916
🟡	Functions	69.3% (-0.16% 🔻)	1291/1863
🟡	Lines	74.82% (-0.11% 🔻)	6779/9060

Show new covered files 🐣

St.❔	File	Statements	Branches	Functions	Lines
🔴	... / natives.ts	14.29%	100%	0%	14.29%

Show files with reduced coverage 🔻

St.❔	File	Statements	Branches	Functions	Lines
🔴	... / errors.ts	42.03% (-1.05% 🔻)	27.27%	14.71% (-0.92% 🔻)	43.94% (-0.13% 🔻)
🟢	... / interpreter.ts	98.77% (-0.27% 🔻)	91.36% (-0.53% 🔻)	98.15%	98.68% (-0.62% 🔻)
🟢	... / index.ts	73.85% (-0.52% 🔻)	52.38% (-0.63% 🔻)	95.83% (+0.18% 🔼)	83.41% (-0.84% 🔻)
🟡	types/errors.ts	63.64% (-0.88% 🔻)	0%	41.38% (-3.07% 🔻)	65.08% (-1.02% 🔻)
🔴	... / extractor.ts	48.65% (-0.13% 🔻)	36.97% (-0.09% 🔻)	45.28%	52.81% (-0.17% 🔻)

Test suite run success

1123 tests passing in 64 suites.

Report generated by 🧪jest coverage report action from 27c6618