Skip to content

Commit 8ea8354

Browse files
RaisinTendanielleadams
RaisinTen
authored and
danielleadams
committed
src: add initial support for single executable applications
Compile a JavaScript file into a single executable application: ```console $ echo 'console.log(`Hello, ${process.argv[2]}!`);' > hello.js $ cp $(command -v node) hello $ npx postject hello NODE_JS_CODE hello.js \ --sentinel-fuse NODE_JS_FUSE_fce680ab2cc467b6e072b8b5df1996b2 $ npx postject hello NODE_JS_CODE hello.js \ --sentinel-fuse NODE_JS_FUSE_fce680ab2cc467b6e072b8b5df1996b2 \ --macho-segment-name NODE_JS $ ./hello world Hello, world! ``` Signed-off-by: Darshan Sen <[email protected]> PR-URL: #45038 Backport-PR-URL: #47495 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Michael Dawson <[email protected]> Reviewed-By: Joyee Cheung <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>
1 parent 956f786 commit 8ea8354

13 files changed

+506
-1
lines changed

‎configure.py

+10
Original file line numberDiff line numberDiff line change
@@ -146,6 +146,12 @@
146146
default=None,
147147
help='use on deprecated SunOS systems that do not support ifaddrs.h')
148148

149+
parser.add_argument('--disable-single-executable-application',
150+
action='store_true',
151+
dest='disable_single_executable_application',
152+
default=None,
153+
help='Disable Single Executable Application support.')
154+
149155
parser.add_argument("--fully-static",
150156
action="store_true",
151157
dest="fully_static",
@@ -1402,6 +1408,10 @@ def configure_node(o):
14021408
if options.no_ifaddrs:
14031409
o['defines'] += ['SUNOS_NO_IFADDRS']
14041410

1411+
o['variables']['single_executable_application'] = b(not options.disable_single_executable_application)
1412+
if options.disable_single_executable_application:
1413+
o['defines'] += ['DISABLE_SINGLE_EXECUTABLE_APPLICATION']
1414+
14051415
# By default, enable ETW on Windows.
14061416
if flavor == 'win':
14071417
o['variables']['node_use_etw'] = b(not options.without_etw)

‎doc/api/index.md

+1
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@
5252
* [Readline](readline.md)
5353
* [REPL](repl.md)
5454
* [Report](report.md)
55+
* [Single executable applications](single-executable-applications.md)
5556
* [Stream](stream.md)
5657
* [String decoder](string_decoder.md)
5758
* [Test runner](test.md)

‎doc/api/single-executable-applications.md

+140
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
# Single executable applications
2+
3+
<!--introduced_in=REPLACEME-->
4+
5+
> Stability: 1 - Experimental: This feature is being designed and will change.
6+
7+
<!-- source_link=lib/internal/main/single_executable_application.js -->
8+
9+
This feature allows the distribution of a Node.js application conveniently to a
10+
system that does not have Node.js installed.
11+
12+
Node.js supports the creation of [single executable applications][] by allowing
13+
the injection of a JavaScript file into the `node` binary. During start up, the
14+
program checks if anything has been injected. If the script is found, it
15+
executes its contents. Otherwise Node.js operates as it normally does.
16+
17+
The single executable application feature only supports running a single
18+
embedded [CommonJS][] file.
19+
20+
A bundled JavaScript file can be turned into a single executable application
21+
with any tool which can inject resources into the `node` binary.
22+
23+
Here are the steps for creating a single executable application using one such
24+
tool, [postject][]:
25+
26+
1. Create a JavaScript file:
27+
```console
28+
$ echo 'console.log(`Hello, ${process.argv[2]}!`);' > hello.js
29+
```
30+
31+
2. Create a copy of the `node` executable and name it according to your needs:
32+
```console
33+
$ cp $(command -v node) hello
34+
```
35+
36+
3. Inject the JavaScript file into the copied binary by running `postject` with
37+
the following options:
38+
39+
* `hello` - The name of the copy of the `node` executable created in step 2.
40+
* `NODE_JS_CODE` - The name of the resource / note / section in the binary
41+
where the contents of the JavaScript file will be stored.
42+
* `hello.js` - The name of the JavaScript file created in step 1.
43+
* `--sentinel-fuse NODE_JS_FUSE_fce680ab2cc467b6e072b8b5df1996b2` - The
44+
[fuse][] used by the Node.js project to detect if a file has been injected.
45+
* `--macho-segment-name NODE_JS` (only needed on macOS) - The name of the
46+
segment in the binary where the contents of the JavaScript file will be
47+
stored.
48+
49+
To summarize, here is the required command for each platform:
50+
51+
* On systems other than macOS:
52+
```console
53+
$ npx postject hello NODE_JS_CODE hello.js \
54+
--sentinel-fuse NODE_JS_FUSE_fce680ab2cc467b6e072b8b5df1996b2
55+
```
56+
57+
* On macOS:
58+
```console
59+
$ npx postject hello NODE_JS_CODE hello.js \
60+
--sentinel-fuse NODE_JS_FUSE_fce680ab2cc467b6e072b8b5df1996b2 \
61+
--macho-segment-name NODE_JS
62+
```
63+
64+
4. Run the binary:
65+
```console
66+
$ ./hello world
67+
Hello, world!
68+
```
69+
70+
## Notes
71+
72+
### `require(id)` in the injected module is not file based
73+
74+
`require()` in the injected module is not the same as the [`require()`][]
75+
available to modules that are not injected. It also does not have any of the
76+
properties that non-injected [`require()`][] has except [`require.main`][]. It
77+
can only be used to load built-in modules. Attempting to load a module that can
78+
only be found in the file system will throw an error.
79+
80+
Instead of relying on a file based `require()`, users can bundle their
81+
application into a standalone JavaScript file to inject into the executable.
82+
This also ensures a more deterministic dependency graph.
83+
84+
However, if a file based `require()` is still needed, that can also be achieved:
85+
86+
```js
87+
const { createRequire } = require('node:module');
88+
require = createRequire(__filename);
89+
```
90+
91+
### `__filename` and `module.filename` in the injected module
92+
93+
The values of `__filename` and `module.filename` in the injected module are
94+
equal to [`process.execPath`][].
95+
96+
### `__dirname` in the injected module
97+
98+
The value of `__dirname` in the injected module is equal to the directory name
99+
of [`process.execPath`][].
100+
101+
### Single executable application creation process
102+
103+
A tool aiming to create a single executable Node.js application must
104+
inject the contents of a JavaScript file into:
105+
106+
* a resource named `NODE_JS_CODE` if the `node` binary is a [PE][] file
107+
* a section named `NODE_JS_CODE` in the `NODE_JS` segment if the `node` binary
108+
is a [Mach-O][] file
109+
* a note named `NODE_JS_CODE` if the `node` binary is an [ELF][] file
110+
111+
Search the binary for the
112+
`NODE_JS_FUSE_fce680ab2cc467b6e072b8b5df1996b2:0` [fuse][] string and flip the
113+
last character to `1` to indicate that a resource has been injected.
114+
115+
### Platform support
116+
117+
Single-executable support is tested regularly on CI only on the following
118+
platforms:
119+
120+
* Windows
121+
* macOS
122+
* Linux (AMD64 only)
123+
124+
This is due to a lack of better tools to generate single-executables that can be
125+
used to test this feature on other platforms.
126+
127+
Suggestions for other resource injection tools/workflows are welcomed. Please
128+
start a discussion at <https://github.com/nodejs/single-executable/discussions>
129+
to help us document them.
130+
131+
[CommonJS]: modules.md#modules-commonjs-modules
132+
[ELF]: https://en.wikipedia.org/wiki/Executable_and_Linkable_Format
133+
[Mach-O]: https://en.wikipedia.org/wiki/Mach-O
134+
[PE]: https://en.wikipedia.org/wiki/Portable_Executable
135+
[`process.execPath`]: process.md#processexecpath
136+
[`require()`]: modules.md#requireid
137+
[`require.main`]: modules.md#accessing-the-main-module
138+
[fuse]: https://www.electronjs.org/docs/latest/tutorial/fuses
139+
[postject]: https://github.com/nodejs/postject
140+
[single executable applications]: https://github.com/nodejs/single-executable

‎doc/contributing/maintaining-single-executable-application-support.md

+81
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# Maintaining Single Executable Applications support
2+
3+
Support for [single executable applications][] is one of the key technical
4+
priorities identified for the success of Node.js.
5+
6+
## High level strategy
7+
8+
From the [Next-10 discussions][] there are 2 approaches the project believes are
9+
important to support:
10+
11+
### Compile with Node.js into executable
12+
13+
This is the approach followed by [boxednode][].
14+
15+
No additional code within the Node.js project is needed to support the
16+
option of compiling a bundled application along with Node.js into a single
17+
executable application.
18+
19+
### Bundle into existing Node.js executable
20+
21+
This is the approach followed by [pkg][].
22+
23+
The project does not plan to provide the complete solution but instead the key
24+
elements which are required in the Node.js executable in order to enable
25+
bundling with the pre-built Node.js binaries. This includes:
26+
27+
* Looking for a segment within the executable that holds bundled code.
28+
* Running the bundled code when such a segment is found.
29+
30+
It is left up to external tools/solutions to:
31+
32+
* Bundle code into a single script.
33+
* Generate a command line with appropriate options.
34+
* Add a segment to an existing Node.js executable which contains
35+
the command line and appropriate headers.
36+
* Re-generate or removing signatures on the resulting executable
37+
* Provide a virtual file system, and hooking it in if needed to
38+
support native modules or reading file contents.
39+
40+
However, the project also maintains a separate tool, [postject][], for injecting
41+
arbitrary read-only resources into the binary such as those needed for bundling
42+
the application into the runtime.
43+
44+
## Planning
45+
46+
Planning for this feature takes place in the [single-executable repository][].
47+
48+
## Upcoming features
49+
50+
Currently, only running a single embedded CommonJS file is supported but support
51+
for the following features are in the list of work we'd like to get to:
52+
53+
* Running an embedded ESM file.
54+
* Running an archive of multiple files.
55+
* Embedding [Node.js CLI options][] into the binary.
56+
* [XCOFF][] executable format.
57+
* Run tests on Linux architectures/distributions other than AMD64 Ubuntu.
58+
59+
## Disabling single executable application support
60+
61+
To disable single executable application support, build Node.js with the
62+
`--disable-single-executable-application` configuration option.
63+
64+
## Implementation
65+
66+
When built with single executable application support, the Node.js process uses
67+
[`postject-api.h`][] to check if the `NODE_JS_CODE` section exists in the
68+
binary. If it is found, it passes the buffer to
69+
[`single_executable_application.js`][], which executes the contents of the
70+
embedded script.
71+
72+
[Next-10 discussions]: https://github.com/nodejs/next-10/blob/main/meetings/summit-nov-2021.md#single-executable-applications
73+
[Node.js CLI options]: https://nodejs.org/api/cli.html
74+
[XCOFF]: https://www.ibm.com/docs/en/aix/7.2?topic=formats-xcoff-object-file-format
75+
[`postject-api.h`]: https://github.com/nodejs/node/blob/71951a0e86da9253d7c422fa2520ee9143e557fa/test/fixtures/postject-copy/node_modules/postject/dist/postject-api.h
76+
[`single_executable_application.js`]: https://github.com/nodejs/node/blob/main/lib/internal/main/single_executable_application.js
77+
[boxednode]: https://github.com/mongodb-js/boxednode
78+
[pkg]: https://github.com/vercel/pkg
79+
[postject]: https://github.com/nodejs/postject
80+
[single executable applications]: https://github.com/nodejs/node/blob/main/doc/contributing/technical-priorities.md#single-executable-applications
81+
[single-executable repository]: https://github.com/nodejs/single-executable

‎lib/internal/main/single_executable_application.js

+55
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
'use strict';
2+
const {
3+
prepareMainThreadExecution,
4+
markBootstrapComplete,
5+
} = require('internal/process/pre_execution');
6+
const { getSingleExecutableCode } = internalBinding('sea');
7+
const { emitExperimentalWarning } = require('internal/util');
8+
const { Module, wrapSafe } = require('internal/modules/cjs/loader');
9+
const { codes: { ERR_UNKNOWN_BUILTIN_MODULE } } = require('internal/errors');
10+
11+
prepareMainThreadExecution(false, true);
12+
markBootstrapComplete();
13+
14+
emitExperimentalWarning('Single executable application');
15+
16+
// This is roughly the same as:
17+
//
18+
// const mod = new Module(filename);
19+
// mod._compile(contents, filename);
20+
//
21+
// but the code has been duplicated because currently there is no way to set the
22+
// value of require.main to module.
23+
//
24+
// TODO(RaisinTen): Find a way to deduplicate this.
25+
26+
const filename = process.execPath;
27+
const contents = getSingleExecutableCode();
28+
const compiledWrapper = wrapSafe(filename, contents);
29+
30+
const customModule = new Module(filename, null);
31+
customModule.filename = filename;
32+
customModule.paths = Module._nodeModulePaths(customModule.path);
33+
34+
const customExports = customModule.exports;
35+
36+
function customRequire(path) {
37+
if (!Module.isBuiltin(path)) {
38+
throw new ERR_UNKNOWN_BUILTIN_MODULE(path);
39+
}
40+
41+
return require(path);
42+
}
43+
44+
customRequire.main = customModule;
45+
46+
const customFilename = customModule.filename;
47+
48+
const customDirname = customModule.path;
49+
50+
compiledWrapper(
51+
customExports,
52+
customRequire,
53+
customModule,
54+
customFilename,
55+
customDirname);

‎node.gyp

+6-1
Original file line numberDiff line numberDiff line change
@@ -153,7 +153,8 @@
153153

154154
'include_dirs': [
155155
'src',
156-
'deps/v8/include'
156+
'deps/v8/include',
157+
'deps/postject'
157158
],
158159

159160
'sources': [
@@ -458,6 +459,7 @@
458459

459460
'include_dirs': [
460461
'src',
462+
'deps/postject',
461463
'<(SHARED_INTERMEDIATE_DIR)' # for node_natives.h
462464
],
463465
'dependencies': [
@@ -531,6 +533,7 @@
531533
'src/node_report.cc',
532534
'src/node_report_module.cc',
533535
'src/node_report_utils.cc',
536+
'src/node_sea.cc',
534537
'src/node_serdes.cc',
535538
'src/node_shadow_realm.cc',
536539
'src/node_snapshotable.cc',
@@ -641,6 +644,7 @@
641644
'src/node_report.h',
642645
'src/node_revert.h',
643646
'src/node_root_certs.h',
647+
'src/node_sea.h',
644648
'src/node_shadow_realm.h',
645649
'src/node_snapshotable.h',
646650
'src/node_snapshot_builder.h',
@@ -683,6 +687,7 @@
683687
'src/util-inl.h',
684688
# Dependency headers
685689
'deps/v8/include/v8.h',
690+
'deps/postject/postject-api.h'
686691
# javascript files to make for an even more pleasant IDE experience
687692
'<@(library_files)',
688693
'<@(deps_files)',

‎src/node.cc

+18
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@
3939
#include "node_realm-inl.h"
4040
#include "node_report.h"
4141
#include "node_revert.h"
42+
#include "node_sea.h"
4243
#include "node_snapshot_builder.h"
4344
#include "node_v8_platform-inl.h"
4445
#include "node_version.h"
@@ -126,6 +127,7 @@
126127
#include <cstring>
127128

128129
#include <string>
130+
#include <tuple>
129131
#include <vector>
130132

131133
namespace node {
@@ -321,6 +323,18 @@ MaybeLocal<Value> StartExecution(Environment* env, StartExecutionCallback cb) {
321323
first_argv = env->argv()[1];
322324
}
323325

326+
#ifndef DISABLE_SINGLE_EXECUTABLE_APPLICATION
327+
if (sea::IsSingleExecutable()) {
328+
// TODO(addaleax): Find a way to reuse:
329+
//
330+
// LoadEnvironment(Environment*, const char*)
331+
//
332+
// instead and not add yet another main entry point here because this
333+
// already duplicates existing code.
334+
return StartExecution(env, "internal/main/single_executable_application");
335+
}
336+
#endif
337+
324338
if (first_argv == "inspect") {
325339
return StartExecution(env, "internal/main/inspect");
326340
}
@@ -1187,6 +1201,10 @@ int LoadSnapshotDataAndRun(const SnapshotData** snapshot_data_ptr,
11871201
}
11881202

11891203
int Start(int argc, char** argv) {
1204+
#ifndef DISABLE_SINGLE_EXECUTABLE_APPLICATION
1205+
std::tie(argc, argv) = sea::FixupArgsForSEA(argc, argv);
1206+
#endif
1207+
11901208
CHECK_GT(argc, 0);
11911209

11921210
// Hack around with the argv pointer. Used for process.title = "blah".

0 commit comments

Comments
 (0)