[clang-repl] Improve flags responsible for generating shared wasm binaries

[anutosh491]

There are a couple changes in this PR that help getting clang-repl to run in the browser. Using a jupyterlite instance for the example pasted below

1. Updating flags responsible for generating shared wasm binaries that need to be dynamically loaded Most Importantly as can be seen in the changes shared and allow-undefined are crucial.

2. While exiting we encounter this.

Now as can be seen here

llvm-project/clang/lib/Interpreter/Interpreter.cpp

Lines 421 to 430 in cd41803

	Interpreter::~Interpreter() {
	IncrParser.reset();
	Act->FinalizeAction();
	if (IncrExecutor) {
	if (llvm::Error Err = IncrExecutor->cleanUp())
	llvm::report_fatal_error(
	llvm::Twine("Failed to clean up IncrementalExecutor: ") +
	toString(std::move(Err)));
	}
	}

We call cleanUP in the destructor. Now cleanUP through IncrementalExecutor tries to deinitialize the JIT which wasn't even intialized as runCtors in wasm.cpp is a no-op

llvm-project/clang/lib/Interpreter/IncrementalExecutor.cpp

Lines 94 to 101 in cd41803

	llvm::Error IncrementalExecutor::cleanUp() {
	// This calls the global dtors of registered modules.
	return Jit->deinitialize(Jit->getMainJITDylib());
	}
	llvm::Error IncrementalExecutor::runCtors() const {
	return Jit->initialize(Jit->getMainJITDylib());
	}

llvm-project/clang/lib/Interpreter/Wasm.cpp

Lines 107 to 109 in cd41803

	llvm::Error WasmIncrementalExecutor::runCtors() const {
	// This seems to be automatically done when using dlopen()
	return llvm::Error::success();

[llvmbot]

@llvm/pr-subscribers-clang

Author: Anutosh Bhat (anutosh491)

Changes

There are a couple changes in this PR that help getting clang-repl to run in the browser. Using a jupyterlite instance for the example pasted below

1. Updating flags responsible for generating shared wasm binaries that need to be dynamically loaded Most Importantly as can be seen in the changes shared and allow-undefined are crucial.

2. While exiting we encounter this.

Now as can be seen here

llvm-project/clang/lib/Interpreter/Interpreter.cpp

Lines 421 to 430 in cd41803

	Interpreter::~Interpreter() {
	IncrParser.reset();
	Act->FinalizeAction();
	if (IncrExecutor) {
	if (llvm::Error Err = IncrExecutor->cleanUp())
	llvm::report_fatal_error(
	llvm::Twine("Failed to clean up IncrementalExecutor: ") +
	toString(std::move(Err)));
	}
	}

We call cleanUP in the destructor. Now cleanUP through IncrementalExecutor tries to deinitialize the JIT which wasn't even intialized as runCtors in wasm.cpp is a no-op

llvm-project/clang/lib/Interpreter/IncrementalExecutor.cpp

Lines 94 to 101 in cd41803

	llvm::Error IncrementalExecutor::cleanUp() {
	// This calls the global dtors of registered modules.
	return Jit->deinitialize(Jit->getMainJITDylib());
	}
	llvm::Error IncrementalExecutor::runCtors() const {
	return Jit->initialize(Jit->getMainJITDylib());
	}

llvm-project/clang/lib/Interpreter/Wasm.cpp

Lines 107 to 109 in cd41803

	llvm::Error WasmIncrementalExecutor::runCtors() const {
	// This seems to be automatically done when using dlopen()
	return llvm::Error::success();

---

Full diff: https://github.com/llvm/llvm-project/pull/116735.diff

3 Files Affected:

• (modified) clang/lib/Interpreter/IncrementalExecutor.h (+1-1)
• (modified) clang/lib/Interpreter/Wasm.cpp (+9-2)
• (modified) clang/lib/Interpreter/Wasm.h (+1)

diff --git a/clang/lib/Interpreter/IncrementalExecutor.h b/clang/lib/Interpreter/IncrementalExecutor.h
index 7954cde36588bd..dbd61f0b8b1ebb 100644
--- a/clang/lib/Interpreter/IncrementalExecutor.h
+++ b/clang/lib/Interpreter/IncrementalExecutor.h
@@ -56,7 +56,7 @@ class IncrementalExecutor {
   virtual llvm::Error addModule(PartialTranslationUnit &PTU);
   virtual llvm::Error removeModule(PartialTranslationUnit &PTU);
   virtual llvm::Error runCtors() const;
-  llvm::Error cleanUp();
+  virtual llvm::Error cleanUp();
   llvm::Expected<llvm::orc::ExecutorAddr>
   getSymbolAddress(llvm::StringRef Name, SymbolNameKind NameKind) const;
 
diff --git a/clang/lib/Interpreter/Wasm.cpp b/clang/lib/Interpreter/Wasm.cpp
index 1001410aa0f279..6f00bac8c0bc46 100644
--- a/clang/lib/Interpreter/Wasm.cpp
+++ b/clang/lib/Interpreter/Wasm.cpp
@@ -72,13 +72,14 @@ llvm::Error WasmIncrementalExecutor::addModule(PartialTranslationUnit &PTU) {
   OutputFile.close();
 
   std::vector<const char *> LinkerArgs = {"wasm-ld",
-                                          "-pie",
+                                          "-shared",
                                           "--import-memory",
                                           "--no-entry",
                                           "--export-all",
                                           "--experimental-pic",
-                                          "--no-export-dynamic",
+                                          "--export-dynamic",
                                           "--stack-first",
+                                          "--allow-undefined",
                                           OutputFileName.c_str(),
                                           "-o",
                                           OutputFileName.c_str()};
@@ -109,6 +110,12 @@ llvm::Error WasmIncrementalExecutor::runCtors() const {
   return llvm::Error::success();
 }
 
+llvm::Error WasmIncrementalExecutor::cleanUp() const {
+  // Can't call cleanUp through IncrementalExecutor as it
+  // tries to deinitialize JIT which hasn't been initialized
+  return llvm::Error::success();
+}
+
 WasmIncrementalExecutor::~WasmIncrementalExecutor() = default;
 
 } // namespace clang
diff --git a/clang/lib/Interpreter/Wasm.h b/clang/lib/Interpreter/Wasm.h
index b1fd88024f14d7..4632613326d39b 100644
--- a/clang/lib/Interpreter/Wasm.h
+++ b/clang/lib/Interpreter/Wasm.h
@@ -28,6 +28,7 @@ class WasmIncrementalExecutor : public IncrementalExecutor {
   llvm::Error addModule(PartialTranslationUnit &PTU) override;
   llvm::Error removeModule(PartialTranslationUnit &PTU) override;
   llvm::Error runCtors() const override;
+  llvm::Error cleanUp() override;
 
   ~WasmIncrementalExecutor() override;
 };

[vgvassilev]

Lgtm! It is still unfortunate that we can’t test this in tree. Do you have any ideas how to test this without having to install a browser or nodejs in the bots?

[anutosh491]

/cherry-pick 752dbd6

[llvmbot]

/pull-request #116766