Workers are not imported correctly despite the worker syntax seemingly being correct, works in parcel

[hybridherbst]

Describe the bug

On an existing project that uses parcel, I switched to vite but that results in web workers not working anymore / returning undefined.

Reference issue with the dev verifying that the worker syntax matches vite's expectations:

• Switching bundler to vite instead of parcel breaks web workers gkjohnson/three-gpu-pathtracer#355

Vite worker docs, which use the same syntax as the project at hand:
https://vitejs.dev/guide/features.html#import-with-constructors

My best guess is that the bug here is that a worker is used from a sub-sub package (three-gpu-pathtracer uses the worker from three-mesh-bvh) and that somehow doesn't work in vite right now?

Links to the source code to easily follow the chain until the actual worker:

• https://github.com/gkjohnson/three-gpu-pathtracer/blob/main/example/index.js#L682
  (identical to the index.js in the repro project)
• https://github.com/gkjohnson/three-gpu-pathtracer/blob/main/src/workers/PathTracingSceneWorker.js#LL10C51-L10C51
• https://github.com/gkjohnson/three-mesh-bvh/blob/master/src/workers/GenerateMeshBVHWorker.js#L9
• https://github.com/gkjohnson/three-mesh-bvh/blob/master/src/workers/generateAsync.worker.js
  (the actual worker)

Reproduction

https://stackblitz.com/edit/vitejs-vite-k1ccfw?file=index.js

Steps to reproduce

Steps to reproduce in the StackBlitz project linked above:

• Run npm run dev
• Note page loads
• Note page stops loading and console logs error Uncaught (in promise) Error: GenerateMeshBVHWorker: undefined at worker.onerror

Compare behaviour to https://gkjohnson.github.io/three-gpu-pathtracer/example/bundle/index.html

Steps to reproduce locally:

• fork https://github.com/gkjohnson/three-gpu-pathtracer
• run npm install --save-dev vite
• in package.json, add "start-vite": "cd example && vite --host", in scripts
• run npm run start-vite
• visit the page
• note loading works fine, then the MeshBVH worker returns undefined and logs to the console: Uncaught (in promise) Error: GenerateMeshBVHWorker: undefined

System Info

Reproduces on any system I tried on.

On StackBlitz:

  System:
    OS: Linux 5.0 undefined
    CPU: (8) x64 Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
    Memory: 0 Bytes / 0 Bytes
    Shell: 1.0 - /bin/jsh
  Binaries:
    Node: 16.14.2 - /usr/local/bin/node
    Yarn: 1.22.19 - /usr/local/bin/yarn
    npm: 7.17.0 - /usr/local/bin/npm
  npmPackages:
    vite: ^4.0.4 => 4.0.4 

Used Package Manager

npm

Logs

Uncaught (in promise) Error: GenerateMeshBVHWorker: undefined at worker.onerror

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Make sure this is a Vite issue and not a framework-specific issue. For example, if it's a Vue SFC related bug, it should likely be reported to vuejs/core instead.
• Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• The provided reproduction is a minimal reproducible example of the bug.

[sapphi-red]

This is similar to #8427.
new Worker(new URL('./foo.js', import.meta.url)) is not supported by the dep optimizer.

A workaround is to set optimizeDeps.exclude: ['three-gpu-pathtracer'].

[hybridherbst]

Thanks @sapphi-red – so are the docs at https://vitejs.dev/guide/features.html#import-with-constructors partially incorrect? I think that's exactly the same syntax.

Seems something like "this works in your local project but not in a dependency" and "this will not work anymore once you release your package to others" would be important to note there, including how to work around.

Additionally, does that mean this is by design? It's unclear to me if / why it's not considered a bug in vite.

[sapphi-red]

Yeah, we should probably have a document that explains dep-optimizer doesn't support every feature on the features page.

IIUC this is just not implemented.

[akazukin5151]

The optimizeDeps.exclude workaround doesn't work if your worker file has imports. For example:

worker.ts:

import { append } from "./util"

self.onmessage = (msg) => {
  self.postMessage(append(msg.data))
}

util.ts:

export function append(s) {
  return s + ' world'
}

Somewhere in main/entry point:

const worker = new Worker(new URL('./worker.ts', import.meta.url))
worker.postMessage('hello')
worker.onmessage = (evt) => {
  console.log(evt.data)
}

And just to be sure, vite.config.ts is:

import { defineConfig } from 'vite';

export default defineConfig({
  optimizeDeps: {
    exclude: ['worker']
  }
});

The browser console reports the following error:

SyntaxError: import declarations may only appear at top level of a module

If you manually inline the append function into worker.ts and remove the import, it works.

[thewtex]

SyntaxError: import declarations may only appear at top level of a module

const worker = new Worker(new URL('./worker.ts', import.meta.url))

Adding { type: 'module' } may be needed?

---

This is similar to #8427. new Worker(new URL('./foo.js', import.meta.url)) is not supported by the dep optimizer.

A workaround is to set optimizeDeps.exclude: ['three-gpu-pathtracer'].

Maybe a package should be able to declare to vite that it should be excluded from optimizeDeps in its package.json? So everything that depends on a given package, three-gpu-pathtracer in this case, does not need to make an entry in optimizeDeps.exclude?

[thewtex]

Maybe a package should be able to declare to vite that it should be excluded from optimizeDeps in its package.json?

Even better, vite analyzes dependencies for new Worker(new URL('./*.worker.js', import.meta.url)), and does not try to optimized the dependency. If / when the functionality is available, the optimization can be re-enabled.

[guillempuche]

With the raise of first-offline database solutions like https://github.com/rhashimoto/wa-sqlite, https://github.com/powersync-ja/powersync-js/blob/main/README.md, https://github.com/sqlite/sqlite-wasm, this is a more pressing issue.

[GibreelAbdullah]

Workaround is not working for me.

I am trying to use - sql.js-httpvfs in a svelte project.

My vite.config.ts looks like this.

export default defineConfig({
    plugins: [sveltekit(), purgeCss({
		safelist: {
			// any selectors that begin with "hljs-" will not be purged
			greedy: [/^hljs-/],
		},
	}),],
    optimizeDeps: {
        exclude: ["sql.js-httpvfs"],
    }
});

I get the following error

ReferenceError: Worker is not defined
    at t.createDbWorker (/home/alfi/Projects/Hadith-Hub/node_modules/sql.js-httpvfs/dist/index.js:1:6857)
    at start (/home/alfi/Projects/Hadith-Hub/src/routes/lang/+page.svelte:21:24)
    at eval (/home/alfi/Projects/Hadith-Hub/src/routes/lang/+page.svelte:37:2)
    at Object.$$render (/home/alfi/Projects/Hadith-Hub/node_modules/svelte/src/runtime/internal/ssr.js:174:16)
    at Object.default (/home/alfi/Projects/Hadith-Hub/.svelte-kit/generated/root.svelte:79:138)
    at Object.default (/home/alfi/Projects/Hadith-Hub/src/routes/+layout.svelte:187:44)
    at eval (/home/alfi/Projects/Hadith-Hub/node_modules/@skeletonlabs/skeleton/dist/components/AppShell/AppShell.svelte:58:145)
    at Object.$$render (/home/alfi/Projects/Hadith-Hub/node_modules/svelte/src/runtime/internal/ssr.js:174:16)
    at eval (/home/alfi/Projects/Hadith-Hub/src/routes/+layout.svelte:150:206)
    at Object.$$render (/home/alfi/Projects/Hadith-Hub/node_modules/svelte/src/runtime/internal/ssr.js:174:16)

I am kinda new to this, any help is appreciated.

[mysterybear]

Yeah I'm having a horrible time trying to use a worker importing Three.js stuff using import statements and it causing the main thread to think that document is not defined? Not sure what's going on...

Edit: calling new SharedWorker instead of new Worker seems to resolve the issue for me? O.o

Edit edit: ah I don't want to use a SharedWorker so that's not ideal D:

Edit edit edit: I was checking window within some function in my code, this was causing my problem (even checking if window undefined from my worker) sorry for the noise, my thing is resolved

[DanielHabenicht]

Also plucking my hair out while trying to make https://github.com/mmomtchev/sqlite-wasm-http work.
Getting Cannot use import statement outside a module no matter what I try to change.

[mysterybear]

@DanielHabenicht I ended up having more issues with this and solved it by splitting the repo into subpath exports and exporting "worker utils" rather than the full worker constructors and letting my consuming app code create the worker using the utils (this latter part might not be as necessary, the first part with the sub-path'ing seemed to help stop Vite from bundling up unnecessary code into the bundle from spaghetti'ish, uncareful imports in the codebase)

e.g. https://github.com/theopensystemslab/buildx-core/blob/c5424ce9fccce93a106edb2d57edaa613c9e04fa/package.json#L5-L16

https://github.com/theopensystemslab/buildx-core/blob/c5424ce9fccce93a106edb2d57edaa613c9e04fa/vite.config.ts

[DanielHabenicht]

Well, I can not influence what is exported from sqlite-wasm-http. So I am stuck with npm build --watch and http-server for now. But thanks anyway!

[TvojTatk0]

Hi. We have some workers with wasm and we solved this issue with worker inline import

import SomeWorker from './SomeWorker?worker&inline';

It is not ideal but it works (as it's described here #15547). One of our customer have integrated our library into application on server with strict CSP. When he wanna use it, he need to allow worker-src blob: and unsafe-eval which is considered as a security risk and it's no go. Is there a way how to use workers and avoid worker-src blob: and unsafe-eval at the same time? Thanks (non of the workaround worked for us so far)

[stefanak-michal]

Hi. I have the same problem with using Vite+React and library memgraph orb.

It generates error:

Failed to load module script: Expected a JavaScript module script but the server responded with a MIME type of "text/html". Strict MIME type checking is enforced for module scripts per HTML spec.
http://localhost:5173/node_modules/.vite/deps/process.worker?worker_file&type=module

Adding library to optimizeDeps.exclude didn't help.

memgraph/orb#94 (comment)