Relative imports in user code combined with imports from wasp/server/operations break wasp start

[sodic]

This is a problem we've had for a while but haven't properly documented. Users have also been known to hit it (example below).

Steps to reproduce

Create a project with Wasp 0.16 and make the following edits:

./src/main.wasp

app nuno {
  wasp: {
    version: "^0.16.0"
  },
  title: "nuno"
}

route RootRoute { path: "/", to: MainPage }
page MainPage {
  component: import { MainPage } from "@src/MainPage"
}

query getAllTasks {
  fn: import { getAllTasks } from "@src/error/queries"
}

./src/queries.js

import { sayHi } from './util'
import { getAllTasks as getAllTasksWasp } from 'wasp/server/operations'

export const getAllTasks = () => {
  console.log(sayHi, getAllTasksWasp)
  return []
}

./src/util.js

export function sayHi() {
  console.log('`Hello!')
}

Run wasp start and you should see the error:

[ Server ] > [email protected] start
[ Server ] > node --enable-source-maps -r dotenv/config bundle/server.js
[ Server ]
[ Server!] node:internal/errors:496
[ Server!]     ErrorCaptureStackTrace(err);
[ Server!]     ^
[ Server!]
[ Server!] Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/tmp/nuno/.wasp/out/sdk/wasp/dist/ext-src/error/util' imported from /tmp/nuno/.wasp/out/sdk/wasp/dist/ext-src/error/queries.js
[ Server!]     at __node_internal_captureLargerStackTrace (node:internal/errors:496:5)
[ Server!]     at new NodeError (node:internal/errors:405:5)
[ Server!]     at finalizeResolution (node:internal/modules/esm/resolve:327:11)
[ Server!]     at moduleResolve (node:internal/modules/esm/resolve:980:10)
[ Server!]     at defaultResolve (node:internal/modules/esm/resolve:1206:11)
[ Server!]     at ModuleLoader.defaultResolve (node:internal/modules/esm/loader:404:12)
[ Server!]     at ModuleLoader.resolve (node:internal/modules/esm/loader:373:25)
[ Server!]     at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:250:38)
[ Server!]     at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:76:39)
[ Server!]     at link (node:internal/modules/esm/module_job:75:36) {
[ Server!]   url: 'file:///tmp/nuno/.wasp/out/sdk/wasp/dist/ext-src/error/util',
[ Server!]   code: 'ERR_MODULE_NOT_FOUND'
[ Server!] }
[ Server!]

Explanation

Wasp bundles the server with Rollup, meaning that server code (and the user code it imports, e.g., operations) can use extensionless relative imports (e.g., import { sayHi } from './util').

Wasp doesn't bundle the SDK (the wasp package), meaning that code in the SDK cannot use extensionless relative imports.

Since Wasp copies all user code into the SDK (more precisely, into wasp/ext-src), it should mean that user code cannot use extensionless imports after all. But, as long as nothing references the offending code, no one will notice and everything will work.

In practice, this means that user code can use extensionless relative imports as long as some other part of user code doesn't indirectly import the offending code from wasp/ext-src (e.g., by importing wasp/server/operations).

You could be working on your app for weeks, happily using extensionless relative imports. And then, one day, you import something from wasp/server/operations, which breaks your app in 100 places that have nothing to do with the change you just made (this is what happened to a user with OpenSaas)

This behavior was also the root cause of this bug (albeit with slight differences): #2096 (comment)

Why isn't this caught during SDK build

At the time of writing, the SDK is built with TypeScript.

TypeScript can resolve extensionless files and doesn't alter import paths when outputting JS code, meaning that the error can only be caught in runtime.

Solution

The long term solution is figuring out how precisely how we want to treat and build the SDK.
The short-term solution is including the wasp package in the server's bundle (as we do for the client). @infomiho talks about it here: #2446