Unbundled JavaScript builds

[texodus]

This PR is a rewrite of a Perspective's initialization API, in an attempt to address framework and build tool integration complexity. This has been a frequent source of Community frustration #2725 #2795 #1734 #2796 etc.

@finos/perspective-esbuild-plugin and @finos/perspective-webpack-plugin have been removed. In their place is an explicit WebAssembly initialization API inspired by DuckDB Wasm which implements a similar API. To use the new API, you must acquire the .wasm binary assets via your bundler's asset packaging, then initialize Perspective via init_client() and init_server() methods. For Vite, this looks like this:

import perspective from "@finos/perspective";
import perspective_viewer from "@finos/perspective-viewer";

import SERVER_WASM from "@finos/perspective/dist/wasm/perspective-server.wasm?url";
import CLIENT_WASM from "@finos/perspective-viewer/dist/wasm/perspective-viewer.wasm?url";

await Promise.all([
    perspective.init_server(fetch(SERVER_WASM)),
    perspective_viewer.init_client(fetch(CLIENT_WASM)),
]);

The downside of this API is its verbosity - it is regrettably noisy, with IKEA-esque nescience. It is also a non-trivial breaking change for any ESM/bundler integration with Perspective. The upside is it requires no plugin at all to import with a bundler, as Perspective now exports pure JavaScript ES modules which should be compatible with any bundler.

Documentation

With this change comes a rewritten User Guide, extracted from docs.rs by popular demand. The beta release can be found here, note it is currently not linked as it describes an unreleased Perspective version!

https://perspective.finos.org/guide/

We've added new sections covering bundling with various bundlers including Vite, ESBuild and Webpack, and added a full-blown Vite example in /examples/vite-example as well.

Deprecated

To be decided, potentially two API deprecations in addition to the API changes above.

@finos/perspective-esbuild-plugin and @finos/perspective-webpack-plugin as mentioned. They will not work with v3.3.0 Perspective (at least - not as described in their docs), so v3.2.1 would be the last release for these. However, they should no longer be necessary, so if there is a strong argument to keep these, I question the value proposition of the unbundled build at all.

With this change, inline build of Perspective are no longer really necessary, as you may now simply inline the .wasm assets in yourself with your bundler's own inlining feature. We'd like to deprecate these entirely, but for now we're still building them (with an explicit deprecation warning in the documentation and console).

Changes

• New bundle process, as described above.
• New mdbook docs site.
• Fixed perspective-viewer overscroll behavior
• Fixed rust build warnings.
• Debuggable Web Worker deploy internals.
• The internal build process is ultimately much simpler - fewer modules, fewer entry points, fewer variations to test and faster development cycles.
• Better type exports, as the type generation build process now has more complete end-to-end type information.

Partially enabled by #2885, which removed one of the last complex Wasm/JavaScript bridge calls.

Community Call for Testing

We are interested in soliciting community feedback for this change, especially if you've had integration issues with previous Perspective builds. You can try out the new unbundled API today, via the next tag on NPM, by specifying next instead of a version in the CLI or package.json:

npm install @finos/perspective@next

Documentation for the new bundling workflow can be found on the new User Guide. New examples (including the Vite example) can be found in the branch /examples directory.

[MPizzotti]

Amazing Work!
I'll try the new changes and give you feedback by end of the week, with the following tech stack:

• Vite 5.4.11
• svelte 4
• sveltekit 2
• Perspective + perspetive-viewer CSR (not SSR)
• apache arrow IPC Stream API

[jordanskole]

This seems to be much better overall, however there are still issues with typescript and node. The biggest one is that if you have tsc configured to use import statements/ESM with a bundler (tsc) you can't import the node library.

By making node a CommonJS module (require) it creates a lot of snowflake configuration just to import WebSocketServer and table.

This is a pain since I would say the majority of libraries support both or just ESM these days.

[jordanskole]

Okay I think I see what is going on. It looks like /node is trying to export types from /dist/esm/perspective.node.d.ts which doesnt exist/wasnt built.

I was able to work around this by re-exporting table in a local client.ts file like so:

// @ts-expect-error we have to fix this manually until they do
import { table as pspTable } from "@finos/perspective/node";
import { Client } from "@finos/perspective/dist/wasm/perspective-js.js";

const table = pspTable as Client["table"];

export default {
  table,
};

[MPizzotti]

Amazing Work! I'll try the new changes and give you feedback by end of the week, with the following tech stack:

• Vite 5.4.11
• svelte 4
• sveltekit 2
• Perspective + perspetive-viewer CSR (not SSR)
• apache arrow IPC Stream API

I've successfully managed the migration, it wasn't as easy as i tought but still very smooth.

i have created this helper file for handling the instanziation as a "singleton".
please note i'm still not very proficient in UI and JS, since is not my main work area

import type { PerspectiveClient } from '$lib/tsTypes/perspective_types';
import perspective from '@finos/perspective';
import CLIENT_WASM from "@finos/perspective-viewer/dist/wasm/perspective-viewer.wasm?url";
import SERVER_WASM from "@finos/perspective/dist/wasm/perspective-server.wasm?url";

class PerspectiveImportManager {
    private static instance: PerspectiveImportManager;
    private perspectiveClient!: PerspectiveClient;
    private perspectiveBootStrapped = false;
    private perspectiveClientBootStrapped = false;

    private constructor() {
        // Private constructor to prevent external instantiation
    }

    public static getInstance(): PerspectiveImportManager {
        if (!PerspectiveImportManager.instance) {
            PerspectiveImportManager.instance = new PerspectiveImportManager();
        }
        return PerspectiveImportManager.instance;
    }

    public async getPerspectiveClient(): Promise<PerspectiveClient> {
        if(!browser){
            return this.perspectiveClient;
        }
        if (!this.perspectiveBootStrapped) {
            await perspective.init_server(fetch(SERVER_WASM));
            this.perspectiveBootStrapped = true;
        }
        if(!this.perspectiveClientBootStrapped){
            const perspective_viewer = await import('@finos/perspective-viewer');
            await import('@finos/perspective-viewer-datagrid')
            await import('@finos/perspective-viewer-d3fc');
            await import('@finos/perspective-viewer-openlayers');
            await import('@finos/perspective-workspace');
            await perspective_viewer.init_client(fetch(CLIENT_WASM));
        }
        this.perspectiveClient = await perspective.worker();
        return this.perspectiveClient;
    }
}
const clientManager = PerspectiveImportManager.getInstance();
export const perspectiveClient = await clientManager.getPerspectiveClient();

by doing so i'm fixing SSR problem on perspective-viewer (since in my case i don't need it, i'm using perspective mainly for his dataframe and query capabilites)

for using the module above you can simply run it everywhere (server side, client side, inside a component etc) by simply calling
dataFrame = (await perspectiveClient.table(await inputJSON, options)) as DataFrame;

here's some more examples on how i've used it

    const arrowBuffer = await (await inputResponse).clone().arrayBuffer();
    const worker = perspectiveClient;
    const tablePromised = worker.table(arrowBuffer, options) as MaybePromise<DataFrame>;
    return tablePromised;
};

.

        await component.load(await temptable);

I hope this will help fellow Sveltekit devs

[kryaksy]

I’ve tested this with a repo built using [email protected], [email protected], and [email protected], but I ran into the following errors:

TS2307: Cannot find module '@finos/perspective/dist/wasm/perspective-server.wasm' or its corresponding type declarations.
TS2307: Cannot find module '@finos/perspective-viewer/dist/wasm/perspective-viewer.wasm' or its corresponding type declarations.

I couldn’t find examples in previous comments specifically covering this environment setup (nx, react) or addressing TypeScript issues with these imports. That’s why I wanted to contribute this example. The repo used for testing can be found here: https://github.com/kryaksy/perspective-react-unbundled-build.

Any advice on how to resolve these issues would be appreciated! Thanks.

[yuj2i0]

I’ve tested this with a repo built using [email protected], [email protected], and [email protected], but I ran into the following errors:

TS2307: Cannot find module '@finos/perspective/dist/wasm/perspective-server.wasm' or its corresponding type declarations.
TS2307: Cannot find module '@finos/perspective-viewer/dist/wasm/perspective-viewer.wasm' or its corresponding type declarations.

I couldn’t find examples in previous comments specifically covering this environment setup (nx, react) or addressing TypeScript issues with these imports. That’s why I wanted to contribute this example. The repo used for testing can be found here: https://github.com/kryaksy/perspective-react-unbundled-build.

Any advice on how to resolve these issues would be appreciated! Thanks.

Same issue previously the perspective-web-plugin worked perfectly fine for me, but struggling to upgrade for > 3.2.1 onward

[texodus]

Y'all - please stop necroposting to this closed PR. The window for comments on this change has expired - please use the Issue template to file a bug with a repro, or Discussion to ask a question.

@kryaksy I appreciate the repro, but this is too big for me to debug - there are 40 dev dependencies, including 3 different bundlers that I can recognize (webpack, swc, nx) as well as TypeScript stacked on top of Babel. Your error however does not even look like a bundler error - it is a TypeScript error complaining you haven't declared types for whatever format it output by whichever of your bundlers is doing the outputting. Assuming your bundler does not do this automatically you can declare the module or workaround by just @ts-ignore

declare module '*.wasm' {
    const value: <<< I legit don't know what type goes here lol >>>;
    export default value;
}