Support Web Platform

[tanersener]

Provide scripts for a WASM based implementation.
Do no publish binaries if it is not possible to prevent end-users using them on their devices directly. That can create legal problems for us (maintainers).

[Larpoux]

Hi guys and girls,

Some developers requested a Web support for FFmpeg. (Specially on Flutter FFmpeg, but I post here because the question is more general than just flutter).

Today I was almost ready to work on this task, but I am realizing that I do not understand the needs and so, I cannot do something useful.

In fact, I have not a clear understanding of what must be done :

• FFmpeg is a magic tool which handle input files and create output files.
• But a web browser cannot handle files. They do not have any access to a file system. Of course we can emulate a File System in memory (WASM does that), but I do not understand what is the point to do that :
  • decode/encode a sound file found on the web to playback it has probably no interest : I cannot believe that someone put on internet a sound which is not directly playable by all the browsers.
  • decode/encode a sound file recorded on the browser is also probably stupid, because the result will be just for the browser need.
• Is the need for Electron ?
• Is the need for hybrid App, like Cordova ?

Please, confirm that you need this feature and for what reason.

[Larpoux]

OK, Guys & Girls,

if the goal is really to be able to use FFmpeg in a Javascript code (and not for Electron or Cordova),
I can propose something very simple :

• Implement an API verb to create a File from a Javascript ArrayBuffer (or perhaps from a Javascript Blob).
  This file will be created in the Virtual File System emulated in memory by Emscripten. (I am sure that this simple verb is already implemented in many places)

• Implement an API verb to create a Javascript ArrayBuffer (or perhaps a Javascript Blob) from a virtual file managed by Emscripten. (I am sure that this simple verb is already implemented in many places)

• Implement the verbs that will access the Virtual files :

  • ffmpeg()
  • ffprobe()

To start, I will create just a min bundle. We will implement other bundles (audio, video, gpl, full, ...) later.

[yelkamel]

Hello,

For my case, I have a back office of our app "evolum" where we upload audio.
And to simplify the process, I want to be able to get the duration of the audio and the size (Mb) when the file is picked.

For now, I found a solution to get the duration with the package just_audio.
But for the size, people have to put it mannually, I try with this function:

` static Future getEvoSize(Evo evo) async {
final url1 = "https://evolum.s3.eu-west-3.amazonaws.com/${evo.filename}";

final http.Response r = await http.head(
  Uri.parse(url1),
  headers: {'Access-Control-Allow-Origin': '*'},
);
final num totalSize = int.parse(r.headers["content-length"]);

return totalSize / (1024 * 1024).toDouble();

}`

It's work on mobile but with phone I have CORS problem...

so for now internal user have to check the size and write it mannual so sometimes there is mistake ....

[tanersener]

A good use-case, thanks for sharing @yelkamel

[Larpoux]

Thank you @yelkamel for your post.
When I look to your need, I realize now that very often (always) the App does not have the data in a buffer, but just the URL (The URL being for something, probably remote, on the web).

Having to download the data to create a temporary file in memory is not convenient.
I suggest that we do not create the two verbs that I proposed above, to create a file from memory buffer and read a file into a memory buffer, but instead two verbs to :

• create a temporary file from an URL
• create a temporary URL from a temporary file

Example in Javascript (in Dart it will be very similar) :

await FFmpegKitConfig.createTemporaryFile( 'https://evolum.s3.eu-west-3.amazonaws.com/filename', 'file1.mp4');
FFmpegKit.executeAsync
(
          '-i file1.mp4 -c:v mpeg4 file2.mp4', 
          async (session) => 
          {
                    aTemporaryURL = await session.createTemporaryURL(file2.mp4);
                   // Do something with `aTemporaryURL`, like play it with τ Sound
          }
);

If the App just wants to get the duration (like @yelkamel), it would be very much simpler if he/she only need to give the URL without having to process temporary files:

FFprobeKit.getMediaInformationAsync
(
        'https://evolum.s3.eu-west-3.amazonaws.com/filename', 
         async (session) => 
         {
                const information = await session.getMediaInformation();
         }
);

[Aminadav]

You asked for use cases. We need audio from users for transcribe on the web. If they choose a video, we need to extract the audio first by using this library.

[matthieudelaro]

You asked for use cases: clip an audio file between two given timestamps, with a precision of a millisecond (basic web API fails for this level of precision)

[matthieudelaro]

OK, Guys & Girls,

if the goal is really to be able to use FFmpeg in a Javascript code (and not for Electron or Cordova), I can propose something very simple :

* Implement an API verb to create a File from a Javascript ArrayBuffer (or perhaps from a Javascript Blob).
  This file will be created in the Virtual File System emulated in memory by Emscripten. (I am sure that this simple verb is already implemented in many places)

* Implement an API verb to create a Javascript ArrayBuffer (or perhaps a Javascript Blob) from a virtual file managed by Emscripten.  (I am sure that this simple verb is already implemented in many places)

* Implement the verbs that will access the Virtual files :
  
  * ffmpeg()
  * ffprobe()

To start, I will create just a min bundle. We will implement other bundles (audio, video, gpl, full, ...) later.

@Larpoux what is the min bundle you refer to please? Any update on this?

[remcova]

@tanersener @Larpoux Any updates on this? I'm willing to contribute because I also need the web support asap for my project.

[Larpoux]

I am sorry but I am not a Web expert.
What I tried to say is that FFmeg is a magic utility which handles input files and create output files.
But Web App run in a sandbox and do not have access to the files system.
So I think difficult to offer an API 100 % compatible with Flutter Mobile.

But a web app can create and read a temporary memory object that can be accessed using a regular URL (something like "memory://foo.bla". Perhaps ffmpeg-kit should work on these temporary objects. This is what I did for Flutter Sound on Web.

But in fine (this is Latin 😁) the users who need ffmeg-kit knows better than me what they need.

Note : temporary File System can be emulated in memory by WASM.
But this pseudo - file system is probably not directly usable by a regular web app.
We probably want to offer a way to create such a file from a regular URL, and also the opposite : create an object accessible by a standard URL, from a temporary WASM file.

The problem to define a good API is certainly the real challenge. The work itself is probably just a compilation of ffmpeg to WASM code.

[remcova]

I've posted the same answer in this issue https://github.com/tanersener/ffmpeg-kit/issues/8 but I think it belongs here:

I've looked into the web implementation first for CHROME extensions. The option I came across was using ffmpeg in WASM. Unfortunately, Manifest V3.0 brings a lot of issues. With Manifest V2.0 you can get it working but it will stop working in January 2023 in Chrome, so this won't be worth the effort to implement.

To port it to Manifest V3, the code will need to be placed in a worker which doesn't have access to the DOM (document). Also, it can't be published on Chrome Store due to Manifest V2.

I haven't found an alternative yet to create an implementation for the web, it looks like this will be a pain in the ass

[duttaoindril]

I've posted the same answer in this issue #8 but I think it belongs here:

I've looked into the web implementation, the option I came across was using ffmpeg in WASM. Unfortunately, Manifest V3.0 brings a lot of issues. With Manifest V2.0 you can get it working but it will stop working in January 2023 in Chrome, so this won't be worth the effort to implement.

To port it to Manifest V3, the code will need to be placed in a worker which doesn't have access to the DOM (document). Also, it can't be published on Chrome Store due to Manifest V2.

I haven't found an alternative yet to create an implementation for the web, it looks like this will be a pain in the ass

Don't manifests only matter for chrome extensions?

[remcova]

I've posted the same answer in this issue #8 but I think it belongs here:
I've looked into the web implementation, the option I came across was using ffmpeg in WASM. Unfortunately, Manifest V3.0 brings a lot of issues. With Manifest V2.0 you can get it working but it will stop working in January 2023 in Chrome, so this won't be worth the effort to implement.
To port it to Manifest V3, the code will need to be placed in a worker which doesn't have access to the DOM (document). Also, it can't be published on Chrome Store due to Manifest V2.
I haven't found an alternative yet to create an implementation for the web, it looks like this will be a pain in the ass

Don't manifests only matter for chrome extensions?

Yes, I've edited my answer. I was looking for an implementation that is also supported when it's a chrome extension.

[tanersener]

Yes, I've edited my answer. I was looking for an implementation that is also supported when it's a chrome extension.

Thanks for the update. Does this mean that a web application living on www.website.com can load www.website.com/ffmpeg-kit.wasm successfully in Manifest V3?

[loic-hamdi]

Is Web support still something you are working on?

[maRci002]

Fireship has a great video on this topic: https://www.youtube.com/watch?v=-OTc0Ki7Sv0

Here is the source code:
https://github.com/fireship-io/react-wasm-gif-maker/blob/main/src/App.jsx

[polarby]

I guess we could just merge https://pub.dev/packages/ffmpeg_wasm with this package to support web?

[maRci002]

I guess we could just merge https://pub.dev/packages/ffmpeg_wasm with this package to support web?

I was the one who updated the ffmpeg_wasm Dart library (redleafsofts/flutter_ffmpeg_wasm#1). Initially, I considered writing some Dart wrappers around it, but that would only benefit the Flutter ffmpeg_kit version. Therefore, in my opinion, we should first write a JS wrapper around ffmpeg.wasm. This way, React Native can benefit from it too. Afterwards, we can write Dart interop methods for the benefit of Flutter.

[thomasf1]

Hey guys, has there been any progress on this? Would be ideal for Expo / React Native :)

[Aminadav]

No progress yet

[thomasf1]

I´m trying to intergrate it the other way, doing a Expo module that uses ffmpeg-kit for native and ffmpeg_wasm for the web. Unfortunately ffmpeg_wasm with it´s service worker is quite tricky to get working on expo as the file paths of the component are rewritten by expo-router and a couple of other issues...