[ENH] Web support

[Larpoux]

Hi @tanersener ,
Hi everybody,

I am the Flutter Sound main maintainer, and today someone post something about the need of FFmpeg on Flutter Web.
I think that several people already ported FFmpeg to javascript, using webassembly and Emscripten.
Is there any reason for Flutter FFmpeg not being supported on Flutter Web ?
Will you be interested by someone (me?) doing this port and do a Pull Request, in the next few weeks ?

Thank you again for this terrific project.

[tanersener]

I shared my plans in #156. I'm a few months behind. Still working on the ffmpeg-kit-flutter native modules.

But, you're free to implement it and create a PR. From my perspective the main difficulty is around implementing packages. I also have concerns about licensing. If your PR addresses them then I don't see why it can't be merged.

[Larpoux]

If I understand correctely, you are working on Flutter FFmpeg linked with ffmpeg-kit.
Does it mean that it is not a good idea to do developments on the master branch ?
Should it be better to develop on the development branch ? And is the development branch stable enough to develop on it ?

[tanersener]

No, I'm not working on flutter_ffmpeg. I'm implementing a new flutter plugin based on flutter_ffmpeg under ffmpeg-kit. I'll retire flutter_ffmpeg when it is ready.

[Larpoux]

I am not sure to be pleased to work on a development for a plugin which is supposed to be retired soon. 🙄

[tanersener]

Fair enough 👍🏻 But that shouldn't stop you if you want to contribute. The new plugin ffmpeg-kit-flutter doesn't support web too. You can implement web support there.

[Larpoux]

This sounds good. I think it will be possible to work , you and me, on two very different things of the same project.
Did you shared your new project on github or elsewhere ?
I will be free to work on my contribution in something like one week or two.

[tanersener]

Yeah, ffmpeg-kit has a branch named flutter-support. The plugin is there. It's API and Android module is ready. I plan to work on the iOS module in the coming weeks. I just have some problems about my macOS setup.

ffmpeg-kit-test also has a branch named flutter-support. There is a working test-application there. It can be used for testing. You'll see that it is the same test-application we have for flutter. With a few additional features.

[Larpoux]

I am glad that the API is stabilized. Because the first task that I will have to do is to create a flutter interface which will be common for ios/android and web.
(this is the Google recommandations to support web).

Now I let you work. I will not disturb you anymore until 2 or 3 weeks.

[tanersener]

Thanks for working on this 👍🏻

[Larpoux]

I will have some difficulties when I will implement ffmpeg-kit on web :

• We want the App run same on ios/android and web
• FFmpeg is a utility that read input files and write output files. But Files are not supported on a Browser sandbox.

On Flutter Sound I tried to emulate temporary files with internal memory URI. But I am not completely pleased by what I did.

If someone has suggestions ... it will be helpful.

[tanersener]

We want the App run same on ios/android and web

Well, we must provide the same api to the users in the end. But we don't have to worry about implementing all API methods in the first iteration. We can take an iterative approach there. There are a lof of details inside the API.

FFmpeg is a utility that read input files and write output files. But Files are not supported on a Browser sandbox.

Can any of these File System APIs help you with that?

[Larpoux]

Can any of these File System APIs help you with that?

Yes. ffmpeg.js and ffmpegwasm use also the "in memory" MEMFS file system emulated by emscripten.
BTW : Do you prefer to use one of these libraries (LGPL), or do you prefer to completely master the library code and compile ourself the FFmpeg code with emscripten ?

Also, I am realizing that I will not have to build the Plateform Interface. You already did it 7 days ago 👍 . Cool!

[tanersener]

Well, this is the part which answers your first question, why web support wasn't implemented before?

There are two main reasons.

• flutter_ffmpeg (and ffmpeg-kit) doesn't provide ffmpeg alone. It also builds more than 25 external libraries and enables them in ffmpeg. Depending on the external libraries enabled there are 8 different packages users can install. So, adding a new platform also means supporting those external libraries on that new platform. Which requires a lot of effort.

• flutter_ffmpeg's parent mobile-ffmpeg and ffmpeg-kit both don't build executable binaries ffmpeg and ffprobe. They build shared or static ffmpeg libraries (libav...) that can be imported by apps. For a new platform the approach should be the same. Why? Because this is the objective of these projects. Building ffmpeg libraries and creating a platform specific mobile-ffmpeg or ffmpeg-kit wrappers to use them. flutter and react-native plugins are just wrappers written on top of native mobile-ffmpeg or ffmpeg-kit wrappers. For web support the same principle must be followed. libav... libraries of ffmpeg must be ported to web. And, I don't think existing js and webasm ports do that.

Based on these two approaches taken when the projects were originally created, we don't too many options unfortunately. As I said in my previous post, we can take an iterative approach for web support. We don't have to implement everything in one go. But also it doesn't make sense to spend time to implement something that is not in line with project goals and will be removed later. So, I don't want to limit your options or ask you to spend on something that requires a lot of effort. But this is where we are in terms of supporting a new platform.

[Larpoux]

Can any of these File System APIs help you with that?

1. The problem is that emscripten MEMFS is not easy to be accessed directly from Flutter. We will have to offer Flutter verbs to open, read, write and convert MEMFS files to/from flutter ByteArrays. Those verbs should probably be offered by a separate plugin, because they are not specific to FFmpeg. But in final, the App code will be different than with ios/android, where it will use directly Dart IO.

2. Actually, Mobile FFmpeg is completely independent from Flutter, and can be used from React Native. I like this architecture/design. I guess that in a similar way, you will want to be able to access your Javascript libraries from other frameworks than flutter (React-js, Vue-js, ...)

[tanersener]

We will have to offer Flutter verbs to open, read, write and convert MEMFS files to/from flutter ByteArrays. Those verbs should probably be offered by a separate plugin, because they are not specific to FFmpeg. But in final, the App code will be different than with ios/android, where it will use directly Dart IO.

Do we have to implement them in android/ios? We can have helper API methods specific to only web platform.

Actually, Mobile FFmpeg is completely independent from Flutter, and can be used from React Native.

Yeah, in the end web platform should be accessible directly without flutter. Similar to other platforms supported under ffmpeg-kit. It is one of the project goals.

[Larpoux]

As I said in my previous post, we can take an iterative approach for web support. We don't have to implement everything in one go.

👍 I really like this way of doing things : better to have a clear idea of what we want to have in final, but we do not have to implement everything in one go. I just like to have a complete view of what we want before starting. This is what is called Functional Specifications in my country.

[Larpoux]

One of the problems I see, is that Mobile FFmpeg is huge with all its libraries.
If we ask the Flutter developer to import all the libraries in his/her html page, it will be long to load the App.
If we ask the Flutter developer to use a loader like Webpack, then it could be difficult to manage that for a flutter user who is possibly not an expert with Web developments.

Note:
I did some search with Google and it seems that Webpack and Flutter are not friends together.

[tanersener]

At the moment, in flutter_ffmpeg, we're not asking the developers to import another file to use one of the external libraries. The entry point to the library is always single and the same for all of them. If there is a separate file that must be imported for that package, we're doing that in library's dependency files. So, for web we should aim the same. And, using Webpack or another bundler is not crucial right now. It can be changed later.

Also, we cannot solve all problems for the developers. All these ffmpeg projects implement packages to give users different options. But users don't have to use them. There is always an option to build your own version of the library and optimise it not to include external libraries/features you don't need. That is an advanced feature of course. But it is there.

[tanersener]

@Larpoux Are there any updates on your end regarding web support. I completed testing the new ffmpeg-kit-flutter plugin. I plan to release it soon with three platforms (android, ios, macos).

[Larpoux]

@tanersener ,

I am sorry, I was busy those last weeks, releasing the first version of an android/ios app that I have worked on during 3 years,
and by forking the Flutter Sound project to a GPL branche.

Actually, some τ Sound users have problems to build their App for an IOS emulator when upgrading their Flutter SDK to 2.,5, and it seems that the responsible is MobileFFmpeg (i386, not M1)

We actually have a big problem : Flutter Sound and τ Sound FULL version depend on Flutter FFmpeg, and Flutter FFmpeg depends on Mobile FFmpeg, and Mobile FFmpeg is not supported any more. This is a serious problem for us.

Today, I downloaded MobileFFmpeg, to see if I can build a iOS library that can be linked to τ sound without errors.
The build, just for ARM64 and just for iOS has actually run for 5 hours ... and is not finished !!!!
.....
TLDR; the short version is : "No, nothing ready from me ready for release".
.....
BTW, I think that supporting Flutter Web is not the good priority for FFmpeg Kit, because of the lack of a packager on Flutter, and the huge number of libraries handled by FFmpeg-kit.
Probably better to support first JS (React JS, Vue JS, ...) and after, seing what can be done for flutter.

Take care, @tanersener ,
I have not much time, but perhaps I will still be able to give you something in the next month.

[tanersener]

Thanks for the update @Larpoux. I know web is not an easy platform to work on. I really appreciate your effort on it.

Let me know if you need help about the mobile-ffmpeg stuff.

[Larpoux]

Hi Guys and Girls waiting for a FFmpeg support by Flutter App on web.
Could you look to this thread and eventually post something on it ?

[tanersener]

We are retiring the project. Consider switching to FFmpegKit.