Describe how to include/build this into an actual android app

[degill]

Hello there :-)

I want to build an Android Application which produces MIDI events (from an on-screen virtual keyboard) and plays these MIDI notes with a custom sf2 soundfont. Therefore I stumbled upon fluidsynth which seems to be what I am looking for.

Sadly its not easy to compile it for android - especially since I have not enough knowledge in doing these kind of things.

I successfully included another fork of fluidsynth into my project: fluidsynth-android. But it does not have support for actually playing anything since the Audio Drivers aren't included. (Or maybe I am just using it wrong?)

Then I stumbled upon your repositories and this one here seems to have support for OpenSL ES. But I don't know how to actually build and/or include it in my android app.

I know this section of GitHub is for issues - but I thought I reach out to you and ask :-) Maybe you can help me.

[atsushieno]

My opensles implementation is exactly what you want :-) Actually I also found your fork which strips glib dependencies which would be suitable (or even the best fit) to incorporate my changes.

Have you had a look at my other project? https://github.com/atsushieno/android-fluidsynth/ It might give you some hints on how I built the entire library.

And since my application is written in C# (as I'm also in Xamarin.Android land), these are the only examples I can show you - https://github.com/atsushieno/nfluidsynth/blob/e5b4a79442ec03737f056fc0a2e102e9854cf4ed/NFluidsynth.MidiAccess.Shared/FluidsynthMidiAccess.cs and https://github.com/atsushieno/fluidsynth-midi-service/blob/master/FluidsynthMidiServices/FluidsynthMidiReceiver.cs

(I'm using libfluidsynth.so as cross-platform synthesizer, so I wrote common code for the desktop library, and then reused it on Android.)

If you want to import my additions to your source tree, the first thing would be to get the diffs. Here is the diff between f52597b and a665cc7 (before my changes until opensles deriver addition), which I believe is the cleanest set of changes: https://gist.github.com/atsushieno/539e4e14d4eafcad724e446ad73dc1ad

After these changes I added support for custom SF loader (as there is no decent way to provide file path to native libs in Android), but the original fluidsynth developers seem to have added the same functionality so I'm not sure if you want them.

Unfortunately, my implementation is based on somewhat old source from sourceforge, and the build system had changed there (they switched to cmake, while mine is based on autotools and cerbero) which makes it difficult to catch up with the upstream (cerbero, cmake and Android NDK r15+ don't blend well). It is my ongoing attempt when I have free time (which is kind of pushed away until I finish my ongoing book project...).

So far I post this for some inputs for you. You are welcomed to keep asking questions here :-)

[degill]

Thank you very much for your quick reply!

I tried incorporating your changes (described within your patch) into the source of the other fork since that fork is compiling correctly for android (at least for me), which seemed like a good starting point.

But I can't get it to compile. Here are the changes I made:

1. I omitted changes to the following files, since they are not present in fluidsynth-android:

• configure.ac
• src/CMakeLists.txt (fluidsynth-android uses its own CMakeLists.txt where I only added src/drivers/fluid_opensles.c and src/drivers/fluid_adriver.c to the source list)
• src/Makefile.am
• src/config.cmake
• src/utils/fluid_sys.c(it seems you 'only' added logging statements; I have omitted them for now)

2. I added the new file src/drivers/fluid_opensles.c and made the appropriate changes to src/drivers/fluid_adriver.c:

   1. src/drivers/fluid_adriver.c: I removed all other audio driver settings (ALSA, PulseAudio, etc.) and added only those which you added for OpenSL ES (without all #if OPENSLES_SUPPORT directives since I definitely want to include them)
   2. src/drivers/fluid_opensles.c: I added the whole file, but made minor changes because in fluidsynth-android some APIs changed.

At this point I can not build fluidsynth-android with your additions.

I am thinking about setting up a repository on GitHub to have a better place discussing errors and stuff. Only if you're interested in it :-)

[atsushieno]

I guess those changes you made are almost all. Maybe create a git branch and push to your existing repo? I guess that's the easiest :-) (You can remove it later, if you want)

[degill]

I started development on this project here.

It consists of a fresh Android project setup via AndroidStudio 3.0, VolcanoMobiles version of fluidsynth and your changes/additions to support OpenSL ES. Its not compiling right now, but that seems to be a good base to start fixing :-)

My C/C++ knowledge seems to be not enought to fix all syntactic/semantic errors though 🤔

[atsushieno]

That's type mismatch: In your code, fluid_opensles_audio_run() needs to return void* instead of void.

And you'll have to add -lOpenSLES to the linker somewhere (I have no quick idea on where to put it).

[atsushieno]

I got it build successfully with this patch: https://gist.github.com/atsushieno/950eb50626e0009d5e919588a0ca9ef9 (There may be unnecessary changes.)

[degill]

This is great! Thank you very much.

I just pushed the changes. I will now extract the fluidsynth part into its own module inside the project to separate it from the sample app.

I am not sure though why you wanted to remove ${src_DIR}/drivers/fluid_adriver.c from the /android/CMakeLists.txt file. Isn't the fluid_adriver.c necessary?

[atsushieno]

It was specified twice, reported by the compiler.

[degill]

Yes, one of it was from the adjustments made by VolcanoMobile here and the other one was from the original fluidsynth repo here.

I removed the one made by VolcanoMobile since its a NO-OP and I thought the original implementation is somewhat necessary.

[degill]

I stopped extracting the library into its own gradle-module since it seems to be quite complicated to reference a native module from another module which also has a native part all inside the same project.

What I wanted to do:

1. Have the fluidsynth part in its own gradle module
2. Have another module, which will be my sample app, called 'app' (the default name for android applications created by AndroidStudio)
3. Have 'app' depend on the fluidsynth-module
4. Implement a JNI layer inside the sample app to use fluidsynth

I was stuck at 3. My JNI layer could not find the headers provided by fluidsynth.

[degill]

I added some sample code to load a soundfont from the assets and have a noteOn play on a button press.

Currently the app constantly logs warnings from OpenSL ES:
W/libOpenSLES: Leaving BufferQueue::Enqueue (SL_RESULT_BUFFER_INSUFFICIENT)

And the sound sometimes gets 'swallowed' or simply stumbles. Do you have, by any chance, an idea? :-)

NOTE: I know that by this change the project mixes sample-app-code with library-code. But I couldn't separate it while still be able to build the project.