[TRACKER] Integrating ALSA and Pulseaudio into Termux

[ackalker]

Goals

This issue is for tracking progress on integrating ALSA and Pulseaudio into Termux.
Of course, ALSA in Termux will not have direct access to sound hardware on an Android device unless rooted and the vendor uses an ALSA driver in the kernel. The goal is to have a means to communicate with the OpenSL ES subsystem in Android as efficiently as possible. Currently, this is done by streaming a Pulseaudio sink onto the localhost network. In future, it may be possible to write or adapt a Pulseaudio backend driver or an ALSA plugin to talk directly to OpenSL ES, using the so-called "fast mixer", with minimum latency when possible.

Overview of tasks and status

Building and using the required packages

• libpulseaudio (alsa-{lib,utils,plugins} and pulseaudio packages #825)
  • pulseaudio
  • libsndfile
  • fftw (useful?)
  • libspeex (useful?)
  • libsoxr (useful?)
  • liborc (useful?)
  • libwebrtc-audio-processing (useful?)
• alsa-lib (alsa-{lib,utils,plugins} and pulseaudio packages #825, disabled: not working yet)
• alsa-plugins (alsa-{lib,utils,plugins} and pulseaudio packages #825, disabled: not working yet)
• alsa-utils (alsa-{lib,utils,plugins} and pulseaudio packages #825, disabled: not working yet)

Writing OpenSL ES backend drivers and plugins

• for ALSA
• for PulseAudio

Enabling audio related packages to make use of ALSA and/or Pulseaudio

• cmus (Music touches people's hearts. I need CMUS. #209)
  • libmad (in repository)
  • libvorbis (in repository)
  • libflac (in repository)
  • mp4v2 (needed?)
  • ...
• mpv
• mpd
• espeak
• ...

Detailed description

Building and using the required packages

All packages build without problems on my system. They're mostly in a minimal, PoC state, I'm open for discussion on what extra dependencies to include, any tweaks to make.

libpulseaudio is usable with Pulseaudio enabled packages, even without pulseaudio installed:

• Install the XServer XSDL app from Google Play
• Make sure to add the following to ~/.bash_profile:

export PULSE_SERVER=tcp::4712

• Open the XServer XSDL app
• In Termux, run your Pulseaudio enabled application as usual

Known issue: Sound gets muted when device screen turns off or XServer XSDL is not running in the foreground.

pulseaudio is a really bare-bones, minimal affair, no speex (I doubt if it is worth the CPU resources on Android), no ORC (AFAICT it currently only supports an architecture with MMX/SSE instructions anyway). I've also disabled Neon optimization: the build generates assembly which the assembler seems not to be able to handle (it spews errors about unknown registers, q0 among them). I symlinked the shared libraries into $TERMUX_PREFIX/lib to avoid having to write an ugly wrapper. If there are any conflicting filenames, I'd like to hear about them.
To get sound output from applications running in Termux, you can adapt and run the following script:

setup-pulseaudio.sh:

#!/data/data/com.termux/files/usr/bin/sh

TERMUX_PREFIX=~/../usr

# Do not shutdown server when idle.
sed -i $TERMUX_PREFIX/etc/pulse/daemon.conf \
    -e '/^; exit-idle-time =/{s/^; //;s/ 20/ -1/}'

# When no sinks are available, Pulseaudio automatically adds a null sink named 'auto_null'.

# Stream the 'auto_null' sink over TCP.
# This can be played by Simple Protocol Player app on Google Play.
sed -i $TERMUX_PREFIX/etc/pulse/default.pa \
    -e '$aload-module module-simple-protocol-tcp source=auto_null.monitor port=12345 record=true'

# In addition, or alternatively,
# add a null sink named 'rtp' and stream it over RTP.
# This can be played by VLC, etc.
# sed -i $TERMUX_PREFIX/etc/pulse/default.pa \
#     -e '/^#load-module module-null-sink sink_name=rtp /s/#//' \
#     -e '/^#load-module module-rtp-send source=rtp.monitor$/s/#//'

To play sound from a TCP stream:

• Install Simple Protocol Player from Google Play
• in Termux, run pulseaudio --daemon
• Open the Simple Protocol Player app
• Make sure that IP Address is set to 127.0.0.1 and Audio Port is set to 12345 (or whatever value was set in the setup-pulseaudio.sh script
• Tap the Play button
• In Termux, run your Pulseaudio enabled application as usual

To play sound from an RTP stream:

• Install VLC for Android (or a similar app which can play back RTP streams) from Google Play
• In Termux, run pulseaudio --daemon
• Open VLC or similar media player app
• Choose to play back an RTP stream (how to do this depends on the media player app) from host 127.0.0.1, port ??? (TODO: Add simple way to discover RTP port number)
• In Termux, run your Pulseaudio enabled application as usual

alsa-lib is pretty complete, I've worked around a problem where the path /etc/asound.conf is hardcoded, which may conflict with system ALSA and isn't useful on non-rooted devices anyway.

alsa-utils: Also pretty complete, not sure if we need all of the utilities. Fun tidbit: alsamixer runs really well in Termux terminal emulator, it's even possible to drag with a finger to change the volume, yay!

alsa-plugins: Only built the Pulseaudio plugin. I'm not sure if we need the samplerate plugin because Pulseaudio and OpenSL ES can both resample audio.

Writing OpenSL ES backend drivers and plugins

pulseaudio is one of the targets for a dedicated OpenSL ES driver, I haven't looked into that yet. There appears to be a (reportedly buggy) backend available, maybe we can adapt that.

alsa-plugins is another target for more direct interaction with OpenSL ES, this time as a (pair of?) plugin(s), a virtual PCM and CTL device.

Enabling audio related packages to make use of ALSA and/or Pulseaudio

cmus runs on Termux. Some of its optional dependencies (support for various file types) still need to be packaged.

Resources

ALSA

• PCM External Plugin SDK
• ALSA project - the C library reference: External I/O plugin SDK

Pulseaudio

• Modules

Android

• High-Performance Audio
• High-Performance Audio (2017)
• gkasten/high-performance-audio (code used in Google IO 2014 talk)
• Audio Latency
• Measuring Audio Latency
• SO answer about low-latency playback on Android (IMPORTANT!)
• slesTestFeedback.cpp (FIFO implementation in audio-utils needs porting, uses system APIs which are not exposed by the NDK).
• audio-echo
• native-audio
• Android audio streaming with OpenSL ES and the NDK
• Lock-free audio IO with OpenSL ES on Android

Third-party libraries

• cubeb (cubeb/cubeb_opensl.c)
• openal-soft (Alc/backends/opensl.c)
• opensl_stream (opensl_stream/opensl_stream.c)

[fornwall]

Nice!

I've also disabled Neon optimization: the build generates assembly which the assembler seems not to be able to handle (it spews errors about unknown registers, q0 among them).

This could be due to clang trying to use its integrated assembler. Does adding -no-integrated-as to CFLAGS fix this and allow neon optimizations?

[its-pointless]

Using gcc fixes that neon issue

[ackalker]

Thanks for your very quick reply! I will look into it.

[ackalker]

Woops, sorry about that! Reopened.

[DoktorStein]

Very interesting the fact that this project is beyond Termux scope. It would be great if eventually it became a official port.

[ackalker]

I hope to push the package build scripts to my fork for testing soon, but I'm hitting a problem with a bogus buffer size when using the ALSA Pulseaudio plugin which causes speaker-test to crash with a Floating point exception. Stay tuned 😄

[ackalker]

@DoktorStein Yes, I agree that it goes beyond Termux as such, but as Termux is such a wonderful project and has a very convenient build system, I think it is only fitting to let Termux be the first to benefit 😉

[ackalker]

Added some resources. slesTestFeedback.cpp in particular looks very interesting, it appears to be some kind of reference test which firmware developers can use to check audio latency on their device before they ship a new ROM. This also explains why it uses some private APIs which are not exposed by the NDK (but can be ported I think).

[ackalker]

I have got cmus running on Termux and playing sound. Still need to package some extra optional dependencies.

[pvagner]

I have just found out a patch adding opensles sink to pulseaudio here: https://github.com/glance-/pulseaudio-android-ndk/blob/master/pulseaudio-patches/0003-WIP-opensl-sink.patch
I haven't yet tried to build this I have just looked at it briefly and I have found out it's a different implementation to the one you have mentioned in the first comment here.

[pvagner]

Apparently there is issue with some missing symbols. I'm on LineAgeOS Android 7 and I'm getting it too. I have tried to install glib in case it may help but it appears the missing symbol comes from the android runtime...
Here is where I have seen the issue mentioned for the first time... cmus/cmus#688

[Grimler91]

I'm also experiencing the missing symbol issue on arm android 7.0 with a AOSP-rom.

[pvagner]

hmm. Looking through the list of packages, I assume libandroid-glob is somehow supposed to address this. However I haven't yet understood how to most easily build a package. Originally I was about to build it on the device with termux running however when looking at termux-package repo, I can only see packages and scripts to use in its specific build environment. So I need to learn all this before I can try doing something usefull here.
Edit: Linking with libandroid-glob the same way we are linking with liblog should be enough I think. At least this is how it looks like when comparing this with other packages relying on libandroid-glob. I don't have the dev envo configured yet, but I imagine this will be a simple thing to do.

[Grimler91]

@pvagner I tried building with the option as you suggested and it solved the error! (I'm still not getting audio though so there are more problems. gdb cmus warns about a couple of missing symbols).

Would you like to open a pull request to solve the issue (and add libandroid-glob as a dependency)?

[pvagner]

@Grimler91 have you configured your pulseaudio in termux as proposed by @ackalker via a simple script setup-pulseaudio.sh and are you using either Simple Protocol Player to play back the audio through local network or VLC to play an rtp stream? Apparently no one else besides @ackalker got thus far as we are only discovering all this after a few months.
If you have got this far, perhaps you can make the pull request earlier. I am still getting all the stuf setup and I would like to get it building for me before submitting anything. It might take me a few hours or a few days, I am not sure yet how I can manage...

[Grimler91]

@pvagner Errrm, no I haven't. That could be my problem, I should start reading the instructions before testing stuff.

I tried it again and managed to play local music files using cmus and "simple protocol player" :)
Cool that it works!

I'll open a pull request shortly. (And ask if you have any questions about how to setup the build environment)

[pvagner]

@Grimler91 thank you! I've got it building and the sound is playing over here as well.

[Grimler91]

For anyone using cmus + simpleprotocolplayer: it is possible to start simpleprotolplayer for command line in termux using the intents described in this simpleprotocolplayer issue.

I am now using a small script to allow me to fully control my music playing device when only connected to it over ssh:

bash-4.4$ more bin/am_protocolplayer.sh
#!/data/data/com.termux/files/usr/bin/bash
if [ $1 == "play" ]; then
    am startservice --user 0 -a com.kaytat.simpleprotocolplayer.action.PLAY -e ip_addr 127.0.0.1 --ei audio_port 12345 --ei sample_rate 44100 --ez stereo true --ei buffer_ms 500 com.kaytat.simpleprotocolplayer/.MusicService
elif [ $1 == "stop" ]; then
    am startservice --user 0 -a com.kaytat.simpleprotocolplayer.action.STOP com.kaytat.simpleprotocolplayer/.MusicService
fi

am_protocolplayer.sh play starts the streaming and am_protocolplayer.sh stop stops the streaming.

[twaik]

Hi! Have working OpenSLES output if pulseaudio writes data to pipe. It can be used instead of simpleprotocolplayer. For now I'm trying to integrate it into module-pipe-sink (using anonymous pipe as buffer). https://gist.github.com/twaik/2c766116950e668989f3497a03b29862
Can anyone help me with that?

[JanuszChmiel]

And what about this so library which is The part of The Exagear strategy or PC? Unfortunately. .so library is closed source.

https://github.com/akbarri/ExaGear-SoundFix
May be, that if setting up correctly, it would be possible to use ALSA sound driver and fast sound responsiveness could be reached.

[twaik]

There are two options how to use ALSA in termux:

1. Use the following config to force ALSA output sound to Pulseaudio: (libasound_module_pcm_pulse.so is a part of libasound2-plugins in Ubuntu/Debian).

/etc/asound.conf

pcm.pulse {
  type pulse
}
ctl.pulse {
  type pulse
}
pcm.!default {
  type pulse
}
ctl.!default {
  type pulse
}

2. Implement ALSA sink working on top of SLES in Termux. Read sparkle sources for reference (1, 2). Of cource if you are sure you will not use this in chroot/proot container it is possible to write ALSA plugin which outputs sound directly to SLES.

[tomty89]

@JanuszChmiel if you need low-latency, you can try to lower the pulse-side buffer length with the module argument latency (i.e. put latency=SOME_VALUE_IN_MS at the end of the load-module line in default.pa):

termux-packages/packages/pulseaudio/module-sles-sink.c

Lines 584 to 588 in 7b7f3c4

	pa_modargs_get_value_u32(ma, "latency", &latency);
	u->block_usec = latency
	? PA_USEC_PER_MSEC * latency
	: PA_USEC_PER_MSEC * 125;
	pa_sink_set_fixed_latency(u->sink, u->block_usec);

The (higher) default is to make sure it works with slow audio output like bluetooth. There's termux-audio-info in termux-api that gives you some hint on how low it can be (e.g. convert the number in frames to ms and divide it by 2) and which sample rate that could give you faster audio (yes that could matter, on some devices even a lot).

You can try the aaudio sink instead if you are on Android 8.1 or above. Theoretically it will use the fastest sample rate and minimum buffer length out of the box (and even adapt when the output device is changed).

EDIT: with proot-distro or alike (i.e. this script from @krmanik), an even bigger source of latency might the the use of TCP (and maybe also the overhead in proot itself, idk). I think it might help if you make it use the unix socket instead by exposing the file to the proot and changing the PULSE_SERVER env var inside. (I've been away from this kind of proot setup for too long to recall whether that was actually possible though.)

[JanuszChmiel]

Thank all of you for very useful comments. I will test The latency with Speech-dispatcher and Espeak-ng in Chroot container executed by using Proot.

[JanuszChmiel]

Dear MR tomty89. I have spent several years while searching for fast responsiveness between Speech-dispatcher will receive some textual content and before Espeak-ng will speak. I have found out, that even if I Am using Speech-dispatcher for Termux, Pulseaudio for Termux and even if I Am using Speech-dispatcher from Termux directly without Proot and some external Linux distribution, lag is 2000 Milliseconds. So may be, that it is bug of sound chip Android kernel driver. But I do not think so, because if I Am using Speech engines from Android directly, responsiveness is much more better. I have also found out, that long responsiveness exist only if I Am using Speech-dispatcher speech server. If I Am running espeak from command line or Espeak-ng lag is significantly less.
It is very interesting programmers adventure.

[tomty89]

speech-dispatcher in Termux doesn't work properly on my devices at all. I can't even run spd-say for a second time until I kill everything. It probably has something to do with the "crippled" pthread stuff in bionic (Android libc). But with proot you would be using GNU libc in the distro.

Anyway, does the espeak itself/alone in proot lag less than the speech-dispatcher there though? Or does only espeak on the outside work quicker?

[JanuszChmiel]

Sad reality is, that espeak from Termux also gives some lag, approximately One second. The lag delay depend on The fact, if synthesizer has speaked in The time, when it speak new text. I have tried to remove The # character in The last line of /etc/pulse/default.pa to enable aadiosink module. Unfortunately, on my device model with Android 8.0 it do not work. Espeak do not speak and app must be terminated by pressing CTRL+c. Please dear MR tomty89commented. Would you be so kind and could you send me whole line of /etc/pulse/default.pa to setup smaller buffer of Pulseaudio than The default is used? I Am recommending value which is 50 % less than The default preconfigured inside Pulseaudio source code. It is possible to run so many many apps, Mate desktop environment if it is being executed from Alpine Linux even gives so much less error messages on The SSH console, because Alpine do not depend on Systemd and some Linux services can be even started on Chroot environment by using Proot thanks to touch word. Sure, healthy thinkink is required. It is not very useful to start Udev on Android environment or other real Linux kernel dependent services. And if somebody would try to help Me to reduce The lag of Espeak, it would be really excellent. Because I can use so many apps including Firefox, Tuner Internet radio app, Pidgin. And it is really revolutionar, because Android phone charger consumes much less of energy than standard Notebook charger. And Android devices are very portable.

[JanuszChmiel]

Proot is very probably not causing long lag. I AM also aware, that I will have to tolerate some build in audio chip device kernel driver features and build in powermanagement inside Android kernel. If delay could be reduced to The 50 % or current time, I will be very happy. I must find which command is being used to cound The time delay of some app operation. I will write exact values so values will be objective numbers.

[JanuszChmiel]

Here are scientifists objective values.
When I have executed Espeak-ng from Debian installed byusing proot-distro install debian.
time espeak-ng "testing"

real 0m1.204s
user 0m0.060s
sys 0m0.100s
This delay exists when no sound have been plaied for more than 30 Seconds.
Now I Am running spd-say testing
time spd-say "testing"

real 0m0.775s
user 0m0.020s
sys 0m0.010s
And now The same above commands executed from Termux directly outside Debian distribution.
~ $ time espeak "testing"

real 0m0.981s
user 0m0.060s
sys 0m0.040s
ASpd-say can not me measured. I must press CTRL+c, some multithreading issue.

[tomty89]

If you are talking about pulse start being slow, make sure you uncomment (i.e. remove ;) the exit-idle-time line in $PREFIX/etc/pulse/daemon.conf and change the value to -1. (It might consume a bit more battery power though.)

If you are using Android 8.0 instead of 8.1, better not bother with aaudio since there's a bug in it back then.

[JanuszChmiel]

Dear tomty89, is there any chance to speed up The speech responsiveness? Or it is very probably impossible at all?
Did I have set buffer of Pulseaudio correctly when I have added
load-module module-sles-sink latency=50
of my /etc/pulse/default.pa
Or do I have to add other separate line?
May be, that it is impossible to tune Pulseaudio for Termux by this way. And that only standard Pulseaudio for Linux can be setup by this way in combination with The Simple protocol player.
But it have not been fast too.

[JanuszChmiel]

I have set exit-idle-time to -1 allready.
I have also acquired wakelock from system tray notification area from Android.

[JanuszChmiel]

I have set latency to The 30. 10 produces speech distortion. 30 produces only very rare distortions. Some slowdowns will be also caused by The absence of 3D graphics acceleration so I must find some .so libraryes for Snap dragon to enable 3D graphics support for VNC session.

[JanuszChmiel]

tomty89 your deep knowledge related to Sound Android APIS and sound infrastructure in general is really outstanding. I have followed your previous advice. I have uncommented aaudiosink line in /etc/pulse/default.pa on my second mobile phone with Android 9.0. And really, AAudio is working smoothly. Really, on Android 8.0 it is impossible to use it with Pulseaudio for Termux. I have also found out, that my complains related to slow responsiveness are more caused by The new shared memory function incorporated to Firefox or Seamonkey. It is causing so big slowdowns, because Proot can not implement The complete shared memory to prevent EEMS chips deteriorations in Android devices. So sound responsiveness is really not so fast like on my notebook, but I can not compare. So thank all of you for your patience with Me.

[licy183]

I think this issue should be opened.

[twaik]

Should we keep this issue opened? Seems like development in this direction stopped because pulseaudio is enough for us.

[sylirre]

I'm closing this.

This issue is inactive and status in the description is outdated.