0

Developing a Musical Instrument App for Android

I am currently developing my first app on Android, which can be quite frustrating as an iOS developer (more on that in another post perhaps).

In this post I will focus on how to develop a multi-touch enabled musical instrument app with low latency playback of audio samples.

Prerequisites

The app should be able to play about 50 different sound samples, at least 5 of them simultaneously. Playing a sound should be instant, so low latency is required.

Android Sound Libraries

Android offers three different sound APIs: MediaPlayer, SoundPool and AudioTrack. MediaPlayer is for playback of longer audio files and video, so this API is not suitable for our needs.

SoundPool’s documentation tells us that it should be used for repeated, simultaneous low-latency playback of multiple short sound samples. That’s exactly what we need! Unfortunately it turned out to be very laggy when playing. So the last resort was Android’s low level audio API AudioTrack.

Using AudioTrack

To play a sound with AudioTrack, the file has to be read into a buffer which is then pushed directly to the audio hardware. Playing multiple sounds simultaneously requires a thread for each sound.

Playing a Sound

To encapsulate audio playback, I created the AudioTrackSoundPlayer class:

public class AudioTrackSoundPlayer
{
    private HashMap<String, PlayThread> threadMap = null;
    private Context context;

    public AudioTrackSoundPlayer(Context context)
    {
        this.context = context;
        threadMap = new HashMap<String, AudioTrackSoundPlayer.PlayThread>();
    }

The two members are a HashMap to store the threads of the currently playing sounds and a reference to the activity context which is needed later to create AudioTrack objects.

    public void playNote(String note)
    {
        if (!isNotePlaying(note))
        {
            PlayThread thread = new PlayThread(note);
            thread.start();
            threadMap.put(note, thread);
        }
    }

    public void stopNote(String note)
    {
        PlayThread thread = threadMap.get(note);
        if (thread != null)
        {
            thread.requestStop();
            threadMap.remove(note);
        }
    }

When playing a note a new thread is started and put into the map. Notes are only played if they are not already playing. Stopping a note (if the user releases the button) fetches the thread out of the map and requests it to stop.

    public boolean isNotePlaying(String note)
    {
        return threadMap.containsKey(note);
    }

To check if a note is already playing we simply have to look for the kay in the map.

Now to the fun part: the PlayThread class.

    private class PlayThread extends Thread
    {
        String note;
        boolean stop = false;
        AudioTrack audioTrack = null;

        public PlayThread(String note)
        {
            super();
            this.note = note;
        }

The class stores the note it plays, a stop flag which is set when the thread should be terminated (and thus stop playing the sound) and the AudioTrack instance used to play the sound.

        public void run()
        {
            try
            {
                String path = note + ".wav";

                AssetManager assetManager = context.getAssets();
                AssetFileDescriptor ad = assetManager.openFd(path);
                long fileSize = ad.getLength();
                int bufferSize = 4096;
                byte[] buffer = new byte[bufferSize];

                audioTrack = new AudioTrack(AudioManager.STREAM_MUSIC, 22050, AudioFormat.CHANNEL_CONFIGURATION_MONO, AudioFormat.ENCODING_PCM_16BIT, bufferSize, AudioTrack.MODE_STREAM);

First the file name is constructed by adding “.wav” to the note. Then the file size is read by opening the file with AssetManager. Next we create a 4096 byte buffer to store the audio data. Adapt the buffer size to your needs. 4096 seemed to work best for me. The AudioTrack object is then created with the appropriate configuration, depending on the sample files you use.

                audioTrack.play();

                InputStream audioStream = null;

                int headerOffset = 0x2C;
                long bytesWritten = 0;
                int bytesRead = 0;

                while (!stop) // loop sound
                {
                    audioStream = assetManager.open(path);
                    bytesWritten = 0;
                    bytesRead = 0;

                    audioStream.read(buffer, 0, headerOffset);

                    // read until end of file
                    while (!stop && bytesWritten < fileSize - headerOffset)
                    {
                        bytesRead = audioStream.read(buffer, 0, bufferSize);
                        bytesWritten += audioTrack.write(buffer, 0, bytesRead);
                    }
                }

                audioTrack.stop();
                audioTrack.release();
            }
            catch (IOException e)
            {
                e.printStackTrace();
            }

        }

Next audio playback of the AudioTrack object is started. Now we have to continuously feed audio data to the object. In a standard WAVE file the raw audio data starts at offset 0x2C, so we skip the header. The outer while loop is used for looping the sound sample indefinitely. The inner while loop reads the actual audio data and feeds it to the AudioTrack object until the end of the file is reached. If the stop flag is set, the loop stops execution and the thread is terminated. The requestStop method sets the stop flag.

        public synchronized void requestStop()
        {
            stop = true;
        }
    }
}

Conclusion

The AudioTrackSoundPlayer class gives us a very simple interface for usage in musical instrument apps. Simultaneous low-latency playback of sound samples using Android’s AudioTrack API can be done quite easily by spawning threads for each sample. I hope this saves you some of the trouble I went through while figuring out Android audio.

To further improve the class, additional configuration settings could be added as well as support for compressed file formats like mp3 or ogg. Currently only PCM data (WAV) can be played.

The complete project including the code used for multi-touch handling can be downloaded here: PianoTest.zip.