pixxid

Annoyances with Android: MediaPlayer [Part 1]

For the past six months I’ve been a member of Grooveshark’s Android team. Before working on the Android project I was a Flash and Flex developer, working on the main Grooveshark web application and was moved to Android to, as I recall, help fix some of the problems with the 1.0 version of the application. Soon after joining the Android team my team member and I decided that the best way to fix 1.0 was to completely scrap it and start over. Many of the problems in the first version were because of some fundamental design decisions that did not fit well with the Android platform and to fix these problems would almost necessitate  a complete rewrite either way.

Around this time others in the mobile group began planning the next versions of all the mobile applications so my teammate and I rushed to begin work on the 2.0 release of Grooveshark for Android. My primary responsibilities were with the player and its various systems–the player cache, offline cache, and other related things.

I learned Java and the Android framework at the same time and along the way encountered what I consider to be several annoyances and shortcomings with and of the Android platform. While on the whole I find both Java and Android a pleasure to work with, some things need improvement.

I initially wanted to compile a Top n list of problems but after writing the first few I found that I had too much for a single post so I’ve chosen to spread these out across multiple entries. Of any of my complaints I’m more than willing to admit to ignorance if it turns out that I’m just doing it wrong. So find fault if you can.

1. MediaPlayer doesn’t accept an InputStream

   Grooveshark for Android does much more with music than just play it. We save to cache recently-played songs, pre-buffer enqueued songs, and allow users to save songs to an explicit “offline” cache for listening when they’re not connected to a high-speed data network. Two of these, saving to a recent cache and saving to an offline cache require that we do more things with an MP3 than simply play it. The Android MediaPlayer object accepts only three sources of audio data, or rather, supports only three ways to play an audio file. One can give the MediaPlayer a URL to a file, open a local file, or use a ContentProvider, which is useful if the audio data is embedded into the application’s installation package (APK).

   It would seem that the first two methods, that of playing from a URL and from a file would suit the Grooveshark application’s needs well: for playing live streams we would provide a URL to the song on one of Grooveshark’s stream servers, and for playing offline cached songs we’d simply direct MediaPlayer to open a file on disk. However in both scenarios these input sources do not provide for everything we require.

   In the first instance we can play from a URL without problem but once this song is played, we cannot access the downloaded data in order to save it to a recently-played cache. Doing this is necessary because if a user has a few songs enqueued, plays one, skips to the next song, then skips back to a previously listened-to song, we do not want to download this song again. Users do not expect that we’d need to download the song again, having just listened to it. When skipping back there would be a noticeable lag as the song buffered again and we’d be unnecessarily transferring data that we’ve already downloaded. Not everyone has an unlimited data plan and it seems that in the near future unlimited data plans may become unavailable anyway.

   Because there’s no way to access the MediaPlayer’s data once downloaded, we could use a URL InputStream to download the file and pipe the stream both into the MediaPlayer and out to the disk. This would allow us to both play the song and save it for later use. But because MediaPlayer does not accept an InputStream, this cannot be done. The only thing you can do with a URL is to play it.

   In the other instance, that of saving a file to disk for offline playback, we could download the file then pass a FileDescriptor to the MediaPlayer. MediaPlayer does allow for this. However we need to encrypt the file because we wouldn’t want users to simply be able to mount their phones on their computer and copy their offline songs elsewhere. If MediaPlayer accepted an InputStream, we could wrap the File’s InputStream in a FilterInputStream which would decrypt the file as it is played. But again we cannot do this. Arguably we could first decrypt the file to a temporary file, then play this file, but this would make available an unencrypted MP3 for a time and on older phones would produce a noticeable lag as the file was decrypted and written-out to disk before playback could begin.

   Others have found this MediaPlayer deficiency to be a problem and a bug was filed with a supposed fix. InputStreams can now be played with the new AudioTrack object first made available in Android 1.5, but AudioTrack can only play raw PCM data, not an MP3 source, and as far as I can tell, to play an MP3 using AudioTrack the MP3 would need to be decoded in software as no hooks to the underlying MP3 hardware is provided (if there even is any, I really don’t know).

   A proposed solution to this shortcoming is to run a local HTTP server which can manipulate the incoming and outgoing data in any way it likes. Ultimately this is what I ended up implementing. The application runs a thread which downloads a song, encrypts it as it writes it out to disk, and passes it along to the MediaPlayer object, sending the necessary HTTP headers as it connects to localhost. A stupid fix but it works.

2. There’s a major bug in OpenCORE

   OpenCORE is the underlying media framework on Android. It’s what the MediaPlayer and other objects use to play audio and video data. Like other parts of the Android API it ships with the phone and save firmware upgrades and patches from Google (assuming Google sends patches over-the-air, which I don’t even know if they do, and they probably do not), what comes on the phone is what you get.

   Unfortunately there’s a major bug in OpenCORE that prevents high-bitrate files from playing.

   The MediaPlayer object has several states that it enters into and leaves when it is directed to play audio data. When you provide a data source to the MediaPlayer, it enters into its “preparing” state. At this state it buffers data until it detects that it has enough to play the data source, audio or video, uninterrupted. Or rather, that it has buffered enough data that it can start playback without playing past its buffered amount and then pausing while it fetches more data.

   When trying to play certain high-bitrate MP3s, MediaPlayer never leaves its preparing state, never enters into the “prepared” state, and so playback never begins. The song will appear to buffer forever. In some instances it will buffer the song almost completely but it never believes it has enough data. This is a documented bug and I don’t know if there’s any way around this.

   The Grooveshark “Now Playing” screen shows buffer progress and player progress and when this bug exhibits itself, to the user the application just looks broken. It’s infuriating, embarrasing, and unfortunately seemingly out of our control. This seems to affect upwards of fifty percent of high-bitrate files, but the most frustrating part is that this problem cannot be reliably reproduced for any given file. You can usually stop and start playback (by pressing pause and play) a few times and the song will eventually play. If the problem was because of how the MP3 is encoded, I could possibly manipulate the MP3 data as it comes across the wire before it is sent to the MediaPlayer, but I can’t seem to find a pattern or a trigger for this problem. So I’m stumped.

   In the eleventh hour before the release of Grooveshark 2.0 for Android I added a “prefer low-bitrate files” option, defaulted to enabled, to the application’s settings. This is only a band-aid of course, but it at least diminishes the problem. Unfortunately all songs on Grooveshark aren’t yet transcoded, so the problem still manifests itself from time to time. One possible fix is to run a watcher thread that monitors bytes downloaded and MediaPlayer state, and if the file is high-bitrate and a threshold percentage of the total file is downloaded but the MediaPlayer has never entered the prepared state, to start and stop playback until it does.

More to come in part two.

Tags: android, development, java