I know that we are all excited about the mythical Apple tablet materializing tomorrow, but with all the pontification already spread across the far corners of the media, I don’t feel I have much to add to the story. On the other hand, I have been thinking a bit about iPhone OS 4.0. I don’t think we will hear about 4.0 tomorrow if there really is a tablet. It seems like that would just be too much to unload all at once. (I do allow that we might hear about an incremental update such as iPhone OS 3.2 though). Still, we’ll be all out of things to speculate on after tomorrow, so perhaps we can kickstart the next round now?
User Desire
As a user, my hope for iPhone OS 4.0 is a way for third party developers to provide background media playing applications. I understand why Apple wants to be careful with allowing any ol’ process to be granted the ability to run in the background. Sure… power users will be able to manage background processes by hand, no problem. As we have seen on the desktop though, ordinary people just don’t want to be bothered with that level of device tweaking. We have been playing with a few Android devices here at the labs and its clear its pretty easy to start installing apps that quietly add background processes that start to suck down the battery without realizing what you have done.
On the other hand, applications like Pandora on the Android phones are so much better than their iPhone equivalents. Being able to tune in that Pandora radio station and having it play in the background just like the built-in iPod application when you launch another app would change the value of these applications immensely.
My suggestion for how Apple can do this without opening Pandora’s box (zing!) is to create a media service API that allows third party apps to hand media streams off to the iPod player. So the flow would work something like this:
You launch Pandora and use their UI to navigate their music offerings. You tune in the station that suits your current mood and give a bit of a listen. Under the hood Pandora is playing their stream via this new service API that is using the iPod application to serve up the audio. Now, you get the urge to catch-up on your news and launch NetNewsWire for some tasty RSS. When Pandora’s process is terminated the music keeps on playing, because it is the iPod application that is playing the audio. Its all Apple code running in the background, managing how much power is consumed just the way Apple wants, certified to the level of quality Apple is comfortable with, but we get audio from other providers.
There are of course some wrinkles to work out. If I launch the iPod app while a background stream is running, what is ‘Now Playing’? Perhaps the iPod application shows an appropriate image indicating the Pandora app is the current offering and allows you to quickly switch to it. One can even imagine a more sophisticated approach where the Pandora app could provide a plugin bundle that hosts its UI inside the iPod app, though I doubt this is appealing to Apple. There also needs to be some interesting mechanism for the iPod application to ask the originating service for the next or previous media file so that mini-controls that can be used to control the iPod in the background work as expected. Perhaps this is solved through a call-back into a simple code-bundle provided by the third party application? I’ m confident though there is some low impact way to achieve Apple’s goals and still enrich the media-playing experience on the device.
Developer Desire
So that’s my biggest desire as an iPhone user, but what about as an iPhone developer? What I would love to see next is a basic suite of video editing APIs. The problem with the iPhone OS and video right now is that if you want to do anything to process frames of video you need to have your own code to decode the video and re-encode after you have done your processing. This is not exactly trivial code to do well and open source starting points are potentially fraught with licensing and patent peril.
Even if you roll your own codec, perhaps even getting down and dirty with some ARM assembly, you still can’t get access to the built-in hardware assist that allows Apple’s software to capture and encode video at decent performance. The last thing we want to do as an iPhone developer is invest a lot of time in developing a codec for the iPhone and find that the best we can do is H.263 at a miserable frame rate. We also don’t want to force the user to wait around in our application for an encode to finish. Once again we need some way to hand this off and allow the OS to encode in the background for us so our users are not trapped in our application waiting for their edit to finish.
A basic QuickTime derived API that would allow us to take a video file, access individual frames and provide our own sequence of frames to encode into a new movie would be just the ticket. I realize that the advanced encoders typically have licensing requirements and perhaps Apple’s current license for their iPhone software doesn’t allow for this to be opened up. Hopefully it is something that can be addressed in the next version of the OS anyway?
Surely we are in for a interesting ride with the big announcement tomorrow. No doubt the next 6 months will be an interesting one for those of playing with Apple technologies. I suspect WWDC will be chock full of new content this year!
Postage | RogueSheep | iPhone
