Some time back Adobe announced an interesting new technology for image processing on GPUs called Hydra. That technology has since been renamed to Pixel Bender and components of it are now available for the CS4 products and Flash Player 10.

Pixel Bender is very similar to Apple’s Core Image technology available on MacOS Tiger and later. Both Pixel Bender and Core Image allow developers to create image processing algorithms in a programming language that is abstracted from the hardware the filter will run on. A developer can create their image processing code in a domain-specific language and allow the runtime implementation in Core Image or Pixel Bender to implement the algorithm optimized for the hardware present in the system when the program actually runs. By leveraging the power of the modern Graphics Processing Unit on today’s common laptops and desktops these technologies offload the image processing tasks to specialized hardware, when present, and provide amazing image editing performance for the user. Often real time edits are possible for still images or video.

The two technologies are remarkably similar. Core Image of course requires the MacOS, but otherwise any application on the platform can be developed to incorporate Core Image. Its also possible for users to install Core Image filters so that all Core Image enabled applications can access the filters. Pixel Bender is currently only available in a few core Adobe applications : Photoshop, After Effects, Flash Pro and Flash Player. Having access to Pixel Bender effects in Flash Player does imply though that any application developed on the Flash platform (presumably including AIR and FLEX applications) can take advantage of the technology. The potential advantage here of course is that Flash, and therefore Pixel Bender, are cross-platform including web applications.

At the heart of each is is the concept of a ‘kernel’ which defines the core algorithm. The kernel describes how to produce a pixel of the output image given some portion of the input image. Both Core Image and Pixel Bender use a C-like language to create kernels. Both are based on the OpenGL Shading Language with the kernel an effective parallel to a fragment shader in GLSL. Adobe’s implementation seems on the surface to be a more complete implementation with less restrictions than Core Image. This can really help when trying to implement a particular image processing effect, but there is one potentially big gotcha : Pixel Bender kernels can only use a very restricted set of kernel language features when running in the Flash Player. If your target is a desktop Adobe application this should not be a problem, but if you are looking at a web application this could present problems.

Another interesting component to Pixel Bender is the idea of a processing graph that allows a developer to chain several kernels together into a single operation. This is quite useful because many interesting effects are only effectively realized as combinations of other effects. The chaining is smart in that the runtime performs analysis to make sure only the minimal amount of pixel processing is performed for a given chain. For instance if there is a crop at the end of your chain, none of the upstream filters will bother to compute output pixels that are outside the crop. Pixel Bender adds Graph language to describe and distribute these multi-pass filters (though once again this is not available in the Flash Player). Core Image includes the same concept of chaining filters with efficient execution, but only recently added the ability to persist these chains for reuse in Leopard.

I’m excited to learn more about Pixel Bender going forward. We have used Core Image extensively here at RogueSheep and have found it to be a great solution. Pixel Bender has the potential to open up some of our image processing solutions to the Windows platform or even web-based applications.

If you want to try out Pixel Bender yourself check out the Adobe Lab’s download here. They have a rudimentary IDE that lets you implement and edit a kernel and see the results live. There is also a plug-in for Photoshop CS4 that adds a Pixel Bender gallery filter which can host PB filters in Photoshop.

Also interesting is support for the visual graphics processing package Conduit from the fine minds at Lacquer. Conduit is a visual graphics processing package, similar to Quartz Composer, that can be hosted in a variety of applications. Their beta support for Pixel Bender allows you to create PB filters using their visual processing language, all without having to dirty your hands in the relatively low-level shader language. You can see some cool samples of it in action in your web browser (with the latest Flash Player).