The Cell SDK from IBM includes a useful library called ALF, the Accelerated Library Framework.

ALF provides a task oriented abstraction for having parallel work done on the Cell's SPU units. ALF allows a developer to define a task consisting of a compute kernel, essentially a block of code that will be executed on the SPU, and a set of work blocks that the task should be performed on. The library will handle SPU allocation and DMA transfers (including support for double-buffering).

ALF was included in the Cell SDK 2.0 but did not seem to get much mention. In the Cell SDK 2.1 it has been updated (some of the API's have changed), has much better documentation, and examples.

For example, if you need to compute the average of a large list of numbers you could write a compute kernel that takes in an integer array of fixed size that is small enough to fit on the memory of the SPU. Your function would the loop through the input array, calculate the average and store the output in an output buffer. This function would get compiled for the SPU and can take advantage of any SIMD instructions you like.

Your main program, which runs on a PPU, would create a task that calls your compute kernel and divide your input list into segments equal to the fixed input size you choose for your compute kernel. Each of these segments is a work block that the ALF library will then schedule and execute on an available accelerator (SPU). Your main program just has to wait for the jobs to complete (alf_task_wait) and can average the results together (which could be treated as another set of work blocks) to get a final answer.

ALF seems to be a good way get into parallel task oriented programming on a Cell. It allows you to concentrate on how your problems can be broken down into parallel tasks instead of getting too involved in the details of SPU affinity, DMA transfers, and mailboxes. Someone could write an ALF implementation for distributed computing or SMP but I don't know if anyone has.

If your going to be using ALF with 2.0 SDK it is worth looking at the examples posted on the ibm forum I had to gunzip the file twice before extracting it. The examples contain corrections to some errors in the 2.0 documentation and have make files with the correct compile flags (linking in the spu code correctly requires special options).