Gputools for R: how to interpret the experimental procedure?

Question

The following paper describes an implementation of R in parallel on a graphics processing unit (GPU).

• Buckner et al., The gputools package enables GPU computing in R, BIOINFORMATICS, Vol. 26 no. 1 2010, pages 134–135

In the experimental section, on a 4-core computer, they compare the performance of the program running on the GPU with the performance without the GPU. The write the following:

We chose to use a single thread of the R environment in our test, since this is the way that most users interact with R.

So, the authors find the baseline by running their experiments using a single core (running in serial).

But, the experimental conditions for the GPU side is unclear (to me). When using a GPU, for efficiency we should simultaneously make use of the CPUs. If the authors used the remaining CPUs in the computer (which would be sensible to do in an optimised algorithm), then the speedup would be based on additional CPUs as well as GPUs over the baseline (and thus be artificially inflated by a factor slightly less than 4).

How should this experiment be interpreted?

In particular, I would like to know if my above interpretation is the correct one, and if so, what does this experiment actually tell us.

Answer 1

But, the experimental conditions for the GPU side is unclear (to me). When using a GPU, for efficiency we should simultaneously make use of the CPUs.

That is not generally true, and in particular is not true for the gputools R package which offers an 'everything to the GPU' mode with new functions gpuMatMult(), gpuQr(), gpuCor() etc. In other words, it offers you new functions that shift the computations completely to the GPU.

But your intuition is good. There should be a mixed mode with hybrid operations between the GPU and the CPU -- and the Magma library aims to offer just that. Better still, the magma R package brings this to R.

Moreover, I have a benchmarking paper / vignette / small package almost completed that compares these as well as several BLAS such as Atlas, Goto and the MKL. I'll update this entry with a URL in a couple of days.

Edit on 16 Sep: The paper I mentioned is now out and on CRAN with its own package gcbd; I and I wrote a brief blog entry about it too.

Answer 2

There is fundamental difference between parallel computing in CPUs and GPUs. Essentially, the CPU has been designed to do clever things on behalf of the programmer. For example, Instruction level parallelism. The GPU on the other hand, stips away this useful stuff and instead contains many more cores. It's a trade off between the processor helping you out and giving you more cores. Therefore, to use the GPU effectively, you need to submit as many threads (as memory allows as possible). The reason for this is because the GPU doesn't do any clever scheduling. So when it requests data for one thread, you want to have another one in the thread queue waiting to take over.

Example

Suppose you have a for loop that you want to make parallel:

#f(i) does not depend on f(j)
#for any j != i
for(i in 1:100000)
    w[i] = f(i)

You can submit N=1000000 threads (spread over the number cores) to the GPU. Now you may think that you could let n threads be done on the multi-core CPU, but:

1. There's quite a lot of extra programming baggage for at most little gain. GPU programming is hard (at least I think so), so combing it with multi-core CPUs is something you want to avoid.
2. The f(i) that you submit to the GPU tends to be a very simple function, say multiplying two elements of a matrix together.
3. You will get a time penalty if you use the GPU and CPU together since they both have to ask each other if they are finished.
4. By reducing the number of threads used on the GPU, you could easily be reducing efficiency, i.e. it takes the same amount of time to do N-n operations as it does for N operations!

Of course there are situations when you may want to use both the GPU and CPU, but typically you don't use them for the same operation.

---

Unfortunately, I don't have access to the paper at the moment. It's on my (long) list of things to read. So the above is more a general discussion on CPU and GPUs. I'll try and read it in the next day or two.