Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [parallel-computing]

42 questions
28
votes
5 answers

Any suggestions for making R code use multiple processors?

I have R-scripts for reading large amounts of csv data from different files and then perform machine learning tasks such as svm for classification. Are there any libraries for making use of multiple cores on the server for R. or What is most…
mohans
  • 283
  • 3
  • 5
20
votes
1 answer

Physical/pictoral interpretation of higher-order moments

I'm preparing a presentation about parallel statistics. I plan to illustrate the formulas for distributed computation of the mean and variance with examples involving center of gravity and moment of inertia. I'm wondering if there is a physical…
James Koppel
  • 241
  • 1
  • 6
16
votes
4 answers

Who uses R with multicore, SNOW or CUDA package for resource intense computing?

Who of you in this forum uses ">R with the multicore, snow packages, or CUDA, so for advanced calculations that need more power than a workstation CPU? On which hardware do you compute these scripts? At home/work or do you have data center access…
Heinrich
  • 111
  • 1
  • 7
15
votes
2 answers

Random numbers and the multicore package

When programming in R, I've used the multicore package a few times. However, I've never seen a statement about how it handles it's random numbers. When I use openMP with C, I'm careful to use a proper parallel RNG, but with R I've assume that…
csgillespie
  • 11,849
  • 9
  • 56
  • 85
11
votes
1 answer

Combining multiple parallel MCMC chains into one longer chain

Let's say that one has run $m$ parallel MCMC chains where each chain has had burn-in. Let the resulting chains be denoted by $$ x_1^{(i)},\dots,x_N^{(i)} \quad \text{ for } i=1,\dots,m,$$ where $N$ is the length of each chain after burn-in. If one…
Lotus3000
  • 125
  • 1
  • 6
10
votes
2 answers

Parallelizing the caret package using doSMP

UPDATE: caret now uses foreach internally, so this question is no longer really relevant. If you can register a working parallel backend for foreach, caret will use it. I have the caret package for R, and I'm interesting in using the train…
Zach
  • 22,308
  • 18
  • 114
  • 158
10
votes
2 answers

RNG, R, mclapply and cluster of computers

I'm running a simulation on R and a cluster of computers and have the following problem. On each of X computers I run: fxT2 <- function(i) runif(10) nessay <- 100 c(mclapply(1:nessay, fxT2), recursive=TRUE) There are 32 computers, each with 16…
user603
  • 21,225
  • 3
  • 71
  • 135
7
votes
2 answers

Gputools for R: how to interpret the experimental procedure?

The following paper describes an implementation of R in parallel on a graphics processing unit (GPU). Buckner et al., The gputools package enables GPU computing in R, BIOINFORMATICS, Vol. 26 no. 1 2010, pages 134–135 In the experimental section,…
Douglas S. Stones
  • 6,931
  • 4
  • 16
  • 18
6
votes
1 answer

What is the current 'standard' for modern statistical computing hardware?

I am in the market for a new system (probably a laptop) that would be be used primarily for Bayesian/MCMC analyses. If I had unlimited funds I would obviously buy very high end hardware and be done with it. Unfortunately, this is the real world…
user25658
6
votes
2 answers

Parallel solving Ax=b?

Cross posted on StackOverflow. I have some extremely large sparse matrices created using spMatrix function from the matrix package. Using the solve() function works for my Ax=b issue, but it takes a very long time. Several days. I noticed that…
Robert
  • 275
  • 3
  • 6
5
votes
1 answer

How can I run a Zero-Inflated Poisson/Negative Binomial Mixed Model with Gaussian Process

After having visited stats stack exchange countless times, I'm finally asking a question! For my research, I am try to run a model of the form: $$ Y = f(X,B)+ g(X) + \epsilon$$ Where $f(X,B)$ is a zero-inflated poisson/negative binomial density,…
user271536
5
votes
1 answer

Estimation of quantile given quantiles of subset

Let's say we have sets $X=\{x_1, x_2, \ldots, x_m\}, Y=\{y_1, y_2, \ldots, y_n\}$ and we have some estimate (or exact) quantile information about them at some level $a$. How could we approximate the quantiles at that same level $a$ for the union of…
bayesian
  • 53
  • 4
3
votes
1 answer

Parallel logistic regression

I need to perform stepwise binary logistic regression (The horror! The horror!) on 1.5 million observations. This takes far too long in SAS, so I'm wondering if I can use R to process it in a multicore environment. Apparently package gmulti…
William Shakespeare
  • 161
  • 1
  • 8
3
votes
1 answer

Simple cloud computing to run R + JAGS simulations

I want to simulate the frequentist properties of a Bayesian model. So, for example, I might want to fit a Bayesian model 1,000 times to 50 different configurations each of which takes about 10 seconds to fit on my machine. i.e., total computing time…
Jeromy Anglim
  • 42,044
  • 23
  • 146
  • 250
3
votes
3 answers

parallelism in data mining softwares

I'm working on a data set for order prediction/classification and a close deadline upcoming. Fortunately, my university has a super-computer with restricted access. I was thinking of using a few nodes (each node is a 16-core processor, 2.3 GHz each…
Soroush
  • 139
  • 2
1
2 3