Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [ram]

Random-access memory (RAM) is a form of computer data storage. A random-access device allows stored data to be accessed directly in any random order.

Random-access memory (RAM) is a form of computer data storage. A random-access device allows stored data to be accessed directly in any random order. [Wikipedia]

On this site, tag is used in the discussion of computation issues.

5 questions
8
votes
1 answer

Incremental learning with decision trees (scikit-learn)

I'm trying to train a regression tree with some very large data I have: approx 3Tb. I'm using scikit-learn and of course there is no way I can load that amount of data on memory. Doing some online research I found that some scikit-learn algorithms…
Ambesh
  • 303
  • 4
  • 10
3
votes
1 answer

RAM requirements for multivariate regression in SAS and R?

Is there a "rule of thumb" for calculating the amount of RAM required on a computer to conduct a multivariate regression in run-time $t$ with a $p \times n$ design matrix? Assuming I have to find the product $X^TX$ when solving for OLS…
RobertF
  • 4,380
  • 6
  • 29
  • 46
0
votes
1 answer

scikit-learn SVC with custom precomputed kernel matrix uses too much memory

I need to implement a custom kernel for the sklearn.svm.SVC learner. My custom kernel consists in multiplying every element of the kernel matrix except the main diagonal by a fixed constant (the "correction" factor). According to scikit-learn docs I…
df342
  • 1
0
votes
0 answers

Tips/References For Building Boosting Trees Or Neural Networks With Large Datasets?

I have a lot of data that can be used to train a model - so much that I am not sure if my computer (16GB Ram) can handle all of the data at once. What are some ways to deal with this issue, given my hardware limitations, to build a gradient…
confused
  • 2,453
  • 6
  • 26
0
votes
1 answer

Memory consumption in NLP tasks

I am trying to apply a simple Naive Bayes or SVM (libSVM) algorithms to a large data set, which I've constructed as an .arff file. The number of features in my set is ~180k and there are ~6k examples. Also there are 8 classification classes. The…
Michael
  • 167
  • 7