Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [numerics]

Also known as Numerical Analysis, Numerics aims to provide methods and algorithms for numerical computations.

139 questions
50
votes
6 answers

What algorithm is used in linear regression?

I usually hear about "ordinary least squares". Is that the most widely used algorithm used for linear regression? Are there reasons to use a different one?
Belmont
  • 1,273
  • 3
  • 12
  • 16
45
votes
7 answers

Why does Andrew Ng prefer to use SVD and not EIG of covariance matrix to do PCA?

I am studying PCA from Andrew Ng's Coursera course and other materials. In the Stanford NLP course cs224n's first assignment, and in the lecture video from Andrew Ng, they do singular value decomposition instead of eigenvector decomposition of…
DongukJu
  • 593
  • 1
  • 5
  • 5
31
votes
2 answers

How to compute SVD of a huge sparse matrix?

What is the best way to compute singular value decomposition (SVD) of a very large positive matrix (65M x 3.4M) where data is extremely sparse? Less than 0.1% of the matrix is non zero. I need a way that: will fit into memory (I know that online…
Sonia
  • 413
  • 1
  • 4
  • 5
25
votes
2 answers

Efficient calculation of matrix inverse in R

I need to calculate matrix inverse and have been using solve function. While it works well on small matrices, solve tends to be very slow on large matrices. I was wondering if there is any other function or combination of functions (through SVD, QR,…
jitendra
  • 468
  • 2
  • 6
  • 12
22
votes
1 answer

Updating SVD decomposition after adding one new row to the matrix

Suppose that I have a dense matrix $ \textbf{A}$ of $m \times n$ size, with SVD decomposition $$\mathbf{A}=\mathbf{USV}^\top.$$ In R I can calculate the SVD as follows: svd(A). If a new $(m+1)$-th row is added to $\mathbf A$, can one compute the new…
user1436187
  • 523
  • 2
  • 4
  • 10
21
votes
3 answers

Weird correlations in the SVD results of random data; do they have a mathematical explanation or is it a LAPACK bug?

I observe a very weird behaviour in the SVD outcome of random data, which I can reproduce in both Matlab and R. It looks like some numerical issue in the LAPACK library; is it? I draw $n=1000$ samples from the $k=2$ dimensional Gaussian with zero…
amoeba
  • 93,463
  • 28
  • 275
  • 317
19
votes
1 answer

What are efficient algorithms to compute singular value decomposition (SVD)?

The Wikipedia article on principal component analysis states that Efficient algorithms exist to calculate the SVD of $X$ without having to form the matrix $X^TX$, so computing the SVD is now the standard way to calculate a principal components…
svd
  • 191
  • 1
  • 3
18
votes
5 answers

Essential papers on matrix decompositions

I recently read Skillicorn's book on matrix decompositions, and was a bit disappointed, as it was targeted to an undergraduate audience. I would like to compile (for myself and others) a short bibliography of essential papers (surveys, but also…
gappy
  • 5,390
  • 3
  • 28
  • 50
18
votes
3 answers

What fast algorithms exist for computing truncated SVD?

Possibly off topic here, but there exist several (one, two) related questions already. Poking around in the literature (or a google search for Truncated SVD Algorithms) turns up a lot of papers that use truncated SVDs in various ways, and claim…
John Doucette
  • 2,113
  • 1
  • 15
  • 24
16
votes
2 answers

Kullback-Leibler Divergence for two samples

I tried to implement a numerical estimate of the Kullback-Leibler Divergence for two samples. To debug the implementation draw the samples from two normal distributions $\mathcal N (0,1)$ and $\mathcal N (1,2)$. For a simple estimate I generated…
Jimbob
  • 185
  • 1
  • 2
  • 9
13
votes
1 answer

Softmax overflow

Waiting the next course of Andrew Ng on Coursera, I'm trying to program on Python a classifier with the softmax function on the last layer to have the different probabilities. However, when I try to use it on the CIFAR-10 dataset (input : (3072,…
Dlmss
  • 143
  • 1
  • 6
12
votes
1 answer

Default lme4 optimizer requires lots of iterations for high-dimensional data

TL;DR: lme4 optimization appears to be linear in the number of model parameters by default, and is way slower than an equivalent glm model with dummy variables for groups. Is there anything I can do to speed it up? I'm trying to fit a fairly large…
Ben Kuhn
  • 5,373
  • 1
  • 16
  • 27
11
votes
3 answers

Approach and example of graph clustering in "R"

I am looking to group/merge nodes in a graph using graph clustering in 'r'. Here is a stunningly toy variation of my problem. There are two "clusters" There is a "bridge" connecting the clusters Here is a candidate network: When I look at the…
EngrStudent
  • 8,232
  • 2
  • 29
  • 82
9
votes
5 answers

Making square-root of covariance matrix positive-definite (Matlab)

Motivation: I'm writing a state estimator in MATLAB (the unscented Kalman filter), which calls for the update of the (upper-triangular) square-root of a covariance matrix $S$ at every iteration (that is, for a covariance matrix $P$, it is true that…
Gilead
  • 381
  • 1
  • 2
  • 8
9
votes
0 answers

Possible causes for the state noise variance to become negative in a Kalman Filter?

I am having some trouble debugging an application of a linear discreet Kalman Filter. From time to time, I find that there are diagonal elements of the covariance matrix that become negative. This is clearly erroneous and causes the whole filter to…
Bogdanovist
  • 6,059
  • 1
  • 23
  • 28
1
2 3
…
9 10