Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [nonparametric-regression]

Nonparametric regression is a form of regression analysis where the form of the functional dependence of the response on the predictors is not assumed. It subsumes many kinds of models, like spline models, kernel regression, gaussian process regression, regression trees or random forrests, and others.

Wikipedia has an article with further references.

62 questions
9
votes
1 answer

Cross validation with nonparametric smoothing regressions

When I use regression models I feel leery of defaulting to an assumptions of linear association; instead I like to explore the functional form of relationships between dependent and explanatory variables using nonparametric smoothing regression…
Alexis
  • 26,219
  • 5
  • 78
  • 131
8
votes
1 answer

Rank and z-transform instead of Wilcoxon?

Andrew Gelman in a recent post in his blog suggests using a rank, transforming the rank to a z-score, and then using parametric tests and tools instead of performing non-parametric tests. I never heard of that before. A search on Google pointed me…
Jacques Wainer
  • 5,032
  • 1
  • 20
  • 32
5
votes
0 answers

Is calculating a moving average a good way to approximate k-nearest neighbor regression?

Given i.i.d samples (x1,y1), ... (xn,yn) such that yi = f0(xi) + $\epsilon$i, i = 1,... n for some f0 Suppose I want an estimate $\hat{f}$ of f0 using k-nearest-neighbors regression in the neighborhood of each xi in my dataset. So for each xi, I…
Moss Murderer
  • 739
  • 4
  • 12
5
votes
2 answers

Local polynomial regression: Why does the variance increase monotonically in the degree?

How can I show that the variance of local polynomial regression is increasing with the degree of the polynomial (Exercise 6.3 in Elements of Statistical Learning, second edition)? This question has been asked before but the answer just states it…
warsaga
  • 112
  • 8
4
votes
0 answers

Intuition of the regression model under fixed design case (nonparametric regression)

Let $(x_1,Y_1), \dotsc, (x_n,Y_n)$ be a random sample from the regression model $$Y_t=m(x_t)+\epsilon_t.$$ When authors want to develop the asymptotic properties of the local linear estimator of $m$ when the design points are fixed (for example,…
Celine Harumi
  • 963
  • 5
  • 12
4
votes
1 answer

Non parametric estimators for noisy functions

Suppose there is a function $f(a,b,c,\ldots)$ of $M$ variables (fixed numbers, not random variables). Add some Gaussian noise to this function: $$ g(a,b,c,\ldots) = f(a,b,c,\ldots) + \varepsilon(a,b,c,\ldots) $$ where $\varepsilon(a,b,c,\ldots) \sim…
Bob Mortimer
  • 61
  • 5
3
votes
1 answer

Calculating local variance

I have some data, and I assume it can be modelled by $y_i = f(x_i) + \epsilon $, where $\epsilon \sim \mathcal{N}(0,\sigma_0^2)$ where $f$ and $\sigma_0^2$ are unknown. I understand that I can estimate the variance of the noise by calculating the…
user112495
  • 297
  • 1
  • 9
3
votes
1 answer

Why isn't every nonparametric model with random model design an additive noise model?

Let $Y$ be a real random variable and $X$ be a real random vector. In a nonparametric model with additive noise, we assume the relationship $$Y = f(X) + \epsilon$$ for some unknown regression function $f$ and noise $\epsilon$. This assumption is in…
rkvymvqt
  • 41
  • 2
3
votes
1 answer

A particular method for estimating the gradient of a log-density from samples

Suppose I have $N$ samples $x^1, \ldots, x^N$ which were drawn iid from an unknown density $P(x)$. Suppose I am interested in estimating the vector-valued function $g(x) = \nabla \log P (x)$. One approach to this could be the following: for $h$ in…
πr8
  • 1,286
  • 5
  • 22
3
votes
1 answer

Rates of convergence for estimating population mean squared error

Suppose I have an i.i.d. sample $\{(Y_i, X_i)\}_{i=1}^n$ on which I am trying to estimate a conditional expectation model: $$Y = g(X) + \varepsilon,\quad \mathbb E[\varepsilon | X] = 0$$ There is a lot of literature on how different estimators of…
stats_model
  • 1,184
  • 4
  • 16
3
votes
0 answers

Multiple regression for left-censored independent and dependent variables

I am interested in developing a predictive multiple regression model which predicts a concentration of one compound based on the measured concentrations of several other compounds. Both the dependent and independent variables contain "non-detects",…
Ryan Conway
  • 41
  • 2
3
votes
0 answers

Nonparametric regression with missing data

Let $(y_i,x_i,b_i)$ be data at hand, where $y_i$ is a response variable, $x_i$ is covariates, and $b_i$ is an indicator for missing: if 1, then $y_i$ is observable, 0 otherwise. Then, under missing at random --$p(b=1|x_i,y_i) = p(b=1|x_i)$--…
user1292919
  • 637
  • 3
  • 8
3
votes
1 answer

What is the density of $X$ under fixed design?

We observe an i.i.d. sample $(X_1, Y_1), \ldots (X_n, Y_n).$ Let $m(x) = E(Y|X=x),$ $\sigma^2(x) = \operatorname{Var}(Y|X=x)$ and let $f(\cdot)$ be the density of $X.$ Under some regularity conditions, the conditional variance of the local linear…
Epiousios
  • 218
  • 1
  • 8
3
votes
0 answers

Options for non-linear or non-parametric count data regression in R?

I am trying to move from a current parametric Poisson regression to a non-parametric count regression and would appreciate views on the best way to do this. Current state of analysis I have a time series of 8,000 items x 4 years of data = 32,000…
alpha137
  • 151
  • 4
3
votes
1 answer

Why instrumental variables? Or: why not nonparametric regression?

Usually instrumental variables are introduced as a means to solve the problem $E(u|X)\neq 0$ in the model $Y = X'\beta + u$. This may happen if we omit important variables from the covariate vector $X$, for instance. However, it is always the case…
user127022
  • 103
  • 5
1
2 3 4 5