Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [nonlinearity]

This tag is deprecated because it is too broad. Please find a more specific tag.

57 questions
18
votes
6 answers

Why is increasing the non-linearity of neural networks desired?

On the wikipedia page of convolutional neural networks, it is stated that rectified linear units are applied to increase the non-linearity of the decision function and of the overall network:…
user158528
12
votes
3 answers

In general, does normalization mean to normalize the samples or features?

I'm just getting into machine learning, and I have seen two conflicting practices for normalization. To be concrete, let's suppose that we have a $n \times d$ matrix containing our training data, where $n$ is the number of samples and $d$ is the…
Jim Koff
  • 121
  • 1
  • 3
9
votes
4 answers

A linear pattern occurs on my residual plot: what can I do?

I'm a bit stuck with a problem here and any kind of help would help a lot :) Just to give a clue about my data. I have 6 independant variables (IV) which are: $X_1$ = Population -within a block- $X_2$ = Households -within a block- $X_3$ = Total…
Aziz
  • 91
  • 2
9
votes
1 answer

How to interpret the direction of the Harvey-Collier test and Rainbow test for linearity?

I implemented both those tests with R, using the lmtest package. Both tests directionally say the same thing (I think) with a very similar p-value of very close to 0. But, are those tests saying that the underlying regression model's residuals are…
Sympa
  • 6,862
  • 3
  • 30
  • 56
7
votes
0 answers

Spinograms vs. conditional densityplots

I have a binary response variable (hail) and multiple continuous predictor variables. My aim is to understand the linear/non-linear relationship of the predictors to the response to be able to justify the use of a linear or non-linear model. I got…
pat-s
  • 479
  • 2
  • 10
6
votes
1 answer

How to detect nonlinear relationship?

I have two continuous variables that may have nonlinear relationship. Scatter plot of two variables showed an ellipse shape. Furthermore, both Pearson correlation coefficient and Spearman's rank correlation coefficient were calculated and they were…
SAN
  • 61
  • 1
  • 4
5
votes
1 answer

Why aren't neural networks used with RBF activation functions (or other non-monotonic ones)?

In most work I've seen, MLPs (multilayer perceptron, the most typical feedforward neural network) and RBF (radial basis function) networks are compared as distinct models, where MLP neuron outputs $\sigma(\mathbf{w}^\top \mathbf{x})$. $\sigma$…
Christabella Irwanto
  • 577
  • 5
  • 11
4
votes
1 answer

Anova on logistic regressions linearity

I'm trying to find out if my numeric predictors have a linear relation to the logit of my logistic regression. I tried to use the lrm fit in the rms package where I have used 3 knot cubic spline on all numeric predictors like this: > fit <- lrm(y ~…
ChuckP
  • 743
  • 7
  • 17
4
votes
4 answers

How can I test a nonlinear vs a linear regression model?

I've got a panel regression model where the Ys assume a curved shape when plotted over time. A histogram of the residuals shows they are normally distributed but a residual-vs-fitted plot shows a pattern (see image 1). When I log-transform the Y…
Nick Cain
  • 138
  • 1
  • 1
  • 11
3
votes
1 answer

Neural Network - Estimating Non-linear function

I am fairly new to neural networks. I am trying to empirically show that a neural network can work better than logistic regression when the underlying function is non-linear. In my simulation study, the true probability of assignment to the…
Alex
  • 487
  • 2
  • 8
3
votes
0 answers

Relu6 and vanishing gradients problem

In some recent machine learning papers (e.g. mobileNetV2), ReLU6, defined as $Relu(x)=\min(\max(0,x),6)$ is used instead of regular Relu non-linearities. Doesn't such a function result in the same gradient vanishing problem as sigmoid functions? My…
Ash
  • 239
  • 1
  • 10
3
votes
2 answers

Is this a nonlinear time series?

Could someone please help me to find out whether a time series is linear? And if it's nonlinear, what degree of nonlinearity? I searched for an appropriate function in Matlab, but it seems there's no function which takes a vector as an argument and…
Vitaly Zinchenko
  • 133
  • 6
2
votes
0 answers

Nonlinear function of two Gaussians - Stein's Lemma

Let $g,h$ be independent standard normal variables ($\cal{N}(0,1)$). Fix $\sigma>0$ and pick $f:\mathbb{R}\rightarrow \mathbb{R}$. Under what conditions on $f$, we have that $$ \mathbb{E}[f(g+\sigma h)h]=\sigma \mathbb{E}[f(g+\sigma h)g] $$ I…
ecstasyofgold
  • 121
  • 2
2
votes
1 answer

Why do nonlinearities in deep neural nets give rise to very high derivatives?

In the book "Deep Learning" by Goodfellow, Bengio, and Courville, I do not understand the following statement about why nonlinearities in deep neural nets give rise to very high derivatives: The objective function for highly nonlinear deep neural…
samra irshad
  • 571
  • 3
  • 12
2
votes
1 answer

Multiple Regression: Finding which variables are non linearly related to the outcome

I have a dataset with 10 predictors and 1 outcome variable. Looking at the Residual Vs Fitted Plot, I suspect a Non-Linearity that I am missing. But how can I check out of the 10 predictors, which are linearly and which are non-linearly related to…
Nithya Subramanian
  • 51
  • 2
1
2 3 4