0

I was going through the following discussions

Is this a linear or non linear model, and why?

How to identify models as linear or non-linear?

In the above discussions, users have submitted some equations and want to know if they represent a linear or a non-linear model along with the reason.

By looking at these examples, I am curious to know how we classify a model to be linear /non-linear when we are given the model description in vectored form. Note that the examples given in the above discussions were not presented in vectored form (expressed as multiplication of matrices, their dot products, inverses etc.) rather they were given in equation form.

What aspects can be used to separate a linear model from a non-linear model when they are presented in vectored form. Any useful tips/observations based on properties of matrices etc.

For example, consider a very trivial case, $F(x,y) = x^{T}Wy$. Can we classify this model as linear or non-linear using matrix properties etc.

Upendra01
  • 1,566
  • 4
  • 18
  • 28
  • also is linearity measured with respect to features or parameters? In one of the papers, authors comment that a function F is bilinear in x and y but is linear with respect to parameter W; function F is defined as $F=x^{T}Wy$ – Upendra01 Mar 14 '18 at 07:07

1 Answers1

0

I suggest to add this answer to your reading list: How to tell the difference between linear and non-linear regression models?

If the result of a vector calculation is a vector (a collection of values) and not a scalar, then we call it linear if every coordinate of the result is linear, and we call it nonlinear if at least one coordinate is nonlinear.

If $W\in\mathbb{R}^{n\times n}$, $x,y\in\mathbb{R}^n$ (we have many realisations and every realisation is $n$ dimensional), then $$F(x,y)=x^T W y = \sum_{i=1}^n \sum_{j=1}^n W_{ij} x_i y_j\in\mathbb{R}.$$

There is ambiguity in how to read what you are asking.

If $x$ and $y$ are both independent variables/regressors/explanatory variables (and you just haven't written out the dependent variable/observation/response variable $z\in\mathbb{R}$, which is a scalar), then the model $$z=F(x,y)+\varepsilon = \sum_{i=1}^n \sum_{j=1}^n W_{ij} x_i y_j +\varepsilon$$ with error $\varepsilon$ is a linear model of a nonlinear relationship: it is linear in the parameters $W_{ij}$ but whenever $W_{ij}\neq 0$, $W_{ij} x_i y_j$ is a nonlinear term. (See what I linked in at the beginning.)

If you meant something like $F(x,y)=c+\varepsilon$ for a constant c, where $x$ is the vector of independent variables and $y$ is the vector of response variables, then that just defines a system of equations (like in the other questions) and you'd need to examine it in detail to understand this implicitly given statistical model.

Bence Mélykúti
  • 445
  • 1
  • 3
  • 10