Possible to do Fourier decomposition using Linear Regression?

Question

I was wondering whether it's possible to do Fourier decomposition using linear regression.

Let me explain this.

Given the samples $(x_1 y_1), \dots, (x_n, y_n)$ with $x_i, y_i \in \mathbb R$, we can maximize the likelihood $$ \begin{pmatrix} y_1 \\ \vdots \\ y_n \end{pmatrix} \sim \mathcal N\left(Xw, I\sigma^2\right) $$ of $y_1, \dots, y_n$ w.r.t. the vector $w$, where $$ X = \begin{pmatrix} \varphi\left(x_1\right) \\ \vdots \\ \varphi\left(x_n\right) \end{pmatrix} $$ and $\varphi : \mathbb R \to \mathbb R^m$ is a non-linear transformation, by choosing $$ w = \left(X^\mathsf{T} X\right)^{-1} X^\mathsf{T} y \text. $$

Clearly, the vector $w$ weights the contribution of the features $\varphi_1, \dots, \varphi_m$ to the linear model.

Now, assume that all $x_i \in \left[0; 1\right]$ and that we choose $$ \varphi_i\left(x\right) = \cos\left( \frac{2\pi}{i} x + \beta_i \right) \text. $$ I believe, that if we knew the correct phase $\beta_i$ of the frequency $2\pi/i$ in the Fourier spectrum $\hat x$ of $x$ on $\left[0; 1\right]$, then $w$ would recover the magnitudes of the frequencies $2\pi/1, \dots, 2\pi/m$ in $\hat x$, wouldn't it? If yes, is there any chance to estimate $\beta_i$? Of course, without actually computing $\hat x$.

Answer 1

Of course you can do this, meaning neither the regression nor the Fourier transformation police will come and bust your house. But would it make sense? I think not.

I guess you are trying to decompose a function(!) of x, where $y_i=f(x_i)$. But this is not what regression is about. Regression assumes that you do not see $f(x_i)$ directly but only noisy observations $y_i=f(x_i) + \epsilon_i$ of it. And only from these errors you get the normal distribution of $y$.

On the other hand you can always define a subspace $span\{\phi_i, i=1,\ldots,n\}$ and project $f$ onto this subspace orthogonally to get coefficients $w$ which are then least squares optimal. But this makes only sense if there is something meaningful or interesting about your subspace. Furthermore this only works for finite dimensional subspaces and Fourier analysis is really more about infinite dimensional problems.

Answer 2

You can do that (estimate the magnitudes via regression), but you can actually estimate the magnitudes and the phases using regression, so it will still work when the phases are unknown, and it will work in the presence of noise in $y$ (in the sense at least that you can still estimate those coefficients, though themselves with some noise).

Specifically, rather than just fitting $\cos$ terms, if you fit both $\sin$ and $\cos$ terms, you can obtain both magnitude and phase.

Using trigonometric terms in this way is discussed here, which discusses fitting several harmonics of the period, as well as estimation of both phase and amplitude.

If frequency is also unknown you have a nonlinear problem and in that case you would use other approaches (such as nonlinear regression discussed in another answer to the first question linked here).