1

Question: (1) Given $N$ IID zero-meaned 3-D (x, y, z) points on a plane, find the equation for the plane. The plane must pass through (0,0,0). (2) Solve problem (1), but now assume every point has some small IID Gaussian error.

Using ordinary least squares, I know that centering $x,y,z$ means that $$ \sum_{i}^N x_i = \sum_{i}^N y_i = \sum_{i}^N z_i = 0 $$ The plane is of the form $z = \hat{\beta}_1x + \hat{\beta}_2y + \hat{\beta}_0$ (here, I am assuming $z$ is the response variable, and $x$ and $y$ are the regressors).

Since $x$ and $y$ are centered, this means $$ \hat{\beta}_0 = \bar{z} $$ Since $z$ is centered, we have $$ \hat{\beta}_0 = 0 \\ \therefore \ \ \text{the plane is guaranteed to pass through the origin} $$

Why would the problem additionally specify that the plane must pass through the origin? Centering $x,y,z$ seems to make the plane pass through the origin by definition. Am I missing something?

For part (2), it's not clear to me how perturbing each sample by a small gaussian error that is IID affects the problem, other than that in the first problem, the least squares estimators are unbiased because there's no error in the points, but bias is possibly introduced when perturbing each sample. Does perturbing each sample make OLS more susceptible to overfitting?

24n8
  • 847
  • 3
  • 13
  • 1
    Your initial attempt is not sufficiently general, because the plane could be vertical. Use, therefore, the formulation $\beta_1 x + \beta_2 y + \beta_3z = 0$ and make sure not to confuse parameters with their estimates, for then you will get lost in problem (2). BTW, "zero-meaned" does not mean the data are centered. It only means the data arise from a distribution whose mean is $(0,0,0).$ In problem (1) the points are *guaranteed* to be coplanar but in problem (2) they are almost surely not coplanar when $N\ge 3.$ – whuber Jul 01 '20 at 13:35
  • @whuber Ahh, you're right -- my formulation was wrong. https://stackoverflow.com/questions/1400213/3d-least-squares-plane#:~:text=9%20Answers&text=Then%20solve%20Ax%20%3D%20b%20for,function%20of%20x%20and%20y. has a similar question, but the top 2 answers seem to have approached it the same way I did initially. I another issue with part (1) is if the points are all collinear, then $X^TX$ is rank deficient, and you end up having non-unique solutions for the plane equation. – 24n8 Jul 01 '20 at 16:31
  • @whuber What is the point of being provided that the data comes from a zero-meaned distribution? That information doesn't seem to be used for parts (1) and (2). – 24n8 Jul 01 '20 at 16:31
  • It can be used in (2) to narrow the model specification: you know the true intercept is $(0,0,0)$ and do not have to estimate it. – whuber Jul 01 '20 at 16:43
  • @whuber But don't you know the true intercept is (0,0,0) based purely on the fact that the plane passes through the origin? – 24n8 Jul 01 '20 at 17:02
  • In problem (2), "the plane" is an ambiguous construct: if the model includes an intercept, it probably does *not* pass through the origin. – whuber Jul 01 '20 at 18:06
  • @whuber Sorry, I think I still don't understand something. So here is what I think so far: (1) Part 1, because the points are specified to be exactly on the plane, there's no unobserved error, hence solving the least squares equations give the population parameters. The "zero-meaned" distribution of $x,y,z$ isn't used. The "origin passes through plane" specification guarantees that $\beta_0 = 0$. – 24n8 Jul 01 '20 at 21:14
  • @whuber (2) Part 2, You perturb each data point with some noise. Solving the least squares equations now gives you an estimate of the population parameters found in part (1), but in addition, we now have to solve for the intercept term $\hat{\beta}_0$? – 24n8 Jul 01 '20 at 21:15
  • Part of the potential for confusion is that this question describes at least three models leading (I believe) to three different solutions: yes, the $\epsilon$ are *iid* Gaussian, but that involves two parameters $\mu$ and $\sigma:$ which one(s) do you suppose you need to estimate? If $\mu$ is not assumed to be $0,$ the points will be offset on average some nonzero distance in the direction $(1,1,1)$ and might not even come close to the origin. Thus, the *model plane* contains the origin, but a *least squares fit to the points* need not contain the origin. – whuber Jul 01 '20 at 21:31
  • @whuber, Iamanon I don't think you can use least square to solve $\beta_1x + \beta_2y + \beta_3z = 0$. Consider a plane $x + y + z = 0$. It is obvious that $z$ is a linear combination of $x$ and $y$. So if you try to formulate your data matrix $X$ where the columns are $x,y,z$, the column corresponding to $z$ becomes a linear combination of the columns for $x$ and $y$. – roulette01 Jul 01 '20 at 21:50
  • @dd22205 On the contrary, least squares directly applies and shows that a solution is given by an eigenvector $(\beta_1,\beta_2,\beta_3)$ of either an SSP or covariance matrix corresponding to the smallest eigenvalue. (Which matrix to use depends on whether you assume the $\epsilon$ have zero mean or not, respectively.) – whuber Jul 02 '20 at 12:45
  • @whuber I actually asked about this question on the Mathematics SE https://math.stackexchange.com/questions/3741830/fitting-least-squares-to-3-d-points-to-get-an-equation-for-a-plane?noredirect=1#comment7694716_3741830. Basically I used the plane $x + y + z = 0$ to generate 4 points, and then tried to use the 4 points to reconstruct the plane using least squares. But the issue is, as dd22205 said above, you get linearly dependent columns in your data matrix, $X$, because the equation $x + y + z = 0$ is structured such that one of the variables is a linear combination of the other 2. – 24n8 Jul 02 '20 at 16:13
  • You're not solving the problem correctly. Use PCA. Your question is basically a duplicate of https://stats.stackexchange.com/questions/16327/. – whuber Jul 02 '20 at 16:16

0 Answers0