4

Suppose you have 3 variables: height, weight and salary. Can you first attempt to fit a 3 dimensional probability distribution to this data - then, if someone gives you a height and weight measurement, could you use the probability distribution to estimate the probability of observing a range of salaries for a fixed value of weight and height?

This would in effect produce a conditional probability distribution for this specific combination and you can take the expected value of this distribution to predict the most likely salary?

Here was my thought: once the 3 dimensional distribution was fit to this data, for a height = 180 cm and weight = 100 kg:

Probabilith (Salary = $10,000 | height = 180 cm, weight = 100 kg) = 0.2

Probabilith (Salary = $10,500 | height = 180 cm, weight = 100 kg) = 0.1

Probabilith (Salary = $9,000 | height = 180 cm, weight = 100 kg) = 0.01

Etc.

Evaluating many such values of salary combinations, you can derive a posterior distribution. There would be no need for beta regression coefficients in this approach.

Is this correct?

Or this is a flawed idea and is it better to just fit a regression model to this problem?

Thanks!

stats_noob
  • 5,882
  • 1
  • 21
  • 42

2 Answers2

3

I am fairly sure it's valid to treat regression model as a joint probability distribution. Let $s,h,w$ be salary, height, weight, then a linear regression

$$ s = \beta_0 + \beta_1h + \beta_2w + \epsilon \\ \epsilon \sim N(0, \sigma^2) $$ where $\beta_0, \beta_1, \beta_2$ are regression coefficients posits the distribution

$$ s \sim N(\beta_0 + \beta_1h + \beta_2w, \sigma^2) $$

The mean salary for a specific combination of weight and height is exactly what is being modelled in a linear regression, and this is also exactly what a conditional expectation of salary for a given weight and height would aim to provide in your approach.

Of course the accuracy of this prediction depends on the normality of the error term $\epsilon$; if this assumption is violated then predictions will not be accurate.

alan ocallaghan
  • 1,108
  • 7
  • 19
  • Thank you for your answer! Can you please see the updates? I tried to re-explain my problem there. Thank you! – stats_noob Sep 19 '21 at 02:37
  • 1
    You could use Bayesian linear regression to derive a closed form estimate of the posterior predictive distribution that you're describing in that more detailed example. Perhaps it would be good to explain why you wish to avoid using regression and how you would fit a 3-dimensional distribution to the data instead? – alan ocallaghan Sep 19 '21 at 03:06
  • Thank you for your answer! I do not want to avoid using a regression model - I am just curious if the approach I described is logical. thank you! – stats_noob Sep 19 '21 at 04:09
  • 1
    @stats555: Please register &/or merge your accounts (you can find information on how to do this in the **My Account** section of our [help]), then you will be able to edit & comment on your own question. – kjetil b halvorsen Sep 19 '21 at 14:12
3

You can, but not without consequences.

  • Linear regression is a pretty flexible model in terms of your ability to define the functional relationship between the features and the dependent variable. If you use multivariate distribution, you are limited by what kind of relationships between variables are possible under the distribution.
  • For fitting the distribution you need to make more assumptions, for example, if you choose multivariate normal distribution you assume that all the variables follow normal distribution vs only $Y$ (conditionally) as in linear regression.
  • For fitting the distribution you may need much more data. Think of discrete distribution: in conditional distribution case (regression) you need only enough data to observe relations of the other variables with $Y$, with the joint distribution you need data for all the combinations of all the levels of all the variables.

It is easier to focus only on the conditional distribution and conditional expectation, as we do with linear regression.

Tim
  • 108,699
  • 20
  • 212
  • 390
  • Regarding the 2nd bullet point, in regression, we make assumptions about the *conditional* distribution of $Y$, not the marginal. – Richard Hardy Sep 19 '21 at 11:04
  • To directly model a continuous univariate probability distribution you can use semiparametric regression models such as the proportional odds model. These models encode the entire empirical cumulative distribution function as model intercepts. – Frank Harrell Sep 19 '21 at 11:17
  • @RichardHardy right, corrected. – Tim Sep 19 '21 at 11:17