0

Today we started arguing at work, and couldn't come to conclusion. Let's say that we have a population of 1000 observations about various people. 50 of these people went bankrupt (1 - bankrupt, 0 - did not went bankrupt). Can we take sample of 100 people (50 bankrupt, 50 not bankrupt) and use them to make a model of bankruptcy (using linear regression or MDA)? Or must we take a random sample of 100 people, which should include around 5 people that went bankrupt?

Do we have to keep the same proportions as in population in modelling sample, to use the model on population ?

What problems would occur with 50-50 sample ?

Thanks!

PolBM
  • 1,363
  • 11
  • 19
Testeris
  • 83
  • 1
  • 1
  • 4
  • Basically: in 50-50 sample your model will see data where 50% people go bankrupt, so it will (correctly given data!) assume that the chance of getting bankrupt is 50%. I guess this is not what you want your model to conclude... See e.g. https://stats.stackexchange.com/questions/306122/rare-events-logistic-regression and https://stats.stackexchange.com/questions/283170/when-is-unbalanced-data-really-a-problem-in-machine-learning for the other side of the coin. – Tim Nov 08 '17 at 14:08
  • 1
    @Tim, Your conclusion will be true only for some models. If you have two artificially created clusters where one has 1000 data samples and the other one has 100. Assuming that they are linearly separable (+noise), training model using 50 samples from each clusters shouldn't produce model that make 50%-50% prediction. – itdxer Nov 08 '17 at 14:25
  • @itdxer true, but the question asks about potential risks and mentions regression, so this is one of the potential risks. – Tim Nov 08 '17 at 17:37

2 Answers2

0

Depends on the question you want to answer,

Taking 50-50 may not give you the result and would be valid conclusion, statistically and again taking random 100 may not give you enough power to catch enough bankrupt subjects as is taking 5 bankrupt subjects. The sample size should be determined ideally based on assumption and will make sure that 1) you have enough power to detect the association that you are testing 2) makes generalisation to the whole population valid.

Taking 50 bankrupt and random sample of another 50 who are bankrupt could make sense. I would say that the best way would be to sample a number of n subjects based on the strength of association you hypothesize (power calculation) and then you can easily generalize it to the whole population.

Sanni
  • 21
  • 1
0

The technique of artificially adding positive (bankrupt) data points to your sample for developing a model is called seeding. It is necessary to develop a model with the appropriate threshold, especially when positive cases are rare (think cancer screening). The sensitivity and specificity of the model can be reported based on the seeded data set. However, the model’s positive predictive value and negative predictive value will require an estimate of the prevalence of the positive cases in the real world. They are calculated using Bayes’ theorem.

jeffalltogether
  • 534
  • 4
  • 12