11

Population stability index quantifies the change of a distribution of a variable by comparing data samples in two time periods. It is very commonly used to measure shifts in scores.

It is calculated as follows:
1) The sample from base period is discretized. Usually it is partitioned into deciles
2) The sample from target period is discretized using the same intervals as in first step
$PSI = \sum_{i} (A_{i} - B_{i}) \cdot ln(\frac{A_{i}}{B_{i}})$
Where:
$A_{i}$ - share of i-th bin in base period. $B_{i}$ - share of i-th bin in target period.

Question: What should be done when one of the bins from target sample is empty?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
Tomek Tarczynski
  • 3,854
  • 7
  • 29
  • 37
  • I usually join the class with no elements with an adjacent class that has elements. Hope that helps. C. – CSands Dec 13 '13 at 12:49
  • @CSands: Thanks for answer, but if I understood correctly then in case when all observations from ninth decile shifts to the tenth decile your solution would join ninth and tenth decile and PSI would be 0, where in reality the distribution has changed significantly. Am I right? – Tomek Tarczynski Dec 13 '13 at 13:54
  • Rather than using bins, is it possible to use density estimates and replace the sum with an integral? – dsaxton Mar 22 '16 at 02:53
  • 1
    You could probably make a better estimator by adapting the ideas from my answer here: https://stats.stackexchange.com/questions/211175/kullback-leibler-divergence/248657#248657 – kjetil b halvorsen Apr 20 '17 at 21:00

3 Answers3

3

I guess you could consider the empty bins as filled with a very small number. This retains the information and avoids division by zero. And, of course, this way you keep the original bins, which is a good thing.

rjeronimo
  • 31
  • 2
  • 4
    This recommendation needs some analysis. For instance, how sensitive is the result to the choice of that "very small number"? If it's relatively insensitive, then why not propose taking the limit as that small number approaches zero from above? If it is sensitive, then the answer would effectively be arbitrary. – whuber Sep 29 '15 at 15:37
  • Agreed. ln(Ai/Bi) shouldn't grow too fast, because of the logarithm, but still the 'small Bi' should be chosen so that it doesn't invalidate the PSI measure. – rjeronimo Sep 29 '15 at 21:45
  • I don't see how that is mathematically possible. – whuber Sep 30 '15 at 13:10
  • for example, if you only consider the % in each bin rounded to 2 decimal places, you could use 0.01 instead of zero. – rjeronimo Sep 30 '15 at 21:31
  • "very small number" could be [Machine epsilon](https://en.wikipedia.org/wiki/Machine_epsilon) – sds Jun 01 '16 at 17:37
0

One way is to assign one count to such bins and then calculate the fraction considering this new point:

counts[counts==0] = 1
fract = counts/sum(counts)

If your sample is large enough that few counts are negligible, then I think it is a good approximation. The information that you have after adding these "phantom" counts is almost the same (always considering the validity of new points in your sample): very very very ... few points from your sample fall down in these bins.

xer-t
  • 1
-2

Could you skip it? That is, could you understand it as zero？

The division by zero is uniquely and reasonably determined as 1/0=0/0=z/0=0 in the natural extensions of fractions. We have to change our basic ideas for our space and world Division by Zero z/0 = 0 in Euclidean Spaces Hi roshi Michiwaki, Hiroshi Okumura and Saburou Saitoh International Journal of Mathematics and Computation Vol. 28(2017); Issue 1, 2017), 1 -16. 　 http://www.scirp.org/journal/alamt 　 　 http://dx.doi.org/10.4236/alamt.2016.62007 http://www.ijapm.org/show-63-504-1.html http://www.diogenes.bg/ijam/contents/2014-27-2/9/9.pdf http://okmr.yamatoblog.net/division%20by%20zero/announcement%20326-%20the%20divi http://okmr.yamatoblog.net/ Relations of 0 and infinity Hiroshi Okumura, Saburou Saitoh　and Tsutomu Matsuura： http://www.e-jikei.org/…/Camera%20ready%20manuscript_JTSS_A… https://sites.google.com/site/sandrapinelas/icddea-2017

Saburou.Saitoh
  • 1