6

I have two overlapping frequency distribution, one of the buyers' demand or willingness to pay and the other one is seller's reservation price frequency distribution.

The two distributions overlap and I'd like to estimate the overlapping area.

freq_dist

What statistical properties / methods can I use to estimate this area?

kms
  • 530
  • 2
  • 17
  • 1
    Do you have access to the PDF of each distribution? – mhdadk Jan 18 '21 at 09:20
  • Yes, we can assume I can find the PDF of each distribution. – kms Jan 18 '21 at 13:22
  • Related: https://stats.stackexchange.com/questions/271582/is-the-intersection-area-between-2-pdfs-a-probability/271590#271590 – kjetil b halvorsen Jan 18 '21 at 14:47
  • 1
    See: https://stats.stackexchange.com/questions/103800/calculate-probability-area-under-the-overlapping-area-of-two-normal-distributi/ – wolfies Jan 18 '21 at 15:54
  • 1
    I wonder if someone could explain what does it actually mean to calculate overlap of two distributions. What is the quantity that you're trying to derive? Also: what exactly is on the horizontal axis? I presume something related to price? A price that an agent is willing to pay/accept? – user1079505 Jan 18 '21 at 22:36

2 Answers2

6

The distributions $X\sim\mathsf{Norm}(100, 15)$ and $Y\sim\mathsf{Norm}(110,15)$ overlap, as in your figure. The total overlap probability is $$P(Y \le 125)+P(X> 125)\\ = P(Y \le 125) + 1 - P(X \le 125)\\ \approx 0.048 + 0.048 = 0.096.$$

enter image description here

R code for figure:

hdr="Densities of NORM(100,15) and NORM(150,15)"
curve(dnorm(x,100,15), 50, 200, ylab="PDF", main=hdr)
curve(dnorm(x,150,15), add=T)
abline(h=0, col = "green2")
abline(v=125, col = "red", lty="dotted")

R code for probability computation, where pnorm is a normal CDF:

pnorm(125, 150, 15)
[1] 0.04779035
1 - pnorm(125, 100, 15)
[1] 0.04779035
pnorm(125, 150, 15) +  1 - pnorm(125, 100, 15)
[1] 0.0955807

Note: The two probabilities might be Type I and Type II error for a test of a hypothesis.

BruceET
  • 47,896
  • 2
  • 28
  • 76
  • +1 -- if we can assume normality. – Bernhard Jan 18 '21 at 12:14
  • Hi - You make use of the point P at 125 but the OP doesn't say s/he has it. Can you check my answer to see if it's a valid alternative? – dariober Jan 18 '21 at 14:44
  • OP's curves seem normal and bits of available context support that guess. In any case, there must be a crossover point such as my 125 (or it's a different problem) and it seems easiest to take advantage of that. // Many texts have such plots as this to illustrate Type I and II error in hypothesis testing.// So I don't think my answer 'cheats'. Your idea seems to pose and solve a more general problem--assuming both curves are densities. – BruceET Jan 18 '21 at 19:55
2

Perhaps this is a more general solution than @BruceET's which doesn't assume normality or a preset reference point P. The OP says s/he has the PDF of the two distributions, so for example these may be:

pdf1 <- function(x, mean= 100, sd= 15) {
    pdf <- (1 / (sd * sqrt(2 * pi))) * exp(-0.5 * ((x - mean)/sd)^2)
    return(pdf)
}

pdf2 <- function(x, mean= 150, sd= 15) {
    pdf <- (1 / (sd * sqrt(2 * pi))) * exp(-0.5 * ((x - mean)/sd)^2)
    return(pdf)
}

(These are Gaussian but they may be any PDF)

At a grid of data points on the x-axis calculate each PDF, take the smallest of the two densities, weight it for the step size of the grid and sum across to get the intersection (this is a very crude integration - I'd like to see a better solution...):

step <- 0.1
at <- seq(20, 250, by= step)
x1 <- pdf1(at, 100, 15)
x2 <- pdf2(at, 150, 15)

area <- 0
for(i in 1:length(at)) {
    area <- area + ifelse(x1[i] > x2[i], x2[i], x1[i]) * step
}
print(area)
0.09558193
dariober
  • 2,805
  • 11
  • 14
  • Didn't take time to follow/verify all details, but numerical integration approach seems correct. A figure would make this clearer. // Same two normal distributions as in my Ans and essentially same numerical result, so details must be OK. – BruceET Jan 18 '21 at 19:52