TLDR: I need to estimate what is the minimum sample size to get a W-wide confidence interval of daily Response Time's P95 of which I only get a sample of events.
Detailed: Due to a Service Level Agreement, I'm trying to estimate the confidence interval for the 95th percentile of different services each day. Each one has Mi number of daily requests (large population), but I only get telemetry of some of the events (Ni randomly) which are my samples each day ( Mi >> Ni ).
Alternative, I could just get the sample size required to have T milliseconds range in my confidence interval around the sampled P95 and then use that as a threshold to be considered.
So far I was using Cox Method to estimate the CI of the mean, but observing the distribution of the sampled P95 values I get, it's right skewed and, as I understand, that invalidates Cox Method. (Not to mention, I'm not actually sure that Cox could be used to any parameter or just the mean)
On a different point of view, a colleague believes that given P95 is in fact determined by the top 5% of the samples, I could just use the standard confidence interval formula for unknown variance, but with n = Ni/20.
Any comments would be much appreciated.
