Suppose we have a Bernoulli process with failure probability $q$ (which will be small, say, $q \leq 0.01$) from which we sample until we encounter $10$ failures. We thereby estimate the probability of failure as $\hat{q}:=10/N$ where $N$ is the number of samples.
Question: Is $\hat{q}$ a biased estimate of $q$? And, if so, is there a way to correct it?
I'm concerned that insisting the last sample is a fail biases the estimate.
It is true that $\hat{q}$ is a biased estimate of $q$ in the sense that $\text{E}(\hat{q}) \neq q$, but you shouldn't necessarily let this deter you. This exact scenario can be used as a criticism against the idea that we should always use unbiased estimators, because here the bias is more of an artifact of the particular experiment we happen to be doing. The data look exactly as they would if we had chosen the number of samples in advance, so why should our inferences change?
Interestingly, if you were to collect data in this way and then write down the likelihood function under both the binomial (fixed sample size) and negative binomial models, you would find that the two are proportional to one another. This means that $\hat{q}$ is just the ordinary maximum likelihood estimate under the negative binomial model, which of course is a perfectly reasonable estimate.
It is not insisting that the last sample is a fail which biases the estimate, it is taking the reciprocal of $N$
So $\mathbb{E}\left[\frac{N}{10}\right] =\frac{1}{q}$ in your example but $\mathbb{E}\left[\frac{10}{N}\right] \not = q$. This is close to comparing the arithmetic mean with the harmonic mean
The bad news is that the bias can increase as $q$ gets smaller, though not by much once $q$ is already small. The good news is that bias decreases as the required number of failures increases. It seems that if you require $f$ failures, then the bias is bounded above by a multiplicative factor of $\frac{f}{f-1}$ for small $q$; you do not want this approach when you stop after the first failure
Stopping after $10$ failures, with $q=0.01$ you will get $\mathbb{E}\left[\frac{N}{10}\right] = 100$ but $\mathbb{E}\left[\frac{10}{N}\right] \approx 0.011097$, while with $q=0.001$ you will get $\mathbb{E}\left[\frac{N}{10}\right] = 1000$ but $\mathbb{E}\left[\frac{10}{N}\right] \approx 0.001111$. A bias of roughly a $\frac{10}{9}$ multiplicative factor
As a complement to dsaxton's answer, here are some simulations in R showing the sampling distribution of $\hat{q}$ when $k=10$ and $q_0 = 0.02$:
n_replications <- 10000
k <- 10
failure_prob <- 0.02
n_trials <- k + rnbinom(n_replications, size=k, prob=failure_prob)
all(n_trials >= k) # Sanity check, cannot have 10 failures in < 10 trials
estimated_failure_probability <- k / n_trials
histogram_breaks <- seq(0, max(estimated_failure_probability) + 0.001, 0.001)
## png("estimated_failure_probability.png")
hist(estimated_failure_probability, breaks=histogram_breaks)
abline(v=failure_prob, col="red", lty=2, lwd=2) # True failure probability in red
## dev.off()
mean(estimated_failure_probability) # Around 0.022
sd(estimated_failure_probability)
t.test(x=estimated_failure_probability, mu=failure_prob) # Interval around [0.0220, 0.0223]
It looks like $\mathbb{E}\left[ \hat{q}\right] \approx 0.022$, which is a rather small bias relative to the variability in $\hat{q}$.
