Sample is almost the same as the population

Question

I'm interested in a situation where I have a small list of events, somewhere in the range ten to a few hundred.

1. Each event can be a 1 or 0 outcome.
2. I have a good reason to believe that the factors controlling the 1 or 0 outcome are complicated and a "1" does not necessarily occur a constant rate.
3. I already know the outcome of all events in the list to date.
4. There will be future events whose outcome is unknowable at present.

I want to use all the information available to help me estimate the rate of "event outcomes = 1", especially for the next event to occur. I actually also have a number of lists A, B, C etc, each where I am interested to understand how similar or different their rates are.

I've looked at these questions and their answers:
statistical-inference-when-the-sample-is-the-population

what-is-the-difference-between-a-population-and-a-sample

Help needed with terminology

I had originally assumed that the current list of events that have taken place is the "sample" and that the "population" refers to some larger group, that could be the current list + 1, or more. I don't mind how we define it as long as we don't lose sight of the practical human language and meaning that I need to operate in the real world.

I am interested in the next event specifically, and probably won't become interested in the one after that until the next has occurred.

Discussion

I follow that if population = sample then there is no role for statistical tests. I think that perhaps population = sample + 1 because I am interested in the next event. However if the sample is almost the same size as the population then there is still little scope for the statistic to vary from sample to sample, especially if the sample cannot be held to be a random sample of the population.

Question: To establish some measure of belief in the rate of event outcomes, the proportion p of population n, should I use a measure such as

1. $$ \pm \sqrt{ \frac{p ( 1-p)} {n}} $$

2. some other measure based on the changes to p that would occur upon re-evaluation after then next future event, should it turn out to be either a 0 or a 1.

Answer 1

I suspect that your events are not independent but rather involve some sequential dependencies, because you mentioned that the rate is not constant and don't know any controlling variables. The sample is the portion of the data randomly sampled from the population. Since you use a deterministic range to define your sample (e.g. select first N events out of the list of N+1 events) you can not treat it as a valid sample.

Thankfully, as you already noted, you can compute your population statistics directly. Yet those statistics wouldn't lead to any reasonable outcomes, because you've also noted that the rate is not constant - and those statistics can provide you only with a constant.

You can approach the prediction problem in multiple ways. I'll provide you one just to give you a hint. Your problem is actually to model a probability $p(e_t = 1 | e_{t-1}, e_{t-2}, ..., e_0)$ of observing the event at time $t$ given the information about all previous events. It is often the case that it is not necessary to consider all of the previous events, but only limit yourself the last K. You end up with $p(e_t = 1 | e_{t-1}, e_{t-2}, ..., e_{t-K})$. For simplicity you could just assume, that this probability equals to the rate of the events within this K-length window:

$$p(e_t = 1 | e_{t-1}, e_{t-2}, ..., e_{t-K}) = \frac{1}{K}\sum_i^K e_{t-i}$$

You can then evaluate this model on different windows sampled from your list in order to find the best window size. You can then improve this model by introducing new assumptions and/or parameters. Take a look at language models and kernel density estimation.