0

I'm trying to understand how I would decide whether a policy actually had a meaningful impact by comparing the population itself, not samples.

An example that comes to mind is comparing the number of accidents on a specific road during $n$ months prior to enacting a law to reduce the speed on that road, to $n$ months after enacting this law.

I would think that I could use a paired t-test using biased standard deviation, since I'm looking at absolute values of the actual population, not samples, but I see a few problems with that: how would I calculate the Standard Error for the t-value if there is no error because I'm looking at the population itself? How would I calculate standard deviation, if there is no mean?

What I'm really trying to understand is: say in my pre-test I get an $x$ number of accidents and in the post-test I got a $y$ number of accidents where $0 < x < y$. How could I know if this change was statistically significant? If someone could just point me to the right direction I can study the subject. I just don't know where to look.

Danilo Souza Morães
  • 123
  • 3

2 Answers2

1

It seems to me that if your research question is so specific that it's only about the number of accidents that happen on a given street in a certain time period, there's no element of random chance skewing your results. Therefore, you don't have to worry about standard deviations and things being significant or not. If you have all the possible data, the difference observed is significant.

Benji
  • 129
  • 9
1

I've seen results reported based on a census of the population, nicely accompanied by t-test significance values. These tests are designed to make inferences from a random sample of the population. As Benji said, when you cover the entire population, effects are "significant" by default. But how then do you separate the wheat from the chaff? By interpreting their effect size.

That being said, you didn't sample the population: you have a sample of months, in a population of many more months that could have been sampled. Your data are probably zero-inflated poisson distributed.

Caspar van Lissa
  • 121
  • 5