0

I'm working on a cross-country study where I look at the impact of different regulatory variables on bank stability (so the dependent variable is at the country-bank-year level and the variable of interest is at the country-year level). My data set is an unbalanced panel which consists of 5000 banks operating in 39 countries over 2000-2015. I did my analysis by accounting for both bank and year fixed effects and clustering standard errors at the bank level, in which case, my variable of interest is significant at 5%. However, a few people suggested to me that since my variable of interest is at the country level, I should cluster standard errors at the country level. When I did that, the level of significance of my variable of interest drops to 10%.

However, I'm concerned that 39 clusters is too few, specially in an unbalanced panel data set like mine (I read somewhere that the rule of thumb is at least 50 clusters) and that it would distort the standard errors. Do you think that using a few clusters will be an issue here?

What do you think is the most appropriate unit of clustering that I should use? Country-level or bank-level?

Thank you in advance.

Ama Perera
  • 11
  • 3
  • Welcome. Once you start to get under 50 clusters then cluster robust uncertainty estimators don’t perform well. There is some literature on this but no hard and fast rule. See [here](http://cameron.econ.ucdavis.edu/research/Cameron_Miller_JHR_2015_February.pdf) for more information. Do your regulatory variables affect all banks within a country? – Thomas Bilach Aug 12 '20 at 14:26
  • It would help if you were more specific about which clustering algorithm you intend to use. Using an unsupervised method like K-means and given that the 39 country-level estimates are based on 5,000 banks, the small sample size is quite robust and, therefore, not a theoretical problem. If you were clustering, e.g., 39 consumers, it would be a different, more problematic issue. One alternative would be to use a supervised method such as latent class clustering, an all-in-one approach. The structure of the model would be the same but your unit of analysis for the classes would be the country – Mike Hunter Aug 12 '20 at 18:46
  • Are your 39 counties the population of interest that you want to study? In other words, if you went out and got more data, would the number of counties increase? – dimitriy Aug 13 '20 at 06:12
  • Yes, that would be the case. – Ama Perera Aug 13 '20 at 15:04
  • Does it have an impact on the level of clustering? – Ama Perera Aug 15 '20 at 08:11

1 Answers1

1

I'm working on a cross-country study where I look at the impact of different regulatory variables on bank stability (so the dependent variable is at the country-bank-year level and the variable of interest is at the country-year level).

Your banks are nested within countries, but the regulatory variables of interest are at the country level. Regulation more than likely impacts all banks within that particular region/country. And, the within-country errors are likely not independent of each other. This supports clustering at the country level.

My data set is an unbalanced panel which consists of 5000 banks operating in 39 countries over 2000-2015.

Cluster-robust uncertainty estimators perform poorly with scanty clusters. In your case, 39 is getting a little low in my estimation. Applied research in this area suggests anything north of 40 is sufficient for the cluster variance formula to be accurate. Chapter 8 of Mostly Harmless Econometrics by Angrist and Pische (2009) offers a thorough appraisal of clustering issues and serial correlation in panel data models. You could also check out their blog.

Here is a quote from Cameron and Miller's practitioner's guide to cluster-robust inference on page 3:

There is no clear-cut definition of “few”; depending on the situation “few” may range from less than 20 to less than 50 clusters in the balanced case.

I would review some of the finite sample corrections recommended in this guide. And, I would venture to say anything below 50 clusters (definitely 40) is cause for concern.

However, a few people suggested to me that since my variable of interest is at the country level, I should cluster standard errors at the country level. When I did that, the level of significance of my variable of interest drops to 10%.

I agree with this advice. You're primarily interested in regulation instituted at the country level—so cluster there. In general, uncertainty estimates will be more conservative if you cluster at this higher level. Review this post for more information.

I would also avoid obsessing over "significance" in applied work. Recent scholarship is exhorting researchers to eschew the dichotomization of p-values. It puts you in a position where you're fishing for significant results. I would try out a couple of different uncertainty estimators (clustering at the country level), maybe even a pairs cluster bootstrap approach, and see if your results change.

Again, 39 clusters ins't big and it isn't small. I doubt anyone will fault you for attempting other finite-cluster corrections. I think others on this forum will agree with a country clustering scheme. If there is any disagreement then I am sure others will offer their input.

I hope this helps!

Thomas Bilach
  • 4,732
  • 2
  • 6
  • 25