Why is there no standard script or software package for regression which explains and interprets results in layman's terms

Question

This is a very basic question, but I find myself constantly frustrated with the same questions as I review regression results and read statistics forums. The question is whether or not some sort of software or automated approach exists to determine:

1) Which type of regression to run. For example, based on the distribution of the dependent variable, x would work. There seem to be many types to run, and it's never clear cut.

2) As a result of regression, how to interpret the results. Statistics programs give you a p-value, R-squared, and sometimes other terms, which you then need to look up. It's never clear if what it says to be significant is actually significant, or the result of some flaw in the data that then needs to be uncovered.

3) Analysis of results. There are all sorts of charts and plots, but all I really ever want to know is what model yields the best results to explain or predict the data. There seem to be endless diagnostics, but I never know what to run or how to do it.

I realize this is a very generic question, and perhaps I'm looking at this all wrong, but I imagine there would be a standard approach to doing this, and you should be able to provide a few columns of data, run tests, and get results that can be interpreted. Instead, what I find is that results come out using terms that are confusing to understand in plain English, and then there are further tests that than either validate or invalidate these results, which lead to more obscure terms. I get that this is why there are experts, but I'm looking for some explanation of how to approach an exercise where you are building a regression model to predict or explain something.

I'm a beginner here, but statistics seems like a world of never ending confusion, and often statisticians will dispute models and approaches, so that further compounds my problem. Typically when faced with these types of topics that are overly confusing, I arrive at "this is probably nonsense or something misleading is going on here." At least that's what I thought about complex derivatives in finance, and felt validated by 2008 (off topic, but this is a bit of a rant here). Apologies if this is not the appropriate forum. Please advise.

Answer 1

There are many good reasons why you want is mostly impossible and not even a good idea in the first place.

1) Which type of regression to run. For example, based on the distribution of the dependent variable, x would work. There seem to be many types to run, and it's never clear cut.

No; it's rarely possible to tell from the distribution of the dependent variable whether a particular kind of regression will work, meaning work well. It's not even a formal assumption of any kind of regression that the dependent variable has a particular kind of distribution. You gave that as an example, but similar comments would apply, I assert, to any other kinds of example. For any state given as a modelling assumption -- in my view, usually better stated as an ideal condition for a particular model to work as well as possible -- one could concoct examples which are poorly behaved in the sense of that assumption, but regression works fine (and likely vice versa too). This is one reason why regression texts are so often so long (and incomplete too, even when they are so long). Compound that with all possible assumptions and you have a multiple tree of possible decisions and actions.

2) As a result of regression, how to interpret the results. Statistics programs give you a p-value, R-squared, and sometimes other terms, which you then need to look up. It's never clear if what it says to be significant is actually significant, or the result of some flaw in the data that then needs to be uncovered.

One could quibble a lot about the wording here -- which Olympian stance allows anyone to determine what is "actually significant" versus anything else -- but while the impulse to know how to think about results is admirable, this is a fiendishly difficult problem. You'd need to teach the program everything known about the data, difficulties in sampling and measurement, etc., etc. This is, in some sense, a goal of some statistical people: to build models that incorporate all kinds of uncertainty in a substantial project. Suffice it to say those are usually multi-member team, multiple year projects.

3) Analysis of results. There are all sorts of charts and plots, but all I really ever want to know is what model yields the best results to explain or predict the data. There seem to be endless diagnostics, but I never know what to run or how to do it.

Specifically, I have a guideline which is that most formal tests are misguided and the best approach is graphical, but I can hardly establish that with this sentence alone. Best to explain or predict? Bang on as a concisely stated goal with which most can agree as a starting point, but the detailed discussions start there. It's not even a matter of unanimity that models should explain! Some see every modelling exercise, especially with observational data, as essentially descriptive and that we are fooling ourselves if we pretend otherwise. There are numerous single-valued measures to guide model choice, except that no one much likes anyone's criterion except their own.

Note that your 2) and 3) contradict each other as the main point of the diagnostics in 3) is to help the thinking in 2) about what can be believed and what is trivial or artefactual.

A dark fact about regression, even in 2017, is that even so-called experts can disagree strongly about fundamentals, so the scope for a simple, unified, easily usable program that makes it all trivial is negligible. For example, my default position is to work on logarithmic scale, but I've seen people of similar or greater experience fight shy of any kind of transformation.

Don't you think that others would love that program too if it existed and its name would be known all over the internet? It's the statistical equivalent of world peace desired by every contestant.

Note I have to add a rider about "layman's terms". The difficulty of solving these problems with ordinary English (or French, Chinese, Hindustani, or any other language) is precisely why we need special notation, terminology and concepts. Any implications that experts are just obscure on purpose are, generally, unhelpful if not offensive. What are these layman's terms any way? Many scientists, for example, know a lot of mathematics; they just may not know much modern statistics. The call for layman's terms is ultimately self-defeating, because there is always someone less educated who can shout that they don't understand it. Undesirable, unfortunate, but it can't be a limit on statistics any more than it is in any other field.

Answer 2

Elements of this do exist, but it's extremely difficult to write down a detailed strategy for explaining what's going on with any possible data set and modeling approach.

That said, aspects of this could be quite valuable, and there have been some efforts to make it work. Two (very different) examples I'm aware of:

• The explainr package for R, which provides a framework for automatically explaining statistical results from specific models to a lay audience
• The "automatic statistician" (paper; code; example report), which performs very sophisticated analyses and writes detailed reports (in a small domain of problems).