What is Shapley value regression and how does one implement it?

Question

I have seen references to Shapley value regression elsewhere on this site, e.g.:

Alternative to Shapley value regression

Shapley Value Regression for prediction

Shapley value regression / driver analysis with binary dependent variable

What is it exactly?

My guess would go along these lines. For a game where a group of players cooperate, and where the expected payoff is known for each subset of players cooperating, one can calculate the Shapley value for each player, which is a way of fairly determining the contribution of each player to the payoff. I assume in the regression case we do not know what the expected payoff is. Instead, we model the payoff using some random variable and we have samples from this random variable.

Answer 1

The Shapley Value Regression: Shapley value regression significantly ameliorates the deleterious effects of collinearity on the estimated parameters of a regression equation. The concept of Shapley value was introduced in (cooperative collusive) game theory where agents form collusion and cooperate with each other to raise the value of a game in their favour and later divide it among themselves. Distribution of the value of the game according to Shapley decomposition has been shown to have many desirable properties (Roth, 1988: pp 1-10) including linearity, unanimity, marginalism, etc. Following this theory of sharing of the value of a game, the Shapley value regression decomposes the R2 (read it R square) of a conventional regression (which is considered as the value of the collusive cooperative game) such that the mean expected marginal contribution of every predictor variable (agents in collusion to explain the variation in y, the dependent variable) sums up to R2.

The scheme of Shapley value regression is simple. Suppose z is the dependent variable and x1, x2, ... , xk ∈ X are the predictor variables, which may have strong collinearity. Let Yi ⊂ X in which xi ∈ X is not there or xi ∉ Yi. Thus, Yi will have only k-1 variables. We draw r (r=0, 1, 2, ... , k-1) variables from Yi and let this collection of variables so drawn be called Pr such that Pr ⊆ Yi . Also, Yi = Yi∪∅. Now, Pr can be drawn in L=kCr ways. Also, let Qr = Pr ∪ xi. Regress (least squares) z on Qr to find R2q. Regress (least squares) z on Pr to obtain R2p. The difference between the two R-squares is Dr = R2q - R2p, which is the marginal contribution of xi to z. This is done for all L combinations for a given r and arithmetic mean of Dr (over the sum of all L values of Dr) is computed. Once it is obtained for each r, its arithmetic mean is computed. Note that Pr is null for r=0, and thus Qr contains a single variable, namely xi. Further, when Pr is null, its R2 is zero. The result is the arithmetic average of the mean (or expected) marginal contributions of xi to z. This is done for all xi; i=1, k to obtain the Shapley value (Si) of xi; i=1, k. The In the regression model z=Xb+u, the OLS gives a value of R2. The sum of all Si; i=1,2, ..., k is equal to R2. Thus, OLS R2 has been decomposed. Once all Shapley value shares are known, one may retrieve the coefficients (with original scale and origin) by solving an optimization problem suggested by Lipovetsky (2006) using any appropriate optimization method. A simple algorithm and computer program is available in Mishra (2016).

Answer 2

There are two good papers to tell you a lot about the Shapley Value Regression:

Lipovetsky, S. (2006). Entropy criterion in logistic regression and Shapley value of predictors. Journal of Modern Applied Statistical Methods, 5(1), 95-106.

Mishra, S.K. (2016). Shapley Value Regression and the Resolution of Multicollinearity. Journal of Economics Bibliography, 3(3), 498-515.

I also wrote a computer program (in Fortran 77) for Shapely regression. It is available here.

Answer 3

In statistics, "Shapely value regression" is called "averaging of the sequential sum-of-squares." Ulrike Grömping is the author of a R package called relaimpo in this package, she named this method which is based on this work lmg that calculates the relative importance when the predictor unlike the common methods has a relevant, known ordering.