Linear Regression with Diminishing Returns

Question

So I have been doing some research on how to implement a linear regression forecast with diminishing returns

I've been looking through a number of other articles and have come across several different (possible?) solutions

1.) include the squares of variables Why is functional form so important when specifying models?

2.) fit the best model and obtain the first derivative Whether to include $x$ and $x^2$ in regression model examining diminishing returns when only $x^2$ is significant?

3.) Use a linear-log transformation for all variables http://www.dummies.com/how-to/content/the-linearlog-model-in-econometrics.html

Not truly sure which method is best, or honestly how each work so would be very grateful for a working example. I have provided some fake data below...

 Period    Budget     Ad Space
 1          100          100
 2          200          200
 3          300          300
 4          400          375
 5          500          450      
 6          600          500
 7          700          550
 8          800          575     
 9          900          600
 10         1000          ?
 11         1100          ?
 12         1200          ?
 13         1300          ?
 14         1400          ?

Answer 1

There is no general answer to your question. 'Best' is a very vague term and dummy data does not do justice to any data unless it replicates some specific trend with reasonable accuracy. What these papers are probably trying to do is to make the transformed result look reasonably linear with the error term having near constant variance (seemingly).

Answer 2

Your question is a marketing science concern. You don't mention them, but chances are good that you have additional marketing factors that could be added to the model. Positing this as an issue for finding the correct functional model form is a good start. One of the best discussions of these issues is in Lee Cooper's book Market Share Analysis, which is available for free download on his UCLA website (http://www.anderson.ucla.edu/faculty/lee.cooper/MCI_Book/BOOKI2010.pdf , see pages 34++). Cooper's frame of reference are elasticities and cross-elasticities -- very useful tools for marketing decision-making.

To your point, in marketing it is quite reasonable to assume diminishing returns to scale as expenditures increase and, conversely, kind of unreasonable to assume that vehicle effectiveness can increase linearly without limit. This decline can be a function of many things: size, market saturation and penetration, the limits to marketing vehicle effectiveness, and so on.

Finding the right mathematical expression for that is the challenge. Two approaches stand out: the MCI (multiplicative competitive interaction) and the MNL (multinomial logit) models. For the MCI model, responsiveness decreases monotonically as the X or marketing instrument increases, while for the MNL model response increases to a point, and then diminishes.

As Cooper notes, "Which one is better depends on the marketing variable."

Answer 3

1. It seems you may have a known 0 in your relationship (0 Budget implies 0 AdSpace); When interpolating that may not be important but when extrapolating it may sometimes be worth considering.

2. If you're looking to extrapolate outside the observed data (as your example data implies), a quadratic would be dangerous, because instead of displaying diminishing marginal returns it must eventually show negative marginal returns:

   

   As we see, the quadratic models do fine inside the range where we have data, but beyond the data become increasingly unrealistic; the negative returns occur inside the values you want to predict for.

   Some other choices of predictor than $x^2$ may allow you to fit a least squares model that doesn't decrease again.

3. You don't mention nonlinear least squares models; these would be common in situations similar to this in some application areas

4. With something like ad space the general form of the relationship is often known (at least for particular sellers of ad space); any such knowledge should usually inform the model.

5. Because Ad Space is a variable that takes only positive values (and could only take non-negative values), and because you might expect variability to increase somewhat as the means get large, you might consider a GLM (perhaps a gamma model, for example).

6. If you are looking to interpolate, a spline model may be a better choice.

7. In some situations you might consider transforming one of the variables. For example taking the square root of budget leads to a nearly linear relationship -- but also reveals an interesting feature not obvious in the earlier plot:

   

   Here we see that the relationship is very close to linear, but we also see that there's an apparent change in slope near the end.

   One very useful property of a model that linearly relates $\text{Ad Space}$ to $\sqrt{\text{Budget}}$ is that diminishing returns (of Ad Space as a function of Budget) will be a natural outcome of the model. (This doesn't imply a good fit necessarily -- in particular we wouldn't anticipate it would be good in this specific case where we've already seen there's a change in slope in the data)