6

We are modeling a discrete choice scenario, with alternative-specific coefficients. We also break the assumption of independence of irrelevant alternatives. To model this, we are using an alternative-specific multinomial probit regression. This is implemented in Stata as asmprobit. However, as described in the documentation, there is a limit on the number of alternatives: 20! We have up to 120 alternatives per case! Uh oh.

Why is there this limit? Is there anything that can be done to increase it? Is this model (or a good alternative) implemented in R or elsewhere, presumably without this limit?

jeffalstott
  • 459
  • 1
  • 4
  • 18
  • Are you going to ask a person to choose among 120 alternatives? It's crazy. (Why so much? did you create an orthogonal or optimal design to reduce the number of alternatives actually presented to a person?) – ttnphns Mar 28 '15 at 13:48
  • The data we are analyzing do not involve people. – jeffalstott Mar 28 '15 at 17:20

1 Answers1

4

The main issue with asmprobit is the flexibility it provides which relaxes the independence of irrelevant alternatives (IIA) assumption but it comes at the cost of increased computing power. In this sense you allow the odds of choosing one alternative over some other alternative to depend on the remaining alternative, though this involves evaluation of probabilities from the multivariate normal distribution. Since there is no closed form solution to those you have to rely on simulation techniques. That's the bottle neck.

The simulation method used by asmprobit in order to solve the simulated maximum likelihood is the Geweke-Hajivassiliou–Kean multivariate normal simulator (GHK documentation) which allows only for dimension $m\leq 20$. That's where the restriction in asmprobit comes from because for more alternatives the simulation time becomes unmanageable. For a detailed description of this you can also see the "Simulated Likelihood" part in the Methods and Formulas section of the asmprobit documentation.

Given that the reason for the limit is a computational one rather than one that is concerned with implementation I would not be too hopeful for a better estimation routine in R. If there was one then probably also Stata would have implemented it by now. By the way, this restriction is also a problem for other probit models of discrete choice (e.g. mprobit allows max 30 distinct choices) for the same reason as outlined above.

A useful reference for you should be

  • Train, Kenneth E. 2007. Discrete Choice Models with Simulation. New York: Cambridge University Press.

which is probably the main reference on this topic. If I remember correctly he also discusses cases where the number of choices is very large. I'm not sure, however, if you can have the best of both worlds, i.e. relaxing IIA and allowing for many alternatives. Certainly probit models will not get you far because of the multivariate normal but perhaps other models that also relax the IIA assumption may be useful. For instance, the mixed logit model also relaxes this assumption so it might be worth to have a look at the Stata options for estimating these kinds of models (see for instance this presentation for an overview).

Andy
  • 18,070
  • 20
  • 77
  • 100
  • 1
    +1. Train's book for free from his website here: http://eml.berkeley.edu/~train/distant.html – Bill Mar 30 '15 at 14:39
  • The `amsprobit` documentation has a line that makes clear how increasing the number of alternatives causes the computational load to explode. Where J is the number of alternatives: "Quasi–Monte Carlo integration is carried out by `asmprobit` by replacing the uniform deviates with either the Halton or the Hammersley sequences. These sequences spread the points more evenly than the uniform random sequence and have a smaller order of discrepancy, `O{(log N)^(J−1)}/N` and `O{(log N)^(J−2)}/N`, respectively." So increasing J is terrible. – jeffalstott Mar 31 '15 at 14:19
  • That presentation is great! One thing is unclear to me, however: Do the mixed logit models allow for the number of alternatives to vary across cases? In our cases the alternatives are being chosen sequentially, each once and only once, and which alternatives have been chosen previously affects which alternative looks best in the future (hence the IIA disruption). `asmprobit` allowed the number of alternatives to vary, which was great; it's unclear to me that mixed logit allows this. – jeffalstott Mar 31 '15 at 14:23
  • It's certainly possible to estimate mixed logit models using a random subset of the choices which eases computational burden if the number of alternatives is large. See this paper on the demand for frozen pizza: http://www.nuffield.ox.ac.uk/economics/papers/2012/Random%20Choice%20sets%20-%20Final%20-%20Oct%203,%202012.pdf – Andy Mar 31 '15 at 14:27
  • What if it's not a random subset? In our case, there are multiple steps at which choices are made. The first step has choices {1, 2, 3, ..., 120}. Then the second step has the same set of choices, minus the one chosen in the first step. For example, if 2 was chosen in the first step, the choices in the second step would be {1, 3, ... 120}. – jeffalstott Apr 02 '15 at 09:17
  • It's difficult to answer your original question now that new information keep coming up. At some point you have to make a decision: complexity of the model v.s. computational feasibility. You can't have everything with the current technology. 100+ choices, relaxing IIA, sequential choices - all of these are tricky issues in their own right and at least to my knowledge there is yet no model that can do all of them at the same time. – Andy Apr 02 '15 at 09:39