I'm having a discussion with my colleague about an exam question.
Let's say we have a questionnaire in which, among others, the preference for a political party is asked for. How would we model this, do we use a fixed or random effect? I think we should go for a random effect as it's a questionnaire and we expect the effect to change with the sample demographics (age, sex,...).
One reason that some people (e.g. Andrew Gelman) call these models "multi-level models" is to avoid these confusing terms.
In general, a random effect is one that we would expect to differ if we ran the experiment again. So "subject" (that is, the person's name or id) is usually a clear random effect. E.g. in a poll or survey, we would (usually) not expect to get the same person again. But we would expect to get (in the USA) a lot of Democrats, Republicans and Independents and a few from other parties. By that definition, "party" is a fixed effect.
A more useful (at least to me) way of looking at it is that one sort of effect goes into the model because we are interested in modeling its relationship to the dependent variable; another goes in because we have to account for it due to non-independence.
Yet another way of thinking about it is that a multilevel model can be distinguished from a "regular" regression model by the form:
Regular $Y = X\beta + \epsilon$
Multilevel: $Y = X\beta + Z\gamma + \epsilon$
and then thinking about whether you want "party" as part of X or Z.
The same effect could be random in one situation and fixed in another: Suppose you are interested in whether men or women are better at teaching math. One way to investigate would be to look at many men and women professors across many schools. Then "subject" (the particular professor) would be random. Another way to investigate would be to look at all the professors of math in one school; then "subject" would be fixed.
So, if you are measuring, say, who a person voted for in the 2012 election (Obama, Romney, Other, None) I would count "party" as a fixed effect.
