3

My question relates to https://math.stackexchange.com/questions/547217/determine-if-being-on-honor-roll-gives-an-advantage-in-being-assigned-to-the-mat/984178#984178 (I've seen similar questions involving hits on a webpage) which I believe should be solved using a chi-squared test. My question regards what happens if there is more than one class. To make things specific here is my problem.

Consumers are interested in purchasing item $i$ where $i=1,\ldots,n$. Consumers interested in item $i$ are randomly sent to a webpage with color $A$ or $B$. We observe the number of consumers that purchase and do not purchase the item under each condition.

Did the color affect purchase decisions? Was there a difference across items?

What about if consumers are interested in purchasing item $i$ where $i=1,\ldots,n$. Consumers interested in item $i$ are randomly sent to webpage of color $A$ or $B$ and randomly shown font of type $C$ or $D$. We observe only the number of consumers that purchase the item under each condition.

Did the color or font affect purchase decisions? Was there a difference across items?

How would you analyze these questions? Is there a standard technique to solving such problems? Can the questions be analyzed with a regression using dummy variables?

Community
  • 1
user103828
  • 231
  • 2
  • 12
  • 1
    Yes, the chi-squared test extends to more than 2 classes. – Eric Farng May 06 '15 at 01:20
  • @EricFarng Do you mean running ch-squared test for each item? Do you have a reference? Thanks. – user103828 May 06 '15 at 10:10
  • 1
    I apologize for the short answer. Wikipedia has an [example of a k-class chi-squared test](http://en.wikipedia.org/wiki/Chi-squared_test#Chi-squared_test_for_independence_and_homogeneity_in_tables) – Eric Farng May 06 '15 at 10:27
  • @EricFarng Thanks the link is useful... what about if the object is three-dimensional. I thought about dividing into two dimensional tables and using chi-squared in each table but this seems a wrong... If there are the three dimensions colors, items, and fonts. Is there a way to put this into a three dimensional table and then applying a chi-squred test? Can this be modeled as part of a regression? – user103828 May 07 '15 at 07:35
  • Also if you want to put this as an answer I can mark it correct. – user103828 May 07 '15 at 07:36
  • I'm afraid I haven't completely answered your question. The chi-squared test will tell only you if there "was a difference across items." Do you know the total number of customers shown color A and color B? Or do you know if customers were equally likely to be shown color A or color B? – Eric Farng May 08 '15 at 10:03
  • Customers equally likely to be chosen. Is there a way to estimate the "effects" of color / item (and not just that there's a difference)? – user103828 May 09 '15 at 18:12

2 Answers2

1

Here are a bunch of leads to explore. Without the actual data, it is difficult to suggest a single thing to try.

  • Did the color affect purchase decisions?

A linear regression might work here. You probably already know but they can be tricky to fit properly. Make sure your data actually has a linear relationship and check your residuals for problems. Since you only have two colors, the Wilcoxon signed-rank test has easier assumptions to meet. There is also a permutation test for paired data which also has easy assumptions.

  • Was there a difference across items?

A chi-squared test is good for this. By creating a table of items vs color, you'll get to see which items don't have the same color effect as other items. Just check the assumptions and make sure your sample sizes are large enough.

  • Did the color or font affect purchase decisions?

Linear regression will extend easily to answer this question. You may want to check for interaction effects. However, it may not be necessary to put both variables in the same model. Consider using a chi-squared test of color vs font to determine independence then running two separate tests if they are.

  • Was there a difference across items?

There are chi-squared tests for 3-way tables. There is also the Cochran-Mantel-Haenszel to check for conditional independence, e.g. Was there a difference across items by color given the font?

Community
  • 1
Eric Farng
  • 1,585
  • 10
  • 17
0

Since this question is so general my answer must be quite general as well.
One way we can analyse these types of questions is to look at the differences in terms of percentages. If a large percentage of customers buy the item when using one color and a small percentage buy the item when using a different color this implies a correlation. If the percentages are closer together then we need more data to determine if there is a difference. As more data is obtained we know that the mean value of the data is getting closer to the true value due to the law of large numbers. If we are getting different percentages for each option then we can assume at some point that they are significantly different.

Math Man
  • 101
  • 1