4

As I understand, the learning curve is normally plotted as accuracy vs. sample size. However, accuracy takes true negatives into account. When doing association rule mining, true negatives aren't very significant because there are usually a lot of them. Would it then make sense to use the lift measure instead of the accuracy measure to plot a learning curve? Or should I use something else?

The lift (or interest) measure is for a rule $X\to Y$ defined as $$\frac{P(X\land Y)}{P(X)P(Y)}.$$

1 Answers1

1

First the good news. In different project there are different goals. These goals are best served by different measures (or graphs). Feel free the construct the one that fits your need best.

Now for some other points. The learning curve is usually used when you have data sets of different size and a single learner. In these scenario, it makes sense to see how the accuracy improves given access to more data.

In association rules you get many rules (and usually a single data set). What kind of graph are you looking for? Do you plan to use all association rules as points? Do you plan to take many data set and choose the rule with the highest support in each case? In not familiar with the goal of your analysis but it is not clear to me which goal such graphs serve.

The records of the true negatives in such scenario are called "null records". There are indeed plenty of them (e.g., at Amazon the vast majority of clients didn't buy "Harry Potter" though it is a best seller). Including them leads to extremely high accuracy. Since in such scenarios the goal is usually not high accuracy, this is not a proper measure.

The Jaccard measure might fit your needs It is like accuracy when omitting the true negatives and therefore suitable to imbalance datasets.

DaL
  • 4,462
  • 3
  • 16
  • 27