18

I'm trying to understand matrix factorization models for recommender systems and I always read 'latent features', but what does that mean? I know what a feature means for a training dataset but I'm not able to understand the idea of latent features. Every paper on the topic I can find is just too shallow.

Edit:

if you at least can point me to some papers that explain the idea.

Jack Twain
  • 7,781
  • 14
  • 48
  • 74
  • Here is simple example that might help you http://www.quuxlabs.com/blog/2010/09/matrix-factorization-a-simple-tutorial-and-implementation-in-python/ – Akavall Jul 16 '14 at 14:31

4 Answers4

13

Latent means not directly observable. The common use of the term in PCA and Factor Analysis is to reduce dimension of a large number of directly observable features into a smaller set of indirectly observable features.

samthebest
  • 835
  • 5
  • 11
  • 1
    so the reduced dimensions are then the latent features? In the case of PCA, the eigen vectors of the covariance matrix, i.e the principal components, right? – Jack Twain Jul 16 '14 at 10:30
  • 1
    Correct @AlexTwain – samthebest Jul 16 '14 at 15:20
  • Can you provide me with a tutorial/paper that mentions that? I'm not able to find any systematic tutorial/paper! – Jack Twain Jul 16 '14 at 16:48
  • Well the wiki page is pretty good, you can follow the references there if you really want http://en.wikipedia.org/wiki/Latent_variable – samthebest Jul 17 '14 at 10:51
  • 2
    @JackTwain the correct PCA analogy is that the latent features are the eigenvectors. The principal components are the weights assigned to each observation for the principal eigenvectors. In other matrix factorisation models the latent features play the role of the eigenvectors. This may sound pedantic, but the mistake creates no end of confusion for people. – conjectures Mar 06 '17 at 18:29
4

In the context of Factorization Method latent features are usually meant to characterize items along each dimension. Let me explain by example.

Suppose we have a matrix of item-users interactions $R$. The model assumption in Matrix Factorization methods is that each cell $R_{ui}$ of this matrix is generated by, for example, $p_u^T q_i$ — a dot product between latent vector $p_u$, describing user $u$ and a latent vector $q_i$, describing item $i$. Intuitively, this product measures how similar these vectors are. During training you want to find "good" vectors, such that the approximation error is minimized.

One may think that these latent features are meaningful, that is, there's a feature in user's vector $p_u$ like "likes items with property X" and corresponding feature in item's vector $q_i$ like "has property X". Unfortunately, unless it's somehow enforced, it's hard to find interpretable latent features. So, you can think of latent features that way, but not use these features to reason about the data.

Artem Sobolev
  • 2,571
  • 13
  • 17
  • I've read papers where the latent features (say the "user vector") are used to predict some target variable(s), let's use gender as an example. It "works" in that a predictive model can be built this way. My question is what is the difference between the "user vector" and, say, averaging the "item vectors" for all the items a user has "visited"? IOW, would you expect the predictive model mentioned above to be better or worse with one vs the other? Thanks (if you ever see this). – thecity2 Apr 04 '17 at 21:10
  • @thecity2, you can average user's items, and this might actually be useful when you're dealing with newcomers for whom you don't have precomputed user vectors (though it should be hard to run a few optimization iterations to compute it). There's also an issue with plain averaging: the more items user has consumed – the closer to zero their average item vector is likely to be (because of typical L2 regularizer, and maybe other nasty properties of high-dimensional spaces). Finally, having a separate vector is more flexible: your model can learn such averaging. – Artem Sobolev Apr 05 '17 at 21:50
  • That said, there're attempts to use user's history to model user's vector. For example see the paper "Build Your Own Music Recommender by Modeling Internet Radio Streams" – Artem Sobolev Apr 05 '17 at 21:55
1

Here your data is ratings given by various users to various movies. As others have pointed out, latent means not directly observable.

For a movie, its latent features determine the amount of action, romance, story-line, a famous actor, etc. Similarly, for another dataset consisting of handwritten digits, the latent variables may be angle of edges, skew, etc.

explorer
  • 113
  • 4
0

I would say that factors are more representative than principal components to get a perception of 'latency'/hiddenness of a variable. Latency is one of the reasons why behavioral scientists measure perceptual constructs like feeling, sadness in terms of multiple items/measures and derive a number for such hidden variables which cannot be directly measured.

KarthikS
  • 1,066
  • 1
  • 9
  • 18