Sparse coding is described as "given an input $X$, finding a latent representation $h$ such that h is sparse and the input can be reconstructed as well as possible." (source: https://www.youtube.com/watch?v=7a0_iEruGoM)
My question is why do we want to find a latent representation ? I mean, what's the benefit of sparse coding? Why do we want to reconstruct the input if we have it in the first place? To which type of problem would we say: "Ok! I am going to use sparse coding for that!"
Parsimony. Sparse representations of a signal are easier to describe because they're short and highlight the essential features. This can be helpful if one wants to understand the signal, the process that generated it, or other systems that interact with it.
Denoising. In this context, the measured signal is a mixture of some underlying/true signal and noise. The goal is to remove the noise. If the underlying signal is sparse in some basis (which is often the case for interesting signals) and the noise is not (e.g. white noise), then denoising can be done by constructing a sparse approximation of the measured signal.
Data compression The goal here is to store the signal, transmit it over a communication channel, or perform further processing on it. These operations require memory, communication, and computational resources that scale with the size of the signal. Sparse coding can be used to compress a set of signals, reducing the resources needed.
Compressed sensing The goal here is to measure signals efficiently by exploiting knowledge about their structure. This allows more efficient storage and transmission, and may also allow measurements to be made more quickly. Typically, specialized hardware is involved. If a signal is known to be sparse in some basis, it's possible to acquire it using fewer measurements than would otherwise be necessary. The original signal can then be reconstructed from the reduced set of measurements. Sometimes a class of signals is known a priori to be sparse in a particular basis. For example, natural images are sparse in the wavelet basis. In this case, the known basis can be used to design the measurement and reconstruction procedure. But, if the basis isn't known, it can be learned from a set of example signals using sparse coding (aka dictionary learning).
"In numerical analysis and computer science, a sparse matrix or sparse array is a matrix in which most of the elements are zero." Wikipedia
There are some datasets where instances have a large number of attributes. This dataset can be thought of as a sparse matrix if most of the recorded attribute are zero. In this scenario we could potentially have a very large file containing the dataset without an equivalent amount of "information."
One way to reduce the size of the dataset files without losing any information is to use a sparse file format. For example, an ARFF file can be stored in either dense or sparse format.
From Weka's documentation, the header information is the same between the two formats. The difference is in how the instances are represented. The instances in the dense representation look like this:
0, X, 0, Y, "class A"
0, 0, W, 0, "class B"
While a sparse representation of the same instances looks like this:
{1 X, 3 Y, 4 "class A"}
{2 W, 4 "class B"}
It can be seen that the first instance, where most of the attributes are nonzero, becomes longer in the sparse representation. The second instance, however, has mostly zeros as attributes and is represented more efficiently. If most of your dataset is like the second instance - if the dataset is a sparse matrix - then a sparse file format might make sense to reduce the size of storing the dataset with no loss in information.
