23

Suppose I have $n$ independent sources, $X_1, X_2, ..., X_n$ and I observe $m$ convex mixtures: \begin{align} Y_1 &= a_{11}X_1 + a_{12}X_2 + \cdots + a_{1n}X_n\\ ...&\\ Y_m &= a_{m1}X_1 + a_{m2}X_2 + \cdots + a_{mn}X_n \end{align}

with $\sum_j a_{ij} = 1$ for all $i$ and $a_{ij} \ge 0$ for all $i,j$.

What's the state of the art in recovering $X$ from $Y$?

PCA is out of the question because I need the components to be identifiable. I've looked at ICA and NMF - I can't find any way to impose nonnegativity of the mixing coefficients for ICA, and NMF doesn't seem to maximize independence.

amoeba
  • 93,463
  • 28
  • 275
  • 317
anonymous
  • 755
  • 1
  • 4
  • 13
  • 2
    I would think that this should be called "nonnegative independent component analysis", but it seems that this name has been used for ICA with the nonnegativity constraint on the sources $X$, not on the mixing matrix $A$ (https://www.eecs.qmul.ac.uk/~markp/2003/Plumbley03-algorithms-c.pdf). So this does not apply to your case. Interesting question. – amoeba Dec 29 '14 at 22:51
  • Don't you want the sums to run over j instead of i? Can you assume the sources are approximately gaussian? if they are unimodal and have sufficiently fast decay, it is possible that fitting a GMM would suffice. – Yair Daon Jan 06 '15 at 22:16
  • @YairDaon Ah yes thanks, good catch. Unfortunately the sources are discrete and don't even look like mixtures of gaussians. But perhaps I could coarsely approximate them as gaussian mixtures and then further refine. But it would be nice to have something more general/robust – anonymous Jan 06 '15 at 22:24
  • Your question looks like a difficult open research problem. It would be really cool if it gets answered here, but alas, there are not many specialists on ICA here (as manifested by the very low number of views and only a single upvote of your question), perhaps even none. Was a brave move to offer a bounty! – amoeba Jan 07 '15 at 14:06
  • @amoeba I was worried it might just be open. But I'll hold out hope for the next 6 days! – anonymous Jan 07 '15 at 17:44
  • 1
    What ICA algorithms have you tried? I'm a little rusty, but think the non-negativity assumption of the mixing coefficients can be imposed in some algorithms that assume certain models for the signals like the Weights-Adjusted Second Order Blind Identification (WASOBI) algorithm, as it assumes you can model the signals as AR processes and, thus, you can impose conditions in the coefficients. – Néstor Jan 10 '15 at 23:10
  • (I'm not saying it is implemented; just that it can) – Néstor Jan 10 '15 at 23:16
  • @user43228 Can you give some context for the problem (ie what the sources are, why you need to solve it)? Also, more specifically: do the sources have a known, finite support? – Andy Jones Jan 11 '15 at 02:08
  • 1
    The sources are all supported on the set {1,2,...,96} – anonymous Jan 12 '15 at 17:13
  • Can you explicitly state in your question whether you are looking to discern independent signals, i.e. the matrix $A$ containing the coefficients $a_{ij}$ should be orthogonal? – means-to-meaning Jan 13 '15 at 13:55
  • @means-to-meaning The mixing matrix is nonnegative and the convex mixtures are true mixtures, i.e. this is not a vector quantization problem, so I don't think A should be orthogonal. As stated in the question, I am looking to as best as possible recover the signals under the assumption they are independent. – anonymous Jan 13 '15 at 18:42
  • @Néstor These signals aren't really time series; I'm not sure they qualify as AR processes. But I don't know almost anything about the area so that could be wrong. – anonymous Jan 13 '15 at 18:45
  • It doesn't matter if they are time series or not. AR models work fine on any indexed series, not only on time series. – Néstor Jan 13 '15 at 19:08
  • 1
    This is not unique. https://link.springer.com/article/10.1007/s11634-014-0192-4 Mixture distributions are ubiquitous and also problematic for those organic applications that I have examined. I have yet to see a circumstance in which they are preferred over dependent processes like convolutions of components upon testing. – Carl Feb 27 '18 at 03:17

1 Answers1

0

It could be achieved by using an exponential non-linearity instead of the typical/default tanh(), if X is also non-negative.

Formula 40 in https://www.cs.helsinki.fi/u/ahyvarin/papers/NN00new.pdf and available in most implementations.

E.g. in sklearn simply use fun='exp' https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.FastICA.html

Henrique Mendonça
  • 23
  • 5
  • 1
    Welcome to Stats.SE. Can you please edit your answer and expand it in order to explain the key steps of the links you provide? This way, the information is searchable in here (and sometimes links break). You may want to take a look at some [formatting help](https://stackoverflow.com/editing-help). While you're at it, you can use [LaTeX / MathJax](https://math.meta.stackexchange.com/q/5020). – Ertxiem - reinstate Monica May 11 '19 at 16:17