3

I have a 5000 X 32 X 10 3D array of gene expression data that I would like to apply clustering and dimensionality reduction on.

The dimensions represent the following:

I have 5000 genes, measured in 32 different mutant strains, each in one of 10 environmental conditions. (mean = 0, var = 1)

I'm trying to do something akin to biclustering or SVD, but have no experience with 3D arrays.

I found a CV post talking about SVD on 3D arrays , but was hoping to get some more information about how to get started, and what I should try first.

Fellow researchers have suggested that I simply flatten the array into a 5000 X 320 2D array, but I am hesitant because I feel like I am loosing information about the relationships between columns.

kmace
  • 737
  • 4
  • 17

1 Answers1

2

As most algorithms which you probably have considered completely ignore the order of axes, flattening is just fine.

Only if you define, e.g., a similarity of environmental conditions or genes and compare different variables with each other, then order becomes important. Quadratic form distances are an example for this. You could then may define a "quadratic tensor" distance, but even that can probably be flattened. You just need to know the positions after rearrangement.

With Euclidean and similar distances, because of commutativity, the 32x10 columns can be arbitrarily rearranged or flattened.

Has QUIT--Anony-Mousse
  • 39,639
  • 7
  • 61
  • 96
  • Most methods make a distinction between rows and columns, if that were not true, one could perform analysis on a 1d representation on the data (for example PCA on a vector) however that is not possible. Therefore the relationships between dimensions is taken into account – kmace Oct 13 '17 at 20:03
  • Yes, rows are special, because these are the instances (don't get stuck join thinking in matrixes - all the methods need is a collection of vectors, not necessarily a matrix). Columns can be rearranged, that is trivial to prove. – Has QUIT--Anony-Mousse Oct 13 '17 at 20:05
  • Define what similarity you want to use. Then check if you have commutativity. – Has QUIT--Anony-Mousse Oct 13 '17 at 20:07
  • In some applications rows are not strictly instances. For example, in SVD, there are row eigenvectors and column eigenvectors. In the Netflix problem, each row is an instance of user, and each column is an instance of movie. Both are equally valid instances. But if I were to flatten the data, how would I go about selecting the row? ie. Which dimensions should I merge to flatten? – kmace Oct 13 '17 at 20:10
  • Depends on what you want to achieve. Choose the method to solve a problem, don't just run whatever you find fancy. – Has QUIT--Anony-Mousse Oct 14 '17 at 15:03