If $X$ is a random $N \times D$ matrix where $N > D$, then why is the rank of X - mean(X, 1) $D$ while the rank of X' - mean(X', 1) $D - 1$? My initial thought was that both should be $D$, but then I found this answer which mentions that the rank is reduced by one when the rows of X are centered. This makes some sense, but I'm confused as to why that doesn't apply to both cases. Here is some example Matlab code:
X = randn(100, 5);
rank(X - mean(X, 1))
rank(X' - mean(X', 1))
The rank will always be bounded by the smaller of the dimensions. For the $X$ with dimensions $N\times D$, centering by the columns (i.e., find the mean of each column, and then subtract the mean from each value in that column) reduces the rank by column: now you are comparing $N$ and $D-1$...$D-1$ is smaller, so that is your rank.
But, for the transposed $X^T$, the dimensions are $D\times N$ and mean centering affects the columns: now you want the minimum of $D$ and $N-1$...which is $D$.
Hope this helps.
