Principle Component Analysis for pre-grouped variables

Question

I have a dataset that has many overlapping factors related groups of factors, and factors that we would like to investigate if there is a relationship. 57 measured factors total, in 50 individuals (lets ignore my low n...) For instance, 1) Blood Pressure: systolic, diastolic, mean, & pulse pressure (height of BP wave) 2) Blood Flow: min, max, mean, & height for blood flow.
3) Size: Individual's weight, length, and circumference.
Each of these groups has substantial collinearity, as you may have guessed, since they are simultaneous measurements of related features. Blood flow is also related to blood pressure, and size always matters. Of course, I have less related features thrown in the mix also, including diet, organ weights, mother's weight... I can certainly select the 'most meaningful' factor from each group, but I dislike ignoring the potential importance of the removed factors.
My goal is Multiple Linear Regressions for areas of interest.

My question is, can I (and how) be selective about my groupings in PCA? Looking at the component plot, the groups naturally cluster together - but in instances where they aren't as clustered as would be ideal, or a stray variable that isn't blood pressure happens to join the blood pressure cluster, what are my options? Do I have to add all variables to PCA, or can I choose those that I would like to group?

Additionally, can someone please explain why units matters for correlations? I was told if I change my g measurements to kg (thus changing 167 to 0.167), it would impact the coefficients/relationships found in MLR. That doesn't logic to me.

I hope this is specific enough! I see a lot of answers that have to account for a variety of interpretations of the question! Use of PCA analysis to select variables for a regression analysis this question gets close to mine, but I wasn't sure what was being asked.

Answer 1

In principle you have a lot of freedom what to do. So yes, you can use Principal Components to summarise different subgroups of your variables. There is no fixed rule how to do this, but I think your intuition to perform PCA separately for groups of related variables (maybe representing each group by one or two of them) makes sense to me. Whether this is the best you can do is a different matter. PCA doesn't involve the $y$ and in terms of optimising prediction error methods that involve the $y$ often do better, however selecting variables not involving the $y$ avoids a bias of the resulting regression coefficients and p-values (if you're interested in them). If you reduce dimensionality and therefore information for the variable groups but you take the "stray variables" as they are (which is a legitimate thing to do in principle), you may implicitly upweight the stray variables, although I'm not sure this is a bad thing. With 50 individuals chances are you want to bring the final number of variables down to around 10 or even less, so you may have to be a bit radical. Alternatively you could look at things like Lasso.

Regarding measurement units, PCA in its raw form is based on the covariance matrix, not the correlation matrix. In this form it is obviously dependent on measurement units (this is sensible if measurement units are the same and higher variances imply higher variable importance). However PCA is these days probably more often performed on correlations, in which case the measurement units can be changed without changing the result (your PCA function should allow you to specify whether it runs on correlations or covariances).

Regarding the MLR (which I see is what the units question is actually about), the regression coefficient can be interpreted as the expected change in $y$ if a certain $x$ is changed by 1 unit assuming all other $x$ are held constant. Obviously this changes if the unit of an $x$ (or $y$) changes, however this is not a problem because MLR is affine equivariant, meaning that the regression coefficients change only accordingly to account for the measurement unit change, and the model fit is otherwise unaffected.