I am applying a multiple linear regression on a data set, where some of the predictors are "transformations" of others (however, I'm not entirely sure if they are linear transformations or not).
For the sake of an example, let's say that we have three predictors, $A$, $B$, and $C$ that completely explain the variance of some dependent variable $Y$. However, $B$ and $C$ are highly correlated since $C$ is a transformation of $B$.
The transformation is $C_i = \sum_{j=0}^{23} V_j B_{i-j}$ where V is a vector of 24 numbers.
My questions for you are the following:
Is it possible to completely eliminate multicollinearity among the predictors seeing that I know exactly how some are transformations of others?
If so, how do I go about eliminating this multicollinearity?
Thank you!
Edit: thanks to @curiositasisasinbutstillcuriou, I am getting very close to the solution of my question; however, I need confirmation that my Python code to retrieve the original coefficients of my predictors makes sense; the majority of the following code was taken from https://www.statology.org/principal-components-regression-in-python/ while the last line is my attempt at retrieving the original coefficients.
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.preprocessing import scale
from sklearn import model_selection
from sklearn.model_selection import RepeatedKFold
from sklearn.model_selection import train_test_split
from sklearn.decomposition import PCA
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error
### Fit the PCR Model ###
# X is a dataframe of the original predictors
# y is a dataframe of the dependent variable
# scale predictor variables
pca = PCA()
X_reduced = pca.fit_transform(scale(X))
# define cross validation method
cv = RepeatedKFold(n_splits=10, n_repeats=3, random_state=1)
regr = LinearRegression()
mse = []
# calculate MSE with only the intercept
score = -1*model_selection.cross_val_score(regr,
np.ones((len(X_reduced),1)), y, cv=cv,
scoring='neg_mean_squared_error').mean()
mse.append(score)
# calculate MSE using cross-validation, adding one component at a time
for i in np.arange(1, len(X.columns)):
score = -1*model_selection.cross_val_score(regr,
X_reduced[:,:i], y, cv=cv, scoring='neg_mean_squared_error').mean()
mse.append(score)
# plot cross-validation results
plt.plot(mse)
plt.xlabel('Number of Principal Components')
plt.ylabel('MSE')
plt.title('hp')
# determine n_components based on the lowest MSE
### Use the Final Model to Make Predictions ###
# split the dataset into training (70%) and testing (30%) sets
X_train,X_test,y_train,y_test = train_test_split(X,y,test_size=0.3,random_state=0)
# scale the training and testing data
X_reduced_train = pca.fit_transform(scale(X_train))[:,:n_components]
X_reduced_test = pca.transform(scale(X_test))[:,:n_components]
# train PCR model on training data
regr = LinearRegression()
regr.fit(X_reduced_train, y_train)
# calculate RMSE
pred = regr.predict(X_reduced_test)
np.sqrt(mean_squared_error(y_test, pred))
# find coefficients for original predictors
orig_coef = np.matmul(regr.coef_, pca.components_[:n_components,:])
