4

Suppose I have some inputs on which I first apply some feature engineering and then apply a machine learning algorithm such as random forest to make predictions.

steps of applying machine learning

Now, if I want to interpret/explain the model for example using perturbation importance or partial dependence plot etc, should I consider the whole feature engineering step and random forest as one model or should I only try to explain the random forest using the model explainability methods. The problem with the second approach is that the features going into the random forest are transformed and it is difficult to interpret them. What is the formal way?

sara iftikhar
  • 53
  • 3
  • For model agnostic explainability methods (partial dependence etc.), I usually start to process the data into interpretable features. Additional preprocessing steps (dummy encoding etc.) then "belong to the model". I don't think there is an official way. A good question though. For model specific explainability methods, one is less flexibel. – Michael M Dec 12 '21 at 20:06

2 Answers2

0

This is a very pertinent question as while pre-processing steps are widely considered to be part of the overall model and should be included in its validation procedure, explainability tasks (through LIME, SHAP, PDPs, etc.) rely on the pre-processed model inputs and are thus unable to account for changes done during pre-processing.

Some further discussion: we know we should zero-centre our data using only the training set derived means - e.g. see Question about subtracting mean on train/valid/test set for a further discussion). When the learning algorithm is then fitted on the pre-processed training set, that learner's "world" is already preprocessed without the learner knowing that. When we use some post-hoc explainability routine, the routine can only account for the behaviour of the learner, a behaviour that it is fixed on the preprocessed training set. As such, we need to make a clear and accurate description of the preprocessing steps done. Some steps might be very straight-forward (e.g. zero-centring) some steps might be more complicated (e.g. imputation of missing values). In all cases, the impact of the preprocessing will/should be captured by the model validation but when explaining the learner's output we focus on the final learner. I have personally tried to created PDPs across each of the folds done by a resampling procedure, as a way to visualise the variability of the PDP and thus of the underlying model, but that was really model specific and definitely ad-hoc. I am sure this can be done with other explainability routines too (e.g. SHAP).

Having said the above, unfortunately there is no formal way of presenting the effect of preprocessing steps; there are some data science project methodologies you might want to consider when structuring and/or presenting a DS project (e.g. CRISP-DM, TDSP, etc. - see CRISP-DM Twenty Years Later: From Data Mining Processes to Data Science Trajectories (2019) by Martinez-Plumed et al. as a first start) but those are not exactly what you want. (I did found them helpful when structuring the project delivery though.)

usεr11852
  • 33,608
  • 2
  • 75
  • 117
0

Depending on what exactly the preprocessing steps are. If you have applied PCA and then feed the principal components as features to the random forest model, then by design you have greatly reduced the interpretability of the machine learning model due to the preprocessing. This is one drawback of applying preprocessing method like PCA.

However, there are other types of preprocessing where the reduction in interpretability isn't as serious.

user344849
  • 106
  • 5