Feature Engineering with Focus on KNN

Question

I have seen a number of helpful posts such as this one on feature engineering, but I am specifically looking for something that may be helpful when using KNN. In my experience, some features work best with one model but when transferred to KNN fail to produce good results. Thanks in advance.

Answer 1

KNN suffers from the curse of dimensionality. For this reason, it will almost never perform well in high dimensional feature space. So in terms of feature engineering, it's important to have a small set of meaningful features.

Additionally, if your model is going to measure distance using the Euclidean distance, it's important to map your numerical features to a standard range, usually $[0,1]$.