I want to implement a regressor (such as a neural network) with multiple outputs, based on a vector of inputs. It would be trained on a big initial dataset, before being used by some human user.
This regressor should be able to receive some feedback from the user, to customize it. The way I considered would be to add a new point (i.e. [inputs outputs]) to the dataset and retrain the model, after every use.
But the size of the initial dataset would mean that newly-added points would be drowned out. I need a mechanism for giving these more recent data points more weight. Is there a way to do that? I know about using weighted cross-entropy loss, but that is used for classification only.
I could just add the user-generated points twice, but then, in the long term, it will be difficult to manage the dataset, and I would have to add later points more than twice.
I would avoid using models such as k-NN as I want to keep the model small.
