I've a classical dataset with real attributes and I want to perform a regression.
But, not all the entries in the training dataset are trustworthy; there is an attribute that I can turn into a confidence value (between 0 and 1) to say how much I can trust the label value. For example, if the confidence value is 0, I mustn't take the entry into consideration.
Is there a way to do this?
Generally, if you want to include a prior knowledge in your model, than the first thing that should come to your mind in Bayesian approach. In your situation however it seems that you have some a priori knowledge about how trustworthy is each of the observations in your sample and possibly a different approach cold be applied in here. Often statistical software enables you to include in your estimation information about survey weights. This may not sound like a thing that you should be interested in this case, but if you think of it, survey weights are used to re-weight the estimation result taking into consideration how representative is your sample comparing to the population. To do this, each observation has its own weight that describes how often such observations are observed in population comparing to your sample and their influence to the overall estimate is accordingly adjusted. If you think of it, the basic idea in here is that different observations carry different amount of information about population and so their influence is adjusted and this is exactly your situation! In regression case, the common approach it to use weighted least squares. This should be described in software manual of the statistical package that you are using, e.g. in R methods such as lm have weights parameter.
