I have seen appearance of negative coefficients where they do not make sense (the data is related to costs where negative coefficient should not appear). If regression models are fitted to individual predictors, each one of them is positive as expected (and the correlation is also positive with the target), but in the presence of all predictors, they become negative.
I know the data and based on that I can say that the coefficients of some variables are negative since the data is seeing them with low cost and low occurrence of high-costing parameters.
For example, if the target is the cost of a visit to mechanic to repair cars, and the variables include electronic problems and engine problems, with engine problems costing a lot more. And the data is arranged in such a way that most people mostly bring their cars with only one type of problem, either engine or electronic, with engine problems costing a lot more to fix. This is making the electronic problems coefficient negative because the where there is high occurrence of electronic problems, there is low occurrence of engine problems, and the negative coefficient is therefore telling that high occurence of electornic problems decrease cost, since most of the cars that come for repairs have only one type of problem.
The aforementioned explanation is similar to the actual data set. My questions are:
- How can the negative coefficients be explained when they should not appear?
- Is there a possibility that my understanding of the cause is correct, and if so, how to prove it?
- What are the exploratory analysis directions generally taken to explain such occurrences?
