0

I want to implement some kind of confidence measure for my stock prediction model, which predicts the next day's trend (whether the price would rise or fall). As in, a trade order should be placed only if the model is certain about its prediction up to some degree. Now decision_function() does provide a confidence measure but I can't use it, as a threshold cannot be set without actually seeing the decision_function() values of test instances.

Scikit-learn's documentation of SVC says that there are some "theoretical issues" with Platt Scaling - the method used to obtain prediction probabilities of classifiers.

Is it okay to use predic_proba() as a confidence measure?

Aditya Kulkarni
  • 115
  • 5
  • Is your model a classification or a regression? I would expect you to be predicting asset price (or percent gain/loss) on a continuum. – Dave Apr 30 '21 at 14:05
  • It is a classification model. Predicts whether the price would rise or fall. – Aditya Kulkarni Apr 30 '21 at 14:10
  • There’s another question on here that I suspect you e read that explains why this might not be a great idea: https://stats.stackexchange.com/q/519339/247274. Sycorax makes some points that are valid here; importantly, your classes aren’t really discrete. – Dave Apr 30 '21 at 14:22
  • You should examine your model's calibration if you want to know if `predict_proba` returns reliable probability estimates. – Demetri Pananos Apr 30 '21 at 14:23
  • Furthermore a probability estimate is not a confidence estimate. If you have only one training sample, the model can predict 100% probability but with confidence intervals of [0,100%] – seanv507 Apr 30 '21 at 18:23

0 Answers0