weak-learner is often defined relating to PAC learning. However, to the extent I know, I have never seen the definition of weak-learner when regression.
That is, the definition of weak-learner on wikipedia is as following. "A weak learner is defined to be a classifier that is only slightly correlated with the true classification", which is defined only when $f$ is classifier.
Is this definition extended to regression case?
According to Bishop in Pattern Recognition and Machine Learning:
Boosting can give good results even if the base classifiers have a performance that is only slightly better than random, and hence sometimes the base classifiers are known as weak learners.
See also the What is meant by 'weak learner'? thread.
You can replace “classifier” with “algorithm” or “regressor” and the definitions would not change their meaning. In fact, people often use classifiers as a general case in machine learning literature and generalize from it to regression, or sometimes loosely use “classifier” to mean machine learning algorithm in general. This is also what Bishop does:
Boosting is a powerful technique for combining multiple ‘base’ classifiers to produce a form of committee whose performance can be significantly better than that of any of the base classifiers. [...] Originally designed for solving classification problems, boosting can also be extended to regression [...]
What we use as weak learners for regression are often regression trees vs classification trees for classification.
weak-learner is often defined relating to PAC learning.
PAC learning only consider binary classification to build a learning theory. But it can be applied to any supervised learning setting.
Is this definition extended to regression case?
Naively, we can extent this definition to regression models. Since, by weak-learning we imply poorly performing model but slightly better than random guess. A regression model can be considered a weak-learner, with the following conditions.
