I am having a hard time trying to understand what weights and biases mean in a neural network and how do they help the neural network make right prediction . I know how weights and biases are changed by backtracking but what does their value mean and what is it's significance.
Asked
Active
Viewed 108 times
0
-
Weights are factors, biases are addends. Both are parameters in a non-linear model. Beside that, there's hardly a need for a more intuitive approach imho. – davidhigh Jul 22 '21 at 21:29
-
2Do you know what the parameters do in a linear regression? It would benefit you immensely to learn what the parameters (including the intercept) mean in a simple linear regression. A neural network is, in some sense, a big web of linear regression models. – Dave Jul 23 '21 at 04:18
1 Answers
0
Each machine learning model has some features $\mathbf{x} = (x_1, x_2, \dots, x_k)$. Those features may come from data, or be generated by a model itself (output of a previous layer of a neural network). Different models have different ways of using those features to learn patterns from the features and be able to make predictions. Some models, like linear regression, generalized linear models (including logistic regression), support-vector machines, or neural networks make the predictions by using at some point linear combinations of the features
$$ \boldsymbol{w}^T\mathbf{x} + b = w_1 x_1 + w_2 x_2 + \dots + w_k x_k + b $$
we call the parameters that we multiply the features with $\boldsymbol{w} = (w_1, w_2, \dots, w_k)$ weights and the parameters that we add biases $b$ to them. You can also see weights being called just parameters (this may include biases as well), and the bias term in regression models (linear regression, GLM's) is traditionally called an intercept.
Weights control how strongly the feature would influence the output (by multiplying it by a value) and biases help to ensure that the result is not too big or too small on average by moving it up (adding a positive constant) or down (adding a negative constant, so subtracting a constant). You usually want to use the bias term in the model.
This is a basic building block often used in machine learning. Many models follow linear combinations of features with non-linear functions that transform them (GLM's, SVM's, neural networks). Some models (neural networks) can use such blocks many times in different places. Finally, there are models using nonlinear functions (e.g. nonlinear regression) where weights are the learned parameters of more complicated functions.
Tim
- 108,699
- 20
- 212
- 390