Reducing the weights limits the range of possibilities the model can achieve.
Imagine a simple linear model like this:
$$Y = b_0 + b_1x_1$$
For example's sake say $Y$ is the height of the person and $x_1$ is the weight of the person.
If we don't limit the range of $b_1$ then the model can fit any value to it. Based on what data we use to estimate the parameters, the model might decide that with each additional cm of height the weight of the person increases by 10kg.
By adding a penalty to high values of weights we prevent the model from choosing extremely high estimates. The model is more constrained - it becomes regularized. This limits overfitting because the range of possible values the model can have now is limited.
In turn when you have a lot of features such a constraint will prevent the model from basing all of it's results on a single feature. In the optimization step where the possible solutions will be between choosing one extremely reliable feature and several less reliable ones - the model will use several features. Just because putting big weight on one of them will become too costly. This again might reduce overfitting, because the model that uses several features often can be more robust.
Bottom line: by adding regularisation the space of possible models you can achieve is reduced. And simpler models are less variable, hence less overfitting (but potentially more biased).
The same reasoning should transfer to neural networks.