In many places, undergrads in statistics learn to derive common statistical procedures, so any standard text that has basic mathematical statistics - of which there are dozens of reasonably suitable choices - covers the basics of this. It requires little more than basic calculus and a little linear algebra to cover many of the tests discussed in a basic intro stats course.
A text like Wackerly, Mendenhall and Scheaffer, Mathematical Statistics with Applications has derivations for a number of basic tests (some might be partly covered in exercises). Pretty much any edition will do - you don't need a recent one, and there are any number of good alternatives at a similar level.
I'd probably start with something like that.
If you seek a little more depth (within a still-widely-used text), you might consider Casella and Berger, Statistical Inference. I don't claim either book is the best of its kind, just that they're widely used books.
I'd supplement those with a book on nonparametric statistics. Possibly something like Conover's Practical Nonparametric Statistics, which gives just about enough background to derive a number of popular tests, and to understand something of how permutation tests work.
What you'd need beyond that is a reasonable text on regression. I'll try to come back with a suggestion for that, but from my recollection Fox's book Applied Regression analysis and Generalized Linear Models was reasonable and if I recall correctly has some of the derivations.
If you want to look at the bootstrap, perhaps Davison and Hinkley, Bootstrap Methods and their Application.
What else you might look at (e.g. Bayesian stats, time series, survival analysis etc etc) depends on how broad a coverage you want.
