48

During the first half of 2015 I did the coursera course of Machine Learning (by Andrew Ng, GREAT course). And learned the basics of machine learning (linear regression, logistic regression, SVM, Neuronal Networks...)

Also I have been a developer for 10 years, so learning a new programming language would not be a problem.

Lately, I have started learning R in order to implement machine learning algorithms.

However I have realized that if I want to keep learning I will need a more formal knowledge of statistics, currently I have a non-formal knowledge of it, but so limited that, for example, I could not properly determine which of several linear models would be better (normally I tend to use R-square for it, but apparently that is not a very good idea).

So to me it seems pretty obvious that I need to learn the basics of statistics (I studied that in uni but forgot most of it), where should I learn, please note that I don't really need a fully comprehensive course, just something that within a month allows me to know enough so I can get eager and learn more :).

So far I have read about "Statistics without tears", any other suggestion?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
Juan Antonio Gomez Moriano
  • 1,171
  • 1
  • 12
  • 16
  • 2
    For Statistics: Casella, G. and R. L. Berger (2002): Statistical Inference, Duxbury. For Econometrics: Hayashi, F. (2000): Econometrics, Princeton University Press. For another point of view: http://stats.stackexchange.com/questions/91863/what-to-learn-after-casella-berger – Guilherme Salomé Oct 13 '15 at 02:10
  • I added the `references` tag. You might like to scan through the first page of [hits](http://stats.stackexchange.com/questions/tagged/references) on that topic. – Glen_b Oct 13 '15 at 02:55
  • 3
    I don't see that this should be closed. I do see an argument for making it CW, though. – gung - Reinstate Monica Oct 13 '15 at 04:25
  • 2
    From my point of view, knowledge will be biased if you start learning statistics without the acquaintance of probability theories before. – Metariat Oct 13 '15 at 08:09
  • 2
    I would like to add one cautionary word. I feel sure you understand this already to some degree, but I just want to say it. I am an MD/PhD student. With my MD degree, I plan to practice internal medicine. For my PhD, I am studying biostatistics. I want you to know that you can no more get a mastery of statistics in 1 month than you can a mastery of medicine in one month. I am by no means trying to discourage you from learning statistics. Quite the opposite, I hope you understand it magnificently. But just understand it is no less involved than wanting to be a developer, for instance. – Vincent Laufer Oct 14 '15 at 18:01
  • Two years have passed, I'm curious to know where you're at on your road to ML? – Aksakal Feb 19 '16 at 14:22
  • I think you made a mistake, two years have not passed :) – Juan Antonio Gomez Moriano Feb 23 '16 at 04:32

7 Answers7

26

I would suggest you a basic road-map about how to go about it:

Bonus:

A wonderful site for such road maps is Metacademy, which I personally would vouch as one of the best Data Science resources on the web.

Gitxiv is another beautiful site, which connects the Arxiv research papers on Data Science with the relevant open source implementations/libraries.

Dawny33
  • 2,239
  • 1
  • 21
  • 37
  • 3
    OP has already taken Ng's course, that's what prompted him to ask the question in the first place. – Aksakal Oct 13 '15 at 15:24
  • 4
    @Aksakal I have noticed it. But, included it as a part of the road map. Wouldn't really make a difference, so I thought including it would help others who are reading this post. – Dawny33 Oct 13 '15 at 15:44
12

Have you checked out either Think Stats or Think Bayes--they are both (free) stats books geared towards programmers and with plenty of Python code.

Also, if you're interested in learning R then CRAN has a lot of (free) pdfs that you might want to check out, such as Introduction to Probability and Statistics Using R. There's also a Coursera course that uses R which a lot of people really love (they use this textbook, which you might want to check out as well, and have labs on DataCamp, I believe).

Also, if you want to brush up on a few Stats topics you can always watch a couple videos on Khan Academy.

Steve S
  • 1,064
  • 8
  • 17
  • I like Think Stats and Think Bayes but they deliberately avoid a lot of the formal statistical theory in favour of getting things done through code. Great for getting an intuitive grasp on the subject, but not as good if your goal is to understand the underlying theory. – Marius Oct 13 '15 at 11:00
  • @Marius: I know what you mean. I was thinking, however, that because he's already a programmer and also because he seemed to want "something small, simple and quick" that it might be more of what he's looking for. – Steve S Oct 13 '15 at 11:33
8

If you were ever, even in distant past, able to solve problems in this list, then you should attempt to study applied stats "properly". I'll give you a simple two step algorithm.

First, get up to speed with probability theory. There are many great books. My favorite is the classic book by Feller. It's called "Introduction" but don't be fooled by the title, it's as deep as you wish to go, yet very well written and simple if you just want to skim the surface.

The second step is statistics. Again, there's a ton of great books. I'll give you one that I used, a decent intro text by Gujarati "Basic Econometrics", Fourth Edition. Econometrics is statistics applied to economics. For a reference, a guy who everyone thinks said that data scientist is going to be a sexiest job in next 10 years is Hal Varian, a Berkeley economist. A lot of machine learning stuff is based on basic statistics, regressions etc. All that is covered in this book, and you don't need to read it all, it's written in a way that you can pick chapters in your own order.

You'll be surprised to see how many gaps left open after Ng's class are filling out quickly while reading these texts.

As a practitioner, you don't need too much theory after these two steps. You can keep learning ML techniques specifically reading the books in this field. It's important not to get too deep in the beginning into probability and stats. Get your code going for ML first, and fill in the gaps as you go.

Aksakal
  • 55,939
  • 5
  • 90
  • 176
4

Everyone is recommending Casella & Berger, which is almost universally used in graduate statistics programs. It's not a bad reference book, but I'm not sure I'd do more than scan the first 4-5 chapters. I don't think you need the theory of how to construct a Neyman-Pearson type test before delving into "statistics" i.e. data analysis.

Instead, I would focus on learning methods. My graduate program used Applied Linear Statistical Methods for the frequentist tests, and it's a pretty decent comprehensive reference, but might not be the most approachable book from a self-teaching standpoint. A course or two from MIT or coursera might be a better way to start in on that, because you'll get a broader overview with more examples than you might from reading a book.

For Bayes, the book I've seen used most often is Doing Bayesian Data Analysis, which comes with puppy pictures (clearly, this makes the book superior to other Bayesian introductory textbooks). I've never used the book myself, but I've paged through it and it seems pretty decent - much better than Gelman's book, which I found somewhat incomprehensible AFTER two classes in Bayesian statistics - the explanations are terrible.

srvanderplas
  • 149
  • 3
  • 1
    The first 5 chapters of C&B are not actually statistics at all, more like background ... The concept of a statistic is addressed in the beginning of chapter 6! More to the point, learning methods will probably not help this specific person. it would help him apply statistics, not understand it, which is what he needs. if he has advanced mathematical training, he can probably skip it to some degree, but his answer suggests he is currently unable to understand the underpinnings of ML ... which strongly suggests his math is limiting (to me, at least). C&B might not be a bad place to start. – Vincent Laufer Oct 14 '15 at 18:11
  • 1
    They may not be statistics, but the background on probability distributions is essential in order to do any sort of modeling - you need to know what a bernoulli distribution is and what its properties are before you can understand logistic regression, for instance. I still reference C&B occasionally, but I don't think I've ever used anything beyond chapter 6 outside of the class I took that used that book. – srvanderplas Oct 15 '15 at 21:49
  • 1
    i quite agree with what you said, but it relates to the digression instead of the major point - which is my fault for adding the digression in the first place. at any rate, the main point is that as several others have suggested, what OP actually needs to do is get a better understanding of theoretical math and statistics. no where in the post does it indicate he needs any help with applying more statistical tests. he can do that. he wishes to understand them more deeply. for this, C&B is better than learning more hands on application oriented preparation. – Vincent Laufer Oct 17 '15 at 19:10
3

This is not intended to be a complete answer, it's just a suggestion. If you want to learn more about statistics (the foundation), you could read:

Casella, G. and R. L. Berger (2002): Statistical Inference, Duxbury

This is a pretty standard book for statisticians and it has a lot of interesting results. You don't need to go through all the proofs of the theorems, but you might want to do some exercises in order to feel more secure with the results.

If you want to learn more about econometrics (models for data), you could take a look at:

Hayashi, F. (2000): Econometrics, Princeton University Press

Someone else actually asked something similar to what you asked and got a nice answer: What to do after "Casella & Berger".

Furthermore, if you really intend on reading these books, this syllabus of an econometrics course can give you a quite good direction and pace on what to read (CB & Hayashi) and when to read.

Community
  • 1
Guilherme Salomé
  • 739
  • 7
  • 16
  • Thanks for the suggestion, however the first book you mention is around 660 pages... I have read bigger books, but is there something small, simple and quick so I can get a basic grasp of it? – Juan Antonio Gomez Moriano Oct 13 '15 at 02:40
  • 3
    Casella and Berger will give you a chunk of the *theory* of statistics, but you'll learn very little about data analysis. – Glen_b Oct 13 '15 at 02:51
  • 1
    @JuanAntonioGomezMoriano how small were you after? I was always a fan of [How to lie with statistics](http://www.amazon.com/How-Lie-Statistics-Darrell-Huff/dp/0393310728) as a starting point. – icc97 Oct 13 '15 at 20:44
  • (-1) Those sound like perfect choices for someone who prefers a mathematical or theoretical approach to statistics, pretty much the opposite of what the OP requested. – Gala Oct 13 '15 at 21:19
  • 1
    He did say that he needed a more "formal" knowledge and basic knowledge about statistics. – Guilherme Salomé Oct 13 '15 at 21:23
2

I'd suggest a new book that came out since the original question: Statistical Rethinking: A Bayesian Course with Examples in R and Stan by Richard McElreath, CRC Press.

It's very well written and uses a Bayesian approach. It's very interactive, and you'll want to work the problems or you may get halfway through and begin to get lost.

It starts very basic and ends up with multi-level models, and it's aimed at fairly advanced scientists who have some statistical knowledge but don't feel comfortable overall with statistics as it was taught to them. So I can't exactly say it's a beginner's book, but it does start very simply and he has a wonderful arc and style.

The "Stan" part of the title is a general-purpose Bayesian sampling tool. Essentially, it's a programming language that compiles automatically to C++ and then gets compiled to an executable. (Bayesian inference is general, unlike alternatives, so you can have a generalized tool.)

Wayne
  • 19,981
  • 4
  • 50
  • 99
1

Figured I'd throw this answer in for posterity, even if it's likely too late to be useful to you. Larry Wasserman's All Of Statistics was conceived as a course for people with a background in machine learning, other comp sci disciplines, or math who didn't have any formal statistics training -- i.e., people in pretty much exactly your current situation. Having a similar lack of formal stats, a few friends and I formed a self-study group to go through it in grad school. I think I really benefited from that experience.

The extra topics Wasserman throws in beyond the typical "probability and statistical inference" course material, like graphical models and bootstrapping, are particularly relevant to someone working in machine learning. I should say that the book can be pretty terse compared to something like Casella & Berger, so if you want more detail or motivation for certain parts (especially proofs) you may have to supplement it with other reading material. That said, I also found the book to be clearly-written with a good number of practice problems, and it's an excellent quick reference.

One month is not a lot of time. If you set a very aggressive pace, though, I think you can certainly get a lot out of this text in one semester: we did our self-study group over the summer, for instance. That's especially true if you're mostly interested in linear modeling, which you'll hit by Ch. 13-14.

Patrick B.
  • 173
  • 6