1

With the advent of Machine Learning, Big data and Artificial Network can models learn over a period of time to take its own decisions?

Harish
  • 11
  • 2
  • 2
    It is possible that sometime in the future, the fitting of models for only predictive purposes on a completely understood data pipeline will be automated. This would leave the other 90% of the job in human hands. – Matthew Drury Mar 06 '16 at 07:17
  • Not until there is strong AI. At which point, basically everything will be automated. – gung - Reinstate Monica Mar 06 '16 at 10:26
  • @MatthewDrury the key components for that type of automation are already out there at the moment. – Marc Claesen Mar 06 '16 at 12:07
  • See [What do statisticians do that can't be automated?](http://stats.stackexchange.com/q/22572/17230). And "threat" seems to imply a rather pessimistic view of technological progress. – Scortchi - Reinstate Monica Mar 06 '16 at 14:05

1 Answers1

1

Well, the greatest part can be automated (and already is) :

  • Fitting (cross validating) the model

Many R packages or Python's sk-learn do it quite well.

  • Encoding the variables

The factors (per example) can be viewed as dummy variables / integers / mean response of the target and sent to a cross validated model. Thus allowing to choose the best encoding of the features (with respect to the CV score)

  • Selecting the best subset of variables

Is really easy to automate as well.

But...

  • Given the current time needed to train the models (the process presented above is really time consuming), some opinion based choices are necessary

  • The metric to optimize is a human decision

  • Feature engineering is hard to automate (and often needs a domain knowledge)

And this second part is usually the one that takes most time...

RUser4512
  • 9,226
  • 5
  • 29
  • 59