Interactive reinforcement learning in R?

Question

Which library in R allows to do interactive temporal (depending on sequence of events) learning? I need to perform a task similar to game playing as shown in this example:

http://cs.stanford.edu/people/karpathy/convnetjs/demo/rldemo.html

Answer 1

The correct term you are looking for is "Online Reinforcement Learning" (instead of Interactive). The temporal aspect is implicitly present in nearly each RL problem, hence not mentioned explicitly.

Due to the fact that multiple subdomains have been connected here, I guess that there is no single R package providing all the functionality. The problem solved by Mnih et. al. in Playing Atari with Deep Reinforcement Learning can be broken in multiple pieces.

• Input is a single frame / image. So you need a package to transform it into a real matrix and more basic image preprocessing (cropping, scaling etc.)
• The State-Action-Value-Function is computed using a deep convolutional neural network, so you need an R package for that one. Architecture described depends on the type of the images.
• The surrounding Q-Learning code is very easy (see page 5 for the algorithm), hence I would not bother with wrangling it into calls to an RL package, but write it yourself.

If you are not restricted to R you may find pointers in the other attempts to reproduce the results from the paper

• https://github.com/kuz/DeepMind-Atari-Deep-Q-Learner
• https://github.com/kristjankorjus/Replicating-DeepMind
• https://github.com/spragunr/deep_q_rl

Weakly linked Question: R libraries for deep learning