2

Most of the resources that I find in the internet deal with Q-learning or SARSA in the discrete sense, when state spaces and actions are finite. There are variety of examples and tutorials for this case.

I am looking for the case when state space is infinite. In this case, it is impossible to list all the states. Hence, an important technique would be to make a features-based representation. However, the resources are few and far in between.

If you know of resources, websites, examples, lectures, even video lectures, especially on the Q-learning/SARSA with features-based representation, it be great to have a look.

Sean Easter
  • 8,359
  • 2
  • 29
  • 58
cgo
  • 7,445
  • 10
  • 42
  • 61
  • 1
    The classic Sutton and Barto book talks a lot about state space approximation. It is available freely online. – Don Reba Dec 04 '15 at 07:36
  • 2
    Possible duplicate of [How to fit weights into Q-values with linear function approximation](http://stats.stackexchange.com/questions/187110/how-to-fit-weights-into-q-values-with-linear-function-approximation) – mlwida Dec 23 '15 at 09:35

1 Answers1

1

Georgia Tech has recently released a reinforcement learning course that covers this in a section called "Generalization." See also Gordon's paper "Stable function approximation in dynamic programming", and Ladoukis & Parr's paper on least-squares policy iteration.

You may also find value in the BURLAP continuous-domain tutorials. (BURLAP is a Java library "for the use and development of single or multi-agent planning and learning algorithms and domains to accompany them.") In particular, the lunar lander example uses SARSA($\lambda$).

Sean Easter
  • 8,359
  • 2
  • 29
  • 58