5

I have made optimizations in continuous time that belong to the control theory, for example one case:

$\max(\min)V[u(t)]=\int_0^Tf(t,x(t),u(t))dt$

constraint to: $\dot x=g(t,x(t),u(t))$

Where:

$x(t)$: state variable.

$u(t)$: control variable.

And given this problem, one way of solving it is to form the Hamiltonian expression. Which is:

$ H(t,x(t),u(t),\lambda (t))=f(t,u(t),x(t))+\lambda g(t,x(t),u(t)) $

Now, we have to solve for the first order conditions:

$\frac{\partial H}{\partial u}=0$

$\frac{\partial H}{\partial x}=-\dot \lambda$

Now my question is, when I have an equivalent problem in discrete time (this could be expressed in different ways):

$\max(\min)=\sum_{i=1}^{T}f(t+i,x_{t+i},u_{t+i})\;\;\;\;\;$ Where $t$ is defined in a discrete space.

Constraint to $x_{t+1}=g(t,x_t,u_t)$

Now I can set (in a similar fashion as $H$) a lagrangian:

$L(t,x_t,u_t,\lambda)=\sum_{i=1}^{T}\left[ f(t+i,x_{t+i},u_{t+i})-\lambda_{t+i}[g(t+i,x_{t+i},u_{t+i})-x_{t+1+i}] \right]$

Now, what are the FOC for state $x$, and control $u$ variables in this case? I'd be thankful if you could apart from the answer, give some literature about it. Thanks!

manifold
  • 715
  • 1
  • 13
  • 2
    For discrete-time finite-horizon problem, you have the usual Theorem of Lagrange-type approach (or KKT, for inequality constraints). Notice your $L(t,x_t,u_t,\lambda)$ is exactly a standard Lagrangian---the FOC's are obtained in the usual way. The continuous time analogue of this approach is the Maximum Principle via the Hamiltonian (not surprisingly), which you started with in the beginning of the question. Second approach is dynamic programming (for Markovian constraints), whose continuous-time extension is also called dynamic programming (HJB equation). – Michael Sep 14 '20 at 20:49

0 Answers0