Suppose that the $ k$th stage has passed. What information is available? It is assumed that at every stage, a sensor observation is made. This results in a sensing history,

$\displaystyle {\tilde{y}}_k = (y_1,y_2,\ldots,y_k).$ (11.12)

At every stage an action can also be applied, which yields an action history,

$\displaystyle {\tilde{u}}_{k-1} = (u_1,u_2,\ldots,u_{k-1}).$ (11.13)

Note that the action history only runs to $ u_{k-1}$; if $ u_k$ is applied, the state $ x_{k+1}$ and stage $ k+1$ are obtained, which lie beyond the current stage, $ k$. By combining the sensing and action histories, the history at stage $ k$ is $ ({\tilde{u}}_{k-1},{\tilde{y}}_k)$.

Steven M LaValle 2012-04-20