11.1.2 Defining the History Information Space

This section defines the most basic and natural I-space. Many others will be derived from it, which is the topic of Section 11.2. Suppose that $X$ , $U$ , and $f$ have been defined as in Formulation 10.1, and the notion of stages has been defined as in Formulation 2.2. This yields a state sequence $x_1$ , $x_2$ , $\ldots$ , and an action sequence $u_1$ , $u_2$ , $\ldots$ , during the execution of a plan. However, in the current setting, the state sequence is not known. Instead, at every stage, an observation, $y_k$ , is obtained. The process depicted in Figure 11.2.

Figure 11.2: In each stage, $k$ , an observation, $y_k \in Y$ , is received and an action $u_k \in U$ is applied. The state, $x_k$ , however, is hidden from the decision maker.

In previous formulations, the action space, $U(x)$ , was generally allowed to depend on $x$ . Since $x$ is unknown in the current setting, it would seem strange to allow the actions to depend on $x$ . This would mean that inferences could be made regarding the state by simply noticing which actions are available.^11.1Instead, it will be assumed by default that $U$ is fixed for all $x \in X$ . In some special contexts, however, $U(x)$ may be allowed to vary.