Hasan Rasay

Markov decision processes, also known as stochastic dynamic programs or stochastic control problems, are models for sequential decision making when outcomes are uncertain. In this paper, first, the stock option model is briefly introduced. Then, an optimality equation is obtained using Markov decision process. The optimality equation is a recursive equation and it is concluded that there is no simple rule for obtaining an explicit solution for the optimality equation. Anyway, this equation has some properties that yield the structure of the optimal policy. These properties are proved using induction approach. Finally application of this new proof is illustrated using two examples.