What is the difference between fully observable MDPs and POMDPs?
In a fully observable MDP the agent can directly observe which state it is in. In a POMDP the agent can not directly observe the state, but it can make observations that are dependent on the state.