Try to know the probability distribution over the hidden state
All estimates are decisions!
Example Gone Fishin
binary hidden state problem we are going to explore. You need to decide which side to fish on--the hidden state. We know fish like to school together. On different days the school of fish is either on the left or right side, but we don’t know what the case is today. We define our knowledge about the fish location as a distribution over the random hidden state variable. Using our probabilistic knowledge, also called our belief about the hidden state, we will explore how to make the decision about where to fish today, based on what to expect in terms of gains or losses for the decision. The gains and losses are defined by the utility of choosing an action, which is fishing on the left or right. The details of the utilities are described
Utility Function
likelihood function
Correlation
Covarying probability distribution
The relationship between the marginal probabilities and the joint probabilities is determined by the correlation between the two random variables - a normalized measure of how much the variables covary. We can also think of this as gaining some information about one of the variables when we observe a measurement from the other.
相关性越强，共享的信息就越多。如下图，相关性为1的话，两变量捆绑出现，某些组合出现的概率为0
Marginalization
与联合分布中的边缘概率如出一辙，通常我们会算某个观测在各种状态的条件下出现的概率。
example
joint probability
marginal likelihood
通过观测可以得出一个m，s的联表，表中的值即为likelihood，其结构与联合分布类似（但不同，因为先验概率的存在，联表内概率之和不为1），但在Bayes里面还存在一个先验概率，是我们对s分布的猜测，即p(s)。
marginal likelihood p(m) = p(m|s)*p(s)
Marginalization is going to be used to combine our prior knowledge, which we call the prior, and our new information from measurement, the likelihood.
Compute Posterior
Normalizing by this P(m) means that our posterior is a complete probability distribution that sums or integrates to 1 appropriately.
The normalization term, P(m) , is the probability of the measurement. This does not depend on state so is essentially a constant we can often ignore. We can compare the unnormalized posterior distribution values for different states because how they relate to each other is unchanged when divided by the same constant.
we can then use this as new prior
when prior is weak(close to random, i.e. 0.5), likelihood is likely to have more influence.
Combining all the things together
Summary
learned about combining prior information with new measurements to update your knowledge using Bayes Rule, in the context of a fishing problem.
- That the likelihood is the probability of the measurement given some hidden state
- That how the prior and likelihood interact to create the posterior, the probability of the hidden state given a measurement, depends on how they covary
- That utility is the gain from each action and state pair, and the expected utility for an action is the sum of the utility for all state pairs, weighted by the probability of that state happening. You can then choose the action with the highest expected utility.
- There are actually many (infinite) combinations that can produce the same expected utility for both actions: but the posterior probabilities will always have to balance out the differences in the utility function. So, what is important is that for a given utility function, there will be some 'point of indifference'
- What matters is the relative information: if the prior is close to 50/50, then the likelihood has more infuence, if the likelihood is 50/50 given a measurement (the measurement is uninformative), the prior is more important. But the critical insite from Bayes Rule and the Bayesian approach is that what matters is the relative information you gain from a measurement, and that you can use all of this information for your decision.
- The model gives us a very precise way to think about how we should combine information and how we should act, GIVEN some assumption about our goals. In this case, if we assume we are trying to maximize expected utility--we can state what an animal or subject should do.
- There are lots of possible extensions. Humans may not always try to maximize utility; humans and animals might not be able to calculate or represent probabiltiy distributions exactly; The utility function might be more complicated; etc.