This paper examines how the probabilistic inference for learning control (PILCO) method incorporates uncertainty into its decision-making process and investigates the impact of uncertainty on its optimization performance. First, the various sources of uncertainty within the function model under a given policy are identified and quantified. Then, variance is adopted as the measure of uncertainty, and the law of total variance is applied to decompose the overall cost uncertainty into two components. Finally, a gold-standard Monte Carlo scheme is introduced to approximate PILCO’s dynamics model by propagating trajectories, thereby enabling the separate quantification of aleatoric and epistemic uncertainties in the cost. Simulation results indicate that effective PILCO learning is associated with the selection of policies in which the epistemic cost uncertainty constitutes a high proportion of the total cost uncertainty.