The paper shows how to use optimal control to compute optimal time-consistent Markovian government policies in nonlinear dynamic general equilibrium models. It extends Cohen and Michel's (1988) results for the linear-quadratic case. The method involves replacing private agents' costate variables with flexible functions of current state variables in the government's maximization problem. The functions hold in equilibrium to an arbitrarily close approximation. They can be found numerically by perturbation or projection methods. A stochastic model of optimal public spending illustrates the technique.