It's not widely known, but the derivative of a gaussian is also a gaussian. Similarly, the conjugate prior of the exponential function