Humans estimate the 3D motion of the self and of objects in the natural environment from the 2D images formed in the two eyes. To understand how this problem should be solved, we trained Bayesian observers on naturalistic binocular videos to solve two tasks: 3D speed estimation and 3D direction estimation. The resulting normative models leverage two cues used by humans--the changing disparity and the interocular velocity difference cues--and show that quadratic combination of linear filter responses is an optimal computation for speed estimation but not for direction estimation. The models reproduce the psychophysical response patterns that characterize human performance in 3D motion estimation tasks, including biases, discrimination thresholds, and counter-intuitive towards-away confusions. These results suggest that, rather than resulting from a haphazardly constructed system, the sometimes surprising performance patterns in human 3D motion perception result from optimal visual information processing.