Retrieving spectral wave parameters such as the peak wave direction and wave period from marine radar backscatter intensity is very well developed. However, the retrieval of significant wave height is difficult because the radar image spectrum (a backscatter intensity variance spectrum) has to be transferred to a wave spectrum (a surface elevation variance spectrum) using a modulation transfer function (MTF) which requires extensive calibration for each individual radar setup. In contrast to the backscatter intensity, the Doppler velocity measured by a coherent radar is induced by the radial velocity (or line-of-sight velocity) of the surface scattering and its periodic component is mainly the contribution of surface waves. Therefore, the variance of the Doppler velocity can be utilized to retrieve the significant wave height. Analyzing approximately 100 days of Doppler velocity measurements of a coherent-on-receive radar operating at X-band with vertical polarization in transmit and receive, a simple relation was derived and validated to retrieve significant wave heights. Comparison to wave measurements of a wave rider buoy as well as an acoustic wave and current profiler resulted in a root mean square error of 0.24 m with a bias of 0.08 m. Furthermore, the different sources of error are discussed and investigated.