Heading error analysis of a pulsed $^{\mathrm{87}}$Rb magnetometer at geomagnetic fields

In scalar atomic magnetometer operation at geophysical magnetic fields heading errors cause unwanted dependence of the measured field on the orientation of the sensor. We use a pulsed $^{\mathrm{87}}$Rb magnetometer using a short pumping pulse and monitoring of free spin precession to study the dependence of the heading error on the initial spin polarization and on the field orientation with respect to the pump laser. The heading error due to the orientation dependence can be calculated by a simple analytical expression in the limit of high spin polarization. For the polarization dependence, we predict a nonlinear shift in the spin precession frequency as a function of the field at lower spin polarizations. It is due to the nuclear magnetic moment which splits the center frequencies of the two hyperfine manifolds and generates interference between them. We experimentally confirm both of these predictions by carefully making frequency measurements as a function of magnetic field magnitude and angle and by taking into account hysteresis of the magnetic shielding.