Observation of an unconventional collective mode on the surface of a topological insulator

Using a high resolution electron energy loss spectroscopy (HREELS) system with the function of two-dimensional energy and momentum mapping, we studied the surface collective modes of a prototypical three-dimensional topological insulator Bi$_{\mathrm{2}}$Se$_{\mathrm{3}}$ with Dirac surface state, and observed a new surface collective mode with energy dispersing from 0 to 10 meV. In contrast, on the surface of Mn-doped Bi$_{\mathrm{2}}$Se$_{\mathrm{3}}$, the mode does not exist due to the absence of the Dirac surface state because of magnetic doping. Thus we conclude the observed low energy collective mode is originated from the surface Dirac electrons. This new HREELS system allows us to detect the collective modes in a large momentum range up to the second Brillouin zone center of Bi$_{\mathrm{2}}$Se$_{\mathrm{3}}$. Unlike the plasmons from regular conducting electrons, which decay into incoherent electron-hole excitations (Landau damping) when entering the electron-hole pair continuum, and disappear at large momentum, this new mode remains prominent in a large momentum range far beyond the electron-hole pair continuum.