Nontrivial spin correlations in disordered Sb$_2$Te$_3$ films

Disorder plays a fundamental role in low-dimensional electronic systems; it can lead to electron localization and quantum phase transitions. Two-dimensional (2D) metallic Dirac surface states of 3D topological insulators (TIs) are expected to be robust against nonmagnetic disorder, but the range of this robustness is still unclear. Here we show that under \textit{strong nonmagnetic disorder}, the 3D topological material Sb$_2$Te$_3$ develops nontrivial spin correlations that break time reversal symmetry, which may set the limits to the topological state. To measure magnetization $M(T)$, thin (20-50 nm) films were exfoliated atop custom-designed on-chip micro-Hall sensors. The disorder $W$ was tuned from amorphous to crystalline state by thermal annealing, and tracked by Raman spectroscopy and TEM. $M$ onsets sharply at $T \sim 200 $ K, and for large $W$ at low $T$ is surprisingly large ($m \simeq 0.001g\mu_B$, \textit{g}-factor $\sim 50$). It is history and time dependent, and is associated with the 3D character of measured negative magnetoresistance in the Anderson localization (hopping) regime. $M$ vanishes at the 3D-2D transition into a topological state, which occurs at a threshold disorder level $W = W_t$.