Higgsinos with masses near the electroweak scale can solve the hierarchy problem and provide a dark matter candidate, while detecting them at the LHC remains challenging if their mass splitting is $\mathcal{O}(1 \text{GeV})$. This Letter presents a novel search for nearly mass-degenerate Higgsinos in events with an energetic jet, missing transverse momentum, and a low-momentum track with a significant transverse impact parameter using 140 fb$^{-1}$ of proton-proton collision data at $\sqrt{s}=13$ TeV collected by the ATLAS experiment. For the first time since LEP, a range of mass splittings between the lightest charged and neutral Higgsinos from $0.3$ GeV to $0.9$ GeV is excluded at 95$\%$ confidence level, with a maximum reach of approximately $170$ GeV in the Higgsino mass.