The long-range collective flow of particles produced in oxygen-oxygen (OO) and neon-neon (NeNe) collisions is measured with the CMS detector at the CERN LHC. The data samples were collected at a center-of-mass energy per nucleon pair of 5.36 TeV, with integrated luminosities of 7 nb$^{-1}$ and 0.8 nb$^{-1}$ for OO and NeNe collisions, respectively. Two- and four-particle azimuthal correlations are measured over nearly five units of pseudorapidity. Significant elliptic ($v_2$) and triangular ($v_3$) flow harmonics are observed in both systems. The ratios of $v_n$ coefficients between NeNe and OO collisions reveal sensitivity to quadrupole correlations in the nuclear wave functions. Hydrodynamic models with $\textit{ab initio}$ nuclear structure inputs qualitatively reproduce the collision-overlap dependence of both the $v_n$ values and the NeNe to OO ratios. These measurements provide new constraints on hydrodynamic models for small collision systems and offer valuable input on the nuclear structure of $^{16}$O and $^{20}$Ne.