CERN-LHC, ATLAS. Searches for the exclusive decays of the Higgs boson to an $\omega$ meson and a photon or a $K^{*}$ meson and a photon can probe flavour-conserving and flavour-violating Higgs boson couplings to light quarks, respectively. Searches for these decays, along with the analogous $Z$ boson decay to an $\omega$ meson and a photon, are performed with a $pp$ collision data sample corresponding to integrated luminosities of up to 134 fb$^{-1}$ collected at $\sqrt{s}=13$ TeV with the ATLAS detector at the CERN Large Hadron Collider. The obtained 95% confidence-level upper limits on the respective branching fractions are ${\cal B}(H\rightarrow\omega\gamma)< 1.5\times 10^{-4}$, ${\cal B}(H\rightarrow K^{*}\gamma)< 8.9\times10^{-5}$ and ${\cal B}(Z\rightarrow \omega\gamma)<3.8\times 10^{-7}$. The limits for $H\rightarrow \omega\gamma$ and $Z\rightarrow \omega\gamma$ are 100 times and 17 times the Standard Model expected values, respectively. The result for $Z\rightarrow \omega\gamma$ corresponds to a three-orders-of-magnitude improvement over a previously set limit.