A test of CP invariance in Higgs boson production via vector-boson fusion in the $H\rightarrow\tau\tau$ decay mode is presented. This test uses the Optimal Observable method and is carried out using $36.1\,\mathrm{fb}^{-1}$ of $\sqrt{s}=13\,\mathrm{TeV}$ proton-proton collision data collected by the ATLAS experiment at the LHC. Contributions from CP-violating interactions between the Higgs boson and electroweak gauge bosons are described by an effective field theory, in which the parameter $\tilde{d}$ governs the strength of CP violation. No sign of CP violation is observed in the distributions of the Optimal Observable, and $\tilde{d}$ is constrained to the interval $[-0.090, 0.035]$ at the 68% confidence level (CL), compared to an expected interval of $\tilde{d} \in [-0.035, 0.033]$ based upon the Standard Model prediction. No constraints can be set on $\tilde{d}$ at 95% CL, while an expected 95% CL interval of $\tilde{d} \in [-0.21,0.15]$ for the Standard Model hypothesis was expected. Object definition:- Electron candidates are required to pass the medium likelihood-based identification selection, to have transverse momentum $p_T > 15$ GeV and $|\eta| < 2.47$, excluding the region $1.37< |\eta| < 1.52$. - The muon candidates are required to have $p_T > 10$ GeV and $|\eta|$ <2.5 and to pass the medium muon identification requirements. - The tau candidates are required to have $p_T > 20$ GeV, one or three associated tracks, an absolute electric charge of one and $|\eta| < 2.47$, excluding the region $1.37< |\eta| < 1.52$. They have to pass the medium identification requirement in the $\tau_{\mathrm{lep}}\tau_{\mathrm{had}}$ channel and the tight identification requirement in the $\tau_{\mathrm{had}}\tau_{\mathrm{had}}$ channel. - Jets are reconstructed from topological clusters in the calorimeter using the anti-Kt algorithm with a radius parameter value R = 0.4, and are required to have $|\eta| < 4.9$. The VBF inclusive region is characterized by the presence of two jets with $p_T > 30$ GeV. In addition, the two jets are required to be in opposite hemispheres of the detector with a large pseudorapidity separation of $|\Delta \eta_{jj}| > 3$ and their invariant mass $m_{jj}$ is required to be larger than 300 GeV. To construct a signal region enriched in VBF signal events, BDTs trained to discriminate between the VBF signal and backgrounds are used in all channels. Kinematic variables used in the BDT training can be categorized as follows: properties of the Higgs boson which discriminate against all background processes without a Higgs boson, properties of a resonant di-tau decay which discriminate against processes with mis-identified tau-decay candidates, and properties of the VBF topology. The most important variables in the training are $m_{\tau\tau}^{\mathrm{MMC}}$, $m_{jj}$ and $C_{jj}(\tau)$. A threshold on the BDT score is used to define the final signal region in each channel. This threshold is chosen to yield a high signal significance.