In this paper we provide a number of tests that are designed to be powerful against structural breaks while allowing for a variety of nonlinear specifications for the dynamic model. It is clear that in the presence of nonlinearity standard tests of structural breaks for linear models may not have the expected performance under the null hypothesis of no breaks because the model is misspecified. We therefore proceed by approximating the conditional expectation of the dependent variable through a neural network. Then, the residual from this approximation is tested using standard residual based structural break tests. We investigate the asymptoptic behaviour of residual based structural break tests in nonlinear regression models. Monte Carlo evidence suggests that the new tests are powerful against a variety of structural breaks while allowing for stationary nonlinearities.