Logistic Regression (LR) has become a widely used and accepted method to analyze binary or multiclass outcome variables, since it is a flexible tool that can predict probability for the state of a dichotomous variable. A recently proposed LR method is based on the hybridization of a linear model and Evolutionary Product Unit Neural Network (EPUNN) models for binary classification. This produces a high number of coefficients, so two different methods for simplifying the structure of the final model by reducing the number of initial or PU covariates are presented in this paper, both being based on the Wald test. The first method is a Backtracking Backward Search (BBS) method, and the other is similar, but it is based on the standard Simulated Annealing process for the decision steps (SABBS). In this study, we used aerial imagery taken in mid-May to evaluate the potential of two different combinations of LR and EPUNN (LR using PUs (LRPU), as well as LR using Initial covariates and PUs (LRIPU)) and the two presented methods of structural simplification of the final models (BBS and SABBS) used for discriminating Ridolfia segetum patches (one of the most dominant, competitive and persistent weed in sunflower crops) in a naturally infested field of southern Spain. Then, we compared the performance of these methods to six commonly used classification algorithms; our proposals obtaining a competitive performance and a lower number of coefficients.