Linear discriminant analysis (LDA) is a versatile method in all pattern recognition fields but it suffers from some limitations. In a multi-class problem, when samples of a class are far from other classes samples, it leads to bias of the whole decision boundaries of LDA in favor of the farthest class. To overcome this drawback, this study is aimed at minimizing this bias by redefining the between- and within-class scatter matrices via incorporating weight vectors derived from Fisher value of classes pairs. After projecting the input patterns into a lower-dimensional space in which the class samples are more separable, a new version of nearest neighbor (NN) method with an adaptive distance measure is employed to classify the transformed samples. To speed up the adaptive distance routine, an iterative learning algorithm that minimizes the error rate is presented. This efficient method is applied to six standard datasets driven from the UCI repository dataset and test results are evaluated from three aspects in terms of accuracy, robustness, and complexity. Results show the supremacy of the proposed two-layer classifier in comparison with the combination of different versions of LDA and NN methods from the three points of view. Moreover, the proposed classifier is assessed in the noisy environment of those datasets and the achieved results confirm the high robustness of the introduced scheme when compared to others.