In this article, we propose an automated construction of knowledge based artificial neural networks (KBANN) for the recognition of restricted sets of handwritten words or characters. The features that better describe the chosen vocabulary are first selected, according to the characteristics of the used script, language and lexicon. Then, ideal samples of lexicon elements (words or characters) are submitted to a feature extraction module to derive their description using the chosen primitives. The analysis of these descriptions generates a symbolic knowledge base reflecting a hierarchical classification of the words (or characters). The rules are then translated into a multilayer neural network by determining precisely its architecture and initializing its connections with specific values. This construction approach reduces the training stage, which enables the network to reach its final topology and to generalize. The proposed method has been tested on the automated construction of neuro-symbolic classifiers for two Arabic word lexicons.