Optimization problems depending on a probability measure correspond to many applications. These problems can be static (single-stage), dynamic with finite (multi-stage) or infinite horizon, single- or multi-objective. It is necessary to have complete knowledge of the "underlying" probability measure if we are to solve the above-mentioned problems with precision. However this assumption is very rarely fulfilled (in applications) and consequently, problems have to be solved mostly on the basis of data. Stochastic estimates of an optimal value and an optimal solution can only be obtained using this approach. Properties of these estimates have been investigated many times. In this paper we intend to study one-stage problems under unusual (corresponding to reality, however) assumptions. In particular, we try to compare the achieved results under the assumptions of thin and heavy tails in the case of problems with linear and nonlinear dependence on the probability measure, problems with probability and risk measure constraints, and the case of stochastic dominance constraints. Heavy-tailed distributions quite often appear in financial problems \cite {Meer 2003)} while nonlinear dependence frequently appears in problems with risk measures \cite {Kan (2012a),Pflu (2007)}. The results we introduce follow mostly from the stability results based on the Wasserstein metric with the "underlying" L1 norm. Theoretical results are completed by a simulation investigation.