Abstract:

The traditional approach to sequential decision-making under uncertainty is an optimization problem to minimize the expected value of the accumulated loss. However, decision-makers are often risk-averse as they would rather minimize the chance of having a very low reward than focus purely on the average. This is a rational behavior when failure can have large consequences. For instance, if a corporation suffers a disastrous loss, they may go out of business. Or in many cases, low performance entails safety issues. Hence, it is natural to move beyond average-case analysis and optimize a risk-aware objective function. Various risk measures have been proposed in the literature, e.g., mean-variance, Value-at-Risk, conditional Value-at-Risk, spectral risk measures, prospect theory and its later enhancement, cumulative prospect theory.

In this work, we consider the problem of estimating a spectral risk measure (SRM) from independent and identically distributed samples, and propose a novel method that is based on numerical integration. We show that our SRM estimate concentrates exponentially, when the underlying distribution has bounded support. Further, we also consider the case when the underlying distribution is either Gaussian or exponential, and derive a concentration bound for our estimation scheme. Finally, we solve a SRM-sensitive multi-armed bandit problem using the best arm identification (BAI) paradigm. BAI is suitable because of simulation optimization, and also the fact that SRM relates to rare events, making samples hard to obtain in real-world settings. Further, we practically validate our algorithm using SUMO, a state-of-the-art traffic simulator in a vehicular traffic routing application.

Regards,
Ajay Kumar Pandey