Behavior of Generative AI While Interpreting Risk-Sensitive Forecasts

Abstract

The adoption of Generative AI in business has been increasing recently. This increase in demand is evident by the huge investments coming up in data centers and IT infrastructure across the world to support LLM models. But the core question that remains unaddressed in the decision science domain is whether LLM models are good enough to interpret and take business level decisions on inventory, planning and operations based on forecasts. A core tenet is that the forecasts itself are not always necessarily accurate. So, it is important to investigate whether generative AI understands the uncertainty associated with the forecast, on which its decision is based. When a group of humans take a decision, if all of them have the same understanding of the uncertainty, even though expressed in different forms or wordings, the final decision always remains the same. This is how normal decision theory works. This paper investigates whether generative AI models give consistent decisions, understand the risks of uncertainty and remain committed to the decision regardless of changes in phrasing.