Improve cooperation between AI and human planners

Supply chain planners have the ability to modify forecasts produced by self-learning systems using artificial intelligence. Does it add value? And on which predictions can we trust more: those of the software or those of the man? In a groundbreaking study, KLU researchers Naghmeh Khosrowabadi, Prof. Kai Hoberg and Prof. Christina Imdahl (Eindhoven University of Technology) get to the bottom of these questions. According to their study, human intervention – on average – did not increase forecast accuracy.

For the study, the team analyzed data from 30 million forecasts for every product unit, per store, and every day from a leading AI vendor and a major European food retailer.

Planners’ optimism leads to more inaccurate forecasts

The results show that planners, on average, do not contribute to forecast accuracy. “Instead, planners even tend to overcompensate for effects like weather or a discount that have already been taken into account by the AI ​​system,” Khosrowabadi explains. In the study, only 50% of human interventions led to better outcomes.

A closer look at the data further reveals that around 5% of AI-generated forecasts were adjusted by supply chain planners. “We wanted to know why the planners decided to adjust the AI-generated forecast,” says Naghmeh Khosrowabadi. “Our results show that product characteristics such as price, freshness, or discounts are key drivers of how often planners adjust to AI forecasts.”

If, for example, the AI ​​system gives a forecast for a very expensive product, planners tend to pay more attention to it and are more likely to intervene, for example by adjusting the forecast. “Furthermore, our results show that large increases over AI predictions, such as when the human prediction of items for sale on a given day at a specific store is twice as high as the AI ​​prediction, are more frequent but also often inaccurate. Too much optimism on the part of planners seems to be a problem here,” says Professor Kai Hoberg. Drops from AI forecasts, on the other hand, were less likely but more accurate.

Improve cooperation between human planners and AI

“Human planners will continue to play an important role in AI-based forecasting processes,” says Professor Hoberg, “in some cases, human planners have knowledge that is not accessible to an AI system, for example local events or competitor actions, which increase the chances of a better forecast to more than 70%. This is why we need to strengthen the cooperation of planners and AI.”

For this, the team recommends more exchanges between traders and AI providers: the better planners understand how the system makes its predictions, the easier it is for them to decide when to intervene. “Using the results of our study, companies can save money and time,” promises Khosrowabadi, “The key is to help planners decide when they need to intervene – and when the system works well on its own and that they can concentrate on other tasks.”

The study involved Naghmeh Khosrowabadi as part of his doctoral thesis, Professor Dr Kai Hoberg and Professor Dr Christina Imdahl from KLU. They analyzed data from 30 million store-level forecast SKUs from a leading AI vendor and a major European retailer. Data on additional variables such as products, weather or holidays were also taken into account.

The book is published in the European Journal of Operational Research.

Provided by Kühne University of Logistics