Market Forecasting—Part 2

The legendary MLB catcher Yogi Berra once said, “It’s tough to make predictions, especially about the future.” Market Forecasting definitely helps but you need to know How to Do It & Consume It (Part 1). But building the forecast is only half the battle. You also need to know how to make it useful and repeatable. Part 2 of this series shows you how.

Read on to discover 5 key insights that’ll give you better visibility into what drives your forecasts, so you can stay ahead of the curve.

How to make it useful

1. Translate the forecast into the actions you should take

No matter how obvious a future state may be, translating the forecast from a graph or spreadsheet into what you should actually do is what creates value. Behavioral economists tell us that when a forecast agrees with a user’s intuition, the user will consider it a “good forecast” which reflects what they call “confirmation bias.”

Confirmation bias predisposes a user not to challenge a forecast if it lines up with their expectations, and to challenge it otherwise. (And this is bad.) But if the forecast is translated into a clear comparison of potential actions, this bias tends to be muted. To show how this is done, we’ve graphed four different procurement alternatives for a purchase of Select Beef Ribs for delivery 12 weeks in the future (Fig. 1):

Figure 1 – A comparison of four procurement alternatives for Select Beef Ribs 12 weeks out.

The box and whisker plot in figure 1 shows the expected gross profit for the four different procurement alternatives. The horizontal lines represent the expected value while the vertical lines represent the range of possible values.

We forecasted the cost to buy Select Beef Ribs for delivery 12 weeks out, under four different options. Option A is a forward purchase at a Fixed Price today, while options B, C and D are Spot Buy later, Formula Buy, and a blend of the first three options.

As a decision maker presented with these four alternatives, if you are seeking reward and willing to tolerate relatively high risk, you would go with option D which has the highest potential gross profit. In contrast, if you are risk averse, you might go with option B or option A depending on your risk tolerance. Given this view, translating the forecast into a decision alternative, it is highly unlikely that option C will result in the highest gross profit, and therefore this option should not be seriously considered.

2. Calculate and consider multiple market scenarios

As time passes and we gather new data, the output of the model may change. That’s why it’s critical to rerun the model as new data becomes available in order to see the impact on the forecasted values. Having up-to-date forecasts is useful for decision makers, but this process also builds in a more intuitive market understanding for the user.

Figure 2 – Each of these scenarios will actually change a commodity price forecast.

Here’s an interesting example of relevant scenarios to consider. The important thing is to be able to flex assumptions and see their impact on the decision alternatives we’re facing (Fig. 2).

Each of these scenarios will actually change a commodity price forecast—some by a lot and others by a little. The bottom line is examining alternative scenarios matters. Be sure to have the discipline to create and consider them as a part of the process.

3. Pay attention to other signals, too

The beef industry cares about direction, but they don’t always care as much about the magnitude of the move. It turns out you can build useful commodity forecasting models that simply predict whether the series is going to go up or down. You will be wrong sometimes, but you can get some pretty good accuracy if direction is all you care about.

Other stuff matters too—pay attention to signals other than expected value, like:

Direction of move
Extreme change in forecast
Correlation between data series

4. Backtest and backtest again

Backtesting might be time-and-resource intensive, but it’s critical if you want a scalable, repeatable modeling process to predict future outcomes.

What is backtesting? Backtesting is a method to test the accuracy of a model by going back in time and comparing what it would have forecast at a specific point in time to what actually happened.

How do you do a backtest?

Go to a historical point in time, say 1 year ago.
Forecast the future using only the historical data before that point in time to inform the model.
Compare the forecast to what actually happened to understand model accuracy.
Repeat for multiple points in time.

Figure 3 – Rigorous backtesting of models is fundamental to having confidence in the model in support of business decisions.

Regardless of the approach, don’t forget that the model matters. We can get beautiful models that are all looks and no substance—they explain history quite well, but are useless in making predictions. Backtesting is the best way to know how the model would have performed in the past and inform how it is going to perform in the future.

5. Make forecasting repeatable with automation

Even the best forecast will quickly get stale. For example, third-party data analysts in the beef space release monthly forecasts. That means that towards the end of the month, decision makers are using a forecast that could be 2-3 weeks old. (Honestly, how useful is that?)

The need to automate your modeling and backtesting is driven by the fact that key forecasts should be updated with drumbeat regularity—like daily. This is in stark contrast to how forecasting is often done, which is to run the model only when you need it. It is worth noting here that forecasts created for specific needs often contain more human bias since the modelers naturally have ideas of what answer they’re looking for.

Figure 4 – Automating your modeling process allows you to update the forecast every day and track changes in real-time.

Another reason to automate is to protect the organization from the departure of the person who best understands the modeling process. With the correct automation, if that resource leaves, the knowledge is still accessible to all. It’s that simple.

Automation is also key to organizational learning. Running your forecast models on a regular cadence allows the team to better understand ongoing market dynamics.

Conclusion

At the end of the day, no forecast is perfect. But models learn over time, and just like good wine, they tend to improve with age. By putting valuable insights like translating forecasts into actionable steps, creating multiple scenarios, backtesting, and automation, you go beyond How To Do Market Forecasting & How Consume It—you make it useful and repeatable. And when that happens, decision makers are empowered to better manage business by looking through the windshield—not the rear view mirror.