Stacks of plastic crates in a storage space

Forecast Accuracy Metrics: MAPE, WMAPE, Bias and FVA

Forecast accuracy metrics measure how far a forecast was from actual demand and in which direction. The working set is MAPE or WMAPE for the size of the error, bias for its direction, and forecast value added (FVA) for whether each step in the process improves the number. Measure them at the level and lag where an order is placed, and remember that the same error costs different amounts of inventory depending on lead time, service target and cost.

Which Forecast Accuracy Metrics to Use

Forecast error for one item and period is actual minus forecast. Every accuracy metric is a way of adding those errors up across items and periods, and each way answers a different question. Always score forecasts on periods the model did not see when it was fitted (Hyndman and Athanasopoulos, Forecasting: Principles and Practice).

MetricFormulaWhat it tells youWhere it misleads
MAPEAverage of |actual – forecast| / actual, per lineTypical percentage error of a lineUndefined when actual is zero; slow items dominate
WMAPESum of |actual – forecast| / sum of actualError weighted by volumeCan hide poor accuracy on small items
BiasSum of (forecast – actual) / sum of actualDirection: over- or under-forecastingErrors in opposite directions cancel, so bias can look fine on a weak forecast
FVAError of the previous step minus error of this stepWhether a step (model, override, consensus) helpsOnly as good as the error metric underneath

One common convention reports accuracy as 1 minus WMAPE. Write down the sign convention for bias as well. Here a positive figure means over-forecasting, which builds stock; some tools use actual minus forecast and flip the sign.

MAPE, WMAPE and Bias in a Worked Example

Take three items in one week:

ItemActualForecastAbsolute errorPercentage error
A, fast mover1,00090010010%
B, mid mover1001303030%
C, slow mover4106150%

MAPE is the average of the three percentages: (10 + 30 + 150) / 3 = 63.3%. The slow item, which sold four units, decides most of the score. MAPE is also undefined in any period with zero sales, has extreme values when actuals are close to zero, and penalizes over-forecasts more heavily than under-forecasts (Hyndman and Athanasopoulos; Wikipedia, mean absolute percentage error).

WMAPE divides total absolute error by total actuals: 136 / 1,104 = 12.3%. Item A now carries most of the weight, which is closer to where the money is. Bias is (-100 + 30 + 6) / 1,104 = -5.8%: overall, the forecast ran low, driven by the fast mover.

Each number answers a different question. The planner for item C needs that line’s own error, a finance team valuing inventory looks at WMAPE, and the replenishment team needs to see that the biggest item is under-forecast.

At Which Level and Lag Should You Measure Forecast Accuracy?

Measure at the grain of the decision. Store replenishment orders are placed by SKU, store and day or week, so that is where accuracy matters for store stock. DC purchasing needs SKU and DC by week. Capacity planning can use product family by month. Errors cancel as they are added up, so the same forecast scores at least as well on WMAPE at chain-month level than at SKU-store-week level. A strong chain-level number does not mean the store orders were right. Channels behave the same way: a total can look accurate while store and online forecasts are biased in opposite directions, as our post on omnichannel demand forecasting explains.

Measure at the lag of the decision as well. If a supplier needs six weeks, the purchase order was based on the forecast made six weeks before the sales week. Compare actuals with that frozen forecast (lag 6), not with the forecast revised the week before, which no order could still use. A team can report excellent lag-1 accuracy while its buying runs on a lag-6 forecast that is much worse.

Two data rules prevent misleading scores. Use demand, not shipments, where you can, because shipments reflect what was available. Treat stock-out periods separately, because sales during a stock-out understate what customers wanted and make an under-forecast look accurate.

Forecast Value Added: Is Each Step Improving the Forecast?

Forecast value added is the change in an error metric that can be attributed to one step or participant in the forecasting process (Institute of Business Forecasting). Start with a naive forecast, such as last period’s actual or the same week last year. Then compare each step with the one before it:

StepWMAPE (lag 4)FVA vs previous step
Naive (same week last year)38%
Statistical model27%+11 points
Planner overrides25%+2 points
Sales consensus29%-4 points

In this illustration the model earns its place, planner overrides help a little, and the sales consensus makes the forecast worse. Run the same comparison by segment. Overrides on promoted items can behave differently from overrides on base demand, and the answer decides where planners should spend their time. Short-term signals are a step too: demand sensing should show positive FVA at the short lags it is meant to improve.

Why Forecast Accuracy Alone Does Not Decide Inventory

A forecast is worth what it changes in replenishment, purchasing and production plans. Two items with the same WMAPE can need very different stock.

A common safety stock formula is z x sigma x square root of (lead time + review period), where sigma is the standard deviation of forecast error per period and z is set by the service target. The formula assumes errors are independent from period to period and roughly normal, so treat it as a starting point. It still shows what accuracy leaves out:

  • Lead time. With the same error, an item reviewed weekly with a nine-week lead time needs about 2.2 times the safety stock of one with a one-week lead time (square root of 10 against square root of 2).
  • Service target. Moving from 95% to 99% cycle service raises z from 1.65 to 2.33, about 41% more safety stock for the same forecast.
  • Bias. Safety stock covers the spread of errors around the forecast. A forecast that runs 10% high builds excess no matter how the buffer is set, and one that runs low causes stock-outs that extra buffer only partly covers.
  • Cost. A missed sale on a high-margin item and a written-off unit of fresh product cost different amounts, so the right target differs. Our post on the inventory service level trade-off shows how to price each extra point of service.

So report accuracy and bias next to the inventory and service they produce. A one-point gain in WMAPE on items with short lead times and low margins may change little. The same gain on long-lead, high-value items can free working capital.

How Düzey Connected Forecasting to Inventory Decisions

Düzey, a wholesale FMCG distributor supplying more than 76,000 retail locations from 62 distribution centers and 9 regional warehouses, had excess stock in its regional warehouses while retailers ran out. Marketing, operations, distribution and finance were not planning together. With Solvoyo, safety and cycle stocks were calculated directly from SKU-level forecasts inside the inventory optimization algorithm, replenishment plans took business and operational constraints into account, and transaction data from regions, channels and product lines was combined into one source of truth.

According to the published Düzey case study, the combined process reduced overall inventory investment by 10%, lost sales by 10% and perishable waste by 15%, and improved forecast accuracy by 20%.

How Solvoyo Measures Forecasts

Solvoyo is a supply chain decision automation platform for retailers and manufacturers. Its demand planning module tracks WMAPE, MAPE and bias continuously for system forecasts and for planner overrides side by side, and measures forecast value added to show where overrides improve accuracy. Underperforming models are caught and replaced automatically. Forecasts feed inventory targets and replenishment on the same data model, so the error that is measured is the error the buffers are sized for.

Because each forecast is scored against the alternatives and the stock built on it reflects its measured error, the resulting orders hold up when planners check them: over 95% of Solvoyo’s plans execute untouched. To see accuracy, bias and FVA on your own data, request a demo.

Forecast accuracy and bias metrics in Solvoyo demand analytics

Frequently Asked Questions

What Are the Main Forecast Accuracy Metrics?

The main forecast accuracy metrics are MAPE (the average percentage error per line), WMAPE (total absolute error divided by total actual demand), bias (whether forecasts run above or below actuals) and forecast value added (whether a step in the process reduces error compared with the step before). Use WMAPE or MAPE for the size of errors and bias for their direction, and track both.

What Is the Difference Between MAPE and WMAPE?

MAPE averages the percentage error of each line, so slow-moving items with small actuals can dominate it, and it is undefined when actual demand is zero. WMAPE divides the sum of absolute errors by the sum of actuals, so each item counts in proportion to its volume. WMAPE is the better summary when the question is inventory value, and MAPE is useful item by item.

What Is Forecast Value Added (FVA)?

Forecast value added is the change in forecast error caused by one step in the forecasting process. Start from a naive forecast, such as the same week last year, then compare the statistical model, planner overrides and consensus each with the step before. A negative FVA means that step made the forecast less accurate and should be changed or dropped.


See It Run on Your Own Data

See how Solvoyo turns your supply chain plans into decisions that run on their own.

Solvoyo mascot