When evaluating a load forecasting model, the first metric most people reach for is mean absolute percentage error (MAPE). It is intuitive: on average, how far off is the forecast in percentage terms? A 2% MAPE sounds good. A 5% MAPE sounds acceptable. A 10% MAPE sounds like a problem.
This framing is not wrong, but it is incomplete in ways that matter for dispatch decisions. MAPE treats all errors equally, regardless of when they occur or what the absolute magnitude of the load is at that moment. For dispatch optimization, the errors that kill you are not the median errors. They are the errors at peak events, at the start of a high-price interval, or during periods where the battery SoC is at a critical threshold. A model with 2% average MAPE can still produce operationally damaging forecasts if its error is systematically concentrated in high-stakes intervals.
The MAPE Denominator Problem
MAPE is computed as the mean of absolute percentage errors, where each interval's error is divided by the actual load in that interval. This normalization makes MAPE sensitive to periods of very low load. If a feeder drops to near-zero load at 3 AM on a holiday weekend, even a small absolute error in the forecast produces a large percentage error. Conversely, a large absolute error at peak load, say 50 MW of under-forecast on a summer afternoon, produces a smaller percentage error simply because the denominator is large.
From a dispatch standpoint, what matters is the absolute error at the intervals where dispatch decisions are being made. A 50 MW absolute error at peak is catastrophic for a distribution operator managing voltage stability and capacity allocation. A 5 MW error at 3 AM is almost certainly irrelevant. MAPE's equal weighting of all intervals obscures this operational asymmetry.
This is why we report multiple metrics in production, not just MAPE. Root mean squared error (RMSE) penalizes large errors more heavily, which makes it more sensitive to the tails of the error distribution where operationally damaging events live. Mean absolute error (MAE) in absolute MW terms, stratified by time-of-day and season, tells operators where their forecast accuracy is weakest in the intervals that matter most.
Bias vs. Variance: Different Problems, Different Fixes
Forecast error has two components: bias (systematic tendency to over- or under-forecast) and variance (random error around the true value). For dispatch optimization, these manifest very differently.
A biased forecast causes systematic misalignment between scheduled and actual position. A model that consistently under-forecasts peak load by 8% will cause the dispatch optimizer to position batteries for a smaller ramp than actually materializes. The operator shows up short in high-price intervals day after day. The settlement imbalance is predictable and cumulative.
High-variance forecast errors are less predictable but can be equally costly. A model that sometimes over-forecasts by 15% and sometimes under-forecasts by 15%, with no systematic direction, will cause the optimizer to be over-prepared on some days and under-prepared on others. The net settlement position may be closer to neutral than a biased model, but the variance in daily P&L is higher and the operator has less confidence in using the forecast as a basis for committing capacity in advance.
Detecting bias requires tracking signed error (forecast minus actual) over time. If the mean signed error is consistently negative in afternoon peak hours, you have afternoon-peak downward bias. This is fixable with a systematic correction applied to that time-of-day bucket, which is a much simpler fix than retraining the full model. We apply these interval-level bias corrections in production using a rolling window of recent actuals, which catches drifting bias as load patterns shift across seasons.
Event-Specific Accuracy: What Matters for Batteries
For battery dispatch optimization specifically, the forecast intervals that drive the most value are not distributed uniformly. Batteries generate revenue through arbitrage, which depends on the spread between low-price charging intervals and high-price discharge intervals. The accuracy of the forecast in predicting when and how large those price-driving load events will be is disproportionately important.
Peak event forecast skill is best evaluated with metrics that focus on the distribution tail. For a feeder serving a commercial district, the top 5% of load intervals by magnitude probably drives 30 to 50% of the total battery revenue opportunity. If the model systematically under-forecasts those high-load intervals, it is degrading the most valuable decision inputs.
One practical test we use: take the top 100 load-magnitude intervals in a test year and compute the model's MAPE across just those intervals. Compare that to the overall MAPE. If the model's accuracy is substantially worse on high-load events than on average intervals, that is a flag worth addressing before putting the model into production dispatch. The opposite is also possible: some models specifically trained to capture peak events are better in the tail and worse at average intervals, which is actually acceptable for dispatch-focused applications.
Forecast Horizons and Refresh Cadence
A load forecast is not a single number. It is a time-series output across a planning horizon, typically 24 to 48 hours, and it needs to be refreshed continuously as new information arrives.
Accuracy degrades as forecast horizon extends. A 15-minute-ahead forecast is more accurate than a 4-hour-ahead forecast, which is more accurate than a 24-hour-ahead forecast. This is expected and cannot be fully overcome with better modeling: there is irreducible uncertainty in future weather and load behavior. What matters operationally is understanding where the accuracy cliff is for each application.
For day-ahead market bidding, the relevant forecast horizon is approximately 18 to 36 hours. Accuracy at this horizon determines day-ahead bid quality and the operator's position going into the trading day. For real-time dispatch adjustments, the relevant horizon is 1 to 4 hours, where forecasts are substantially more accurate and can be used to make fine adjustments to the dispatch schedule.
The refresh cadence matters as much as the base accuracy. A forecast that is run once in the morning and held static until the next day misses the signal updates that become available as the day progresses. Weather models update every few hours. Intraday load actuals allow real-time bias correction. Each new data point improves the rolling forecast for the remaining hours of the day. We run forecast updates every 15 minutes, feeding the current actuals and the latest short-term weather into the model to produce a continuously refreshed forward view.
Calibration: Does the Uncertainty Interval Mean What It Says?
Point forecasts alone are insufficient for dispatch optimization. The optimizer needs to know not just what load is expected but how uncertain that expectation is. Probabilistic forecasts expressed as quantile ranges (P10, P50, P90) give the optimizer this information and allow it to make decisions that reflect forecast uncertainty appropriately.
But quantile forecasts are only useful if they are calibrated. A calibrated P90 forecast should be exceeded by actual values only about 10% of the time. If the actual load exceeds the P90 band 30% of the time, the P90 is not a useful measure of risk. The optimizer will allocate insufficient reserve capacity based on falsely narrow uncertainty bounds.
Calibration checking is a standard step in our model validation process. We plot reliability diagrams that show the fraction of actuals falling within each stated quantile interval. Systematic miscalibration in either direction, too narrow or too wide, indicates a calibration correction is needed. This is typically applied as a post-processing step after the base model produces raw quantile estimates.
The practical question is not whether the model is perfect, because no load forecasting model is. The question is whether the errors it makes are the kind the dispatch optimizer can account for, whether the uncertainty estimates are honest, and whether the systematic biases are known and corrected. A forecast that meets those criteria is good enough to dispatch on, even if the headline MAPE number looks unimpressive on a benchmark comparison.