Forecasting power prices: combining fundamentals and machine learning

The evolution of power price forecasting

Electricity price forecasting has rapidly evolved over the past two decades. Early models depended on econometric methods like regression analysis, using fuel prices and demand as explanatory variables. As markets liberalised and data volumes increased, statistical methods were replaced by more advanced machine learning techniques, from gradient boosting to neural networks.

Although these models are impressive, many fail to represent the physical logic of electricity systems accurately, specifically the interaction of supply, demand, and network constraints that influence real prices. This is why the industry is transitioning towards hybrid modelling, where the interpretability of fundamental models is combined with the pattern recognition power of machine learning.

Fundamental vs. statistical models: strengths and limits

Fundamental and statistical models approach the same problem from opposite directions. Fundamental models are structural. They simulate the physical and economic mechanisms that determine prices, using data such as generation capacity, demand forecasts, and interconnector flows. They offer transparency and scenario flexibility, allowing analysts to test policy or fuel price assumptions.

Machine learning models, by contrast, learn directly from the data. Algorithms such as random forests, XGBoost (Extreme Gradient Boosting), or LSTMs (Long Short-Term Memory networks) identify complex, non-linear relationships that humans might overlook. Their strength lies in pattern recognition and their ability to adapt to new data streams.

However, pure machine learning approaches can suffer from overfitting, i.e. where models capture noise rather than genuine signals. They may perform well in-sample but lose accuracy in real-world conditions. Equally, they can lack interpretability, making it difficult to understand why a forecast moves. This has led to growing interest in hybrid frameworks that harness both structural realism and statistical learning.

How hybrid models combine physical and data-driven insight

Hybrid power price forecasting models connect economics and artificial intelligence. The aim is not to replace fundamentals with algorithms, but to utilise each where it is most effective.

Visit our product update page to find out more about the Ensemble forecast from Montel EnAppSys

One common approach is to embed machine learning layers within a fundamental model. Here, the fundamental structure provides the market logic, i.e. supply, demand, and dispatch. At the same time, machine learning corrects residual errors, capturing non-linear effects such as renewables forecast bias or unexpected generator behaviour.

Another technique uses the residuals from a fundamental forecast, which is the difference between simulated and actual prices, as the training data for an ML “refiner”. This secondary model learns systematic deviations and adjusts future forecasts accordingly.

Hybrid frameworks often deliver superior results because they combine:

• Causal structure: grounded in physical and economic relationships

• Pattern learning: capturing dynamic, data-driven trends

• Adaptive calibration: continuously improving with new data

The difficulty is in aligning the two layers so that the machine learning part improves and not distorts, the basic market logic.

Building, training, and validating ML models

Designing the machine learning element of a hybrid forecast involves three core stages: data preparation, model training, and validation.

Data preprocessing and feature engineering

Feature engineering is the most crucial factor for model performance. Inputs usually comprise weather conditions, outages during generation, prices for carbon and fuel, interconnection flow data, and demand forecasts. Techniques like creating lag features, normalising data, and adjusting for seasonality help the data capture the temporal dynamics of power markets.

A strong feature set balances breadth and relevance, capturing the main drivers without overwhelming the model with noise.

Cross-validation and performance metrics

Validation is vital to prevent overfitting. Cross-validation methods like time-series splits help evaluate robustness across various market periods. Model performance is then measured using error metrics including Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Mean Absolute Percentage Error (MAPE).

Although lower error scores point to higher accuracy, analysts should also assess performance stability. A model that performs consistently across different conditions is often more useful than one that only excels in stable markets.

Explainability and model interpretation

Modern forecasting teams are more and more emphasising interpretability. Methods like SHAP values (Shapley Additive Explanations) and partial dependence plots help identify which variables most affect predictions, offering greater transparency for risk teams and decision-makers.

Explainable models foster trust, especially when forecasts directly inform trading or investment strategies.

Best practices for integrating forecasts into trading strategies

The main aim of price forecasting is to enhance trading and hedging results. Forecast accuracy is only valuable when it leads to better decisions. Effective incorporation of forecasts into trading processes relies on three key principles.

• Alignment with strategy: Forecasts should support the trading horizon, i.e. intraday, day-ahead, or long-term, with outputs tailored to that timeframe.

• Quantified uncertainty: Instead of a single point forecast, models should offer probabilistic outputs or confidence intervals to aid risk-adjusted decision-making.

• Continuous feedback: Forecasts must be monitored against realised outcomes, feeding back into model retraining and performance review.

Traders are increasingly evaluating forecast skill based on market value, not only whether a forecast is statistically accurate, but also whether it consistently produces positive profit and loss (P&L) outcomes after accounting for costs and spreads.

Conclusion

The future of power price forecasting lies in the intelligent fusion of physics, data, and human expertise. Fundamental models explain why the market behaves as it does; machine learning identifies how those behaviours evolve. Together, they provide a balanced framework that is both accurate and interpretable.

In a landscape where volatility, renewable energy, and policy shifts continually transform power markets, the most effective forecasters are those who combine economics, data science, and intuition. This approach yields not only price predictions but also insights into potential opportunities.