Tempus price prediction quality
Tempus predictions deliver improved demand response results, as they are tailor-made for this application. In this blog post we show the extent savings can be increased when using Tempus predictions - instead of AEMO - to make energy optimisation decisions.
Below, we summarise the increase - in financial value - of the results we can deliver through our technology. We will then describe the methodology used to evaluate this comparison with AEMO.
Measuring prediction accuracy
Successful demand flexibility requires reliable and accurate predictions for energy prices. However, typical methods of assessing a prediction’s accuracy, such as Mean Absolute Percentage Error, are not appropriate indicators of how useful the prediction will be for generating financial value through demand flexibility.
Application of predictions in demand response
Typical accuracy statistics fail to represent many important aspects:
How do we translate prediction accuracy into a financial gain? We make decisions based on cost benefit, not on error rates.
How successful were the predictions at ranking consecutive prices? Direction of price movement is more important than absolute price.
How effective were the predictions during extreme price periods? Avoiding this is an asymmetric risk.
What is the real world benefit of any increased accuracy? We expect a lower error rate to deliver improvements, but it’s unclear how much.
We overcome these common failings, by using a ‘Forecast Quality Model’ to measure the financial value we are able to extract. This model take a prediction series as input (along with actual prices) and calculates the savings delivered. Inputting AEMO’s predictions sets our baseline. Then, we input our predictions to measure the increased savings.
How the Tempus Forecast Quality Model works
Our Forecast Quality Model simulates a battery connected to the grid. The use of a theoretically perfect battery allows us to evaluate the prediction without considering the complexity of real world flexible assets.
At each ‘prediction step’, an optimal strategy is set - charging when prices are low and discharging when they are high.
The output of the forecast is how much profit the battery makes - allowing us to effectively compare different prediction methods.
Comparing AEMO with Tempus
In each run of the Tempus Forecast Quality Model we evaluate the financial potential using prices between June 2018 and June 2019.
Actual prices are those reported by AEMO, and we use AEMO predictions to set a baseline for the value which can be extracted. Below, we can see how the Forecast Quality Model has operated the battery in response to predicted prices, over a short time period.
We evaluate our model’s ability to predict over a 1 hour forecast horizon and over an 8 hour forecast horizon. Short term forecasts will always have a lower error rate, but long term forecasts allow for greater flexibility; provided they are reliable enough.
In this analysis we have presented a technique for:
Expressing the latent value in energy market volatility for demand flexibility
Evaluating the effectiveness of different price predictions in extracting that value
Using this Forecast Quality Model, we have demonstrated that Tempus’ predictions consistently outperform AEMO’s by a considerable margin, in most cases over 50% more value can be extracted.
While many companies provide predictions, Tempus is unique in its focus on delivering predictions which are most effective for demand response applications.
While the theoretical battery model provides a clean way to evaluate prediction quality, we invite you to give us information about your flexible asset and we can help you understand the available financial value using demand response.
The full results of comparing our model, across 5 regions and these 2 horizons can be found in the downloadable factsheet.