请选择 进入手机版 | 继续访问电脑版
楼主: oliyiyi
1612 7

Time Series Forecasting Performance Measures With Python [推广有奖]

版主

泰斗

0%

还不是VIP/贵宾

-

TA的文库  其他...

计量文库

威望
7
论坛币
272091 个
通用积分
31269.1729
学术水平
1435 点
热心指数
1554 点
信用等级
1345 点
经验
383778 点
帖子
9599
精华
66
在线时间
5466 小时
注册时间
2007-5-21
最后登录
2024-3-21

初级学术勋章 初级热心勋章 初级信用勋章 中级信用勋章 中级学术勋章 中级热心勋章 高级热心勋章 高级学术勋章 高级信用勋章 特级热心勋章 特级学术勋章 特级信用勋章

oliyiyi 发表于 2017-2-13 15:57:54 |显示全部楼层 |坛友微信交流群

+2 论坛币
k人 参与回答

经管之家送您一份

应届毕业生专属福利!

求职就业群
赵安豆老师微信:zhaoandou666

经管之家联合CDA

送您一个全额奖学金名额~ !

感谢您参与论坛问题回答

经管之家送您两个论坛币!

+2 论坛币

本帖隐藏的内容

Time series prediction performance measures provide a summary of the skill and capability of the forecast model that made the predictions.

There are many different performance measures to choose from. It can be confusing to know which measure to use and how to interpret the results.

In this tutorial, you will discover performance measures for evaluating time series forecasts with Python.

Time series generally focus on the prediction of real values, called regression problems. Therefore the performance measures in this tutorial will focus on methods for evaluating real-valued predictions.

After completing this tutorial, you will know:

  • Basic measures of forecast performance, including residual forecast error and forecast bias.
  • Time series forecast error calculations that have the same units as the expected outcomes such as mean absolute error.
  • Widely used error calculations that punish large errors, such as mean squared error and root mean squared error.

Let’s get started.

[color=rgb(255, 255, 255) !important]

Time Series Forecasting Performance Measures With Python
Photo by Tom Hall, some rights reserved.

Forecast Error (or Residual Forecast Error)

The forecast error is calculated as the expected value minus the predicted value.

This is called the residual error of the prediction.

forecast_error = expected_value - predicted_value

The forecast error can be calculated for each prediction, providing a time series of forecast errors.

The example below demonstrates how the forecast error can be calculated for a series of 5 predictions compared to 5 expected values. The example was contrived for demonstration purposes.

expected = [0.0, 0.5, 0.0, 0.5, 0.0] predictions = [0.2, 0.4, 0.1, 0.6, 0.2] forecast_errors = [expected-predictions for i in range(len(expected))] print('Forecast Errors: %s' % forecast_errors)

Running the example calculates the forecast error for each of the 5 predictions. The list of forecast errors is then printed.

Forecast Errors: [-0.2, 0.09999999999999998, -0.1, -0.09999999999999998, -0.2]

The units of the forecast error are the same as the units of the prediction. A forecast error of zero indicates no error, or perfect skill for that forecast.

Mean Forecast Error (or Forecast Bias)

Mean forecast error is calculated as the average of the forecast error values.

mean_forecast_error = mean(forecast_error)

Forecast errors can be positive and negative. This means that when the average of these values is calculated, an ideal mean forecast error would be zero.

A mean forecast error value other than zero suggests a tendency of the model to over forecast (positive error) or under forecast (negative error). As such, the mean forecast error is also called the forecast bias.

The forecast error can be calculated directly as the mean of the forecast values. The example below demonstrates how the mean of the forecast errors can be calculated manually.

expected = [0.0, 0.5, 0.0, 0.5, 0.0] predictions = [0.2, 0.4, 0.1, 0.6, 0.2] forecast_errors = [expected-predictions for i in range(len(expected))] bias = sum(forecast_errors) * 1.0/len(expected) print('Bias: %f' % bias)

Running the example prints the mean forecast error, also known as the forecast bias.

Bias: -0.100000

The units of the forecast bias are the same as the units of the predictions. A forecast bias of zero, or a very small number near zero, shows an unbiased model.

Mean Absolute Error

The mean absolute error, or MAE, is calculated as the average of the forecast error values, where all of the forecast values are forced to be positive.

Forcing values to be positive is called making them absolute. This is signified by the absolute function abs() or shown mathematically as two pipe characters around the value: |value|.

mean_absolute_error = mean( abs(forecast_error) )

Where abs() makes values positive, forecast_error is one or a sequence of forecast errors, and mean() calculates the average value.

We can use the mean_absolute_error() function from the scikit-learn library to calculate the mean absolute error for a list of predictions. The example below demonstrates this function.

from sklearn.metrics import mean_absolute_error expected = [0.0, 0.5, 0.0, 0.5, 0.0] predictions = [0.2, 0.4, 0.1, 0.6, 0.2] mae = mean_absolute_error(expected, predictions) print('MAE: %f' % mae)

Running the example calculates and prints the mean absolute error for a list of 5 expected and predicted values.

MAE: 0.140000

These error values are in the original units of the predicted values. A mean absolute error of zero indicates no error.

Mean Squared Error

The mean squared error, or MSE, is calculated as the average of the squared forecast error values. Squaring the forecast error values forces them to be positive; it also has the effect of putting more weight on large errors.

Very large or outlier forecast errors are squared, which in turn has the effect of dragging the mean of the squared forecast errors out resulting in a larger mean squared error score. In effect, the score gives worse performance to those models that make large wrong forecasts.

mean_squared_error = mean(forecast_error^2)

We can use the mean_squared_error() function from scikit-learn to calculate the mean squared error for a list of predictions. The example below demonstrates this function.

from sklearn.metrics import mean_squared_error expected = [0.0, 0.5, 0.0, 0.5, 0.0] predictions = [0.2, 0.4, 0.1, 0.6, 0.2] mse = mean_squared_error(expected, predictions) print('MSE: %f' % mse)

Running the example calculates and prints the mean squared error for a list of expected and predicted values.

MSE: 0.022000

The error values are in squared units of the predicted values. A mean squared error of zero indicates perfect skill, or no error.

Root Mean Squared Error

The mean squared error described above is in the squared units of the predictions.

It can be transformed back into the original units of the predictions by taking the square root of the mean squared error score. This is called the root mean squared error, or RMSE.

rmse = sqrt(mean_squared_error)

This can be calculated by using the sqrt() math function on the mean squared error calculated using the mean_squared_error() scikit-learn function.

from sklearn.metrics import mean_squared_error from math import sqrt expected = [0.0, 0.5, 0.0, 0.5, 0.0] predictions = [0.2, 0.4, 0.1, 0.6, 0.2] mse = mean_squared_error(expected, predictions) rmse = sqrt(mse) print('RMSE: %f' % rmse)

Running the example calculates the root mean squared error.

RMSE: 0.148324

The RMES error values are in the same units as the predictions. As with the mean squared error, an RMSE of zero indicates no error.

Further Reading

Below are some references for further reading on time series forecast error measures.

Summary

In this tutorial, you discovered a suite of 5 standard time series performance measures in Python.

Specifically, you learned:

  • How to calculate forecast residual error and how to estimate the bias in a list of forecasts.
  • How to calculate mean absolute forecast error to describe error in the same units as the predictions.
  • How to calculate the widely used mean squared error and root mean squared error for forecasts.



二维码

扫码加我 拉你入群

请注明:姓名-公司-职位

以便审核进群资格,未注明则拒绝

关键词:performance Forecasting Time Series Performan measures Series

缺少币币的网友请访问有奖回帖集合
https://bbs.pinggu.org/thread-3990750-1-1.html
w-long 发表于 2017-2-13 16:03:49 |显示全部楼层 |坛友微信交流群
thanks
已有 1 人评分论坛币 收起 理由
oliyiyi + 10 沙发

总评分: 论坛币 + 10   查看全部评分

使用道具

auirzxp 学生认证  发表于 2017-2-13 16:14:07 |显示全部楼层 |坛友微信交流群

使用道具

ekscheng 发表于 2017-2-13 17:38:37 |显示全部楼层 |坛友微信交流群

使用道具

谢谢分享
已有 1 人评分论坛币 收起 理由
oliyiyi + 5 精彩帖子

总评分: 论坛币 + 5   查看全部评分

使用道具

phipe 发表于 2017-2-27 21:08:42 |显示全部楼层 |坛友微信交流群
谢谢分享
已有 1 人评分论坛币 收起 理由
oliyiyi + 5 精彩帖子

总评分: 论坛币 + 5   查看全部评分

使用道具

whdxsn123 发表于 2017-3-11 12:17:12 |显示全部楼层 |坛友微信交流群

使用道具

谢谢楼主

使用道具

您需要登录后才可以回帖 登录 | 我要注册

本版微信群
加好友,备注jltj
拉您入交流群

京ICP备16021002-2号 京B2-20170662号 京公网安备 11010802022788号 论坛法律顾问:王进律师 知识产权保护声明   免责及隐私声明

GMT+8, 2024-3-28 23:55