请选择 进入手机版 | 继续访问电脑版
楼主: oliyiyi
3396 24

Time Series Forecast Case Study with Python [推广有奖]

版主

泰斗

0%

还不是VIP/贵宾

-

TA的文库  其他...

计量文库

威望
7
论坛币
272091 个
通用积分
31269.1753
学术水平
1435 点
热心指数
1554 点
信用等级
1345 点
经验
383778 点
帖子
9599
精华
66
在线时间
5466 小时
注册时间
2007-5-21
最后登录
2024-3-21

初级学术勋章 初级热心勋章 初级信用勋章 中级信用勋章 中级学术勋章 中级热心勋章 高级热心勋章 高级学术勋章 高级信用勋章 特级热心勋章 特级学术勋章 特级信用勋章

oliyiyi 发表于 2017-2-13 10:31:36 |显示全部楼层 |坛友微信交流群

+2 论坛币
k人 参与回答

经管之家送您一份

应届毕业生专属福利!

求职就业群
赵安豆老师微信:zhaoandou666

经管之家联合CDA

送您一个全额奖学金名额~ !

感谢您参与论坛问题回答

经管之家送您两个论坛币!

+2 论坛币

本帖隐藏的内容

Time series forecasting is a process, and the only way to get good forecasts is to practice this process.

In this tutorial, you will discover how to forecast the number of monthly armed robberies in Boston with Python.

Working through this tutorial will provide you with a framework for the steps and the tools for working through your own time series forecasting problems.

After completing this tutorial, you will know:

  • How to check your Python environment and carefully define a time series forecasting problem.
  • How to create a test harness for evaluating models, develop a baseline forecast, and better understand your problem with the tools of time series analysis.
  • How to develop an autoregressive integrated moving average model, save it to file, and later load it to make predictions for new time steps.

Let’s get started.

[color=rgb(255, 255, 255) !important]

Time Series Forecast Case Study with Python – Monthly Armed Robberies in Boston
Photo by Tim Sackton, some rights reserved.

Overview

In this tutorial, we will work through a time series forecasting project from end-to-end, from downloading the dataset and defining the problem to training a final model and making predictions.

This project is not exhaustive, but shows how you can get good results quickly by working through a time series forecasting problem systematically.

The steps of this project that we will work through are as follows:

  • Environment.
  • Problem Description.
  • Test Harness.
  • Persistence.
  • Data Analysis.
  • ARIMA Models
  • Model Validation

This will provide a template for working through a time series prediction problem that you can use on your own dataset.

1. Environment

This tutorial assumes an installed and working SciPy environment and dependencies, including:

  • SciPy
  • NumPy
  • Matplotlib
  • Pandas
  • scikit-learn
  • statsmodels

I used Python 2.7. Are you on Python 3? let me know how you go in the comments.

This script will help you check your installed versions of these libraries.

# scipy import scipy print('scipy: {}'.format(scipy.__version__)) # numpy import numpy print('numpy: {}'.format(numpy.__version__)) # matplotlib import matplotlib print('matplotlib: {}'.format(matplotlib.__version__)) # pandas import pandas print('pandas: {}'.format(pandas.__version__)) # scikit-learn import sklearn print('sklearn: {}'.format(sklearn.__version__)) # statsmodels import statsmodels print('statsmodels: {}'.format(statsmodels.__version__))

The results on my workstation used to write this tutorial are as follows:

scipy: 0.18.1 numpy: 1.11.2 matplotlib: 1.5.3 pandas: 0.19.1 sklearn: 0.18.1 statsmodels: 0.6.1
2. Problem Description

The problem is to predict the number of monthly armed robberies in Boston, USA.

The dataset provides the number of monthly armed robberies in Boston from January 1966 to October 1975, or just under 10 years of data.

The values are a count and there are 118 observations.

The dataset is credited to McCleary & Hay (1980).

You can learn more about this dataset and download it directly from DataMarket.

Download the dataset as a CSV file and place it in your current working directory with the filename “robberies.csv“.

3. Test Harness

We must develop a test harness to investigate the data and evaluate candidate models.

This involves two steps:

  • Defining a Validation Dataset.
  • Developing a Method for Model Evaluation.
3.1 Validation Dataset

The dataset is not current. This means that we cannot easily collect updated data to validate the model.

Therefore we will pretend that it is October 1974 and withhold the last one year of data from analysis and model selection.

This final year of data will be used to validate the final model.

The code below will load the dataset as a Pandas Series and split into two, one for model development (dataset.csv) and the other for validation (validation.csv).

from pandas import Series series = Series.from_csv('robberies.csv', header=0) split_point = len(series) - 12 dataset, validation = series[0:split_point], series[split_point:] print('Dataset %d, Validation %d' % (len(dataset), len(validation))) dataset.to_csv('dataset.csv') validation.to_csv('validation.csv')

Running the example creates two files and prints the number of observations in each.

Dataset 106, Validation 12

The specific contents of these files are:

  • dataset.csv: Observations from January 1966 to October 1974 (106 observations)
  • validation.csv: Observations from November 1974 to October 1975 (12 observations)

The validation dataset is 10% of the original dataset.

Note that the saved datasets do not have a header line, therefore we do not need to cater to this when working with these files later.

3.2. Model Evaluation

Model evaluation will only be performed on the data in dataset.csv prepared in the previous section.

Model evaluation involves two elements:

  • Performance Measure.
  • Test Strategy.
3.2.1 Performance Measure

The observations are a count of robberies.

We will evaluate the performance of predictions using the root mean squared error (RMSE). This will give more weight to predictions that are grossly wrong and will have the same units as the original data.

Any transforms to the data must be reversed before the RMSE is calculated and reported to make the performance between different methods directly comparable.

We can calculate the RMSE using the helper function from the scikit-learn library mean_squared_error() that calculates the mean squared error between a list of expected values (the test set) and the list of predictions. We can then take the square root of this value to give us an RMSE score.

For example:

from sklearn.metrics import mean_squared_error from math import sqrt ... test = ... predictions = ... mse = mean_squared_error(test, predictions) rmse = sqrt(mse) print('RMSE: %.3f' % rmse)
3.2.2 Test Strategy

Candidate models will be evaluated using walk-forward validation.

This is because a rolling-forecast type model is required from the problem definition. This is where one-step forecasts are needed given all available data.

The walk-forward validation will work as follows:

  • The first 50% of the dataset will be held back to train the model.
  • The remaining 50% of the dataset will be iterated and test the model.
  • For each step in the test dataset:
    • A model will be trained.
    • A one-step prediction made and the prediction stored for later evaluation.
    • The actual observation from the test dataset will be added to the training dataset for the next iteration.
  • The predictions made during the iteration of the test dataset will be evaluated and an RMSE score reported.

Given the small size of the data, we will allow a model to be re-trained given all available data prior to each prediction.

We can write the code for the test harness using simple NumPy and Python code.

Firstly, we can split the dataset into train and test sets directly. We’re careful to always convert a loaded dataset to float32 in case the loaded data still has some String or Integer data types.

# prepare data X = series.values X = X.astype('float32') train_size = int(len(X) * 0.50) train, test = X[0:train_size], X[train_size:]

Next, we can iterate over the time steps in the test dataset. The train dataset is stored in a Python list as we need to easily append a new observation each iteration and Numpy array concatenation feels like overkill.

The prediction made by the model is called yhat for convention, as the outcome or observation is referred to as y and yhat (a ‘y‘ with a mark above) is the mathematical notation for the prediction of the y variable.

The prediction and observation are printed each observation for a sanity check prediction in case there are issues with the model.

# walk-forward validation history = [x for x in train] predictions = list() for i in range(len(test)):         # predict         yhat = ...         predictions.append(yhat)         # observation         obs = test         history.append(obs)         print('>Predicted=%.3f, Expected=%3.f' % (yhat, obs))



二维码

扫码加我 拉你入群

请注明:姓名-公司-职位

以便审核进群资格,未注明则拒绝

关键词:Time Series Case study Forecast forecas python Series

缺少币币的网友请访问有奖回帖集合
https://bbs.pinggu.org/thread-3990750-1-1.html
nicacc 在职认证  发表于 2017-2-13 10:32:43 |显示全部楼层 |坛友微信交流群
thank you
已有 1 人评分论坛币 收起 理由
oliyiyi + 10 精彩帖子

总评分: 论坛币 + 10   查看全部评分

使用道具

fengyg 企业认证  发表于 2017-2-13 10:41:00 |显示全部楼层 |坛友微信交流群
kankan
已有 1 人评分论坛币 收起 理由
oliyiyi + 5 精彩帖子

总评分: 论坛币 + 5   查看全部评分

使用道具

astar55 发表于 2017-2-13 10:43:34 |显示全部楼层 |坛友微信交流群
谢谢分享!
已有 1 人评分论坛币 收起 理由
oliyiyi + 5 精彩帖子

总评分: 论坛币 + 5   查看全部评分

使用道具

yazxf 发表于 2017-2-13 13:10:13 |显示全部楼层 |坛友微信交流群
谢谢你的书!
已有 1 人评分论坛币 收起 理由
oliyiyi + 5 精彩帖子

总评分: 论坛币 + 5   查看全部评分

使用道具

ekscheng 发表于 2017-2-13 13:48:57 |显示全部楼层 |坛友微信交流群

使用道具

mencius 发表于 2017-2-13 14:45:56 |显示全部楼层 |坛友微信交流群
thanks.
已有 1 人评分论坛币 收起 理由
oliyiyi + 5 精彩帖子

总评分: 论坛币 + 5   查看全部评分

使用道具

franky_sas 发表于 2017-2-13 14:48:27 |显示全部楼层 |坛友微信交流群
已有 1 人评分经验 收起 理由
unparalleled + 20 精彩帖子

总评分: 经验 + 20   查看全部评分

使用道具

kkkm_db 发表于 2017-2-13 15:01:45 |显示全部楼层 |坛友微信交流群
谢谢分享!
已有 1 人评分论坛币 收起 理由
oliyiyi + 5 精彩帖子

总评分: 论坛币 + 5   查看全部评分

使用道具

huaxuqiao 发表于 2017-2-13 16:05:25 |显示全部楼层 |坛友微信交流群
关于python时序的书不多啊
已有 1 人评分论坛币 收起 理由
oliyiyi + 5 精彩帖子

总评分: 论坛币 + 5   查看全部评分

使用道具

您需要登录后才可以回帖 登录 | 我要注册

本版微信群
加好友,备注jltj
拉您入交流群

京ICP备16021002-2号 京B2-20170662号 京公网安备 11010802022788号 论坛法律顾问:王进律师 知识产权保护声明   免责及隐私声明

GMT+8, 2024-3-29 19:37