Tracking and modelling prices using web-scraped price microdata: towards automated daily consumer price index forecasting

TY - JOUR

T1 - Tracking and modelling prices using web-scraped price microdata

T2 - towards automated daily consumer price index forecasting

AU - Powell, Benedict James

AU - Nason, Guy

AU - Elliott, Duncan

AU - Mayhew, Matthew

AU - Davies, Jennifer

AU - Winton, Joe

N1 - © 2017 The Authors Journal of the Royal Statistical Society: Series A.

PY - 2018/5/24

Y1 - 2018/5/24

N2 - With the increasing relevance and availability of on-line prices that we see today, it is natural to ask whether the prediction of the consumer price index (CPI), or related statistics, may usefully be computed more frequently than existing monthly schedules allow for. The simple answer is ‘yes’, but there are challenges to be overcome first. A key challenge, addressed by our work, is that web-scraped price data are extremely messy and it is not obvious, a priori, how to reconcile them with standard CPI statistics. Our research focuses on average prices and disaggregated CPI at the level of product categories (lager, potatoes, etc.) and develops a new model that describes the joint time evolution of latent daily log-inflation rates driving prices seen on the Internet and prices recorded in official surveys, with the model adapting to various product categories. Our model reveals the differing levels of dynamic behaviour across product category and, correspondingly, differing levels of predictability. Our methodology enables good prediction of product-category-specific CPI immediately before their release. In due course, with increasingly complete web-scraped data, combined with the best survey data, the prospect of more frequent intermonth aggregated CPI prediction is an achievable goal.

AB - With the increasing relevance and availability of on-line prices that we see today, it is natural to ask whether the prediction of the consumer price index (CPI), or related statistics, may usefully be computed more frequently than existing monthly schedules allow for. The simple answer is ‘yes’, but there are challenges to be overcome first. A key challenge, addressed by our work, is that web-scraped price data are extremely messy and it is not obvious, a priori, how to reconcile them with standard CPI statistics. Our research focuses on average prices and disaggregated CPI at the level of product categories (lager, potatoes, etc.) and develops a new model that describes the joint time evolution of latent daily log-inflation rates driving prices seen on the Internet and prices recorded in official surveys, with the model adapting to various product categories. Our model reveals the differing levels of dynamic behaviour across product category and, correspondingly, differing levels of predictability. Our methodology enables good prediction of product-category-specific CPI immediately before their release. In due course, with increasingly complete web-scraped data, combined with the best survey data, the prospect of more frequent intermonth aggregated CPI prediction is an achievable goal.

KW - Dynamic inflation model

KW - High frequency inflation prediction

KW - Inflation estimation

KW - State space model

UR - http://www.scopus.com/inward/record.url?scp=85029434401&partnerID=8YFLogxK

U2 - 10.1111/rssa.12314

DO - 10.1111/rssa.12314

M3 - Article

SN - 0964-1998

VL - 181

SP - 737

EP - 756

JO - Journal of the Royal Statistical Society: Series A (Statistics in Society)

JF - Journal of the Royal Statistical Society: Series A (Statistics in Society)

IS - 3

ER -