MODELING SPATIOTEMPORALITY FOR MULTIVARIATE TIME SERIES IN URBAN APPLICATIONS

MODELING SPATIOTEMPORALITY FOR MULTIVARIATE TIME SERIES IN URBAN APPLICATIONS

Abstract

With the rapid development of smart cities, a large amount of urban data is collected and stored. Most collected data can be formulated as multivariate time series (MTS) with geo-tagged information, such as public transportation, air quality, and weather. Spatialtemporality, defined from both spatial and temporal aspects, is the most important dynamics of urban MTS data. Understanding and analyzing the spatiotemporality of MTS data would benefit a wide range of real-world applications, thus contributing to the urbanization process. The majority of prior work focuses on data characteristics of MTS data from specific tasks (e.g., [103, 81, 102, 79, 106] ), and successfully model spatiotemporal dynamics in proposed models. Despite the initial success of existing methods, real-world MTS urban data has several unique characteristics that have not been fully addressed, which brings both challenges and opportunities for researches on modeling MTS. First, from thedata characteristics perspective, the distribution of real-world MTS data is affected by various hidden factors such as human daily activity. Understanding and modeling those hidden factors as prior knowledge can potentially enhance existing machine learning algorithms for specific applications. Second, from a data quality perspective, in most cases, collected MTS data are imperfect due to a lot of reasons (e.g., broken of sensors). Common low-quality issues include missing value, noisy sequential data, and anomaly samples, which challenges existing MTS models. Third, from the task correlation perspective, many tasks from urban scenarios are highly correlated. Exploiting and reusing knowledge shared in different tasks can benefit multi-tasks while how to efficiently exploit and reuse knowledge across multi-tasks for MTS data remains a challenging problem.

Therefore, in this thesis, we investigate the novel problem of tacking the three unique characteristics of spatiotemporal MTS urban data. In particular, we first will investigate how to leverage domain knowledge for spatiotemporal MTS modeling to explore the first characteristic. We then study the low data quality issue jointly with various tasks (e.g., prediction, classification) and improve the robustness of our framework. The proposed solutions contribute to a wide range of real-world applications, such as public transportation and air quality monitoring.

Chapter 1

Introduction

Multivariate time series (MTS) data are collected from a variety of urban scenarios, such as meteorological facilities [93, 97] , intelligent transportation systems [108, 100, 102] , air quality stations [12, 30] , and solar energy collectors [13] . A better understanding of MTS data benefits those applications, thus contributes to smart city development.

However, modeling MTS data is extremely challenging because of its complex spatiotemporal dependency. When MTS data meets urban applications, we categorize the challenges into three groups:

• Data Characteristic: Because most real-world MTS are collected from human’s daily life, the data distributions are correlated with many hidden factors in the city. Considering related hidden factors in existing models/frameworks would greatly benefit the understanding of data characteristics. For example, MTS of city traffic (e.g., speed, volume) of a certain block has strong periodicity, which can be explicitly captured to improve the forecasting of future traffic [102] . The characteristic of periodicity is actually influenced by urban daily lifewhich serves as the hidden factor that dominated the data distribution of traffic volume. In many cases, understanding the hidden factors that affect data distribution is difficult, and incorporating them into existing models requires specific domain knowledge. For example, the MTS of individual vehicles (i.e., GPS-coordinate, speed, acceleration, etc.) is affected by various constraints, such as road network structure and driving rules. Tang et al. sample MTS data of moving vehicles from a high-fidelity simulator so that the influence of numerous hidden factors are considered jointly. The simulator contains domain knowledge and helps quantities the affect of those hidden factors on MTS data distribution. We are interested in combine domain knowledge and expertise with MTS data for urban applications. Our goal is to find an appropriate knowledge rep-resentation so that existing models (e.g., recurrent neural networks) can leverage special data characteristics for real-world problems.

• Data Quality: The majority of the aforementioned research assumes MTS data are complete. In the real-world, MTS data are usually imperfect due to various reasons such as broken sensors, failed data transmission, or damaged storage. The abnormal cases, including missing values, noisy data, and outlier samples, are widely existing in real-world MTS data. The reduction of MTS data quality significantly increases the difficulty of data understanding. For example, missing values challenge existing models because they make it more difficult to model temporal dependencies and variable correlations [51, 12] ; outliers decrease the robustness of trained model [32] . Therefore, it is vital to design models that can handle the abnormal conditions in multivariate time series for urban tasks.
• Task Correlation: Due to the existing variable dependency in MTS data, many timeseries signals are highly correlated. Modeling the inner correlation between different tasks would benefit urban applications. For example, jointly modeling of air quality and meteorological signals can improve each other [103] . Different air pollution ingredients can transform because of chemical reactions (e.g., NO and NO₂) [22] ; some ingredients are produced jointly by the same human activity. When forecasting future air quality index, the causality between tasks defined on different air pollution ingredients, and on meteorological signals can be utilized for a better accuracy [104] .

Most prior work focus on spatial-temporal dynamics of MTS data in urban applications. Real-world MTS are usually collected with geo-spatial features. The spatial information provides similarity measurement on a new dimension. Utilizing the spatial similarity of MTS is a challenging task, because the spatial dependencies are affected by numerical factors. For example, Yao et al. introduce functional similarity of regions as the spatial correlation measurement for traffic volume pattern; Shi et al. introduce learnable location-variant structure that describes spatial similarity on the space; Wilson et al. directly optimize spatial adjacency of weather monitoring stations in a low-rank matrix factorization approach. On the other hand, variables of MTS data are evolving over time, and bring temporal dependency over different observations. Autoregressive Integrated Moving Average (ARIMA) models are widely used for time series analysis, especially in non-stationary cases [10] . Li et al., Lippi et al. apply ARIMA for traffic prediction. Recently, recurrent neural networks (RNN) have been very successful in a number of time series analysis problems. Chang et al. design an RNN framework based on memory network for MTS forecasting; Li et al. incorporate graph convolutional operator into RNN for citywide vehicle speed prediction.Most pioneer studies only focus on data characteristic (e.g., [47, 72, 108, 104] ), while few of them manage to solve the challenges from low data quality and urban task correlations.

To tackle the above challenges, our major research interests lie in the following directions:

• Low quality MTS data: we aim to build robust frameworks that handle the low-quality issue of urban MTS data from the following aspects: (1) modeling noisy distribution on MTS data, or on labels to improve existing methods; (2) exploring adversarial learning to improve the robustness of the frameworks.
• Utilizing task correlation: we aim to leverage the correlations between different tasks of urban applications. For example, we are interested in modeling the causality of air quality (e.g., NO₂) and meteorological records (e.g., wind speed) and process mutlitask forecasting of air pollution integrate in the future.

The main contributions of this thesis are:

• We formulate the research question of modeling the spatiotemporality for multivariate time series data in urban applications.
• We analyze and categorize the challenges of our research questions from three perspectives.
• We propose solutions to tackle the above challenges for various specific applications and in a general approach.

The rest of the thesis is organized as follows. We survey related work in Chapter 2. In Chapter 3, we show an application on urban traffic MTS that leverages the specific characteristic of periodicity. We further tackle the missing portions of specific MTS (i.e., vehicle trajectories) in Chapter 4. In Chapter 5, we describe an end-to-end framework to model the local and global temporal dependencies of incomplete MTS data.

MODELING SPATIOTEMPORALITY FOR MULTIVARIATE TIME SERIES IN URBAN APPLICATIONS