Now showing 1 - 4 of 4
  • Publication
    Knowledge-based Deep Learning for Modeling Chaotic Systems
    (2022)
    Elabid, Zakaria
    ;
    ;
    Deep Learning has received increased attention due to its unbeatable success in many fields, such as computer vision, natural language processing, recommendation systems, and most recently in simulating multiphysics problems and predicting nonlinear dynamical systems. However, modeling and forecasting the dynamics of chaotic systems remains an open research problem since training deep learning models requires big data, which is not always available in many cases. Such deep learners can be trained from additional information obtained from simulated results and by enforcing the physical laws of the chaotic systems. This paper considers extreme events and their dynamics and proposes elegant models based on deep neural networks, called knowledge-based deep learning (KDL). Our proposed KDL can learn the complex patterns governing chaotic systems by jointly training on real and simulated data directly from the dynamics and their differential equations. This knowledge is transferred to model and forecast real-world chaotic events exhibiting extreme behavior. We validate the efficiency of our model by assessing it on three real-world benchmark datasets: El Niño sea surface temperature, San Juan Dengue viral infection, and Bjørnøya daily precipitation, all governed by extreme events' dynamics. Using prior knowledge of extreme events and physics-based loss functions to lead the neural network learning, we ensure physically consistent, generalizable, and accurate forecasting, even in a small data regime. Index Terms-Chaotic systems, long short-term memory, deep learning, extreme event modeling.
      26  2
  • Publication
    Probabilistic AutoRegressive Neural Networks for Accurate Long-Range Forecasting
    (2023)
    Panja, Madhurima
    ;
    ;
    Kumar, Uttam
    ;
    Forecasting time series data is a critical area of research with applications spanning from stock prices to early epidemic prediction. While numerous statistical and machine learning methods have been proposed, real-life prediction problems often require hybrid solutions that bridge classical forecasting approaches and modern neural network models. In this study, we introduce a Probabilistic AutoRegressive Neural Network (PARNN), capable of handling complex time series data exhibiting non-stationarity, nonlinearity, non-seasonality, long-range dependence, and chaotic patterns. PARNN is constructed by improving autoregressive neural networks (ARNN) using autoregressive integrated moving average (ARIMA) feedback error. Notably, the PARNN model provides uncertainty quantification through prediction intervals and conformal predictions setting it apart from advanced deep learning tools. Through comprehensive computational experiments, we evaluate the performance of PARNN against standard statistical, machine learning, and deep learning models. Diverse real-world datasets from macroeconomics, tourism, epidemiology, and other domains are employed for short-term, medium-term, and long-term forecasting evaluations. Our results demonstrate the superiority of PARNN across various forecast horizons, surpassing the state-of-the-art forecasters. The proposed PARNN model offers a valuable hybrid solution for accurate long-range forecasting. The ability to quantify uncertainty through prediction intervals further enhances the model’s usefulness in various decision-making processes.
      10
  • Publication
    Van der Pol-informed Neural Networks for Multi-step-ahead Forecasting of Extreme Climatic Events
    (2023)
    Dutta, Anurag
    ;
    Panja, Madhurima
    ;
    Kumar, Uttam
    ;
    Hens, Chittaranjan
    ;
    Deep learning has produced excellent results in several applied domains including computer vision, natural language processing, speech recognition, etc. Physics-informed neural networks (PINN) are a new family of deep learning models that combine prior knowledge of physics in the form of high-level abstraction of natural phenomena with data-driven neural networks. PINN has emerged as a flourishing area of scientific computing to deal with the challenges of shortage of training data, enhancing physical plausibility, and specifically aiming to solve complex differential equations. However, building PINNs for modeling and forecasting the dynamics of extreme climatic events of geophysical systems remains an open scientific problem. This study proposes Van der Pol-informed Neural Networks (VPINN), a physics-informed differential learning approach, for modeling extreme nonlinear dynamical systems such as climatic events, exploiting the physical differentials as the physics-derived loss function. Our proposal is compared to state-of-the-art time series forecasting models, showing superior performance.The codes and dataset used for the experiments are made available at https: //github.com/mad-stat/VPINN.
      38  1
  • Publication
    W-Transformers : A Wavelet-based Transformer Framework for Univariate Time Series Forecasting
    Deep learning utilizing transformers has recently achieved a lot of success in many vital areas such as natural language processing, computer vision, anomaly detection, and recommendation systems, among many others. Among several merits of transformers, the ability to capture long-range temporal dependencies and interactions is desirable for time series forecasting, leading to its progress in various time series applications. In this paper, we build a transformer model for non-stationary time series. The problem is challenging yet crucially important. We present a novel framework for univariate time series representation learning based on the wavelet-based transformer encoder architecture and call it W-Transformer. The proposed W-Transformers utilize a maximal overlap discrete wavelet transformation (MODWT) to the time series data and build local transformers on the decomposed datasets to vividly capture the nonstationarity and long-range nonlinear dependencies in the time series. Evaluating our framework on several publicly available benchmark time series datasets from various domains and with diverse characteristics, we demonstrate that it performs, on average, significantly better than the baseline forecasters for short-term and long-term forecasting, even for datasets that consist of only a few hundred training samples.
      45  58