Towards Calibrated, Sharp, and Interpretable Probabilistic Prediction

Predicting the future has always been a fundamental aspiration of humankind. In the era of big data, the sheer volume, diversity, and scale of data pose significant challenges in effective and efficient learning and inference. Moreover, data-driven approaches, particularly deep learning models, often lack interpretability and struggle with uncertainty quantification.

This dissertation aims to enhance calibration, sharpness, and interpretability in probabilistic forecasting, with a focus on solar flare prediction and active region evolution. We develop methods that fuse historical and multimodal data, constructing physically meaningful features, and unlock the potential of deep neural networks in solar forecasting. We also apply modern techniques to estimate multi-way and high-dimensional covariances and use them to predict the evolution of spatio-temporal processes in solar active regions. For post-processing, we develop statistical theories and efficient algorithms for recalibration in binary and multiclass settings with potential distribution shifts, and apply them in solar flare prediction influenced by solar cycles. Additionally, we develop attribution tools to help astrophysicists identify and interpret solar flare precursors discovered by convolutional neural networks.
These advances provide new insights into the statistical theory behind probabilistic prediction and offer practical methods and techniques for handling large and complex datasets.

Chair: Professor Alfred Hero