Gaussian Process Regression

Author: CislunarSpace
Site: https://cislunarspace.cn

Definition

Gaussian Process Regression (GPR) is a non-parametric machine learning method based on the Bayesian framework, inferring function values and their uncertainties at unknown points by assuming function values follow a Gaussian process. GPR provides both prediction mean and confidence intervals, particularly suitable for small-sample, high-dimensional scenarios requiring uncertainty estimation.

Basic Principles

Gaussian Process Definition

Function $f(\mathbf{x})$ follows a Gaussian process:

f(\mathbf{x}) \sim \mathcal{GP}(m(\mathbf{x}), k(\mathbf{x}, \mathbf{x}'))

Where $m(\mathbf{x})$ is the mean function, $k(\mathbf{x}, \mathbf{x}')$ is the covariance function (kernel).

Covariance Functions (Kernels)

RBF (Radial Basis) Kernel

k_{RBF}(\mathbf{x}, \mathbf{x}') = \sigma_f^2 \exp\left(-\frac{||\mathbf{x} - \mathbf{x}'||^2}{2l^2}\right)

Parameter	Meaning
$\sigma_f^2$	Signal variance
$l$	Length scale

Matern Kernel

k_{Matern}(\mathbf{x}, \mathbf{x}') = \sigma_f^2 \frac{2^{1-\nu}}{\Gamma(\nu)} \left(\frac{\sqrt{2\nu}||\mathbf{x} - \mathbf{x}'||}{l}\right)^\nu K_\nu\left(\frac{\sqrt{2\nu}||\mathbf{x} - \mathbf{x}'||}{l}\right)

Commonly used $\nu = 3/2$ or $\nu = 5/2$ .

Prediction Formulas

Training Data

\mathcal{D} = \{(\mathbf{x}_i, y_i)\}_{i=1}^{N}

Prediction Distribution

For new input $\mathbf{x}_*$ :

f_* | \mathbf{x}_*, \mathcal{D} \sim \mathcal{N}(\mu(\mathbf{x}_*), \sigma^2(\mathbf{x}_*))

Mean and Variance

\mu(\mathbf{x}_*) = \mathbf{k}_*^T \mathbf{K}_y^{-1} \mathbf{y}

\sigma^2(\mathbf{x}_*) = k(\mathbf{x}_*, \mathbf{x}_*) - \mathbf{k}_*^T \mathbf{K}_y^{-1} \mathbf{k}_*

Applications in Wind Prediction

Input Features

Feature	Description
Time $t$	Sampling time
Altitude $h$	Altitude layer
Latitude $\phi$	Geographic location
Historical wind speed $v_{t-k}$	Lag features

Uncertainty Quantification

GPR provides $95\%$ confidence intervals:

[ \mu - 1.96\sigma, \mu + 1.96\sigma ]

This is critical for safety-critical control system decisions.

Algorithm Advantages

Advantage	Description
Small sample learning	$N$ can be very small (10-100)
Uncertainty quantification	Automatic prediction variance
Interpretability	Kernel function visualization
Non-parametric	No explicit function form required

References

Rasmussen C E, Williams C K I. Gaussian Processes for Machine Learning[M]. MIT Press, 2006.
Wang H, et al. GPR-based Wind Speed Prediction for Airship Station-keeping[J]. IEEE Transactions on Aerospace Systems, 2025.