State-Space Models & Kalman Filter

In modern finance, we often deal with variables that are not directly observable, such as "true" market trend or the "dynamic" beta of an asset. State-space models provide a framework to estimate these hidden states from noisy observations.

1. State-Space Representation

A state-space model consists of two equations:

  • Measurement Equation: Relates the observed data to the hidden state.
  • Transition Equation: Describes how the hidden state evolves over time.

y_t = H_t * x_t + v_t (Measurement)
x_t = F_t * x_{t-1} + w_t (Transition)

2. The Kalman Filter

The Kalman filter is an optimal recursive algorithm for estimating the hidden state. It operates in two steps:

Predict

Project the current state and error covariance forward in time to obtain the a priori estimates for the next time step.

Update

Adjust the projected estimate by an actual measurement to obtain an improved a posteriori estimate (Kalman Gain).

3. Prophet Model

The Prophet model (developed by Meta) is a robust forecasting tool that handles seasonality, holidays, and trend changes. It is particularly effective for business time series that have strong seasonal patterns and several seasons of historical data.

Why we use it in Yulmong:

  • Automatic Change Point Detection: Identifies shifts in market trends automatically.
  • Uncertainty Intervals: Provides confidence bands for risk management.
  • Robustness: Handles missing data and outliers effectively compared to traditional ARIMA.