Technical Foundations & Architecture

A mathematical document unifying phase-space trajectory reconstruction, multi-expert battery algorithms, online multiplicative weight fusion, and causal stream contracts.

1. Takens' Embedding Theorem & Phase Space Reconstruction

Floris Takens (1981) established that the unobserved multi-dimensional attractor manifold $\mathcal{M}$ of a non-linear dynamical system can be reconstructed from a single observed scalar time series $v_t$ via delay coordinate vectors:

$$\mathcal{X}_t = [v_t, v_{t-\tau}, v_{t-2\tau}, \dots, v_{t-(d-1)\tau}]^T \in \mathbb{R}^d$$

where $\tau$ is the time delay lag and $d$ is the embedding dimension. If $d > 2m + 1$ (where $m$ is the manifold dimension), the delay embedding map $\Phi: \mathcal{M} \to \mathbb{R}^d$ forms a smooth topological immersion that preserves original manifold geometry.

2. Parameter Selection: AMI and FNN

Average Mutual Information (AMI)

Fraser & Swinney (1986) demonstrated that the optimal time delay $\tau$ corresponds to the first local minimum of the Average Mutual Information $I(\tau)$:

$$I(\tau) = \sum_{x, y} P(v_t, v_{t+\tau}) \log_2 \left( \frac{P(v_t, v_{t+\tau})}{P(v_t) P(v_{t+\tau})} \right)$$

This selects a lag $\tau$ where scalar coordinates $v_t$ and $v_{t+\tau}$ are maximally independent in an information-theoretic sense.

False Nearest Neighbors (FNN)

Kennel et al. (1992) introduced False Nearest Neighbors to determine the minimal embedding dimension $d$. Points projected into dimension $d$ may appear adjacent due to geometric overlap rather than manifold proximity. The fraction of FNNs is tracked as $d$ increases:

$$\frac{|v_{t+d\tau} - v_{nn+d\tau}|}{\|\mathcal{X}_t^{(d)} - \mathcal{X}_{nn}^{(d)}\|} > R_{\text{tol}}$$

The optimal dimension $d$ is identified where the FNN fraction falls below $1\%$ (configured via FNN_THRESHOLD in config.py).

Note: compute_ami and compute_fnn are available utilities in representation.py but are not currently wired into the live TSADPipeline. The pipeline uses fixed constructor defaults tau=2, d=8 for all published benchmark results. Wiring AMI/FNN into the pipeline would require a warmup buffer and would change tau/d per stream, invalidating the reported numbers.

3. Covariance Geometry: Ledoit-Wolf Shrinkage

When sample window size $N$ is small relative to embedding dimension $d$, the standard sample covariance matrix $\Sigma_{\text{sample}}$ is ill-conditioned. Ledoit & Wolf (2004) analytical shrinkage computes a well-conditioned estimator $\Sigma_{\text{LW}}$:

$$\Sigma_{\text{LW}} = (1 - \delta) \Sigma_{\text{sample}} + \delta \mu I, \quad \mu = \frac{\text{trace}(\Sigma_{\text{sample}})}{d}$$

where optimal shrinkage intensity $\delta \in [0, 1]$ minimizes expected Frobenius loss, guaranteeing positive-definiteness and invertibility. The robust Mahalanobis distance is evaluated as:

$$D_{\text{mahal}}(Z_t) = \sqrt{(Z_t - \mu_Z)^T \Sigma_{\text{LW}}^{-1} (Z_t - \mu_Z)}$$

4. Surrogate Null Models: IAAFT

To test whether dynamic predictability exceeds linear autocorrelation noise, a surrogate null series is generated using the Iterative Amplitude Adjusted Fourier Transform (IAAFT):

Iterative Amplitude Adjusted Fourier Transform (IAAFT): Schreiber & Schmitz (1996) phase-randomization algorithm that exactly preserves both the empirical amplitude distribution and Fourier power spectrum of the original series. This is the sole surrogate null model currently implemented in evaluation/iaaft.py.

5. The 6-Detector Battery

The ensemble combines six mathematically orthogonal detectors to score each incoming streaming observation:

1. Simplex Projection (EDM)

Sugihara & May (1990) Simplex Projection performs non-parametric forecasting by distance-weighted averaging over $E+1$ nearest phase-space neighbors (not S-Map; no locally-weighted linear fit or $\theta$ parameter): $$w_i = \exp \left( -\frac{\|Z_t - Z_{nn_i}\|}{\|Z_t - Z_{nn_1}\| + \epsilon} \right), \quad \hat{v}_{t+h} = \sum_{i=1}^{E+1} \bar{w}_i v_{nn_i + h}$$ Anomaly score $s_t$ measures normalized prediction error $|v_t - \hat{v}_t|$.

2. Robust Mahalanobis (Ledoit-Wolf)

Measures geometric spatial deviation from the robust covariance centroid in projected space. Covariance is estimated from a fixed 1000-sample block buffer with full recomputation and Ledoit-Wolf shrinkage on each update (not EWMA).

3. Subsequence Motifs (Single-Window Matrix Profile)

Inspired by Yeh et al. (2016) Matrix Profile. The implementation uses a single window $w_{\text{mp}} = \max(5, \tau \cdot d)$ and computes raw (non-z-normalized) Euclidean distance from the current subsequence to its nearest non-overlapping historical neighbor via numpy.lib.stride_tricks.sliding_window_view. Not STUMPY/STOMP; not dual-scale. Peaks identify structural discords.

4. Subspace Isolation (Isolation Forest)

Liu et al. (2008) Isolation Forest isolates point anomalies via axis-aligned random feature splits. Path length $h(x)$ across trees measures anomaly severity: $$s(x, n) = 2^{-\frac{\mathbb{E}(h(x))}{c(n)}}$$

5. AR Linear Ridge Filter

AR(p) linear ridge regression filter measuring stochastic prediction residuals. Coefficients are refit by batch ordinary least squares with ridge regularization ($\lambda_{\text{ridge}} = 10^{-3}$) every 20 observations (not online RLS; no exponential forgetting factor).

6. Transformer Autoencoder (MSE Reconstruction)

Lightweight PyTorch Transformer Encoder ($L=2$ layers, $H=4$ heads, $D_{\text{model}}=32$) acting as a sequence autoencoder: $$s_t = 1 - \exp\left(-\frac{1}{L \cdot D} \sum_{l,d} (x_{l,d} - \hat{x}_{l,d})^2\right)$$

6. Meta-Judge Online Fusion: Hedge Algorithm

Freund & Schapire (1997) Hedge multiplicative weight update algorithm updates expert weights $w_{t, k}$ based on trailing Pearson correlation loss $\ell_{t, k} = 1 - \text{PearsonCorr}(S_k, E_k)$. Pearson is used instead of Spearman because the loss must be sensitive to the magnitude of linear association between anomaly scores and prediction errors, not merely their rank ordering; rank invariance would discard information about detector calibration drift. The learning rate $\eta = 0.10$ is fixed (plain Hedge, not AdaHedge):

$$w_{t+1, k} = \frac{w_{t, k} e^{-\eta \ell_{t, k}}}{\sum_{j=1}^K w_{t, j} e^{-\eta \ell_{t, j}}}, \quad \eta = 0.10$$

Fixed-share mixing (Herbster & Warmuth, 1998) guarantees a weight floor $w'_{k} \ge \frac{\sigma}{K}$ ($\sigma = 0.01$) to prevent expert weight collapse during sudden regime shifts.

7. System Compartments & Streaming Data Flow

The streaming pipeline executes sequentially across five causal compartments:

[Raw Scalar x_t]
      │
      ▼
┌─────────────────────────────────────────────────────────────┐
│ Module 1: Ingestion & Preprocessing                         │
│ - StreamBuffer(window_size=200)                             │
│ - Standardized output: v_t = (x_t - med) / (1.4826 * mad)   │
└─────────────────────────────┬───────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────┐
│ Module 2: Multidimensional Representation                   │
│ - Takens Delay Embedding: X_t in R^d (tau=2, d=8)           │
│ - JL Random Projection: Z_t in R^D_target (D_target=8)      │
│ - HNSW Graph Index: ANN trajectory indexing                 │
└─────────────────────────────┬───────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────┐
│ Module 3: 6-Detector Battery                                │
│ [Simplex, Mahalanobis-LW, MatrixProfile, IForest,           │
│  ARFilterDetector, MSETransformerAutoencoder]               │
│ Output: Scores S_t in [0,1]^K, Forecasts v_hat in R^K       │
└─────────────────────────────┬───────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────┐
│ Module 4 & 5: Meta-Judge & Online Learning Loop             │
│ - Pearson Correlation Loss: ell_{t,k} = 1 - PearsonCorr(S,E)│
│ - Fixed-rate Hedge Update (eta=0.10) + Fixed-Share Floor    │
│ - Output: Fused Anomaly Score A_t, Fused Forecast v_hat*    │
└─────────────────────────────┬───────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────┐
│ Module 6: CUSUM Change Detection Gating                     │
│ - C_t+ > H_c: Freeze adaptation during acute alarms         │
│ - Alarm >= T_drift (200): Flush baseline & reset CUSUM      │
└─────────────────────────────┴───────────────────────────────┘

8. Configuration Schema

Centralized pipeline hyperparameters defined in src/tsad/config.py:

Parameter	Type	Default	Description
`SEED`	`int`	42	Global random seed enforcing 100% execution determinism
`MAX_TAU`	`int`	100	Clamped upper bound for time delay lag $\tau$
`MAX_D`	`int`	20	Clamped upper bound for embedding dimension $d$
`D_TARGET`	`int`	8	Johnson-Lindenstrauss target dimension
`K_DETECTORS`	`int`	6	Total number of expert detectors in battery
`HEDGE_ETA`	`float`	0.10	Hedge learning rate parameter $\eta$
`FIXED_SHARE_SIGMA`	`float`	0.01	Fixed-share mixing floor parameter $\sigma$
`CUSUM_KC`	`float`	0.50	CUSUM allowance parameter $k_c$
`CUSUM_HC_SIGMA_MULT`	`float`	5.00	CUSUM alarm threshold multiplier $H_c = 5.0 \sigma_E$
`T_DRIFT`	`int`	200	Consecutive alarm steps before concept drift reset

9. Zero-Lookahead Leakage Invariant

The pipeline strictly guarantees zero-lookahead temporal isolation:

Causal Buffer Scope: All transformations operate exclusively on historical causal buffers $W_t = \{x_1, x_2, \dots, x_t\}$. No future sample is ever accessed.
NaN/Inf Safety: StreamBuffer.step forward-fills NaN/Inf values at ingestion using the last valid observation. As defense-in-depth, TSADPipeline.step applies np.nan_to_num guards on per-detector scores, per-detector forecasts, and the fused output, so a detector returning NaN/Inf cannot poison the learning loop or the gating module.
Invariant Verification: tests/integration/test_no_lookahead.py confirms that element-by-element streaming execution yields an exact Euclidean distance of 0.0000 compared to sequential batch processing.