Recursive Least Squares: theoretical foundations, practical applications with copper and monetary datasets, structural break analysis, and comparisons with alternative time-varying parameter methodologies.
This notebook provides an in-depth exploration of Recursive Least Squares (RLS), an adaptive filtering technique for estimating time-varying parameters in linear regression models. Unlike traditional batch estimation methods, RLS updates parameter estimates recursively as new data becomes available, making it particularly valuable for real-time applications, structural break detection, and analyzing evolving relationships in economic and financial data.
- Mathematical derivation of RLS algorithm with forgetting factors
- Connection to Kalman filtering framework
- Comparison with Ordinary Least Squares (OLS) and other estimation methods
- Copper Market Analysis: World copper consumption modeling (1951-1975)
- Quantity Theory of Money: Testing Lucas's hypothesis on money growth and inflation
- Structural Break Detection: CUSUM and CUSUM of squares tests
- Linear constraints implementation
- Forgetting factor sensitivity analysis
- Comparison with state space models
- Bayesian and machine learning alternatives
- Predictive performance evaluation
- Time-varying parameter estimation
- Monetary policy analysis
- Commodity market dynamics
- Structural stability testing
- Real-time parameter tracking
- Outlier detection via recursive residuals
- Model stability diagnostics
- Forecast error analysis
- Rolling window OLS comparison
- State space model benchmarking
- Bayesian inference approaches
- Machine learning alternatives
- statsmodels: RLS implementation and statistical testing
- numpy: Numerical computations and array operations
- pandas: Data manipulation and time series handling
- matplotlib: Visualization and plotting
- pandas-datareader: Financial and economic data retrieval
- Recursive Coefficient Tracking: Visualize parameter evolution over time
- Stability Diagnostics: CUSUM and CUSUM of squares tests
- Forecast Evaluation: One-step ahead prediction errors
- Model Comparison: RLS vs OLS vs state space models
- Custom Implementations: Forgetting factor adjustments and constraints
- Library imports and configuration
- Data loading and preprocessing utilities
- RLS algorithm derivation
- Mathematical formulation with forgetting factors
- Advantages and limitations discussion
- Complete analysis of world copper consumption
- Parameter stability assessment
- Alternative estimation method comparisons
- Testing Lucas's hypothesis (1959-2015)
- Structural break detection
- State space model comparison
- Linear constraint implementation
- Forgetting factor analysis
- Predictive performance metrics
- Bayesian approaches
- Machine learning alternatives
- Industry applications
- Implementation guidelines
- Model validation strategies
- Future research directions
- Academic literature
- Software documentation
- Data sources
pip install numpy pandas matplotlib statsmodels pandas-datareaderimport statsmodels.api as sm
# Initialize RLS model
mod = sm.RecursiveLS(endog, exog)
res = mod.fit()
# Access recursive coefficients
coefficients = res.recursive_coefficients.filtered
# Test for structural breaks
res.plot_cusum()
res.plot_cusum_squares()fit(): Estimate model parametersplot_recursive_coefficient(): Visualize parameter evolutionplot_cusum(): Structural break detectionfrom_formula(): Formula interface for model specificationconstrained estimation: Linear constraints on parameters
- RLS effectively detects structural breaks in economic relationships
- CUSUM tests provide visual diagnostics for model stability
- The quantity theory of money shows evidence of parameter instability post-1990
- RLS converges faster than stochastic gradient methods
- Computational complexity scales quadratically with parameters
- Forgetting factors balance responsiveness to new data vs. stability
- Real-time economic monitoring
- Adaptive trading strategies
- Policy evaluation with time-varying effects
This notebook serves as both a practical implementation guide and theoretical reference for:
- Graduate students in econometrics and time series analysis
- Researchers studying time-varying parameter models
- Practitioners implementing real-time estimation systems
- Data scientists working with streaming data and adaptive algorithms
- Algorithmic Understanding: Clear derivation of RLS updating equations
- Methodological Comparison: Contrast with Kalman filtering and Bayesian methods
- Practical Implementation: Real-world case studies with economic interpretation
- Diagnostic Framework: Comprehensive model validation and stability testing
- Computational Complexity: O(k²) per update where k is number of parameters
- Initialization Sensitivity: Results dependent on initial parameter values
- Linearity Assumption: Restricted to linear-in-parameters models
- Memory Requirements: Recursive storage of coefficient matrices
- High-Dimensional RLS: Regularization techniques for many predictors
- Nonlinear Extensions: Kernel methods and neural network integration
- Distributed Computation: Parallel implementation for big data
- Automated Diagnostics: Machine learning for model specification
If using this notebook in research or publications, please acknowledge the methodological framework and cite relevant academic references from Section 8.
Contributions to extend functionality, add new examples, or improve documentation are welcome. Please ensure code follows the existing structure and includes appropriate documentation.
This project is licensed under the MIT License - see the LICENSE file for details.
This notebook is designed for educational and research purposes. Real-world applications may require additional considerations and validation.