FactorizePhys: Matrix Factorization for Multidimensional Attention in Remote Physiological Sensing

🎯 Overview

This repository hosts the GitHub Pages website for FactorizePhys, a novel method for remote photoplethysmography (rPPG) published at NeurIPS 2024. The work introduces Factorized Self-Attention Module (FSAM) that leverages nonnegative matrix factorization to jointly compute multidimensional attention across spatial, temporal, and channel dimensions.

🚀 Key Contributions

• Novel Attention Mechanism: FSAM jointly computes spatial-temporal-channel attention using matrix factorization
• Superior Generalization: State-of-the-art cross-dataset performance across all major rPPG datasets
• Computational Efficiency: ~50x fewer parameters than existing methods while maintaining competitive performance

📊 Results Highlights

• 67% reduction in Mean Absolute Error (MAE) compared to SOTA methods
• 15% improvement in Signal-to-Noise Ratio (SNR)
• 51K parameters vs. 7.3M in competing methods
• 0.998 correlation for heart rate estimation

📝 Authors

• Jitesh Joshi - University College London, UK
• Sos S. Agaian - City University of New York, USA
• Youngjun Cho - University College London, UK

🎓 Citation

@inproceedings{joshi2024factorizephys,
    title={FactorizePhys: Matrix Factorization for Multidimensional Attention in Remote Physiological Sensing},
    author={Jitesh Joshi and Sos Agaian and Youngjun Cho},
    booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
    year={2024},
    url={https://openreview.net/forum?id=qrfp4eeZ47}
}