This repository accompanies our ICML 2025 paper “Exploring Representations and Interventions in Time Series Foundation Models.”
Time series foundation models (TSFMs) are powerful tools for various applications, but their internal representations and learned concepts are not well understood. In this study, we:
- Analyze representation similarity: Investigate the structure and redundancy of representations across various TSFMs
- Perform model pruning: Leverage redundancy in representations to prune layers and improve efficiency
- Identify and localize concepts: Explore what concepts (periodicity, trends) are learned by these models
- Implement concept steering: Manipulate latent space to influence model behavior
representations-in-tsfms/
├── efficiency/ # Representation similarity & pruning tools
│ ├── chronos-forecasting/ # Upstream Chronos implementation
│ ├── tsfm_similarity/ # Similarity metrics & experiments
│ ├── produce_similarity_maps.sh
│ ├── produce_and_time_models.sh
│ └── evaluate_chronos_variants.sh
├── steering/ # Concept discovery & steering
│ ├── configs/ # YAML experiment configs
│ ├── steertool/ # Steering library & CLI
│ ├── run_steering_experiments.sh
│ └── run_separability_analysis.sh
├── environment.yml # Conda environment specification
└── create_env.sh # Helper script for env creation
- Clone the repository and initialize submodules:
git clone --recurse-submodules git@github.com:moment-timeseries-foundation-model/representations-in-tsfms.git
cd representations-in-tsfms
- Make sure that you have
conda
installed and create the environment:
bash create_env.sh
conda activate reps-tsfm
Analyze model representation similarity and prune redundant layers in TSFMs:
# Generate similarity maps between layers of TSFMs
cd efficiency
./produce_similarity_maps.sh
# Produce pruned models and time them
./produce_and_time_models.sh
# Evaluate pruned Chronos model variants
./evaluate_chronos_variants.sh
Results will be available in the results
directory.
Analyze concept separability and steer model behaviour using the provided CLI utilities:
# move to the steering module
cd steering
# Run separability analysis (creates figures under steering/results)
./run_separability_analysis.sh
# Run steering experiments (latent intervention)
./run_steering_experiments.sh
@inproceedings{wilinski2025exploring,
title={Exploring Representations and Interventions in Time Series Foundation Models},
author={Micha{\l} Wili{\'n}ski and Mononito Goswami and Willa Potosnak and Nina {\.{Z}}ukowska and Artur Dubrawski},
booktitle={Forty-second International Conference on Machine Learning},
year={2025},
url={https://openreview.net/forum?id=goVzfYtj58}
}
This project is licensed under the MIT License (see the LICENSE
file for details).