MolmoAct: Multimodal Open Language Model for Action

---

Updates

• [2025/08/15] 🔥 Code for MolmoAct Evaluation on SimplerEnv has been released at allenai/SimplerEnv
• [2025/08/12] 🔥 Datasets used for our pre-training and mid-training have been released
• [2025/08/12] 🔥 Models have been released

Table of Contents

1. Overview
2. Release Notes
    2.1 Datasets
    2.2 Models
3. Training (WIP)
    3.1 Data Processing & Fine-tuning
    3.2 Pre-training
    3.3 Mid-training
4. Evaluation (WIP)
    4.1 SimplerEnv
    4.2 LIBERO Evaluation
    4.3 Real-world Evaluation
5. License and Use
6. Model and Hardware Safety
7. Citation
8. Contacts

Quick Links

Section	Link
Overview	#1-Overview
Datasets	#21-Datasets
Models	#22-Models
Training	#3-Training-WIP
Evaluation	#4-Evaluation-WIP
License	#5-License-and-Use
Safety	#6-Model-and-Hardware-Safety
Citation	#7-Citation
Contacts	#8-Contacts

---

1. Overview

MolmoAct is a repository for training and using AI2’s open-sourced Action Reasoning Model that can reason in space.

Note: Training code, evaluation code, and data processing scripts will be released soon. We’re finalizing them for public release to ensure reproducibility and ease of use.

---

2. Release Notes

2.1 Datasets

Data	Description	Dataset Path
MolmoAct Dataset	MolmoAct dataset in LeRobot format. All contents were collected in-house by AI2.	https://huggingface.co/datasets/allenai/MolmoAct-Dataset
MolmoAct Pre-training Mixture	Data mixture for MolmoAct pre-training. Contains a subset of OXE formulated as Action Reasoning data, auxiliary robot data, and web data.	https://huggingface.co/datasets/allenai/MolmoAct-Pretraining-Mixture
MolmoAct Mid-training Mixture	Data mixture for MolmoAct mid-training. Contains MolmoAct Dataset formulated as Action Reasoning data.	https://huggingface.co/datasets/allenai/MolmoAct-Midtraining-Mixture

2.2 Models

Model	Use Case	Description	Checkpoint Path
MolmoAct-7B-D	Fine-tuning	Best/demo MolmoAct; adapt to real robots by fine-tuning on your datasets.	https://huggingface.co/allenai/MolmoAct-7B-D-0812
MolmoAct-7B-O	Fine-tuning	Most open MolmoAct; adapt to real robots by fine-tuning on your datasets.	https://huggingface.co/allenai/MolmoAct-7B-O-0812
MolmoAct-7B-D-Pretrain	Inference	Checkpoint to replicate zero-shot results on SimplerEnv (Google Robot).	https://huggingface.co/allenai/MolmoAct-7B-D-Pretrain-0812
MolmoAct-7B-D-Pretrain-RT-1	Inference	Checkpoint to replicate RT-1 fine-tuned results on SimplerEnv (Google Robot).	https://huggingface.co/allenai/MolmoAct-7B-D-Pretrain-RT-1-0812

---

3. Training (WIP)

3.1 Data Processing & Fine-tuning

Content coming soon.

3.2 Pre-training

Content coming soon.

3.3 Mid-training

Content coming soon.

---

4. Evaluation (WIP)

4.1 Simpler-Env

We release the SimplerEnv evaluation code for MolmoAct at allenai/SimplerEnv. Please first install the dependencies for SimplerEnv Evaluation environment following allenai/SimplerEnv and dependencies for MolmoAct Inference Setup. After installing all the dependencies, evaluation scripts are located at:

# under the project dir of SimplerEnv/
bash scripts/molmoact_pick_coke_can_visual_matching.sh
bash scripts/molmoact_pick_coke_can_variant_agg.sh
bash scripts/molmoact_move_near_visual_matching.sh
bash scripts/molmoact_move_near_variant_agg.sh
bash scripts/molmoact_drawer_visual_matching.sh
bash scripts/molmoact_drawer_variant_agg.sh

4.2 LIBERO Evaluation

Content coming soon.

4.3 Real-world Evaluation

Content coming soon.

---

5. License and Use

MolmoAct is licensed under Apache 2.0 and intended for research and educational use.
For more information, please see our Responsible Use Guidelines.

---

6. Model and Hardware Safety

MolmoAct can display a visual reasoning trace of its intended actions before execution, enabling proactive auditing and adjustment of behavior. The model’s action space is bounded within the data provided, and compliance is built in to limit excessive force when resistance is detected. Always follow hardware manufacturer guidelines and operate in a safely configured environment.

---

7. Citation

@misc{molmoact2025,
      title={MolmoAct: Action Reasoning Models that can Reason in Space}, 
      author={Jason Lee and Jiafei Duan and Haoquan Fang and Yuquan Deng and Shuo Liu and Boyang Li and Bohan Fang and Jieyu Zhang and Yi Ru Wang and Sangho Lee and Winson Han and Wilbert Pumacay and Angelica Wu and Rose Hendrix and Karen Farley and Eli VanderBilt and Ali Farhadi and Dieter Fox and Ranjay Krishna},
      year={2025},
      eprint={2508.07917},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2508.07917}
}

---

8. Contacts

For questions, collaborations, or support, please contact with:

{haoquanf,jasonl,jiafeid}@allenai.org 

Found a bug or have a feature request? Please open a GitHub issue.