StoryWeaver

This is an official implementation of AAAI 2025 paper StoryWeaver: A Unified World Model for Knowledge-Enhanced Story Character Customization. The proposed StoryWeaver can achieve both single- and multi-character based story visualization within a unified model. For more detailed explanations and supplementary materials, please refer to our arXiv version. At the same time, we will continuously update the arXiv paper with more experiments.

🚀 Overview

🚩 Updates/Todo List

• Adding more experiments results on more stories world, e.g., Flintstons, Pokemon,...
• Code Released.
• Paper Released.

🖥️ Getting Start

Environment

The codebase is tested on

• Python 3.8

For additional python libraries, please install by:

pip install -r requirements.txt

Training StoryWeaver

use the shell script,

bash train.sh

Sample from StoryWeaver

use the shell script,

bash sample.sh

Tip

We highly recommend performing Knowledge-Enhanced Spatial Guidance on the coarse class label too for each character in the text descriptions. For example, the coarse label for character Loopy is beaver. Please refer to the code of the dataset for more details.

Visual Results

Single-Character Story Visualization

Multi-Character Story Visualization

🖊️ Citation

If StoryWeaver is helpful for your research or you wish to refer the baseline results published here, we'd really appreciate it if you could cite this paper:

@article{Zhang_Tang_Zhang_Lv_Sun_2025,
title={StoryWeaver: A Unified World Model for Knowledge-Enhanced Story Character Customization},
volume={39}, url={https://ojs.aaai.org/index.php/AAAI/article/view/33079}, DOI={10.1609/aaai.v39i9.33079},
number={9}, journal={Proceedings of the AAAI Conference on Artificial Intelligence},
author={Zhang, Jinlu and Tang, Jiji and Zhang, Rongsheng and Lv, Tangjie and Sun, Xiaoshuai},
year={2025}, month={Apr.}, pages={9951-9959} }