Official Repository for "BlendX : Complex Multi-Intent Detection with Blended Patterns." [Paper(ACL Anthology)] [Paper(arXiv)]
Yejin Yoon, Jungyeon Lee, Kangsan Kim, Chanhee Park and Taeuk Kim. Accepted to LREC-COLING2024 long paper.
BlendX is now available via Hugging Face Datasets! Check it out here: HYU-NLP/BlendX
You can easily load the dataset using HF datasets library:
from datasets import load_dataset
dataset = load_dataset("HYU-NLP/BlendX", data_dir="v1.0/BlendX/BlendATIS")
print(dataset["test"][0])Task-oriented dialogue (TOD) systems are commonly designed with the presumption that each utterance represents a single intent. However, this assumption may not accurately reflect real-world situations, where users frequently express multiple intents within a single utterance. While there is an emerging interest in multi-intent detection (MID), existing in-domain datasets such as MixATIS and MixSNIPS have limitations in their formulation. To address these issues, we present BlendX, a suite of refined datasets featuring more diverse patterns than their predecessors, elevating both its complexity and diversity. For dataset construction, we utilize both rule-based heuristics as well as a generative tool -- OpenAI's ChatGPT -- which is augmented with a similarity-driven strategy for utterance selection. To ensure the quality of the proposed datasets, we also introduce three novel metrics that assess the statistical properties of an utterance related to word count, conjunction use, and pronoun usage. Extensive experiments on BlendX reveal that state-of-the-art MID models struggle with the challenges posed by the new datasets, highlighting the need to reexamine the current state of the MID field. The dataset is available at https://github.com/HYU-NLP/BlendX.
An overview of the BlendX construction framework. Initially, we preprocess four source datasets: ATIS, Banking77, CLINC150, and SNIPS. We then select single-intent utterances from these datasets. These utterances are combined using both Manual and Generative approaches. It is important to note that utterances are kept separate and not mixed across datasets. Following the merging process, all resultant datasets are compiled to form BlendX. We particularly highlight non-trivial combinations, such as omissions, which are indicated within the blue rounded box on the rightmost side of the framework. Finally, BlendX is evaluated using three methods: custom metrics, baseline evaluation, and visualization.
The BlendX dataset comprises a suite of refined datasets with a focus on multi-intent detection in task-oriented dialogues. It introduces more diverse patterns of utterance formulation, challenging the existing MID models with its elevated complexity and diversity.
./
├── docs/
├── v1.0/
│ ├── BlendX/
│ │ ├── BlendATIS/
│ │ ├── BlendBanking77/
│ │ ├── BlendCLINC150/
│ │ └── BlendSNIPS/
│ └── MixX/
│ ├── MixATIS/
│ ├── MixBanking77/
│ ├── MixCLINC150/
│ └── MixSNIPS/
└── v2.0/
├── KoBlendX/
│ ├── KoBlendATIS/
│ ├── KoBlendBanking/
│ └── KoBlendCLINC/
└── KoMixX/
├── KoMixATIS/
├── KoMixBanking/
└── KoMixCLINC/
📂 The repository contains the following datasets and documentation organized under the v1.0/ and v2.0/ directories:
-
v1.0/– English datasets-
BlendX/– Our enhanced multi-intent dataset created by concatenating these single-intent utterance data sources: -
MixX/– Our version of MixX, incorporating a concatenation strategy from this paper and datasets, also including Banking77 and CLINC150. This version is tailored specifically for intent detection, maintaining the integrity of the original datasets.
-
-
v2.0/– Korean extensionsKoBlendX/– The Korean translation of BlendX, created from ATIS, Banking77, and CLINC150KoMixX/– The Korean version of MixX, constructed in a similar manner
-
docs/– Documentationblendx-poster.pdf: Poster presented at LREC-COLING 2024blendx-slides.pdf: Presentation slides from LREC-COLING 2024koblendx-paper.pdf: Paper on KoBlendX, accepted to KCC2024 long paper (Outstanding Paper Award)
BlendX
@inproceedings{yoon-etal-2024-blendx-complex,
title = "{B}lend{X}: Complex Multi-Intent Detection with Blended Patterns",
author = "Yoon, Yejin and
Lee, Jungyeon and
Kim, Kangsan and
Park, Chanhee and
Kim, Taeuk",
editor = "Calzolari, Nicoletta and
Kan, Min-Yen and
Hoste, Veronique and
Lenci, Alessandro and
Sakti, Sakriani and
Xue, Nianwen",
booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
month = may,
year = "2024",
address = "Torino, Italia",
publisher = "ELRA and ICCL",
url = "https://aclanthology.org/2024.lrec-main.218",
pages = "2428--2439",
abstract = "Task-oriented dialogue (TOD) systems are commonly designed with the presumption that each utterance represents a single intent. However, this assumption may not accurately reflect real-world situations, where users frequently express multiple intents within a single utterance. While there is an emerging interest in multi-intent detection (MID), existing in-domain datasets such as MixATIS and MixSNIPS have limitations in their formulation. To address these issues, we present BlendX, a suite of refined datasets featuring more diverse patterns than their predecessors, elevating both its complexity and diversity. For dataset construction, we utilize both rule-based heuristics as well as a generative tool{---}OpenAI{'}s ChatGPT{---}which is augmented with a similarity-driven strategy for utterance selection. To ensure the quality of the proposed datasets, we also introduce three novel metrics that assess the statistical properties of an utterance related to word count, conjunction use, and pronoun usage. Extensive experiments on BlendX reveal that state-of-the-art MID models struggle with the challenges posed by the new datasets, highlighting the need to reexamine the current state of the MID field. The dataset is available at \url{https://github.com/HYU-NLP/BlendX}.",
}
Yejin Yoon, Jungyeon Lee, Kangsan Kim, Chanhee Park, and Taeuk Kim. 2024. BlendX: Complex Multi-Intent Detection with Blended Patterns. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 2428–2439, Torino, Italia. ELRA and ICCL.
KoBlendX
@inproceedings{yoon-etal-2024-koblendx,
title = "{한국어 발화의 다중 의도 감지 연구}",
author = "윤예진 and 김지수 and 임정민 and 이정연 and 김태욱",
booktitle = "한국정보과학회 학술발표논문집",
year = "2024",
pages = "465--467"
}
@inproceedings{yoon-etal-2024-koblendx,
title = "Multi-Intent Detection for Korean Spoken Language",
author = "Yoon, Yejin and Kim, Jisoo and Lim, Jungmin and Lee, Jungyeon and Kim, Taeuk",
booktitle = "Proceedings of the 2024 Korea Computer Congress (KCC)",
year = "2024",
pages = "465--467"
}
Yejin Yoon, Jisoo Kim, Jungmin Lim, Jungyeon Lee, and Taeuk Kim. 2024. 한국어 발화의 다중 의도 감지 연구. In 한국정보과학회 학술발표논문집, pages 465–467.
- 2025.07.16 Renamed documentation files for clarity and added the KoBlendX paper (Accepted to KCC2024 long paper, Outstanding Paper Award).
- 2025.06.12 Uploaded to HuggingFace Datasets Hub: BlendX, MixX, KoBlendX, KoMixX
- 2024.05.26 KoBlendX is now available in the
v2.0/directory. KoBlendX is the Korean translation of BlendX. Details will be published in an upcoming paper. (Submitted to KCC2024) The presentation materials and posters from the LREC-COLING2024 conference have also been added, along with the citation. - 2024.04.15 BlendX is now available in this repository. We are also planning to release an updated version soon, which will include additional enhancements and features designed to further support research in TOD systems.
- 2024.03.28 BlendX will soon be made available in this repository, offering a comprehensive and detailed dataset designed for enhancing research in task-oriented dialogue (TOD) systems.
This repository and its contents are licensed under the GNU General Public License v2.0. By using, distributing, or contributing to this repository, you agree to the terms and conditions outlined in the LICENSE file.