This repository is the source code for the ECAI paper titled "Learning the value systems of societies from preferences" (soon to be published). The paper presents a novel approach to learning value systems (value-based preferences) and value groundings (domain-specific value alignment measures) of a society of agents or stakeholders from examples of pairwise preferences between alternatives in a decision-making problem domain.
In the paper we utilize the Apollo dataset from https://rdrr.io/cran/apollo/man/apollo_swissRouteChoiceData.html, about train choice in Switzerland. The dataset includes features such as cost, time, headway, and interchanges, which are used to model agent preferences based on values. Although it also works for sequential decision making, in the paper we focus on the non-sequential decision making use case that the Apollo Dataset is about.
Refer to the paper for more details on the methodology and experiments. Then read the notebook for an explanation about the code and how to replicate the experiments. for an explanation about the code and how to replicate the experiments.
To set up the environment and run the code, follow these steps:
-
Install Python: Ensure you have Python 3.13 or higher installed on your system. You can download it from python.org.
-
Create a Virtual Environment:
python3 -m venv .venv
-
Activate the Virtual Environment:
- On macOS/Linux:
source venv/bin/activate - On Windows:
venv\Scripts\activate
- On macOS/Linux:
-
Install Dependencies:
pip install -r requirements.txt
After installation, you can open the execute_experiments.ipynb notebook in your preferred Jupyter environment to explore the code and replicate the experiments.
We added some experiments in the reviewed paper. These are executed with:
python -O train_vsl.py --dataset_name ecai_apollo -ename 'additionalTest' -e apollo -cf='algorithm_config.json' -a='rlhf'
It produces a .txt file that presents all the information summarized in the "Additional results" section of the supplementary material.